Skip to content

Infrastructure Overview

System Architecture

WithinEarth operates a hybrid cloud infrastructure with on-premise servers handling core business operations.

Architecture Components

graph TB
    subgraph "External Traffic"
        CLIENT[Clients/Users]
    end

    subgraph "Load Balancer Layer - HA Pair"
        HAP1[HAProxy Master<br/>10.32.8.36<br/>ACTIVE]
        HAP2[HAProxy Standby<br/>10.32.8.38<br/>STANDBY]
    end

    subgraph "Application Tier - 7 API Servers"
        API1[API-1<br/>10.32.8.134]
        API2[API-2<br/>10.32.8.135]
        API3[API-3<br/>10.32.8.139]
        API4[API-4<br/>10.32.8.137]
        API5[API-5<br/>10.32.8.35]
        API6[API-6<br/>10.32.8.166]
        API7[API-7<br/>10.32.8.167]
    end

    subgraph "Supplier Integration Layer"
        SP1[Proxy-1<br/>10.32.8.41]
        SP2[Proxy-2<br/>10.32.8.42]
        SP3[Proxy-3<br/>10.32.8.43]
        GW1[Gateway-1<br/>10.32.8.3]
        GW2[Gateway-2<br/>10.32.8.4]
        GW3[Gateway-3<br/>10.32.8.10]
    end

    subgraph "Primary Database"
        SQL_PRIMARY[SQL Server Primary<br/>10.32.8.130:1988<br/>withinearthUpdated]
    end

    subgraph "SQL Read Replicas via HAProxy VIP"
        SQL_HAPROXY[HAProxy SQL Cluster<br/>VIP: 10.32.8.5]
        SQL_REP1[Replica 1<br/>10.32.8.143]
        SQL_REP2[Replica 2<br/>10.32.8.149]
        SQL_REP3[Replica 3<br/>10.32.8.85]
        SQL_REP4[Replica 4<br/>10.32.8.37]
        SQL_REP5[Replica 5<br/>10.32.8.39]
    end

    subgraph "Search Key Storage - 3 Independent MongoDB"
        MONGO1[MongoDB-1<br/>10.32.8.51<br/>1.6M docs]
        MONGO2[MongoDB-2<br/>10.32.8.52<br/>800K docs]
        MONGO3[MongoDB-3<br/>10.32.8.53<br/>3.2M docs]
    end

    subgraph "Specialized MongoDB Instances"
        MONGO_ROOM[Room Mapping<br/>10.32.8.96<br/>8.5M docs]
        MONGO_API[API Logs<br/>10.32.8.75<br/>162M docs]
        MONGO_PERF[Supplier Performance<br/>10.32.8.74<br/>216M docs]
    end

    subgraph "Message Queue & Cache"
        RABBIT[RabbitMQ<br/>10.32.8.90<br/>Active]
    end

    CLIENT --> HAP1
    CLIENT -.Failover.-> HAP2

    HAP1 --> API1 & API2 & API3 & API4 & API5 & API6 & API7
    HAP2 -.Standby failover to ALL APIs.-> API1 & API2 & API3 & API4 & API5 & API6 & API7

    API1 & API2 & API3 --> SQL_PRIMARY
    API4 & API5 & API6 & API7 --> SQL_PRIMARY

    API1 & API2 & API3 --> SQL_HAPROXY
    API4 & API5 & API6 & API7 --> SQL_HAPROXY

    SQL_HAPROXY --> SQL_REP1 & SQL_REP2 & SQL_REP3 & SQL_REP4 & SQL_REP5

    API1 & API2 & API3 & API4 & API5 & API6 & API7 --> MONGO1
    API1 & API2 & API3 & API4 & API5 & API6 & API7 -.Cascading Search.-> MONGO2
    API1 & API2 & API3 & API4 & API5 & API6 & API7 -.Cascading Search.-> MONGO3

    API1 --> SP1 --> GW1
    API2 --> SP2 --> GW1
    API3 --> SP3 --> GW1

    API1 & API2 & API3 & API4 & API5 & API6 & API7 --> RABBIT

Server Inventory

Load Balancer Servers

Server IP Address OS Role Status
HAProxy-Master 10.32.8.36 Linux Load Balancer (Active) ✅ Active
HAProxy-Standby 10.32.8.38 Linux Load Balancer (Standby) ✅ Active
Zabbix Server 10.32.8.148 Linux Primary Monitoring ✅ Active
UptimeKuma 10.32.8.102 Linux Uptime Monitoring ✅ Active

Application Servers (7 API Servers)

Server IP Address OS Role Status
API-1 10.32.8.134 Windows Server API Instance ✅ Active
API-2 10.32.8.135 Windows Server API Instance ✅ Active
API-3 10.32.8.139 Windows Server API Instance ✅ Active
API-4 10.32.8.137 Windows Server API Instance ✅ Active
API-5 10.32.8.35 Windows Server API Instance ✅ Active
API-6 10.32.8.166 Windows Server API Instance ✅ Active
API-7 10.32.8.167 Windows Server API Instance ✅ Active

Database Servers - SQL Server

Server IP Address Port Database Purpose Status
SQL-Primary 10.32.8.130 1988 withinearthUpdated Primary Database ✅ Active
SQL-Replica-1 10.32.8.5 1433 withinearthUpdated Read Replica ✅ Active
SQL-Replica-2 10.32.8.143 1433 withinearthUpdated Read Replica ✅ Active
SQL-Replica-3 10.32.8.149 1433 withinearthUpdated Read Replica ✅ Active
SQL-Replica-4 10.32.8.85 1433 withinearthUpdated Read Replica ✅ Active
SQL-Replica-5 10.32.8.37 1433 withinearthUpdated Read Replica ✅ Active
SQL-Replica-6 10.32.8.39 1433 withinearthUpdated Read Replica ✅ Active
SQL-HAProxy-Cluster 10.32.8.5 1433 Replica Cluster VIP HAProxy for Replicas ✅ Active
SQL-HAProxy-Master 10.32.8.7 - - HAProxy Active ✅ Active
SQL-HAProxy-Standby 10.32.8.8 - - HAProxy Standby ✅ Active
SQL-LogTrack 10.32.8.152 1433 Logtracking API Logs ✅ Active
SQL-LogTrack-2 10.32.8.180 1433 Logtracking API Logs ✅ Active
SQL-Supplier 10.32.8.140 1988 savesupplierlog Supplier Logs ✅ Active
SQL-LogServer 10.32.8.16 1433 ErrorLog Error Logs ✅ Active
CM-Server-1 10.32.8.142 1433 Cache Manager Connection Manager ✅ Active
CM-Server-2 10.32.8.34 1433 Cache Manager Connection Manager ✅ Active
OTH-MASTER 10.32.8.11 1433 OTH Database OTH Primary ✅ Active
OTH-2 10.32.8.86 1433 OTH Database OTH Replica ✅ Active
OTH-3 10.32.8.40 1433 OTH Database OTH Replica ✅ Active

Database Servers - MongoDB

Server IP Address Port Purpose Status
Mongo-Cache-1 10.32.8.51 27017 Search Cache ✅ Active
Mongo-Cache-2 10.32.8.52 27017 Search Cache ✅ Active
Mongo-Cache-3 10.32.8.53 27017 Search Cache ✅ Active
Mongo-SupplierLog 10.32.8.18 27017 Supplier Logs ✅ Active
Mongo-RoomMapping 10.32.8.96 27017 Room Mapping ✅ Active
Mongo-UnmappedRooms 10.32.8.101 27017 Unmapped Rooms ✅ Active
Mongo-APIRQRS 10.32.8.75 27017 API Request/Response ✅ Active
Mongo-Performance 10.32.8.74 27017 Supplier Performance ✅ Active

Cache & Message Queue

Server IP Address Port Type Status
Redis-Primary 10.32.8.205 6379 Redis Cache ✅ Active
RabbitMQ 10.32.8.90 5672 Message Queue ✅ Active

Additional Infrastructure Services

Server IP Address Purpose Status
B2B-Master 10.32.8.133 B2B Platform ✅ Active
B2B-HAProxy 10.32.8.251 B2B Load Balancer ✅ Active
Tableau Server 10.32.8.129 Business Intelligence ✅ Active
ELK-HAProxy-Logs 10.32.8.132 Log Aggregation ✅ Active
ELK-Supplier-Logs 10.32.8.84 Supplier Log Analysis ✅ Active
ML-AI-Server 10.32.8.250 Machine Learning/AI ✅ Active
Hotel-Content 10.32.8.179 Hotel Content Service ✅ Active
Reverse-Proxy 10.32.8.99 Reverse Proxy ✅ Active
UptimeRobot 10.32.8.102 Uptime Monitoring ✅ Active
Supplier-Proxy-Gateway 10.32.8.3 Supplier API Gateway ✅ Active
OTH-HAProxy-Master 10.32.8.106 OTH Load Balancer (Active) ✅ Active
OTH-HAProxy-Standby 10.32.8.107 OTH Load Balancer (Standby) ✅ Active

Supplier Proxy API Servers (7 Instances)

Server IP Address Status
SupplierProxy-API-1 10.32.8.41 ✅ Active
SupplierProxy-API-2 10.32.8.42 ✅ Active
SupplierProxy-API-3 10.32.8.43 ✅ Active
SupplierProxy-API-4 10.32.8.48 ✅ Active
SupplierProxy-API-5 10.32.8.45 ✅ Active
SupplierProxy-API-6 10.32.8.46 ✅ Active
SupplierProxy-API-7 10.32.8.47 ✅ Active

Hypervisors (XCP-ng)

Server IP Address Type Status
XCP-1 10.32.8.22 XCP-ng Hypervisor ✅ Active
XCP-2 10.32.8.23 XCP-ng Hypervisor ✅ Active
XCP-3 10.32.8.24 XCP-ng Hypervisor ✅ Active
Proxmox-HAProxy-1 10.32.8.13 Proxmox Management ✅ Active
Proxmox-HAProxy-2 10.32.8.14 Proxmox Management ✅ Active

Network Topology

IP Address Ranges

  • Load Balancer Layer: 10.32.8.36 (Master), 10.32.8.38 (Standby)
  • Application Layer: 10.32.8.35, 10.32.8.134-135, 10.32.8.137, 10.32.8.139, 10.32.8.166-167 (7 API servers)
  • SQL Server Layer: 10.32.8.5, 10.32.8.9, 10.32.8.16, 10.32.8.37, 10.32.8.39, 10.32.8.85, 10.32.8.130, 10.32.8.140, 10.32.8.143, 10.32.8.149, 10.32.8.152, 10.32.8.180
  • MongoDB Layer: 10.32.8.18, 10.32.8.51-53, 10.32.8.74-75, 10.32.8.96, 10.32.8.101
  • Cache/Queue: 10.32.8.90 (RabbitMQ - Active), 10.32.8.205 (Redis - Not Used)
  • Monitoring: 10.32.8.148 (Zabbix), 10.32.8.102 (UptimeKuma), 10.32.8.209 (Docs - Temporary)
  • Other Services: 10.32.8.11, 10.32.8.34, 10.32.8.40, 10.32.8.86, 10.32.8.105-107, 10.32.8.133, 10.32.8.142

Ports in Use

Port Protocol Service
1988 TCP SQL Server (Primary)
1433 TCP SQL Server (Replicas)
27017 TCP MongoDB
6379 TCP Redis
5672 TCP RabbitMQ
80/443 TCP HTTP/HTTPS
8080 TCP HAProxy Stats

Data Flow

Search Request Flow

sequenceDiagram
    participant Client
    participant HAProxy
    participant API
    participant Redis
    participant MongoDB
    participant SQL
    participant Suppliers

    Client->>HAProxy: Hotel Search Request
    HAProxy->>API: Route to API Server
    API->>Redis: Check Cache

    alt Cache Hit
        Redis-->>API: Cached Results
        API-->>Client: Return Results
    else Cache Miss
        API->>MongoDB: Check MongoDB Cache
        alt MongoDB Hit
            MongoDB-->>API: Cached Results
            API->>Redis: Update Redis
            API-->>Client: Return Results
        else MongoDB Miss
            API->>SQL: Get Agent Config
            SQL-->>API: Agent Settings
            API->>Suppliers: Search Hotels (Parallel)
            Suppliers-->>API: Hotel Results
            API->>MongoDB: Save to Cache
            API->>Redis: Save to Cache
            API-->>Client: Return Results
        end
    end

High Availability Strategy

Current HA Configuration

Component HA Method Failover Time Status
API Servers Load Balancer (HAProxy) Immediate ✅ Configured
SQL Server Read Replicas (Manual Failover) 5-15 minutes ⚠️ Manual
MongoDB 3-Node Cluster Automatic ✅ Configured
Redis Single Instance N/A ❌ No HA
RabbitMQ Single Instance N/A ❌ No HA

Planned HA Improvements

  1. SQL Server Always On Availability Groups - Automatic failover in 10-30 seconds
  2. Redis Sentinel - Automatic failover for cache layer
  3. RabbitMQ Clustering - Message queue redundancy
  4. Hybrid DNS + IP Failover - ✅ Implemented (Nov 2025)

Capacity & Performance

Current Capacity

Metric Value Notes
Database Size ~500 GB Primary SQL Server
Total Agents 2,925 Active travel agents
Hotel Bookings 1.5M+ Historical data
Search Records 3.7M+ Search history
MongoDB Documents 10M+ Cache & logs

Performance Targets

Metric Target Current Status
API Response Time < 200ms 150-300ms ⚠️ Variable
Cache Hit Rate > 70% 60-65% ⚠️ Below Target
Database Query Time < 50ms 30-100ms ⚠️ Variable
Concurrent Users 1000+ 500-700 ✅ OK

Security

Network Security

  • All database servers on private 10.32.8.0/24 network
  • Public internet access via HAProxy only
  • SSL/TLS encryption for all external connections

Database Security

  • SQL Server: Windows Authentication + SQL Authentication
  • MongoDB: No authentication (internal network only) ⚠️
  • Redis: No authentication (internal network only) ⚠️

Security Recommendations

  • Enable MongoDB authentication
  • Enable Redis authentication
  • Implement firewall rules between network segments
  • Setup VPN for remote database access

Monitoring & Logging

Monitoring Setup

  • HAProxy Stats: http://10.32.8.209:8080/stats
  • Application Logs: SQL Server (Logtracking database)
  • Supplier Logs: MongoDB + SQL Server
  • Error Logs: SQL Server (ErrorLog database)

Health Checks

  • Database health checks every 30 seconds
  • API endpoint health checks every 10 seconds
  • SSL certificate expiry checks every 6 hours

Backup & Recovery

Backup Schedule

Component Frequency Retention Location
SQL Server (Full) Daily 2:00 AM 30 days Local + Offsite
SQL Server (Differential) Every 6 hours 7 days Local
MongoDB Daily 14 days Local

Recovery Time Objectives (RTO)

  • API Servers: < 5 minutes (load balancer failover)
  • SQL Server: < 30 minutes (manual failover to replica)
  • MongoDB: < 10 minutes (automatic replica election)

Recovery Point Objectives (RPO)

  • SQL Server: < 6 hours (differential backups)
  • MongoDB: < 24 hours (daily backups)
  • Cache Data: 0 (can be regenerated)

Future Roadmap

Short Term (Q1 2025)

  • ✅ Implement hybrid DNS + IP failover
  • ⬜ Setup SQL Server Always On AG
  • ⬜ Enable MongoDB authentication
  • ⬜ Setup Redis Sentinel

Medium Term (Q2-Q3 2025)

  • ⬜ Migrate 30% of data to NoSQL
  • ⬜ Implement connection pooling optimization
  • ⬜ Setup Prometheus + Grafana monitoring
  • ⬜ Improve cache hit rate to 80%+

Long Term (2026)

  • ⬜ Complete SQL to NoSQL migration
  • ⬜ Move to Kubernetes for container orchestration
  • ⬜ Implement service mesh (Istio/Linkerd)
  • ⬜ Multi-region deployment

Last Updated: 2025-11-15