Complete System Architecture¶

🎯 Overview¶

This page provides a comprehensive view of the WithinEarth platform from all architectural perspectives: hardware infrastructure, virtualization layer, service distribution, application architecture, data flow, and integration patterns.

📋 Table of Contents¶

Hardware Infrastructure Layer
Virtualization Layer (Hypervisors & VMs)
Service Distribution Map
Application Architecture
Data Layer Architecture
Integration Layer
Network Architecture
Complete Request Flow
Technology Stack
Architecture by Services

Hardware Infrastructure Layer¶

Physical Server Inventory¶

graph TB
    subgraph DATACENTER["LeaseWeb Data Center - 3 Physical Racks"]
        subgraph CENTRAL["CRITICAL - MAIN DATABASE"]
            MAINDB["MAINDB 10.32.8.130<br/>PHYSICAL DEDICATED<br/>Central Master Database<br/>ALL DATA - ALL WRITES<br/>SINGLE POINT OF FAILURE"]
        end

        subgraph PHYSICAL_API["PHYSICAL DEDICATED - API SERVERS"]
            API1["API-1: 10.32.8.134<br/>Physical Dedicated"]
            API2["API-2: 10.32.8.135<br/>Physical Dedicated"]
            API3["API-3: 10.32.8.139<br/>Physical Dedicated"]
            API4["API-4: 10.32.8.137<br/>Physical Dedicated"]
            API5["API-5: 10.32.8.35<br/>Physical Dedicated"]
            API6["API-6: 10.32.8.166<br/>Physical Dedicated"]
            API7["API-7: 10.32.8.167<br/>Physical Dedicated"]
        end

        subgraph PHYSICAL_HYPERVISORS["PHYSICAL - HYPERVISORS"]
            XCP1["XCP-ng 1: 10.32.8.22<br/>Physical Hypervisor"]
            XCP2["XCP-ng 2: 10.32.8.23<br/>Physical Hypervisor"]
            XCP3["XCP-ng 3: 10.32.8.24<br/>Physical Hypervisor"]
            PROX1["Proxmox 1: 10.32.8.13<br/>Physical"]
            PROX2["Proxmox 2: 10.32.8.14<br/>Physical"]
        end

        subgraph PHYSICAL_OTHER["PHYSICAL - OTHER"]
            B2B["B2B Server<br/>Physical Dedicated"]
        end

        subgraph VMS["VIRTUAL MACHINES (Hosted on Hypervisors)"]
            SQL_REP["SQL Replicas (VMs)<br/>143, 149, 85, 37, 39, 9"]
            MONGO["MongoDB Servers (VMs)<br/>51, 52, 53, 74, 75, 96, 18, 101"]
            OTH_VMS["OTH Databases (VMs)<br/>11, 86, 40"]
            CM_VMS["Channel Managers (VMs)<br/>142, 34"]
            HAPROXY["HAProxy (VMs)<br/>36, 38"]
            ZABBIX_VM["Zabbix (VM)<br/>10.32.8.148"]
            OTHER_VMS["Other VMs:<br/>Proxies, Gateways, RabbitMQ<br/>pfSense, VPN, UptimeKuma"]
        end
    end

    API1 & API2 & API3 & API4 & API5 & API6 & API7 --> MAINDB
    VMS --> XCP1 & XCP2 & XCP3
    HAPROXY --> PROX1 & PROX2

    style MAINDB fill:#ff0000,stroke:#990000,stroke-width:3px
    style CENTRAL fill:#ffcccc
    style PHYSICAL_API fill:#e3f2fd
    style PHYSICAL_HYPERVISORS fill:#e8f5e9
    style VMS fill:#fff3e0

Physical vs Virtual Summary¶

Type	Server	IP	Role
PHYSICAL	MainDB	10.32.8.130	Central Master DB - ALL DATA
PHYSICAL	API-1	10.32.8.134	Dedicated API Server
PHYSICAL	API-2	10.32.8.135	Dedicated API Server
PHYSICAL	API-3	10.32.8.139	Dedicated API Server
PHYSICAL	API-4	10.32.8.137	Dedicated API Server
PHYSICAL	API-5	10.32.8.35	Dedicated API Server
PHYSICAL	API-6	10.32.8.166	Dedicated API Server
PHYSICAL	API-7	10.32.8.167	Dedicated API Server
PHYSICAL	XCP-ng 1	10.32.8.22	Hypervisor
PHYSICAL	XCP-ng 2	10.32.8.23	Hypervisor
PHYSICAL	XCP-ng 3	10.32.8.24	Hypervisor
PHYSICAL	Proxmox 1	10.32.8.13	Hypervisor
PHYSICAL	Proxmox 2	10.32.8.14	Hypervisor
PHYSICAL	B2B Server	-	B2B Portal
VIRTUAL	SQL Replicas	143, 149, 85, 37, 39, 9	Read-only replicas
VIRTUAL	MongoDB (all)	51, 52, 53, 74, 75, 96, 18, 101	NoSQL databases
VIRTUAL	OTH Databases	11, 86, 40	Online Travel Hub
VIRTUAL	HAProxy	36, 38	Load balancers
VIRTUAL	Zabbix	10.32.8.148	Infrastructure monitoring
VIRTUAL	All Others	Various	Proxies, VPN, Channel Managers, etc.

🔲 Virtualization Layer¶

XCP-ng Hypervisor 1 (10.32.8.22) - 11 VMs¶

graph TB
    XCP1["XCP-ng Hypervisor 1<br/>10.32.8.22<br/>Physical Server"]

    XCP1 --> VM1["HAProxy-SQL-Replica-1<br/>Load balancer for SQL reads"]
    XCP1 --> VM2["MongoDB-RoomMapping<br/>10.32.8.96<br/>8.5M documents"]
    XCP1 --> VM3["SupplierProxy-Api-3<br/>10.32.8.43<br/>Supplier integration"]
    XCP1 --> VM4["OTH-3<br/>Online Travel Hub DB"]
    XCP1 --> VM5["Redis-OTH<br/>Cache for OTH"]
    XCP1 --> VM6["CM-1<br/>Channel Manager<br/>7 integrations"]
    XCP1 --> VM7["RabbitMQ<br/>10.32.8.90<br/>Message queue"]
    XCP1 --> VM8["SQL Replica 5<br/>10.32.8.37<br/>Read replica"]
    XCP1 --> VM9["HAProxy-RoomMapping1<br/>Load balancer"]
    XCP1 --> VM10["VPN-Employee<br/>WireGuard VPN"]
    XCP1 --> VM11["XOA<br/>Xen Orchestra<br/>Management"]

    style XCP1 fill:#87CEEB

XCP-ng Hypervisor 2 (10.32.8.23) - 17 VMs¶

graph TB
    XCP2["XCP-ng Hypervisor 2<br/>10.32.8.23<br/>Physical Server"]

    XCP2 --> VM1["MongoDB-ReCheck-2<br/>10.32.8.52<br/>799K documents"]
    XCP2 --> VM2["SupplierProxy-Api-5<br/>10.32.8.45"]
    XCP2 --> VM3["pfSense-2<br/>Firewall & IDS/IPS"]
    XCP2 --> VM4["OTH-HAProxy-Master<br/>10.32.8.106"]
    XCP2 --> VM5["OTH-HAProxy-Standby<br/>10.32.8.107"]
    XCP2 --> VM6["OTH-4<br/>Database server"]
    XCP2 --> VM7["HAProxy-SQL-LB-2<br/>10.32.8.8"]
    XCP2 --> VM8["ELK-SupplierLogs<br/>Log aggregation"]
    XCP2 --> VM9["MariaDB<br/>MySQL compatible DB"]
    XCP2 --> VM10["SupplierProxy-Gateway-2<br/>10.32.8.4<br/>NAT Gateway"]
    XCP2 --> VM11["VPN-2<br/>WireGuard backup"]
    XCP2 --> VM12["Proxmox-MainDB-3<br/>Database"]
    XCP2 --> VM13["HAProxy-RoomMapping2"]
    XCP2 --> VM14["NewHotelMapping<br/>Mapping service"]
    XCP2 --> VM15["OTH-Cache-Standby"]
    XCP2 --> VM16["LoadTest-VM-2"]
    XCP2 --> VM17["OTH-Cache-Master"]

    style XCP2 fill:#FFE4B5

XCP-ng Hypervisor 3 (10.32.8.24) - 14 VMs¶

graph TB
    XCP3["XCP-ng Hypervisor 3<br/>10.32.8.24<br/>Physical Server<br/>10Gbps WAN"]

    XCP3 --> VM1["MongoDB-APILogs<br/>10.32.8.75<br/>162M log documents"]
    XCP3 --> VM2["MongoDB-Supplier-Logs<br/>10.32.8.18"]
    XCP3 --> VM3["UptimeKuma<br/>10.32.8.102<br/>Uptime monitoring"]
    XCP3 --> VM4["API-LogServer-152<br/>Centralized logging"]
    XCP3 --> VM5["Hotel Content API<br/>Content service"]
    XCP3 --> VM6["GitLab<br/>Source control"]
    XCP3 --> VM7["Internal Management<br/>Admin tools"]
    XCP3 --> VM8["SupplierProxy-Api-6<br/>10.32.8.46"]
    XCP3 --> VM9["OTH-2<br/>Database server"]
    XCP3 --> VM10["DataShare-FTP<br/>File transfer"]
    XCP3 --> VM11["LoadTest-VM1<br/>Load testing"]
    XCP3 --> VM12["LogTracking<br/>10.32.8.180"]
    XCP3 --> VM13["LoadTest-VM-2"]
    XCP3 --> VM14["SupplierProxy-Gateway-1<br/>10.32.8.3<br/>NAT Gateway"]

    style XCP3 fill:#90EE90

Proxmox Cluster (HAProxy Aloha VMs)¶

graph LR
    PROX["Proxmox HA Cluster<br/>Nodes: 10.32.8.13, 14"]

    PROX --> HAP1["HAProxy Master<br/>10.32.8.36<br/>Public: 89.149.192.33<br/>Active LB"]
    PROX --> HAP2["HAProxy Standby<br/>10.32.8.38<br/>Public: 89.149.192.33<br/>Failover LB"]

    style PROX fill:#87CEEB
    style HAP1 fill:#90EE90
    style HAP2 fill:#FFE4B5

VM Distribution Summary¶

Hypervisor	VMs	Key Services
XCP-1 (10.32.8.22)	11	RabbitMQ, SQL Replica 5, MongoDB RoomMapping, CM-1, VPN
XCP-2 (10.32.8.23)	17	MongoDB ReCheck, pfSense-2, OTH HAProxy pair, Gateway-2
XCP-3 (10.32.8.24)	14	MongoDB Logs, UptimeKuma, GitLab, Gateway-1
Proxmox Cluster	2	HAProxy Master/Standby pair
Total VMs	44	Mixed services across 5 hypervisors

🗺️ Service Distribution Map¶

Complete Service Mapping Across Infrastructure¶

graph TB
    subgraph PHYSICAL["Physical Servers"]
        direction TB
        subgraph API_PHYSICAL["API Services (7 Physical Servers)"]
            API1["API-1: 10.32.8.134<br/>43 IIS pools<br/>Windows Server"]
            API2["API-2: 10.32.8.135<br/>43 IIS pools"]
            API3["API-3: 10.32.8.139<br/>43 IIS pools"]
            API4["API-4: 10.32.8.137<br/>43 IIS pools"]
            API5["API-5: 10.32.8.35<br/>43 IIS pools"]
            API6["API-6: 10.32.8.166<br/>43 IIS pools"]
            API7["API-7: 10.32.8.167<br/>43 IIS pools"]
        end

        subgraph DB_PHYSICAL["Database (Physical - SPOF)"]
            SQL_P["SQL Primary (MainDB)<br/>10.32.8.130<br/>withinearthUpdated<br/>SINGLE POINT OF FAILURE"]
        end
    end

    subgraph VMS["Virtual Machines (All Others)"]
        direction TB
        subgraph DATABASE_VMS["Database Services (VMs)"]
            SQL_R["6 SQL Replicas (VMs)<br/>143, 149, 85, 37, 39, 9"]
            MONGO_P["8 MongoDB Servers (VMs)<br/>51-53 (Search Keys)<br/>74, 75, 96, 18, 101"]
            OTH["3 OTH Databases (VMs)<br/>11, 86, 40"]
            CM["2 Channel Managers (VMs)<br/>142, 34"]
        end

        subgraph INTEGRATION_VMS["Integration Services (VMs)"]
            PROXY["7 Supplier Proxies<br/>41-47<br/>Supplier API calls"]
            GATEWAY["3 NAT Gateways<br/>3, 4, 10<br/>Public routing"]
        end

        subgraph INFRASTRUCTURE_VMS["Infrastructure Services (VMs)"]
            HAPROXY["2 HAProxy (Master/Standby)<br/>36, 38"]
            RABBIT["RabbitMQ<br/>90"]
            VPN["2 WireGuard VPNs"]
            PFSENSE["pfSense Firewalls"]
        end

        subgraph MONITORING_VMS["Monitoring Services (VMs)"]
            ZABBIX["Zabbix<br/>10.32.8.148<br/>60+ hosts"]
            UPTIME["UptimeKuma<br/>102"]
            ELK["ELK Stack<br/>Log aggregation"]
            GITLAB["GitLab<br/>Source control"]
        end
    end

    style API_PHYSICAL fill:#87CEEB
    style DB_PHYSICAL fill:#ff6666
    style DATABASE_VMS fill:#FFE4B5
    style INTEGRATION_VMS fill:#FFE4B5
    style INFRASTRUCTURE_VMS fill:#90EE90
    style MONITORING_VMS fill:#DDA0DD

Service-to-Server Mapping Table¶

Service Type	Service Name	Server Type	IP Address	Host
Load Balancer	HAProxy Master	VM	10.32.8.36	Proxmox-1
Load Balancer	HAProxy Standby	VM	10.32.8.38	Proxmox-2
Application	API-1	Physical	10.32.8.134	Dedicated
Application	API-2	Physical	10.32.8.135	Dedicated
Application	API-3	Physical	10.32.8.139	Dedicated
Application	API-4	Physical	10.32.8.137	Dedicated
Application	API-5	Physical	10.32.8.35	Dedicated
Application	API-6	Physical	10.32.8.166	Dedicated
Application	API-7	Physical	10.32.8.167	Dedicated
Database	MainDB (Primary)	Physical	10.32.8.130	Dedicated
Database	SQL Replica 1	VM	10.32.8.143	XCP-1
Database	SQL Replica 2	VM	10.32.8.149	XCP-2
Database	SQL Replica 3	VM	10.32.8.85	XCP-2
Database	SQL Replica 4	VM	10.32.8.39	XCP-3
Database	SQL Replica 5	VM	10.32.8.37	XCP-1
Database	SQL Replica 6	VM	10.32.8.9	XCP-3
NoSQL	MongoDB-1 (Search)	VM	10.32.8.51	XCP-1
NoSQL	MongoDB-2 (Search)	VM	10.32.8.52	XCP-2
NoSQL	MongoDB-3 (Search)	VM	10.32.8.53	XCP-2
NoSQL	MongoDB-4 (Perf)	VM	10.32.8.74	XCP-3
NoSQL	MongoDB-5 (Logs)	VM	10.32.8.75	XCP-3
NoSQL	MongoDB-6 (RoomMap)	VM	10.32.8.96	XCP-1
NoSQL	MongoDB-7 (SupplierLog)	VM	10.32.8.18	XCP-3
NoSQL	MongoDB-8 (Unmapped)	VM	10.32.8.101	XCP-2
Integration	Supplier Proxy 1	VM	10.32.8.41	XCP-1
Integration	Supplier Proxy 2	VM	10.32.8.42	XCP-2
Integration	Supplier Proxy 3	VM	10.32.8.43	XCP-1
Integration	Supplier Proxy 4	VM	10.32.8.48	XCP-2
Integration	Supplier Proxy 5	VM	10.32.8.45	XCP-2
Integration	Supplier Proxy 6	VM	10.32.8.46	XCP-3
Integration	Supplier Proxy 7	VM	10.32.8.47	XCP-3
Gateway	NAT Gateway 1	VM	10.32.8.3	XCP-3
Gateway	NAT Gateway 2	VM	10.32.8.4	XCP-2
Gateway	NAT Gateway 3	VM	10.32.8.10	XCP-3
Queue	RabbitMQ	VM	10.32.8.90	XCP-1
Database	OTH-1	VM	10.32.8.11	XCP-1
Database	OTH-2	VM	10.32.8.86	XCP-3
Database	OTH-3	VM	10.32.8.40	XCP-1
Database	Channel Manager 1	VM	10.32.8.142	XCP-1
Database	Channel Manager 2	VM	10.32.8.34	XCP-2
Monitoring	Zabbix	VM	10.32.8.148	XCP-3
Monitoring	UptimeKuma	VM	10.32.8.102	XCP-3
Security	pfSense-1	VM	-	XCP-1
Security	pfSense-2	VM	-	XCP-2
VPN	WireGuard Primary	VM	-	XCP-1
VPN	WireGuard Backup	VM	-	XCP-2
Hypervisor	XCP-ng 1	Physical	10.32.8.22	Dedicated
Hypervisor	XCP-ng 2	Physical	10.32.8.23	Dedicated
Hypervisor	XCP-ng 3	Physical	10.32.8.24	Dedicated
Hypervisor	Proxmox 1	Physical	10.32.8.13	Dedicated
Hypervisor	Proxmox 2	Physical	10.32.8.14	Dedicated
B2B	B2B Portal	Physical	-	Dedicated

🏗️ Application Architecture¶

Complete Application Stack¶

graph TB
    subgraph CLIENTS["Client Layer"]
        AGENTS["345 Travel Agents<br/>Web/Mobile Apps"]
        B2B["B2B Partners<br/>API Integration"]
    end

    subgraph LB_LAYER["Load Balancing Layer"]
        HAPROXY["HAProxy Aloha HA<br/>Master: 10.32.8.36<br/>Standby: 10.32.8.38<br/>VIP: 89.149.192.33"]
    end

    subgraph APP_LAYER["Application Layer (7 Servers × 43 IIS Pools = 301 Pools)"]
        direction LR
        API_POOLS["IIS Application Pools<br/>XConnect API (.NET Core)<br/>Port Range: 19169-19635<br/><br/>Each API server runs:<br/>• 43 isolated pools<br/>• One pool per client<br/>• Dedicated resources"]
    end

    subgraph INTEGRATION_LAYER["Integration Layer"]
        direction LR
        PROXIES["7 Supplier Proxies<br/>Forward supplier requests"]
        GATEWAYS["3 NAT Gateways<br/>Route to internet"]
        SUPPLIERS["141 External Suppliers<br/>Expedia, Agoda, Booking.com<br/>HotelBeds, etc."]
    end

    subgraph CACHE_LAYER["Caching Layer"]
        MONGODB_SEARCH["MongoDB Search Keys<br/>51, 52, 53<br/>Cascading read"]
        MONGODB_OTHER["MongoDB Other<br/>RoomMap, Logs, Performance"]
    end

    subgraph DATA_LAYER["Data Layer"]
        SQL_WRITE["SQL Primary<br/>10.32.8.130<br/>All writes"]
        SQL_READ["SQL Replicas (via HAProxy)<br/>VIP: 10.32.8.5<br/>All reads"]
        OTH_DB["OTH Databases<br/>VIP: 10.32.8.105"]
        CM_DB["Channel Manager DBs<br/>142, 34"]
    end

    subgraph QUEUE_LAYER["Message Queue"]
        RABBIT["RabbitMQ<br/>10.32.8.90<br/>Async processing"]
    end

    CLIENTS --> HAPROXY
    HAPROXY --> APP_POOLS

    APP_POOLS --> PROXIES
    PROXIES --> GATEWAYS
    GATEWAYS --> SUPPLIERS

    APP_POOLS --> MONGODB_SEARCH
    APP_POOLS --> MONGODB_OTHER
    APP_POOLS --> SQL_WRITE
    APP_POOLS --> SQL_READ
    APP_POOLS --> OTH_DB
    APP_POOLS --> CM_DB
    APP_POOLS --> RABBIT

    style HAPROXY fill:#90EE90
    style APP_POOLS fill:#87CEEB
    style SQL_WRITE fill:#FFB6C1
    style SUPPLIERS fill:#FFA500

Application Layer Details¶

Component	Count	Details
API Servers	7	Physical Windows servers
IIS Pools per Server	43	Isolated application pools
Total IIS Pools	301	One pool per client domain
Port Range	19169-19635	Each pool on unique port
Framework	.NET Core	Latest version
Concurrency Model	Async/Await	Optimized for I/O
Connection Pooling	SQL + HTTP	Reuse connections

💾 Data Layer Architecture¶

Database Architecture Overview¶

graph TB
    subgraph WRITE_PATH["Write Path"]
        APP_W["Application Servers"]
        SQL_PRIMARY["SQL Primary<br/>10.32.8.130:1988<br/>withinearthUpdated<br/><b>ALL WRITES</b>"]

        APP_W -->|"INSERT/UPDATE/DELETE"| SQL_PRIMARY
    end

    subgraph REPLICATION["Replication Layer"]
        direction TB
        SQL_PRIMARY -->|"Always On AG"| REP["5 SQL Replicas<br/>Continuous replication"]

        HAPROXY_SQL["SQL HAProxy<br/>VIP: 10.32.8.5<br/>Nodes: 7, 8"]

        REP --> HAPROXY_SQL
    end

    subgraph READ_PATH["Read Path"]
        APP_R["Application Servers"]
        HAPROXY_SQL -->|"Round-robin"| R1["Replica 1<br/>10.32.8.143"]
        HAPROXY_SQL -->|"Round-robin"| R2["Replica 2<br/>10.32.8.149"]
        HAPROXY_SQL -->|"Round-robin"| R3["Replica 3<br/>10.32.8.85"]
        HAPROXY_SQL -->|"Round-robin"| R4["Replica 4<br/>10.32.8.39"]
        HAPROXY_SQL -->|"Round-robin"| R5["Replica 5<br/>10.32.8.37"]

        APP_R -->|"SELECT queries"| HAPROXY_SQL
    end


    style SQL_PRIMARY fill:#FFB6C1
    style HAPROXY_SQL fill:#90EE90

Database Inventory¶

Database Type	Purpose	Server(s)	Documents/Rows	Details
SQL Primary	Main OLTP database	10.32.8.130	9.3M hotels, 33M mappings	All writes
SQL Replicas	Read scaling	5 servers via VIP	Same as primary	Read-only
MongoDB Search Keys	Search key cache	51, 52, 53	5.7M total	Cascading read
MongoDB Room Mapping	Hotel room mappings	96	8.5M	Mapping data
MongoDB Logs	API request logs	75	162M	Historical logs
MongoDB Performance	Performance metrics	74	216M	APM data
OTH Databases	Online Travel Hub	11, 86, 40	Various	OTH system
Channel Manager	CM integrations	142, 34	Various	7 CM systems each
RabbitMQ	Message queue	90	N/A	Async processing

Data Volume Summary¶

Metric	Value
Total SQL Rows	~45M+ rows across 37 tables
Total MongoDB Docs	390M+ documents across 8 servers
MongoDB Search Keys	5.7M (30-60 min retention)
MongoDB Logs	162M (historical)
MongoDB Performance	216M (metrics)
SQL Replicas	5 servers for read scaling
Replication Lag	<1 second (Always On AG)

🔌 Integration Layer¶

Supplier Integration Architecture¶

graph LR
    subgraph API_SERVERS["API Servers"]
        API1["API-1<br/>134"]
        API2["API-2<br/>135"]
        API3["API-3<br/>139"]
        API4["API-4<br/>137"]
        API5["API-5<br/>35"]
        API6["API-6<br/>166"]
        API7["API-7<br/>167"]
    end

    subgraph PROXY_LAYER["Supplier Proxy Layer (VMs)"]
        P1["Proxy-1<br/>41"]
        P2["Proxy-2<br/>42"]
        P3["Proxy-3<br/>43"]
        P4["Proxy-4<br/>48"]
        P5["Proxy-5<br/>45"]
        P6["Proxy-6<br/>46"]
        P7["Proxy-7<br/>47"]
    end

    subgraph GATEWAY_LAYER["NAT Gateway Layer"]
        GW1["Gateway-1<br/>10.32.8.3<br/>Public: 95.211.19.201"]
        GW2["Gateway-2<br/>10.32.8.4<br/>Public: 95.211.19.207"]
        GW3["Gateway-3<br/>10.32.8.10<br/>Public: 95.211.19.211"]
    end

    subgraph SUPPLIERS["External Suppliers (141 Total)"]
        direction TB
        SUP1["Expedia<br/>Hotels, availability, booking"]
        SUP2["Agoda<br/>Hotels, rates, inventory"]
        SUP3["Booking.com<br/>Properties, pricing"]
        SUP4["HotelBeds<br/>Content, bookings"]
        SUP5["137 more suppliers..."]
    end

    API1 --> P1 --> GW1
    API2 --> P2 --> GW1
    API3 --> P3 --> GW1
    API4 --> P4 --> GW2
    API5 --> P5 --> GW2
    API6 --> P6 --> GW3
    API7 --> P7 --> GW3

    GW1 --> SUPPLIERS
    GW2 --> SUPPLIERS
    GW3 --> SUPPLIERS

    style SUPPLIERS fill:#FFA500

Supplier Integration Summary¶

Integration Type	Count	Details
Total Suppliers	141	Aggregated hotel suppliers
Supplier Proxies	7	One per API server
NAT Gateways	3	Public IP routing
Hotels Aggregated	9.3M	Worldwide coverage
Mappings	33M	Hotel-supplier relationships
Channel Managers	7	Derbysoft, RateGain, etc.

🌐 Network Architecture¶

Network Layers¶

graph TB
    INTERNET["Internet"]

    subgraph PUBLIC["Public-Facing Layer"]
        HAPROXY_PUB["HAProxy Public VIP<br/>89.149.192.33"]
        VPN_PUB["WireGuard VPN<br/>95.211.19.203"]
        GW_PUBS["Gateway Public IPs<br/>95.211.19.201, 207, 211"]
    end

    subgraph FIREWALL["Firewall Layer"]
        PFS["pfSense IDS/IPS<br/>95.211.19.202, 205"]
    end

    subgraph INTERNAL["Internal Network (10.32.8.0/24)"]
        direction LR
        SERVERS["60+ Internal Servers<br/>APIs, Databases, Services"]
    end

    INTERNET --> HAPROXY_PUB
    INTERNET --> VPN_PUB
    INTERNET <--> GW_PUBS
    INTERNET --> PFS
    PFS --> INTERNAL
    HAPROXY_PUB --> INTERNAL
    VPN_PUB --> INTERNAL
    INTERNAL --> GW_PUBS

    style PFS fill:#FFB6C1
    style INTERNAL fill:#87CEEB

See Network Architecture for complete details.

🔄 Complete Request Flow¶

End-to-End Request Trace¶

sequenceDiagram
    participant Agent as Travel Agent
    participant HAProxy as HAProxy LB<br/>89.149.192.33
    participant API as API Server<br/>(1 of 7)
    participant MongoCache as MongoDB<br/>Search Keys
    participant SQLRead as SQL Replica<br/>via VIP
    participant Proxy as Supplier Proxy
    participant Gateway as NAT Gateway
    participant Supplier as External Supplier<br/>(Expedia, etc.)

    Agent->>HAProxy: HTTPS Request<br/>search hotels
    HAProxy->>HAProxy: Check rate limit<br/>IP whitelist
    HAProxy->>API: Forward to IIS pool

    API->>MongoCache: Check search key cache<br/>(tries 51→52→53)

    alt Cache Hit
        MongoCache-->>API: Return cached results
    else Cache Miss
        API->>SQLRead: Query permissions<br/>mappings, config
        SQLRead-->>API: Return DB data
        API->>Proxy: Forward to supplier
        Proxy->>Gateway: Route via NAT
        Gateway->>Supplier: External API call
        Supplier-->>Gateway: Hotel availability
        Gateway-->>Proxy: Response
        Proxy-->>API: Supplier data
        API->>MongoCache: Cache search key<br/>(30-60 min TTL)
    end

    API-->>HAProxy: Return results
    HAProxy-->>Agent: HTTPS Response

Request Flow Steps¶

Client Request: Travel agent sends hotel search via HTTPS to public VIP
Load Balancer: HAProxy validates (rate limit, IP whitelist), routes to API pool
Cache Check: API checks MongoDB (51→52→53) for cached search key
Database Query: If cache miss, query SQL replicas for permissions/mappings
Supplier Call: Forward request via Proxy → Gateway → External supplier
Response Assembly: Combine supplier data, cache results, return to client
Total Latency: Target <100ms (P50), currently 300ms

🛠️ Technology Stack¶

Complete Technology Inventory¶

Layer	Technology	Version	Purpose
Hypervisor	XCP-ng	Latest	VM hosting (42 VMs)
Hypervisor	Proxmox	Latest	HAProxy VM hosting
OS (API)	Windows Server	2019/2022	API server OS
OS (Linux)	Ubuntu/Debian	20.04/22.04	VM OS
Web Server	IIS	10.0	Application pool hosting
Application	.NET Core	6.0/8.0	XConnect API
Load Balancer	HAProxy Aloha	Enterprise	Load balancing + HA
Database	SQL Server	2019	Primary OLTP database
Database	MongoDB	4.4/5.0	NoSQL caching/logging
Replication	Always On AG	SQL Server	SQL replication
Message Queue	RabbitMQ	3.x	Async processing
Monitoring	Zabbix	6.x	Infrastructure monitoring
Monitoring	UptimeKuma	Latest	Uptime monitoring
Monitoring	Netdata	Latest	Bandwidth monitoring
Firewall	pfSense	2.6+	IDS/IPS, NAT
VPN	WireGuard	Latest	Secure remote access
Source Control	GitLab	CE	Code repository
Log Aggregation	ELK Stack	8.x	Centralized logging

📊 Architecture Summary¶

Infrastructure at a Glance¶

Aspect	Details
Physical Servers	27+ dedicated servers across 3 racks
Hypervisors	5 (3 XCP-ng + 2 Proxmox)
Virtual Machines	44 VMs distributed across hypervisors
Total Servers	60+ (physical + virtual)
API Servers	7 physical servers, 301 IIS pools
Database Servers	14 (6 SQL + 8 MongoDB)
Network	Flat 10.32.8.0/24, dual 10Gbps switches
Public IPs	15+ for various services
Hosting	LeaseWeb datacenter, 3 racks

Service Count Summary¶

Service Category	Count
Load Balancers	6 pairs (12 instances)
API Servers	7 physical
Application Pools	301 IIS pools
SQL Databases	6 (1 primary + 5 replicas)
MongoDB Instances	8 servers
Supplier Proxies	7 VMs
NAT Gateways	3 VMs
Firewalls	2 pfSense instances
VPN Servers	2 WireGuard instances
Monitoring	3 (Zabbix, UptimeKuma, Netdata)

🏢 Architecture by Services¶

This section provides detailed architecture documentation for each individual service in the WithinEarth platform.

HAProxy Aloha¶

HAProxy Aloha is the enterprise load balancer that serves as the entry point for all API traffic.

graph TB
    subgraph INTERNET["Internet"]
        CLIENTS["B2B Clients<br/>Travel Agents, OTAs"]
    end

    subgraph HAPROXY_CLUSTER["HAProxy Aloha Enterprise Cluster"]
        subgraph ACTIVE["Active Node"]
            HAP1["HAProxy Master<br/>10.32.8.36<br/>VM on Proxmox-1"]
        end

        subgraph STANDBY["Standby Node"]
            HAP2["HAProxy Standby<br/>10.32.8.38<br/>VM on Proxmox-2"]
        end

        VIP["Virtual IP<br/>89.149.192.33<br/>Floats between nodes"]
    end

    subgraph API_BACKEND["API Backend Pool"]
        API1["API-1: 10.32.8.134"]
        API2["API-2: 10.32.8.135"]
        API3["API-3: 10.32.8.139"]
        API4["API-4: 10.32.8.137"]
        API5["API-5: 10.32.8.35"]
        API6["API-6: 10.32.8.166"]
        API7["API-7: 10.32.8.167"]
    end

    CLIENTS -->|"HTTPS :443"| VIP
    VIP --> HAP1
    VIP -.->|"Failover"| HAP2
    HAP1 & HAP2 -->|"Round-robin"| API1 & API2 & API3 & API4 & API5 & API6 & API7

    style HAP1 fill:#90EE90
    style HAP2 fill:#FFE4B5
    style VIP fill:#87CEEB

HAProxy Configuration¶

Setting	Value	Description
Public VIP	89.149.192.33	Floating IP for client access
Master Node	10.32.8.36	Active load balancer
Standby Node	10.32.8.38	Passive failover node
Hosting	Proxmox Cluster	VMs on Proxmox 1 & 2
Edition	Aloha Enterprise	Licensed enterprise version
Management UI	haproxy-ui.withinearth.com	Web-based configuration
Protocol	HTTPS (TLS 1.2+)	SSL termination at HAProxy
Backend Algorithm	Round-robin	Even distribution across API servers
Health Checks	HTTP /health	Every 5 seconds
Session Persistence	None	Stateless API design

Rate Limiting¶

Feature	Configuration
Rate Limiting	Per-client IP based
Applied To	Search endpoint only
Management	haproxy-ui.withinearth.com
Default Limit	Configured per agent

Failover Behavior¶

Master monitors: HAProxy Master sends heartbeats
Failure detection: Standby detects master failure within 2-3 seconds
VIP migration: Virtual IP moves to standby automatically
Traffic resumes: Clients reconnect to same VIP, traffic flows to new active

API Servers¶

The XConnect API runs on 7 dedicated physical servers, each hosting 43 isolated IIS application pools.

graph TB
    subgraph HAPROXY["Load Balancer"]
        LB["HAProxy VIP<br/>89.149.192.33"]
    end

    subgraph API_FARM["API Server Farm (7 Physical Dedicated Servers)"]
        subgraph API1["API-1: 10.32.8.134"]
            POOLS1["43 IIS Pools<br/>Ports 19169-19211"]
        end
        subgraph API2["API-2: 10.32.8.135"]
            POOLS2["43 IIS Pools<br/>Ports 19169-19211"]
        end
        subgraph API3["API-3: 10.32.8.139"]
            POOLS3["43 IIS Pools<br/>Ports 19169-19211"]
        end
        subgraph API4["API-4: 10.32.8.137"]
            POOLS4["43 IIS Pools<br/>Ports 19169-19211"]
        end
        subgraph API5["API-5: 10.32.8.35"]
            POOLS5["43 IIS Pools<br/>Ports 19169-19211"]
        end
        subgraph API6["API-6: 10.32.8.166"]
            POOLS6["43 IIS Pools<br/>Ports 19169-19211"]
        end
        subgraph API7["API-7: 10.32.8.167"]
            POOLS7["43 IIS Pools<br/>Ports 19169-19211"]
        end
    end

    subgraph DEPENDENCIES["Backend Dependencies"]
        SQL["SQL Server<br/>MainDB + Replicas"]
        MONGO["MongoDB<br/>Search Keys + Logs"]
        REDIS["Redis Cluster<br/>SP Data Cache"]
        SUPPLIERS["56+ Suppliers<br/>via Proxies"]
    end

    LB --> API1 & API2 & API3 & API4 & API5 & API6 & API7
    API_FARM --> SQL & MONGO & REDIS & SUPPLIERS

    style API_FARM fill:#e3f2fd
    style SQL fill:#FFB6C1
    style MONGO fill:#90EE90
    style REDIS fill:#ff9999

API Server Inventory¶

Server	IP Address	Type	IIS Pools	Port Range
API-1	10.32.8.134	Physical Dedicated	43	19169-19211
API-2	10.32.8.135	Physical Dedicated	43	19169-19211
API-3	10.32.8.139	Physical Dedicated	43	19169-19211
API-4	10.32.8.137	Physical Dedicated	43	19169-19211
API-5	10.32.8.35	Physical Dedicated	43	19169-19211
API-6	10.32.8.166	Physical Dedicated	43	19169-19211
API-7	10.32.8.167	Physical Dedicated	43	19169-19211
Total	-	7 Servers	301 Pools	-

API Server Specifications¶

Specification	Value
OS	Windows Server 2019/2022
Web Server	IIS 10.0
Framework	.NET 9 / ASP.NET Core
Pools per Server	43 (one per client domain)
Total Pools	301 across all servers
Concurrency	Async/Await pattern
Connection Pooling	SQL + HTTP connection reuse

API Endpoints¶

Endpoint	Purpose	Volume
availability	Hotel search across suppliers	~7.3M/day
ReCheck	Price/availability verification	High
CancellationPolicy	Get cancellation terms	Medium
PreBook	Reserve before payment	Medium
Book	Confirm booking	~2,746/day
BookingDetail	Retrieve booking info	Medium
CancelBooking	Cancel reservations	Low

Deployment Process¶

Stage	Current	Future
Build	Visual Studio manual build	Jenkins automated build
Deploy	Manual copy to each server	GitLab CI/CD pipeline
Rollback	Manual restore	Automated rollback
Testing	Manual verification	Automated test suite

SQL Replica Servers & Redis Cluster¶

SQL Replicas provide read scaling, while Redis Cluster is the new caching layer for supplier-agent mapping data.

graph TB
    subgraph MAINDB["Primary Database (Physical - SPOF)"]
        MASTER["MainDB<br/>10.32.8.130:1988<br/>SQL Server 2019<br/>ALL WRITES"]
    end

    subgraph REPLICATION["SQL Server Replication"]
        direction LR
        MASTER -->|"Always On AG<br/>Async Replication"| R1["Replica 1<br/>10.32.8.143"]
        MASTER -->|"Always On AG"| R2["Replica 2<br/>10.32.8.149"]
        MASTER -->|"Always On AG"| R3["Replica 3<br/>10.32.8.85"]
        MASTER -->|"Always On AG"| R4["Replica 4<br/>10.32.8.39"]
        MASTER -->|"Always On AG"| R5["Replica 5<br/>10.32.8.37"]
        MASTER -->|"Always On AG"| R6["Replica 6<br/>10.32.8.9"]
    end

    subgraph SQL_LB["SQL Read Load Balancer"]
        SQLVIP["HAProxy SQL VIP<br/>10.32.8.5<br/>Nodes: 7, 8"]
    end

    subgraph REDIS["Redis Cluster (NEW - SP Data Cache)"]
        REDIS_M["Redis Master<br/>10.32.8.202"]
        REDIS_R1["Redis Replica 1<br/>10.32.8.203"]
        REDIS_R2["Redis Replica 2<br/>10.32.8.204"]

        REDIS_M -->|"Replication"| REDIS_R1
        REDIS_M -->|"Replication"| REDIS_R2
    end

    R1 & R2 & R3 & R4 & R5 & R6 --> SQLVIP

    subgraph API["API Servers"]
        APIS["7 API Servers"]
    end

    APIS -->|"WRITES"| MASTER
    APIS -->|"READS"| SQLVIP
    APIS -->|"SP Mapping Cache"| REDIS

    style MASTER fill:#ff6666
    style SQLVIP fill:#90EE90
    style REDIS fill:#ff9999

SQL Replica Inventory¶

Server	IP Address	Type	Host	Role
Replica 1	10.32.8.143	VM	XCP-1	Read-only replica
Replica 2	10.32.8.149	VM	XCP-2	Read-only replica
Replica 3	10.32.8.85	VM	XCP-2	Read-only replica
Replica 4	10.32.8.39	VM	XCP-3	Read-only replica
Replica 5	10.32.8.37	VM	XCP-1	Read-only replica
Replica 6	10.32.8.9	VM	XCP-3	Read-only replica

SQL Replication Configuration¶

Setting	Value
Replication Type	SQL Server Always On AG
Sync Mode	Asynchronous
Replication Lag	< 1 second typically
Load Balancer VIP	10.32.8.5
LB Nodes	10.32.8.7, 10.32.8.8
Distribution	Round-robin

Redis Cluster Configuration¶

Setting	Value
Cluster Type	3-node Master-Replica
Master Node	10.32.8.202
Replica Node 1	10.32.8.203
Replica Node 2	10.32.8.204
Hosting	Spread across XCP hypervisors
Purpose	Supplier-Agent mapping data cache
Status	Production - Running

Redis Use Case¶

graph LR
    API["API Server"] -->|"Check Cache"| REDIS["Redis Cluster"]
    REDIS -->|"Cache Hit"| API
    REDIS -->|"Cache Miss"| SQL["SQL Replica"]
    SQL -->|"Data"| API
    API -->|"Cache Result"| REDIS

    style REDIS fill:#ff9999
    style SQL fill:#87CEEB

Cache Data	Description
SP Mapping	Supplier-Agent relationship data
Purpose	Reduce SQL read load
TTL	Configurable per data type
Eviction	LRU (Least Recently Used)

Logging Servers¶

Comprehensive logging infrastructure for API requests, supplier calls, and system events.

graph TB
    subgraph SOURCES["Log Sources"]
        API["7 API Servers"]
        HAPROXY["HAProxy"]
        SUPPLIERS["Supplier Proxies"]
        MAINDB["MainDB"]
    end

    subgraph LOG_SERVERS["Logging Infrastructure"]
        subgraph API_LOGS["API Logging"]
            APILOG["API-LogServer-VM<br/>10.32.8.152"]
            MONGOLOG1["MongoDB-API-Logs<br/>10.32.8.75<br/>162M documents"]
        end

        subgraph ELK_LOGS["ELK Stack"]
            ELK1["ELK-Proxy-Supplier<br/>10.32.8.84"]
            ELK2["ELK-HAProxy-Log<br/>10.32.8.132"]
        end

        subgraph GENERAL_LOGS["General Log Servers"]
            LOG1["LogServer-1<br/>10.32.8.140"]
            LOG2["LogServer-2-MainDB<br/>10.32.8.16"]
            LOG3["LogServer-3-Agoda<br/>10.32.8.88"]
        end

        subgraph SUPPLIER_LOGS["Supplier-Specific Logs"]
            EXPLOG["MongoDB-ExpediaLogs<br/>10.32.8.18"]
        end
    end

    API --> APILOG & MONGOLOG1
    HAPROXY --> ELK2
    SUPPLIERS --> ELK1 & LOG3 & EXPLOG
    MAINDB --> LOG2
    API --> LOG1

    style API_LOGS fill:#87CEEB
    style ELK_LOGS fill:#FFE4B5
    style GENERAL_LOGS fill:#90EE90
    style SUPPLIER_LOGS fill:#DDA0DD

Log Server Inventory¶

Server	IP Address	Type	Purpose
API-LogServer-VM	10.32.8.152	VM	Centralized API request logs
ELK-Proxy-Supplier	10.32.8.84	VM	Supplier proxy call logs
ELK-HAProxy-Log	10.32.8.132	VM	HAProxy access/error logs
LogServer-1	10.32.8.140	VM	General application logs
LogServer-2-MainDB	10.32.8.16	VM	MainDB transaction logs
LogServer-3-Agoda	10.32.8.88	VM	Agoda-specific supplier logs
MongoDB-API-Logs	10.32.8.75	VM	API logs in MongoDB (162M docs)
MongoDB-ExpediaLogs	10.32.8.18	VM	Expedia supplier logs

Log Retention & Volume¶

Log Type	Storage	Retention	Volume
API Request Logs	MongoDB	90 days	162M+ documents
Supplier Logs	ELK/MongoDB	30 days	High
HAProxy Logs	ELK	14 days	Very High
Database Logs	File System	30 days	Medium

MongoDB Servers¶

MongoDB provides NoSQL storage for search keys, room mappings, logs, and performance metrics.

graph TB
    subgraph MONGO_CLUSTER["MongoDB Infrastructure (8 VMs)"]
        subgraph SEARCH_KEYS["Search Key Cache (Cascading Read)"]
            M1["MongoDB-1<br/>10.32.8.51<br/>Primary"]
            M2["MongoDB-2<br/>10.32.8.52<br/>Secondary"]
            M3["MongoDB-3<br/>10.32.8.53<br/>Tertiary"]

            M1 -->|"Try First"| M2
            M2 -->|"If Miss"| M3
        end

        subgraph SPECIALIZED["Specialized MongoDB"]
            M4["MongoDB-Perf<br/>10.32.8.74<br/>216M metrics"]
            M5["MongoDB-Logs<br/>10.32.8.75<br/>162M logs"]
            M6["MongoDB-RoomMap<br/>10.32.8.96<br/>8.5M mappings"]
            M7["MongoDB-SupplierLog<br/>10.32.8.18"]
            M8["MongoDB-Unmapped<br/>10.32.8.101"]
        end
    end

    subgraph API["API Servers"]
        APIS["7 API Servers"]
    end

    APIS -->|"Search Keys"| M1
    APIS -->|"Performance Data"| M4
    APIS -->|"API Logs"| M5
    APIS -->|"Room Mapping"| M6

    style SEARCH_KEYS fill:#90EE90
    style SPECIALIZED fill:#FFE4B5

MongoDB Server Inventory¶

Server	IP Address	Host	Purpose	Documents
MongoDB-1	10.32.8.51	XCP-1	Search Keys (Primary)	~2M
MongoDB-2	10.32.8.52	XCP-2	Search Keys (Secondary)	~2M
MongoDB-3	10.32.8.53	XCP-2	Search Keys (Tertiary)	~1.7M
MongoDB-Perf	10.32.8.74	XCP-3	Performance Metrics	216M
MongoDB-Logs	10.32.8.75	XCP-3	API Request Logs	162M
MongoDB-RoomMap	10.32.8.96	XCP-1	Room Mapping Data	8.5M
MongoDB-SupplierLog	10.32.8.18	XCP-3	Supplier Call Logs	Variable
MongoDB-Unmapped	10.32.8.101	XCP-2	Unmapped Hotel Data	Variable

Search Key Cascading Read Pattern¶

sequenceDiagram
    participant API as API Server
    participant M1 as MongoDB-51<br/>(Primary)
    participant M2 as MongoDB-52<br/>(Secondary)
    participant M3 as MongoDB-53<br/>(Tertiary)

    API->>M1: Query Search Key
    alt Found in M1
        M1-->>API: Return cached result
    else Not in M1
        API->>M2: Query Search Key
        alt Found in M2
            M2-->>API: Return cached result
        else Not in M2
            API->>M3: Query Search Key
            alt Found in M3
                M3-->>API: Return cached result
            else Cache Miss
                M3-->>API: Not found - call supplier
            end
        end
    end

Configuration	Value
Search Key TTL	30-60 minutes
Total Search Keys	~5.7M across 3 servers
Read Pattern	Cascading (51 → 52 → 53)
Write Pattern	Write to all 3

Room Mapping Services¶

Room mapping services handle hotel-supplier-room relationships for accurate booking.

graph TB
    subgraph ROOM_MAPPING["Room Mapping Architecture"]
        subgraph HAPROXY_RM["Room Mapping Load Balancers"]
            HARP1["HAProxy-RoomMapping1"]
            HARP2["HAProxy-RoomMapping2"]
        end

        subgraph MONGO_RM["Room Mapping Storage"]
            MONGORM["MongoDB-RoomMapping<br/>10.32.8.96<br/>8.5M mappings"]
        end

        subgraph MAPPING_SERVICE["Mapping Services"]
            NEWMAP["NewHotelMapping<br/>Service"]
        end
    end

    subgraph API["API Servers"]
        APIS["7 API Servers"]
    end

    APIS --> HARP1 & HARP2
    HARP1 & HARP2 --> MONGORM
    NEWMAP --> MONGORM

    style MONGORM fill:#90EE90
    style HAPROXY_RM fill:#87CEEB

Room Mapping Data¶

Metric	Value
Total Mappings	8.5M documents
Hotels Mapped	9.3M properties
Supplier Mappings	33M relationships
Storage	MongoDB 10.32.8.96
Load Balancer	HAProxy-RoomMapping 1 & 2

OTH (Online Travel Hub)¶

OTH is a specialized service for specific market segments with dedicated infrastructure.

graph TB
    subgraph OTH_INFRA["OTH Infrastructure"]
        subgraph OTH_LB["OTH Load Balancers"]
            OTHLB1["OTH-HAProxy-Master<br/>10.32.8.106"]
            OTHLB2["OTH-HAProxy-Standby<br/>10.32.8.107"]
            OTHVIP["OTH VIP<br/>10.32.8.105"]
        end

        subgraph OTH_SERVERS["OTH Application Servers (VMs)"]
            OTH1["OTH-1<br/>10.32.8.11"]
            OTH2["OTH-2<br/>10.32.8.86"]
            OTH3["OTH-3<br/>10.32.8.40"]
        end

        subgraph OTH_CACHE["OTH Caching"]
            OTHCM["OTH-Cache-Master"]
            OTHCS["OTH-Cache-Standby"]
            REDISOTH["Redis-OTH"]
        end
    end

    OTHVIP --> OTHLB1
    OTHVIP -.->|"Failover"| OTHLB2
    OTHLB1 & OTHLB2 --> OTH1 & OTH2 & OTH3
    OTH1 & OTH2 & OTH3 --> OTHCM & REDISOTH

    style OTHVIP fill:#87CEEB
    style OTHLB1 fill:#90EE90
    style OTHLB2 fill:#FFE4B5

OTH Server Inventory¶

Component	IP Address	Type	Host
OTH-1	10.32.8.11	VM	XCP-1
OTH-2	10.32.8.86	VM	XCP-3
OTH-3	10.32.8.40	VM	XCP-1
OTH-HAProxy-Master	10.32.8.106	VM	XCP-2
OTH-HAProxy-Standby	10.32.8.107	VM	XCP-2
OTH VIP	10.32.8.105	Virtual	-
OTH-Cache-Master	-	VM	XCP-2
OTH-Cache-Standby	-	VM	XCP-2
Redis-OTH	-	VM	XCP-1

OTH Features¶

Feature	Description
Purpose	Specialized travel hub for specific markets
Architecture	Dedicated servers, separate from main API
Load Balancing	HAProxy HA pair with VIP
Caching	Redis + dedicated cache servers
Database	Separate optimized database

Channel Managers¶

Channel Manager integrations receive hotel inventory from 9 external CM providers.

graph TB
    subgraph CM_PROVIDERS["External Channel Managers (9 Integrations)"]
        DEDGE["DEdge<br/>dedge.withinearth.com"]
        RATEGAIN["RateGain<br/>rategain.withinearth.com"]
        RATETIGER["RateTiger<br/>ratetiger.withinearth.com"]
        STAAH["Staah<br/>staah.withinearth.com"]
        DERBYSOFT["DerbySoft<br/>derbysoft.withinearth.com"]
        HOTELRUNNER["HotelRunner<br/>hotelrunner.withinearth.com"]
        TRAVELCLICK["TravelClick<br/>travelclick.withinearth.com"]
        RESELIVA["Reseliva<br/>reseliva.withinearth.com"]
        EXTRANET["ExtranetCM<br/>extranetcm.withinearth.com"]
    end

    subgraph CM_SERVERS["CM Processing Servers (VMs)"]
        CM1["Channel Manager 1<br/>10.32.8.142<br/>Host: XCP-1"]
        CM2["Channel Manager 2<br/>10.32.8.34<br/>Host: XCP-2"]
    end

    subgraph STORAGE["Data Storage"]
        SQL["SQL Server<br/>MainDB"]
        MONGO["MongoDB"]
    end

    DEDGE & RATEGAIN & RATETIGER & STAAH & DERBYSOFT --> CM1
    HOTELRUNNER & TRAVELCLICK & RESELIVA & EXTRANET --> CM2
    CM1 & CM2 --> SQL & MONGO

    style CM_PROVIDERS fill:#e1bee7
    style CM_SERVERS fill:#90EE90

Channel Manager Inventory¶

Provider	Subdomain	Integration Type
DEdge	dedge.withinearth.com	Push rates/availability
RateGain	rategain.withinearth.com	Revenue management
RateTiger	ratetiger.withinearth.com	Channel management
Staah	staah.withinearth.com	Distribution platform
DerbySoft	derbysoft.withinearth.com	Connectivity services
HotelRunner	hotelrunner.withinearth.com	Hotel distribution
TravelClick	travelclick.withinearth.com	Hotel solutions
Reseliva	reseliva.withinearth.com	Booking engine
ExtranetCM	extranetcm.withinearth.com	Extranet connectivity

CM Server Details¶

Server	IP Address	Type	Host	Handles
CM-1	10.32.8.142	VM	XCP-1	5 CM integrations
CM-2	10.32.8.34	VM	XCP-2	4 CM integrations

B2B Portal¶

The B2B Portal provides web-based access for travel agents and administrators.

graph TB
    subgraph USERS["Portal Users"]
        AGENTS["Travel Agents<br/>627+ Active"]
        ADMINS["Administrators"]
    end

    subgraph B2B_INFRA["B2B Infrastructure"]
        B2B["B2B Server<br/>Physical Dedicated<br/>b2b.withinearth.com"]
    end

    subgraph BACKEND["Backend Services"]
        API["XConnect API<br/>api.withinearth.com"]
        SQL["SQL Server"]
    end

    AGENTS -->|"Agent Booking"| B2B
    ADMINS -->|"Admin Dashboard"| B2B
    B2B --> API --> SQL

    style B2B fill:#87CEEB
    style API fill:#90EE90

B2B Portal Features¶

Feature	Description
URL	b2b.withinearth.com
Server Type	Physical Dedicated
Agent Booking	Web-based hotel booking interface
Admin Dashboard	Client management and reporting
Rate Management	Configure markups and pricing
Booking History	View and manage reservations
Credit Management	Balance and payment tracking
Active Agents	627+ in last 30 days

Tableau¶

Tableau provides business intelligence, analytics, and real-time operational dashboards.

graph TB
    subgraph DATA_SOURCES["Data Sources"]
        SQL["SQL Server<br/>MainDB + Replicas"]
        MONGO["MongoDB<br/>Logs & Metrics"]
        API["API Metrics"]
    end

    subgraph TABLEAU["Tableau Server"]
        TAB["Tableau Server<br/>10.32.8.129<br/>Physical Dedicated"]
    end

    subgraph USERS["Dashboard Users"]
        MGMT["Management"]
        OPS["Operations Team"]
        SUPPORT["Support Team"]
    end

    SQL & MONGO & API --> TAB
    TAB --> MGMT & OPS & SUPPORT

    style TAB fill:#FFE4B5

Tableau Configuration¶

Setting	Value
Server IP	10.32.8.129
Server Type	Physical Dedicated
Purpose	BI, Analytics, Real-time Dashboards

Tableau Dashboards¶

Dashboard Type	Purpose
BI Dashboards	Business intelligence and reporting
Data Analytics	Advanced data analysis and visualization
Real-time Monitoring	Live operational dashboards
Booking Analytics	Booking trends and patterns
Supplier Performance	Supplier response times and success rates

Monitoring¶

Comprehensive monitoring infrastructure for all systems.

graph TB
    subgraph MONITORING["Monitoring Infrastructure (All VMs)"]
        ZABBIX["Zabbix Server<br/>10.32.8.148<br/>60+ hosts monitored"]
        UPTIME["UptimeKuma<br/>10.32.8.102<br/>Uptime monitoring"]
        NETDATA["Netdata<br/>Bandwidth monitoring"]
    end

    subgraph TARGETS["Monitored Systems"]
        API["7 API Servers"]
        SQL["SQL Servers"]
        MONGO["MongoDB Servers"]
        HAPROXY["HAProxy"]
        OTH["OTH Servers"]
        CM["Channel Managers"]
    end

    ZABBIX -->|"SNMP/Agent"| TARGETS
    UPTIME -->|"HTTP Checks"| TARGETS
    NETDATA -->|"Metrics"| TARGETS

    style ZABBIX fill:#90EE90
    style UPTIME fill:#87CEEB
    style NETDATA fill:#FFE4B5

Monitoring Server Inventory¶

Server	IP Address	Type	Host	Purpose
Zabbix	10.32.8.148	VM	XCP-3	Infrastructure monitoring (60+ hosts)
UptimeKuma	10.32.8.102	VM	XCP-3	Uptime and availability monitoring
Netdata	-	VM	-	Bandwidth and performance metrics

Monitoring Capabilities¶

System	Metrics Collected
Zabbix	CPU, Memory, Disk, Network, Services, Custom metrics
UptimeKuma	HTTP/HTTPS endpoints, Response times, SSL certificates
Netdata	Real-time bandwidth, Per-process metrics

XCP Servers (Hypervisors)¶

XCP-ng hypervisors host the majority of virtual machines.

graph TB
    subgraph HYPERVISORS["XCP-ng Hypervisor Cluster (3 Physical Servers)"]
        subgraph XCP1["XCP-ng 1: 10.32.8.22"]
            VMS1["11 VMs:<br/>RabbitMQ, SQL Replica 5<br/>MongoDB-RoomMap, CM-1<br/>VPN, HAProxy-SQL-1"]
        end

        subgraph XCP2["XCP-ng 2: 10.32.8.23"]
            VMS2["17 VMs:<br/>MongoDB-ReCheck, pfSense-2<br/>OTH-HAProxy pair<br/>Gateway-2, MariaDB"]
        end

        subgraph XCP3["XCP-ng 3: 10.32.8.24"]
            VMS3["14 VMs:<br/>MongoDB-Logs, UptimeKuma<br/>GitLab, Gateway-1<br/>Zabbix"]
        end

        XOA["XCP Orchestra (XOA)<br/>Centralized Management"]
    end

    XOA --> XCP1 & XCP2 & XCP3

    style XCP1 fill:#87CEEB
    style XCP2 fill:#FFE4B5
    style XCP3 fill:#90EE90

XCP Server Inventory¶

Server	IP Address	Type	VMs Hosted	Key Services
XCP-ng 1	10.32.8.22	Physical	11	RabbitMQ, SQL Replica, MongoDB, CM-1, VPN
XCP-ng 2	10.32.8.23	Physical	17	MongoDB, OTH-HAProxy, pfSense, Gateway
XCP-ng 3	10.32.8.24	Physical	14	MongoDB-Logs, GitLab, Zabbix, UptimeKuma
Total	-	3 Physical	42 VMs	All virtualized services

XCP Management¶

Component	Purpose
XCP Orchestra (XOA)	Centralized VM management UI
Live Migration	Move VMs between hosts without downtime
HA	Automatic VM restart on host failure
Storage	Local and shared storage support

MainDB & New Cluster¶

The MainDB is currently a single point of failure. A new HA cluster is planned to eliminate this risk.

graph TB
    subgraph CURRENT["Current Architecture (SPOF)"]
        MAINDB_OLD["MainDB<br/>10.32.8.130<br/>Physical Dedicated<br/>SQL Server 2019<br/>⚠️ SINGLE POINT OF FAILURE"]
    end

    subgraph PLANNED["Planned Architecture (HA Cluster)"]
        subgraph PROXMOX_NEW1["New Proxmox 1 (Physical Dedicated)"]
            MAINDB_NEW["MainDB VM<br/>10.32.8.130 (same IP)<br/>SQL Server Standard<br/>Primary Node"]
        end

        subgraph PROXMOX_NEW2["New Proxmox 2 (Physical Dedicated)"]
            AG_NODE["AG Sync Node VM<br/>SQL Server Standard<br/>Synchronous Replica"]
        end

        MAINDB_NEW <-->|"Always On AG<br/>Synchronous Replication<br/>Real-time Sync"| AG_NODE
    end

    CURRENT -->|"Migration Plan"| PLANNED

    style MAINDB_OLD fill:#ff6666
    style MAINDB_NEW fill:#90EE90
    style AG_NODE fill:#90EE90

Current MainDB (SPOF)¶

Setting	Value
Server IP	10.32.8.130
Port	1988
Type	Physical Dedicated
SQL Version	SQL Server 2019
Database	withinearthUpdated
Role	ALL WRITES - Central Master
Risk	⚠️ SINGLE POINT OF FAILURE

Planned HA Cluster¶

Component	Specification
Infrastructure	2 New Proxmox Physical Dedicated Servers
Proxmox 1 VM	MainDB (same IP: 10.32.8.130 after migration)
Proxmox 2 VM	AG Synchronous Replica Node
SQL Edition	SQL Server Standard Edition
Replication	Always On Availability Group
Sync Mode	Synchronous (real-time, zero data loss)
Failover	Automatic failover cluster

Migration Benefits¶

Current Risk	After Migration
Single physical server failure = total outage	Automatic failover to replica
No real-time backup	Synchronous replication (zero data loss)
Manual recovery required	Automatic recovery
Hours of potential downtime	Seconds of failover time

Supplier Proxies & Gateways¶

Supplier Proxies handle outbound API calls to 56+ external hotel suppliers. NAT Gateways provide public IP routing for these requests.

graph TB
    subgraph API_SERVERS["API Servers (7 Physical)"]
        API1["API-1<br/>10.32.8.134"]
        API2["API-2<br/>10.32.8.135"]
        API3["API-3<br/>10.32.8.139"]
        API4["API-4<br/>10.32.8.137"]
        API5["API-5<br/>10.32.8.35"]
        API6["API-6<br/>10.32.8.166"]
        API7["API-7<br/>10.32.8.167"]
    end

    subgraph PROXY_LAYER["Supplier Proxy Layer (7 VMs)"]
        P1["Proxy-1<br/>10.32.8.41<br/>Host: XCP-1"]
        P2["Proxy-2<br/>10.32.8.42<br/>Host: XCP-2"]
        P3["Proxy-3<br/>10.32.8.43<br/>Host: XCP-1"]
        P4["Proxy-4<br/>10.32.8.48<br/>Host: XCP-2"]
        P5["Proxy-5<br/>10.32.8.45<br/>Host: XCP-2"]
        P6["Proxy-6<br/>10.32.8.46<br/>Host: XCP-3"]
        P7["Proxy-7<br/>10.32.8.47<br/>Host: XCP-3"]
    end

    subgraph GATEWAY_LAYER["NAT Gateway Layer (3 VMs)"]
        GW1["Gateway-1<br/>10.32.8.3<br/>Public: 95.211.19.201"]
        GW2["Gateway-2<br/>10.32.8.4<br/>Public: 95.211.19.207"]
        GW3["Gateway-3<br/>10.32.8.10<br/>Public: 95.211.19.211"]
    end

    subgraph SUPPLIERS["External Suppliers (56+ Integrations)"]
        direction TB
        SUP1["Agoda (35%)"]
        SUP2["Expedia (28%)"]
        SUP3["Booking.com (12%)"]
        SUP4["HotelBeds"]
        SUP5["WebBeds, Dida, TBO<br/>+48 more suppliers"]
    end

    API1 --> P1
    API2 --> P2
    API3 --> P3
    API4 --> P4
    API5 --> P5
    API6 --> P6
    API7 --> P7

    P1 & P2 & P3 --> GW1
    P4 & P5 --> GW2
    P6 & P7 --> GW3

    GW1 & GW2 & GW3 --> SUPPLIERS

    style PROXY_LAYER fill:#FFE4B5
    style GATEWAY_LAYER fill:#90EE90
    style SUPPLIERS fill:#FFA500

Supplier Proxy Inventory¶

Proxy	Internal IP	Host	Assigned API
Proxy-1	10.32.8.41	XCP-1	API-1
Proxy-2	10.32.8.42	XCP-2	API-2
Proxy-3	10.32.8.43	XCP-1	API-3
Proxy-4	10.32.8.48	XCP-2	API-4
Proxy-5	10.32.8.45	XCP-2	API-5
Proxy-6	10.32.8.46	XCP-3	API-6
Proxy-7	10.32.8.47	XCP-3	API-7

NAT Gateway Inventory¶

Gateway	Internal IP	Public IP	Proxies Routed
Gateway-1	10.32.8.3	95.211.19.201	Proxy 1, 2, 3
Gateway-2	10.32.8.4	95.211.19.207	Proxy 4, 5
Gateway-3	10.32.8.10	95.211.19.211	Proxy 6, 7

Traffic Flow¶

sequenceDiagram
    participant API as API Server
    participant Proxy as Supplier Proxy
    participant Gateway as NAT Gateway
    participant Supplier as External Supplier

    API->>Proxy: Internal request<br/>(10.32.8.x)
    Proxy->>Gateway: Forward request
    Gateway->>Supplier: External call<br/>(Public IP: 95.211.19.x)
    Supplier-->>Gateway: Response
    Gateway-->>Proxy: Response
    Proxy-->>API: Return data

Supplier Distribution¶

Tier	Suppliers	Booking Share
Tier 1	Agoda, Expedia, Booking.com	76%
Tier 2	HotelBeds, WebBeds, Dida, GoGlobal, TBO	18%
Tier 3	48+ regional/specialized suppliers	6%

Proxy Configuration¶

Setting	Value
Proxy Type	Forward proxy
Protocol	HTTP/HTTPS
Timeout	Supplier-specific (5-30 seconds)
Retry Logic	Configurable per supplier
Connection Pooling	Enabled
SSL Verification	Enabled

Gateway Configuration¶

Setting	Value
NAT Type	Source NAT (SNAT)
Public IPs	3 dedicated IPs
Purpose	Supplier IP whitelisting
Failover	Manual failover between gateways
Bandwidth	10Gbps per gateway

Last Updated: 2025-12-02 Complete System Architecture - All Layers Documented