Note: This article focuses on preserving session affinity across multiple service ports. It assumes that TLS, health checks, and general HAProxy deployment practices are already in place.

Building high availability for YSoft SafeQ 6 turned out to be much more challenging than simply placing a load balancer in front of two application nodes.

The official documentation specifies which ports must be routed to the Terminal Server, but it doesn't explain that these seemingly independent connections actually belong to a single logical session. That distinction becomes critical when introducing high availability. The following approach proved reliable in production after extensive testing.

The Challenge: Independent Ports, Single Session

Unlike a traditional web application that communicates over a single HTTPS endpoint, YSoft SafeQ 6 establishes multiple concurrent connections over different TCP and HTTPS ports during a single logical workflow (e.g., terminal authentication, print job release, and scanning).

From the printer's perspective, the communication flows to different services running on different ports:

               ┌──► Port 5011 (HTTP Auth)  ──► Frontend TS (insecure)
               │
Printer/Client ┼──► Port 5022 (HTTPS TS)   ──► Frontend TS (secure)
               │
               └──► Port 5555 (TCP SPOC)   ──► Frontend terminal_5555

Inside HAProxy, each of these ports belongs to a different frontend and backend.

  • From HAProxy's perspective, these are completely unrelated TCP sessions.
  • From SafeQ's perspective, they must land on the same physical server (node) to maintain session continuity and avoid state synchronization issues.

If a printer authenticates on Node 1 (port 5011) but its subsequent print release request on port 5022 is routed to Node 2, the workflow may fail because the second node has no knowledge of the existing session.

Why Independent Stick Tables Fail

A straightforward approach to load balancing this architecture is defining independent stick tables for each backend:

backend ts_http
    stick-table type ip size 100k expire 30m
    stick on src
    server NODE1 192.168.1.100:5011 check

backend ts_https
    stick-table type ip size 100k expire 30m
    stick on src
    server NODE1 192.168.1.100:5022 ssl verify none check

Even though both backends track the client by the same source IP (src), their routing tables are completely isolated.

When a printer initiates an HTTP request, it might get mapped to NODE1. A split-second later, when it initiates an HTTPS request, the ts_https backend calculates its own routing decision (e.g., based on leastconn or server load) and might send the request to NODE2. This breaks the application session.

Sharing the Routing Decision

Instead of making an independent routing decision for every backend, HAProxy is instructed to reuse the routing decision made by the first backend. Effectively, the first successful connection establishes the node assignment for the entire application workflow.

Here is the configuration that achieves this:

frontend terminal
    bind *:5011
    bind *:5012 ssl crt /etc/haproxy/ssl/ha-safeq.pem
    bind *:5022 ssl crt /etc/haproxy/ssl/ha-safeq.pem verify none
    bind *:5610 ssl crt /etc/haproxy/ssl/ha-safeq.pem
    mode http
    acl is_ssl ssl_fc
    use_backend TS_ssl if is_ssl
    default_backend TS_insecure

backend TS_insecure
    balance leastconn
    option redispatch
    mode http
    
    # 1. This backend defines and owns the shared persistence table
    stick-table type ip size 100k expire 30m
    stick on src
    server LVIV 192.168.1.100:5011 check inter 5s fall 3 rise 2
    server KYIV 192.168.1.200:5011 check inter 5s fall 3 rise 2

backend TS_ssl
    balance leastconn
    option redispatch
    mode http
    # 2. We reuse the shared table owned by TS_insecure
    stick on src table TS_insecure
    # 3. Server names MUST match the primary backend exactly
    server LVIV 192.168.1.100 ssl verify none check port 5022 inter 5s fall 3 rise 2
    server KYIV 192.168.1.200 ssl verify none check port 5022 inter 5s fall 3 rise 2

How it works:

  1. TS_insecure establishes the initial routing decision. When a printer hits port 5011, HAProxy selects a server (e.g., LVIV) and stores the mapping [Printer IP -> LVIV] in the TS_insecure stick table.
  2. When the printer makes subsequent calls to HTTPS ports (5012, 5022, etc.), they land in the TS_ssl backend.
  3. Rather than performing a new load-balancing decision, TS_ssl consults the shared persistence table (stick on src table TS_insecure) and routes the request to the previously selected backend server.

Note: The balancing algorithm (balance leastconn) is only used when no persistence entry exists. Once a client is present in the shared stick table, subsequent requests bypass the balancing algorithm and follow the stored mapping.

5011 (HTTP Auth) ──► Hits TS_insecure ──► Creates stick record ──► Routed to LVIV
                                                │
5022 (HTTPS TS)  ──► Hits TS_ssl     ──► Reads stick record  ──► Routed to LVIV
                                                │
5610 (HTTPS)     ──► Hits TS_ssl     ──► Reads stick record  ──► Routed to LVIV

Technical Gotchas You Must Avoid

During failover testing in production, I discovered two critical caveats that can completely break this configuration:

1. Server Names Must Be Identical Across Backends

During testing I discovered that all participating backends must use identical server names (for example, LVIV and KYIV). Different names caused session affinity to break because HAProxy could no longer correctly reuse the routing decision across backends. Always ensure that server names match exactly in all backends that share the same persistence table.

2. Failing Over Gracefully (option redispatch)

If a node goes down, HAProxy will eventually mark it as DOWN via health checks. However, clients may still attempt to use an existing persistence entry until HAProxy redispatches the request.

Adding option redispatch forces HAProxy to ignore the stick table mapping if the destination server is dead, instantly routing the client to the remaining healthy node.

For raw TCP backends (like LPR/RAW print ports), also append on-marked-down shutdown-sessions to the server lines. This immediately kills active, hung TCP connections when a node dies, forcing clients to reconnect and fail over without waiting for TCP timeouts.

Lessons Learned

High availability is as much about understanding the application's behavior as it is about load balancer syntax.

HAProxy is designed to balance individual TCP/HTTP connections. However, enterprise systems like YSoft SafeQ 6 expect session consistency across multiple independent ports.

By forcing HAProxy backends to share a single persistence table and configuring aggressive connection reaping during node failures, I achieved a highly resilient environment where failover between geographical regions occurs seamlessly and transparently to the end-users.

Related Notes