vpnfailover.guide A guide by Firezone

VPN failover that does not wait for you

VPN failover moves traffic to a healthy gateway when one gateway fails. This guide explains active-passive and active-active designs, health checks, and failover time. It also lists the single points of failure that people forget.

See how Firezone does failoverBook a demo

How a Client moves to a healthy GatewayA Client connects to a Site that has three Gateways in front of one Resource. The first Gateway fails. The keepalive timer of the Client expires, and the Client asks the control plane for a new Gateway. Traffic then goes through the second Gateway to the Resource.Siteno replyrequesttunnelGateway 1failedClientkeepalive timerGateway 2healthyResource10.0.4.20Control planetracks Gateway healthGateway 3healthy
  1. Several Gateways serve one Site

    Each Gateway in the Site can reach the same Resources. No Gateway is a standby.

  2. One Gateway fails

    The control plane stops the use of that Gateway for new connections. The Client stops getting replies.

  3. The Client asks for a new Gateway

    The keepalive timer of the Client expires after 15 seconds. The Client then asks the control plane for a healthy Gateway.

  4. Traffic moves

    The Client makes a new tunnel to a healthy Gateway and reaches the same Resource.

What VPN failover is

VPN failover is the process that moves users from a failed gateway to a healthy one. VPN high availability is the larger goal. It means that remote access continues to work when one part fails.

Remote access is a dependency of all the systems behind it. If the VPN gateway stops, people cannot reach the database, the admin panel, or the tools that they need to repair the failure. An outage of the VPN makes every other outage longer.

Every design for high availability has the same three parts.

  • Redundancy. More than one device can do the work.
  • Detection. Something sees that a device failed.
  • Redirection. Traffic moves to a device that works.

The designs differ in how they do each part, and in how long each part takes.

Active-passive and active-active

Active-passive

In an active-passive design, one gateway carries all the traffic, and a second gateway waits. The two usually share a virtual IP address. When the active gateway fails, the standby takes the address.

This design is simple to understand, and it has costs. You pay for a device that does nothing most of the time. The standby is also untested until the day that you need it. A configuration that drifted, an expired certificate, or an old software version shows up only at failover.

Active-active

In an active-active design, all gateways carry traffic all the time. When one fails, the others take its users. Each gateway proves each day that it works, and you can add gateways to add capacity.

The design needs a method to spread users across the gateways. It also needs spare capacity. If two gateways each run at 80 percent, one gateway cannot carry the load of the two.

DNS failover

DNS failover puts several gateway addresses behind one name. A health check removes the address of a failed gateway from the answer. This method is easy to add, and it is slow.

Resolvers and operating systems cache DNS answers for the time to live (TTL) of the record. Some caches keep an answer for longer than the TTL. A client that has the old address continues to use it until the cache expires and the client connects again.

Health checks and failover time

Failover time is the time from the failure to the moment that traffic flows again. It is the sum of three parts.

  1. Time to detect. A health check runs at an interval, and it usually needs several failures in a row before it acts.
  2. Time to decide. A cluster elects a new active member, or a controller selects a new gateway.
  3. Time to move. Clients learn the new path and make new tunnels.

Who runs the health check

A health check from a central monitor shows that the monitor can reach the gateway. It does not show that a user can. A health check from the client sees the real path. The best designs use the two. They must also make sure that a failure of the monitor does not disconnect users from a gateway that is healthy.

What happens to open sessions

A new connection goes to a healthy gateway. An open session is harder. A TCP connection is tied to addresses and to state on the gateway, such as NAT entries. Unless the gateways copy that state to each other, most open sessions end at failover. The app then connects again.

State that is shared between gateways protects sessions, but it adds a new part that can fail. Many teams accept a short interruption and make sure that apps retry.

The single points of failure that people forget

Two gateways do not give high availability by themselves. Look for each single point of failure on the full path.

  • The gateway. This is the part that everyone remembers. Check also that the two gateways are not on the same host, rack, or availability zone.
  • The authentication server. If users cannot sign in, a healthy gateway does not help. Find out what happens to current sessions when the identity provider or the VPN controller is offline.
  • The link. Two gateways behind one internet connection fail together. The same applies to one firewall, one NAT device, or one public IP address.
  • The shared state. A cluster database or a virtual IP protocol can cause the outage that it exists to prevent.
  • DNS. The name of the VPN endpoint must resolve. So must the internal names that users need after they connect.
  • The upgrade. Planned maintenance causes more downtime than hardware faults. A design that needs all gateways to stop for an upgrade is not highly available.

Good practice

  • Run at least three gateways, so that one can be in maintenance while one fails.
  • Keep enough spare capacity to lose one gateway at the busiest hour.
  • Stop a gateway on purpose, and measure the failover time.

How Firezone does high availability

Firezone is zero trust access built on WireGuard. You run the Gateways, and Firezone runs the control plane. High availability is active-active, and it needs no cluster setup. You deploy more than one Gateway in a Site.

Load balancing

When a Client connects to a Resource, Firezone selects a healthy Gateway in the Site. It selects the Gateway that is nearest to the location of the Client. This method spreads Client connections across all Gateways in the Site. The Client stays on that Gateway until it disconnects or the Gateway becomes unhealthy.

Automatic failover

Firezone uses two independent health checks.

  1. The control plane detects that a Gateway is unhealthy, and it stops the use of that Gateway for new connections.
  2. Each Client keeps a keepalive timer for its Gateway. The timer expires after 15 seconds. The Client then disconnects and asks the control plane for a healthy Gateway.

The 15 seconds are the maximum time before a Client moves to a healthy Gateway. Because the two checks are independent, a network problem between a Client and the control plane does not interrupt a connection to a healthy Gateway.

No shared state

A Gateway keeps no long-term state and needs no persistent storage. It gets its configuration from the control plane. Gateways do not form a cluster, and there is no virtual IP address to manage. The documentation does not promise that an open TCP session survives the move to a new Gateway. Make sure that your apps retry.

If the control plane is offline

Existing connections stay active for about 5 minutes. During that time, you cannot use the admin portal or the API.

What to deploy

The sizing guide recommends at least three Gateways for each Site in production. Three Gateways keep the Site available during rolling upgrades. Gateway load balancing and failover are part of every plan, and that includes the free Starter plan. The link and the network of the Site remain your responsibility.

An active-passive pair, DNS failover, and Firezone

The three approaches detect a failure and move traffic in different ways. The differences decide how long users wait.

Active-passive pairDNS failoverFirezone
All gateways carry trafficNoOne waits as a standbyYesYesNearest healthy Gateway
Who detects the failureThe standby gatewayA central monitorThe control plane and each Client
Time to move existing usersSeconds, if the standby is readyThe TTL of the record, or longer15 seconds at most
Needs no shared state between gatewaysNoA shared address and state syncYesYes
Open sessions survivePartialOnly with state syncNoNoPlan for apps to connect again
More gateways add capacityNoYesYes
Setup workA cluster to configure and testRecords and health checksDeploy a second Gateway in the Site

The table describes the usual design of each approach. A specific product can be different. Read the documentation of the product, and test a failure before you depend on it.

Common questions

What is the difference between active-passive and active-active?

In an active-passive design, one gateway works and a second gateway waits. In an active-active design, all gateways work at the same time. Active-active uses the hardware that you pay for, and it shows each day that each gateway is healthy.

How fast is VPN failover?

The failover time is the time to detect the failure plus the time to move the traffic. A cluster with a shared address can move in a few seconds. DNS failover can take minutes, because clients cache the old address. A Firezone Client moves within 15 seconds.

Do users need to sign in again after a failover?

It depends on where the session lives. If the gateway holds the session, users sign in again unless the gateways share it. In Firezone, the Client asks the control plane for a healthy Gateway by itself. The user does not select a new server.

How many gateways do I need for high availability?

Two is the minimum. Three or more is the usual recommendation for production. With three, you can upgrade one gateway and still survive the failure of a second one.

Is DNS failover enough for a VPN?

DNS failover is enough if a wait of several minutes is acceptable. It does not help a client that already has the address of the failed gateway. A health check on the client gives a faster result.

What happens if the Firezone control plane is offline?

Existing connections stay active for about 5 minutes. You cannot use the admin portal or the API during that time. The Firezone FAQ gives the details.

Add a second Gateway, and failover is on

Firezone balances Clients across the Gateways in a Site and moves them when a Gateway fails. There is no cluster to configure. The Starter plan is free.

See plans and pricingBook a demo