Health checks and Multi-value DNS records in AWS Route53

A lesser understood feature of AWS Route53 is the ability to create multi-value records and control which values are published using health checks.

For reference, a multi-value record is something like:

$ dig -s server1.mydomain.com
2.2.2.2
4.4.4.4
6.6.6.6

Let’s say that whatever it is that responds on 2.2.2.2 has experienced a service interruption. What’s the best way to ensure traffic doesn’t flow to that address?

The typical way to create this type of record is to create a single record in AWS Route53 and add 3 IP addresses to the value, like so:

Screenshot 2022-01-05 at 18.02.51

However, to remove an IP from that record, you would need to edit the record itself, and then re-edit when the service recovers.

Instead, you can create 3 multi-value records like so, associating each with a health check that refers to the IP address in the record:

Screenshot 2022-01-05 at 18.09.59

Each of these records is associated with one of the following health checks:

Screenshot 2022-01-05 at 18.15.00

At this point, you can decide whether you want your health checks to be dynamic, or if you want to disable them and use the inversion feature of health checks to use them as on/off switches for the IP addresses.

If they’re dynamically used, and the health check goes into an unhealthy state, the IP address associated with that record will be dropped from the response. Alternatively, you can invert a disabled health check to achieve the same effect, which may be a more appropriate method to integrate this mechanism into existing tooling.

$ dig -s server1.mydomain.com
4.4.4.4
6.6.6.6

Crucially, if all health checks are unhealthy (or inverted) AWS Route53 will stop making assumptions are publish all the IP addresses associated with the record.

$ dig -s server1.mydomain.com
2.2.2.2
4.4.4.4
6.6.6.6