add post about docker2caddy
This commit is contained in:
155
content/blog/2022-04-docker2caddy-reverse-proxy.md
Normal file
155
content/blog/2022-04-docker2caddy-reverse-proxy.md
Normal file
@@ -0,0 +1,155 @@
|
||||
---
|
||||
title: "Docker2Caddy - An automatic Reverse Proxy for Docker containers"
|
||||
date: 2022-04-25
|
||||
categories:
|
||||
- english
|
||||
tags:
|
||||
- tools
|
||||
- fsfe
|
||||
- server
|
||||
headerimage: /blog/docker.jpg
|
||||
headercredits: Shipping a load of containers requires a reliable infrastructure
|
||||
---
|
||||
|
||||
So you have a number of Docker containers running web services which you would
|
||||
like to expose to the outside? Well, you probably will at least have considered
|
||||
a reverse proxy already. Doing this manually for one, two or even five
|
||||
containers may be feasible, but everything above that will be a PITA for sure.
|
||||
At the [FSFE](https://fsfe.org) we ran into the same issue with our [own
|
||||
distributed container
|
||||
infrastructure](https://git.fsfe.org/fsfe-system-hackers/container-server/) at
|
||||
and crafted a neat solution that I would like to present to you in the next few
|
||||
minutes.
|
||||
|
||||
The result is
|
||||
[Docker2Caddy](https://git.fsfe.org/fsfe-system-hackers/docker2caddy/) that
|
||||
provides a workflow in which you can spin up new containers anytime (e.g. via a
|
||||
CI) and the reverse proxy will just do the rest for you magically.
|
||||
|
||||
## The assumptions
|
||||
|
||||
Let's assume you want to go with reverse proxies to make your web services
|
||||
accessible via ports 80 and 443. There are other possibilities, and in more
|
||||
complex environments there may be already integrated solutions, but for this
|
||||
article we'll wade in a rather simple environment spun up with `docker-compose`[^1].
|
||||
|
||||
Let's also assume you care about security and go with a rootless installation of
|
||||
Docker. So the daemon will run as an unprivileged user. That's possible but much
|
||||
more complex than the default rootful installation[^2]. Because of this, a few
|
||||
other solutions will not work, we'll check that later.
|
||||
|
||||
Finally, each container shall at least have one separate domain assigned to it
|
||||
for which you obviously want to have a valid certificate, e.g. by Let's Encrypt.
|
||||
|
||||
In the examples below, we have two containers running, each running a webserver
|
||||
listening to port `8080`. The first container shall be available via
|
||||
`first.com`, the second via `second.net`. The latter shall also be available via
|
||||
`www.second.net`.
|
||||
|
||||
[^1]: [This is](https://git.fsfe.org/fsfe-system-hackers/minimal-docker) how a
|
||||
very minimal Docker service in the FSFE infrastructure looks like. For
|
||||
Docker2Caddy, only the `docker-compose.yml` file with its labels is
|
||||
relevant.
|
||||
|
||||
[^2]: If you're interested in setting this up via Ansible, I can recommend the
|
||||
[ansible-docker-rootless](https://github.com/konstruktoid/ansible-docker-rootless)
|
||||
role which we integrated in our full-blown
|
||||
[playbook](https://git.fsfe.org/fsfe-system-hackers/container-server/) for
|
||||
the container servers.
|
||||
|
||||
## The problems
|
||||
|
||||
In the described scenario, there are a number of problem for automating the
|
||||
configuration of the reverse proxy in order to direct a domain to the correct
|
||||
container, starting with **container discovery** to **IPv6 routing** to
|
||||
**handling offline containers**.
|
||||
|
||||
The reverse proxy has to be able to discover the currently running containers
|
||||
and ideally monitor for changes regularly so that a newly created container with
|
||||
a new domain is reachable within a short time without manual intervention.
|
||||
|
||||
Before Docker2Caddy we have used
|
||||
[nginx-proxy](https://github.com/nginx-proxy/nginx-proxy) combined with
|
||||
[acme-companion](https://github.com/nginx-proxy/acme-companion) (formerly known
|
||||
as *docker-letsencrypt-nginx-proxy-companion*). These are Docker containers that
|
||||
query all containers connected to the `bridge` Docker network. For this to work,
|
||||
the containers have to run with environment variables indicating the desired
|
||||
domains and local ports that shall be proxied.
|
||||
|
||||
In a rootless Docker setup this finally reaches its limits although discovery
|
||||
still works. But already before that we did not like the fact that we had to
|
||||
connect containers to the bridge network upon creation and therefore lost a bit
|
||||
more isolation (which is dubious in Docker anyway).
|
||||
|
||||
Now, with rootless, IPv6 was the turning point. Even in rootful Docker setups,
|
||||
IPv6 – a 20+ years old, well defined standard protocol – is a pain in the butt.
|
||||
But with rootless, the FSFE System Hackers team did not manage to get IPv6
|
||||
working in containers to the degree that we needed. While IPv6 traffic reached
|
||||
the `nginx-proxy`, it was then treated as IPv4 traffic with the internal Docker
|
||||
IP address. That bits you ultimately if you limit requests based on IP
|
||||
addresses, e.g. for signups or payments. All traffic via IPv6 will be treated
|
||||
as the same internal IPv4 address, therefore triggering the limits regularly.
|
||||
|
||||
The easiest solution therefore is to use a reverse proxy running on the host
|
||||
system, not as a Docker container with its severe limitations. While the first
|
||||
intuition lead us to nginx, we decided to go with
|
||||
[Caddy](https://caddyserver.com/). The main advantages we saw are that a virtual
|
||||
host in Caddy is very simple to configure and that TLS certificates are
|
||||
generated and maintained automatically without extra dependencies like
|
||||
`certbot`.
|
||||
|
||||
In this setup, containers would need to open their webserver port to the host.
|
||||
This "public" port has to be unique per host, but the internal port can stay the
|
||||
same, e.g. port `1234` could be mapped to port `8080` inside the container. In
|
||||
Caddy you would then configure the domain `first.org` to forward to
|
||||
`localhost:1234`. A more or less identical second example container could then
|
||||
expose the port `5678` to the host, again listen on `8080` internally, and Caddy
|
||||
would redirect `second.net` and `www.second.net` to `localhost:5678`.
|
||||
|
||||
But how does Caddy know about the currently running containers and the ports via
|
||||
which they want to receive traffic? And how can we handle containers that are
|
||||
unavailable, for instance because they crashed or have been deleted for good?
|
||||
Docker2Caddy to the rescue!
|
||||
|
||||
## The solution
|
||||
|
||||
I already concluded that Caddy is a suitable reverse proxy for the outlined use
|
||||
case. But in order to be care-free, the configuration has to be generated
|
||||
automatically. For this to work, I wrote a rather simple Python application
|
||||
called [Docker2Caddy](https://git.fsfe.org/fsfe-system-hackers/docker2caddy)
|
||||
that is kept running in the background via a systemd service and writes proper
|
||||
logs that are also rotated nicely.
|
||||
|
||||
This is how it works internally: it queries (in a configurable interval) the
|
||||
Docker daemon for running containers. For each container it looks for specific
|
||||
labels (that are also configurable), by default `proxy.host`, `proxy.host_alias`
|
||||
and `proxy.port`. If one or multiple containers are found – in our case two –
|
||||
one Caddy configuration file per container is created. This is based on a freely
|
||||
configurable Jinja2 template. If the configuration changed, e.g. by a new host,
|
||||
Caddy will be reloaded and will create a TLS certificate if needed.
|
||||
|
||||
But what happens if a container is unavailable? In Docker2Caddy you can
|
||||
configure a grace period. Until this is reached, the Caddy configuration for the
|
||||
container in question is not removed but could forward to a local or remote
|
||||
error page. Only afterwards, the configuration is removed, and Caddy reloaded
|
||||
subsequently.
|
||||
|
||||
So, what makes Docker2Caddy special? I am biased but see a number of points:
|
||||
|
||||
1. **Simplicity**: fundamentally it's a 188 pure lines of code Python script.
|
||||
2. **Configurability**: albeit it's simplicity, it's easy to configure for
|
||||
various needs thanks to the templates and the support for rootless Docker
|
||||
setups.
|
||||
3. **Adaptability**: it should be rather simple to make Docker2Caddy also work
|
||||
for Podman, or even use different reverse proxies. Feel free to extend it
|
||||
before I'll do it myself someday ;)
|
||||
4. **Performance**: while I did not perform before/after benchmarks, Caddy is
|
||||
blazingly fast and will surely perform better on the host than in a limited
|
||||
Docker container.
|
||||
|
||||
If you're facing the same challenges in your setup, please feel free to try it
|
||||
out. Installation is quite simple and there's even a [minimal Ansible
|
||||
playbook](https://git.fsfe.org/fsfe-system-hackers/docker2caddy/src/branch/master/install.yml).
|
||||
If you have feedback, I appreciate reading it via comments on Mastodon (see
|
||||
below), [email](/contact), or, if you have an FSFE account, as a new issue or
|
||||
patch at the [main repo](https://git.fsfe.org/fsfe-system-hackers/docker2caddy).
|
||||
Reference in New Issue
Block a user