feat: add distroless Docker image with built-in healthcheck#6902
feat: add distroless Docker image with built-in healthcheck#6902vyavdoshenko merged 7 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds support for a distroless Docker runtime image by introducing a shell-less healthcheck binary and a lightweight /healthz HTTP endpoint, plus CI to build/push the new image.
Changes:
- Add
tools/packaging/Dockerfile.distroless-devand a GitHub Actions workflow to build/push a distroless image. - Add
/healthzHTTP handler (and allow unauthenticated access like/metrics) plus a standalonedragonfly-healthcheckclient binary. - Add a Linux-only
--userflag to drop privileges after binding ports and (attempt to) chown the data directory.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/packaging/Dockerfile.distroless-dev | New distroless image build (copies dragonfly + dragonfly-healthcheck, adds Docker HEALTHCHECK). |
| tools/docker/fetch_release.sh | Minor output formatting tweak. |
| src/server/main_service.cc | Adds /healthz handler and auth bypass for it. |
| src/server/dfly_main.cc | Adds --user privilege-drop implementation and call site before service.Init(). |
| src/server/dfly_healthcheck.cc | New minimal HTTP healthcheck helper binary for distroless containers. |
| src/server/CMakeLists.txt | Adds dragonfly-healthcheck target. |
| Makefile | Ensures dragonfly-healthcheck is built/packaged in release artifacts. |
| .github/workflows/docker-distroless.yml | New scheduled/manual workflow to build/test/push distroless image and publish multi-arch manifest. |
🤖 Augment PR SummarySummary: Adds a distroless-based Docker image variant and a built-in healthcheck mechanism suitable for shell-less containers. Changes:
Technical Notes: The healthcheck uses an HTTP GET against the main port (via 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
a227553 to
410f7fd
Compare
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
410f7fd to
43e7b8e
Compare
There was a problem hiding this comment.
Pull request overview
Adds support for producing a minimal distroless Dragonfly Docker image by bundling a standalone dragonfly-healthcheck binary and exposing a lightweight /healthz HTTP endpoint.
Changes:
- Add
/healthzHTTP handler and allow unauthenticated access to it (similar to/metrics). - Introduce a new
dragonfly-healthcheckexecutable and wire it into the build/package flow. - Add a distroless Dockerfile and a GitHub Actions workflow to build/push a multi-arch distroless image.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/packaging/Dockerfile.distroless-dev | New distroless image build stage and runtime image with binary healthcheck. |
| tools/docker/fetch_release.sh | Minor script output formatting change. |
| src/server/main_service.cc | Adds /healthz endpoint and auth bypass for it. |
| src/server/dfly_main.cc | Adds --user flag and privilege drop logic on Linux. |
| src/server/dfly_healthcheck.cc | New standalone healthcheck binary (HTTP GET /healthz). |
| src/server/CMakeLists.txt | Adds dragonfly-healthcheck build target. |
| Makefile | Builds/packages the new healthcheck binary. |
| .github/workflows/docker-distroless.yml | New scheduled/manual workflow to build and publish distroless images (multi-arch). |
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
43e7b8e to
2d52e88
Compare
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
2d52e88 to
3b3a716
Compare
There was a problem hiding this comment.
Pull request overview
Adds support for running Dragonfly in a minimal distroless container by introducing a standalone HTTP-based healthcheck binary, wiring a /healthz endpoint into the existing HTTP console handlers, and adding CI to build/publish the new image.
Changes:
- Add a distroless Dockerfile that builds
dragonflyand a newdragonfly-healthcheckbinary and configures DockerHEALTHCHECK. - Add
/healthzHTTP endpoint and exempt it from HTTP auth checks (similar to/metrics). - Add a GitHub Actions workflow to build/push a multi-arch distroless image on a schedule.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/packaging/Dockerfile.distroless-dev | New distroless runtime image build with bundled healthcheck binary and healthcheck config. |
| tools/docker/fetch_release.sh | Minor script formatting fix. |
| src/server/main_service.cc | Adds /healthz handler and exempts it from HTTP auth. |
| src/server/dfly_main.cc | Adds --user flag support to drop privileges after binding ports. |
| src/server/dfly_healthcheck.cc | New minimal standalone healthcheck client binary (no dragonfly_lib). |
| src/server/CMakeLists.txt | Builds the new dragonfly-healthcheck executable. |
| Makefile | Builds/packages dragonfly-healthcheck alongside the main release artifacts. |
| .github/workflows/docker-distroless.yml | New scheduled multi-arch distroless image build/publish workflow. |
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
3b3a716 to
2a94998
Compare
|
please note I do not want to switch to distroless right away - but we can replace internal tools in our current image. |
| @@ -0,0 +1,126 @@ | |||
| // Copyright 2024, DragonflyDB authors. All rights reserved. | |||
There was a problem hiding this comment.
Commonly Used Minimal Health Checkers:
microcheck: A suite of tiny (~75 KB) C-compiled binaries including httpcheck, httpscheck, and portcheck. They offer configurable settings, such as response codes and timeout, and provide standard exit codes (0 for healthy, 1 for unhealthy).
dmikusa/tiny-health-checker: A very small binary written in Go designed to only make HTTP requests to localhost. It is specifically designed to be safe and difficult to misuse if a container is compromised.
cloudlena/healthcheck: An extension of the scratch Docker image that adds a built-in HTTP health check tool, adding only about 2.8MB to the image size.
There was a problem hiding this comment.
We have to add an additional binary regardless of our new binary or a tiny external check.
All these external checks can check the HTTP request or just open a TCP port. In this case, our binary is preferable because it can check the health using the PING command. /healthz handler has already been removed.
There was a problem hiding this comment.
what about tls configured datastore?
There was a problem hiding this comment.
in general I do have preference to package an external binary if it does the same as we do not want to reinvent the wheel and their static tool chain is probably better than of C++.
There was a problem hiding this comment.
Fixed
I added httpcheck and httpscheck in the image, but httpcheck is used by default.
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
2a94998 to
96f9a70
Compare
vyavdoshenko
force-pushed
the
bobik/distroless_container
branch
from
96f9a70 to
a56605e
Compare
There was a problem hiding this comment.
Pull request overview
Adds distroless-based Docker images and a standalone dragonfly-healthcheck binary to enable HEALTHCHECK CMD in shell-less containers, plus a /healthz HTTP endpoint and a Linux-only privilege drop flag.
Changes:
- Add distroless dev/prod Dockerfiles that include
dragonfly+dragonfly-healthcheckand use an exec-form Docker healthcheck. - Add
/healthzHTTP handler and a newdragonfly-healthcheckstandalone binary. - Add Linux
--userflag to drop privileges (and chown the data dir) after binding ports; extend build/CI to build the new binary and distroless flavor.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/packaging/Dockerfile.distroless-prod | New distroless production image that copies release artifacts + libz and configures exec-form healthcheck. |
| tools/packaging/Dockerfile.distroless-dev | New distroless dev image built from source; includes healthcheck binary and libz. |
| tools/docker/fetch_release.sh | Minor formatting/line fix; still stages the release binary into /build. |
| src/server/main_service.cc | Adds unauthenticated /healthz HTTP handler and exempts it from auth. |
| src/server/dfly_main.cc | Adds --user flag and privilege-drop/chown logic on Linux before service.Init(). |
| src/server/dfly_healthcheck.cc | New minimal HTTP /healthz probe binary for distroless HEALTHCHECK. |
| src/server/CMakeLists.txt | Adds dragonfly-healthcheck target. |
| Makefile | Builds and packages dragonfly-healthcheck into release artifacts. |
| .github/workflows/docker-dev-release.yml | Adds distroless to the dev image build matrix. |
Comments suppressed due to low confidence (1)
src/server/dfly_healthcheck.cc:119
- A single
read()into a small buffer is not guaranteed to contain the full HTTP status line; packetization/short reads can makestrchr(resp, ' ')fail even when the server is healthy. Read until you have a full line ("\r\n") or at least enough bytes to parse the status code, retrying on EINTR.
return strncmp(resp, kPongReply, strlen(kPongReply)) == 0 ? 0 : 1;
}
|
|
||
| ABSL_FLAG(uint16_t, tcp_backlog, 256, "TCP listen(2) backlog parameter."); | ||
| #ifdef __linux__ | ||
| ABSL_FLAG(string, user, "", |
There was a problem hiding this comment.
does it mean that distroless containers will have to run with root permissions?
There was a problem hiding this comment.
Current state: The distroless image runs as root, same as the existing images before entrypoint.sh drops privileges. The --user flag allows dropping privileges at runtime after binding ports.
Distroless base ships a built-in nonroot user (uid 65534). We could add USER nonroot to the Dockerfile, but there are trade-offs:
- Volume mounts (-v /host/path:/data) may not be writable by uid 65534 on the host - root doesn't have this problem
- The /data directory ownership needs to be set up at build time (extra COPY --chown layer)
- The --user flag for runtime privilege dropping becomes less useful (can't chown as non-root)
For now, running as root in distroless is less risky than in ubuntu - there's no shell, no package manager, no writable system paths. Users who require non-root can use docker run --user 65534 or Kubernetes securityContext.runAsUser.
I can add USER nonroot if you prefer - let me know.
romange
left a comment
romange
left a comment
There was a problem hiding this comment.
lgtm, we really need to improve our http strategy by going over use-cases like this one and public datastores (with tls) and understand how we should configure http access.
| }; | ||
| do_chown(data_dir); | ||
| std::error_code ec; | ||
| fs::recursive_directory_iterator it(data_dir, ec); |
There was a problem hiding this comment.
I run claude /review on this, as I do not have much knowledge and it gave me good comments, I think:
-
dfly_main.cc:230-238 — When a numeric UID is passed (e.g. --user=1000), getpwuid() is called. If it returns NULL (UID not in /etc/passwd), the function fails and the server exits. This is very common in
containers — distroless only has root and nobody in passwd. A user running --user=65534 (nobody) works, but --user=1000 fails even though it's a perfectly valid UID for setuid. Consider falling back to using the raw UID with setgroups(0, nullptr) when getpwuid returns NULL, similar to the more complete version discussed in earlier review rounds. Without this, the --user flag is
significantly less useful in the very containers it's designed for. -
/healthz requires primary_port_http_enabled — silent failure
The /healthz handler is registered via ConfigureHttpHandlers (main_service.cc:2797), which only runs when the HTTP console is active on the port. The HTTP listener is only created when
primary_port_http_enabled=true (default) or on the admin port (dragonfly_listener.cc:162-163).
worth documenting in the Dockerfile comments.
- HEALTHCHECK --start-period=30s is too short for large snapshots
Dockerfile.distroless-dev:47 — During snapshot loading, gstate() == GlobalState::LOADING (common_types.h:29), so /healthz returns 503. After the 30s start period, Docker begins counting health failures. With
--retries=3 --interval=15s, the container gets 75s total (30 + 3*15) before being marked unhealthy. Large snapshots can take minutes to load. Worth noting in the Dockerfile comment.
There was a problem hiding this comment.
- fixed
- the comment was updated
- the comment was updated
| #ifdef __linux__ | ||
| #include <grp.h> | ||
| #include <pwd.h> | ||
| #include <unistd.h> |
There was a problem hiding this comment.
nit: Since unistd.h is a POSIX header (not Linux-specific), it could stay outside the guard.
| }; | ||
| do_chown(data_dir); | ||
| std::error_code ec; | ||
| fs::recursive_directory_iterator it(data_dir, ec); |
There was a problem hiding this comment.
use error_code overload for iterating:
std::error_code ec;
fs::recursive_directory_iterator it(data_dir, ec);
for (; !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) {
do_chown(it->path());
}
if (ec) {
LOG(WARNING) << "Error iterating " << data_dir << ": " << ec.message();
}
Adds
Dockerfile.distroless-devandDockerfile.distroless-prodfor a minimal shell-less runtime image based ongcr.io/distroless/cc-debian12, and a/healthzHTTP(s) endpoint to Dragonfly for container health checks.Health checking uses microcheck - two tiny static binaries (~75 KB each) copied into the image:
httpcheck— default HEALTHCHECK for plain HTTP deploymentshttpscheck— available for TLS deployments (https://localhost:6379/healthz)The prod image (
Dockerfile.distroless-prod) is ready but requires a release thatincludes the
/healthzendpoint. Local strip experiment: ~70 MB release image.