117 lines
9.1 KiB
Markdown
117 lines
9.1 KiB
Markdown
# PoC Implementation Plan
|
|
|
|
This plan validates the architecture described in `concept.md` end-to-end before adopting it for real services. It was drafted after a scoping discussion; the decisions below reflect that discussion.
|
|
|
|
## Problem statement
|
|
|
|
Validate the `concept.md` architecture (Gitea push → Woodpecker build → Zot registry via ORAS → cross-pipeline trigger → central `deployments.git` pipeline → SSH → `docker compose up` on a target host) end-to-end with a dummy multi-container service, before adopting it for real services.
|
|
|
|
## Scope decisions
|
|
|
|
- Full end-to-end flow is in scope, not just isolated pieces (e.g. just the registry).
|
|
- Two Hetzner VMs, cheapest available tier, hostnames to be provided by the user once created:
|
|
- `ci` host: Woodpecker (server + agent) + Zot, rootful Docker.
|
|
- `target` host: the "production" host, rootful Docker.
|
|
- Ansible in this repo (`repo2cicd2deploy`) provisions both hosts: one inventory, host groups `ci` and `target`, single playbook run.
|
|
- New repos to be created outside this repo, in the user's self-hosted Gitea:
|
|
- `deployments.git` — central deploy pipeline.
|
|
- a dummy service repo — DB container with a root password plus an app container that receives the DB password and its own separate password. All values are dummy/fake, never real secrets, and are SOPS-encrypted regardless.
|
|
- Registry: Zot, with htpasswd basic auth (simplest viable auth for a PoC) and a retention policy configured.
|
|
- Cross-pipeline trigger: official `woodpeckerci/plugin-trigger` plugin, authenticated with a dedicated bot Gitea account's Woodpecker API token (not a personal token).
|
|
- Host alias resolution: kept simple, lives in `deployments.git`, no per-repo allow-list for this PoC.
|
|
- SSH: normal key-based SSH, no forced-command restriction (explicitly deferred, not needed for PoC).
|
|
- Dummy service: two containers (db + app), no real application code. The app container only needs to prove it received the correct env vars, e.g. by logging `env | grep -E 'DB_PASSWORD|APP_PASSWORD'` on startup and then staying up.
|
|
- Rootless Docker is explicitly deferred; the PoC uses rootful Docker on both hosts.
|
|
|
|
## Background / research findings
|
|
|
|
- Woodpecker has its own secret store, separate from Gitea. Gitea Actions secrets are unrelated and unused here; Gitea is only the Git forge and webhook source.
|
|
- Cross-repo pipeline triggering is supported via the official `woodpeckerci/plugin-trigger` plugin, which calls the Woodpecker REST API with a token, can pass arbitrary params, and can trigger a `deploy`-type pipeline event.
|
|
- Zot supports OCI artifacts (ORAS-compatible), unlike Gitea's built-in registry, and supports retention policies; htpasswd is the simplest auth backend for a PoC.
|
|
- Gitea↔Woodpecker integration is a standard OAuth application registration in Gitea (client ID/secret) plus `WOODPECKER_GITEA_URL` / `WOODPECKER_GITEA_CLIENT` / `WOODPECKER_GITEA_SECRET` environment variables on the Woodpecker server.
|
|
- `concept.md` was updated during the scoping discussion to reflect all of the above (Zot as the chosen registry, host-alias resolution living in `deployments.git`, the concrete `plugin-trigger` mechanism with a bot-token note).
|
|
|
|
## Architecture for the PoC
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
subgraph CI["ci host (Hetzner VM 1)"]
|
|
WPS["Woodpecker server + agent"]
|
|
Zot["Zot registry"]
|
|
end
|
|
subgraph TGT["target host (Hetzner VM 2)"]
|
|
Compose["docker compose<br/>dummy-db + dummy-app"]
|
|
end
|
|
Gitea["existing Gitea"] -->|webhook| WPS
|
|
Dev["dummy-service.git push"] --> Gitea
|
|
WPS -->|build image, push| Zot
|
|
WPS -->|ORAS push artifact| Zot
|
|
WPS -->|plugin-trigger| WPS
|
|
WPS -->|ORAS pull artifact, SOPS decrypt, SSH| TGT
|
|
```
|
|
|
|
Both VMs are provisioned by one Ansible playbook (`site.yml`) with two groups: `ci` and `target`. Roles: `docker` (both groups), `zot` and `woodpecker` (`ci` group only).
|
|
|
|
## Task breakdown
|
|
|
|
### Task 1: Ansible skeleton and Docker role
|
|
|
|
Set up `inventory/hosts.yml` with `ci` and `target` groups (placeholder hostnames until provided), a `docker` role installing Docker CE (rootful) via the official apt repository, and a `site.yml` playbook applying it to both groups.
|
|
|
|
- Test: `ansible-playbook site.yml --check` runs without errors against a real inventory entry; `docker run hello-world` succeeds on both hosts after a real run.
|
|
- Demo: both VMs have working rootful Docker, verified with `docker version` over an Ansible ad-hoc command.
|
|
|
|
### Task 2: Zot role and registry smoke test
|
|
|
|
Add a `zot` role deploying Zot via Docker Compose on the `ci` host, with htpasswd basic auth and a retention policy configured. Generate credentials and store the htpasswd file as an Ansible-vault-encrypted variable, or generate it idempotently on the host.
|
|
|
|
- Test: from a workstation, `oras login`, then `oras push`/`oras pull` a trivial test artifact (a text file) against the Zot instance, and `docker push`/`docker pull` a trivial image — confirms Zot handles both image and arbitrary-artifact OCI content as the concept requires.
|
|
- Demo: a manually pushed test artifact and test image are both retrievable by digest from Zot.
|
|
|
|
### Task 3: Woodpecker server/agent role and Gitea integration
|
|
|
|
Add a `woodpecker` role deploying Woodpecker server + agent via Docker Compose on the `ci` host. Register a Gitea OAuth application for Woodpecker (documented manual step, since it requires the Gitea UI — note this in a README rather than automating it). Wire `WOODPECKER_GITEA_URL` / `WOODPECKER_GITEA_CLIENT` / `WOODPECKER_GITEA_SECRET`.
|
|
|
|
- Test: log into the Woodpecker UI via Gitea SSO, activate a throwaway test repo, confirm a trivial `.woodpecker.yml` (e.g. `echo hello`) runs successfully on push.
|
|
- Demo: a test push to a scratch repo triggers a visible, successful Woodpecker pipeline run.
|
|
|
|
### Task 4: Dummy service repo — build and publish
|
|
|
|
Create `dummy-service.git` in Gitea with:
|
|
|
|
- `Dockerfile.db` (e.g. `FROM alpine`, sets up a fake "DB" that just accepts a root password env var)
|
|
- `Dockerfile.app` (e.g. `FROM alpine`, entrypoint script `env | grep -E 'DB_PASSWORD|APP_PASSWORD'; sleep infinity`)
|
|
- `compose.yaml` referencing `${IMAGE_TAG}`-style placeholders for both images
|
|
- `secrets.prod.env` (dummy `DB_ROOT_PASSWORD`, `DB_PASSWORD`, `APP_PASSWORD`) encrypted with SOPS/age (a fresh age keypair generated for this PoC only)
|
|
- `.sops.yaml` and `.woodpecker.yml` building both images, pushing them to Zot, and bundling and pushing the deployment artifact (compose.yaml + encrypted secrets.prod.env + release.env) via ORAS
|
|
|
|
- Test: push triggers Woodpecker; verify both images and the deployment artifact land in Zot with resolvable digests (`oras discover`/`crane digest` or equivalent).
|
|
- Demo: a git push to `dummy-service.git` results in two images and one deployment artifact visible in Zot.
|
|
|
|
### Task 5: Central deployments repo — trigger wiring
|
|
|
|
Create `deployments.git` with `.woodpecker/deploy.yaml` reacting to a `deployment` event, and `hosts/target1` containing the SSH connection details (hostname, user) for the target VM. Add the `plugin-trigger` step to `dummy-service.git`'s pipeline (Task 4), authenticated with a dedicated bot account's Woodpecker token stored as a Woodpecker secret on `dummy-service.git`.
|
|
|
|
- Test: push to `dummy-service.git`; confirm the `deployments.git` pipeline is triggered with the correct `ARTIFACT` and `TARGET` params visible in its build log.
|
|
- Demo: one push to the service repo visibly triggers a second, separate pipeline run in `deployments.git` with the right parameters.
|
|
|
|
### Task 6: Central deployments repo — decrypt, transfer, deploy
|
|
|
|
Extend `deploy.yaml` to: resolve `TARGET` against `hosts/`, ORAS-pull the artifact by the given digest, SOPS-decrypt `secrets.prod.env` using the deployment age key (stored as a repo-level Woodpecker secret on `deployments.git`), transfer the compose file and decrypted env to the target host, then SSH-run `docker compose pull && docker compose up -d --remove-orphans`, then delete the remote plaintext secrets file.
|
|
|
|
- Test: end-to-end run from a `dummy-service.git` push results in both containers running on the target host; inspecting the app container (exec or logs) shows the correct decrypted `DB_PASSWORD`/`APP_PASSWORD` values reaching it, proving the full secrets path.
|
|
- Demo: full pipeline, push-to-running-containers, verified by inspecting the app container's logged env vars on the target host.
|
|
|
|
### Task 7: Wrap-up documentation
|
|
|
|
Add a short `POC-RESULTS.md` (or similar) in this repo capturing what worked, any deviations from `concept.md`, and open items to resolve before using this for a real service (e.g. restricted SSH, per-repo host allow-list, key rotation process). Update `concept.md` if the PoC surfaces anything materially different from what is documented there.
|
|
|
|
- Demo: a short written record for future reference to decide next steps — no new code, just documentation.
|
|
|
|
## Open items deferred beyond this PoC
|
|
|
|
- Restricted SSH (forced-command / dedicated non-shell user) for the deployment account on production hosts.
|
|
- Rootless Docker on production hosts.
|
|
- Per-repository allow-list for which host aliases a service repository may deploy to.
|
|
- Age key rotation process and procedure for onboarding/offboarding DevOps team members.
|