Menu
← All posts

Homerun has release channels now, because I was tired of shipping broken builds

Homerun used to release on every merge to main. Releaser read the commits, bumped the version, tagged, built the image, and every install checking for updates saw a shiny new version in the sidebar. Including the broken ones. Not "a test is flaky" broken, "the dashboard doesn't boot" broken, already :latest, already pulled by whoever clicked Update now in the last few minutes.

So I'd write the fix, push it, and sit through fifteen minutes of CI to ship a patch release whose only job was to apologize for the previous one. 1.4.2: fix what 1.4.1 broke. 1.4.3: fix what 1.4.2 broke fixing 1.4.1. Do that enough and your changelog stops being a history of features and becomes a history of your mistakes.

the actual problem

Shipping bugs isn't the problem. Everyone does. The problem was that "I merged it" and "it's on your server" were the same event, so every merge was a stable release by definition, because it was the only kind of release there was.

The usual answer is "test more before merging". Sure. But with one lane, every minute I added to the end-to-end suite was a minute added to every release, emergency fixes included. So the suite stayed thin, because a thorough one was a tax on every fix. What I actually wanted was a cheap merge and a deliberate release. Two events, not one.

three channels

Pick one in Settings → General → Release channel, or homerun instance channel from the CLI if the dashboard is the thing that's broken.

Nightly ships every merge to main that passes the prek suite and builds. The only thing it skips is end-to-end, so a build that would fail those still ships here. That's the point.

This is what my own instance runs. Not a staging box, not a VM I spin up to feel responsible. The instance I use every day, the one I migrated everything to off Dokploy, the one serving the page you're reading. If a nightly breaks deploys, I find out because I can't deploy. If it breaks the dashboard, I find out because I'm staring at it. Tests catch what I thought to test. Living on it catches the rest, and it catches it on me before it catches it on you.

Anyone else is welcome on nightly too. Expect breakage. I'll be complaining louder, and first.

Canary is the same builds, once they've passed the end-to-end tests. Nightly and canary builds of the same merge share their number, so 1.5.0-nightly.3 and 1.5.0-canary.3 are the same code, one of them just survived the tests.

Stable is the default, and the only channel the installer and homerun update follow. I cut it by hand from a canary that's already been running on my instance and passed the full suite. If it hasn't held up on both fronts, it never becomes stable, and stable users never find out it existed.

Moving to a more stable channel never downgrades you either. Go from nightly back to stable and the update notice stays quiet until stable catches up with what you already run.

the Releaser side

No second release tool, which is good, since the whole point of writing Releaser was not having a pile of them. Just two inputs:

- uses: orochibraru/releaser@v1
  with:
    prerelease: canary
    release-pr: true

prerelease ships each push as X.Y.Z-<id>.N: a tag on HEAD, a GitHub prerelease that never becomes "Latest", an image tagged :canary, never :latest, and no release commit churning CHANGELOG.md on every push. nightly is just another id.

release-pr stops Releaser from committing to main at all. It keeps an open PR carrying the next stable version and its changelog, rebuilt on every push so it never conflicts. When a canary has earned it, I merge that PR, and that push is the stable release: tag, release notes, :X.Y.Z and :latest. One click, on purpose, not whenever I happen to merge a typo fix.

the updater doesn't trust me either

Channels decide what gets offered. They don't stop a bad build from getting installed, and on nightly I've explicitly promised bad builds.

So before switching versions, the updater boots the new one next to the running one as a throwaway candidate and waits for /api/v1/ready, which needs the database and the auth layer actually working. If it doesn't answer within a couple of minutes, your files go back and nothing changes. If the switch happens and the new version falls over anyway, it rolls back. A build that can't boot never replaces one that can, which matters a lot when the instance taking the risk also hosts my website.

the tests get to be slow

This is the part I didn't see coming. Nightly skips end-to-end, so end-to-end is off the critical path. A fix reaches my instance as soon as it passes prek and builds, however long the e2e suite takes.

Which means I can stop rationing it. If the suite grows to 45 minutes because it covers the slow flows I used to skip, full deploys, rollbacks, updates, I don't care. The only thing waiting on it is canary, and canary is supposed to wait. Every minute that suite runs goes into making stable boring, and boring is all I want from stable.

Before, a thorough test suite and a fast fix came out of the same budget and I had to pick. Now they're on separate lanes and I get both.

Merging is cheap again. Releasing is a decision again. Docs for the channels and the Releaser setup if you want to steal it.