• Necessary packages docker-registry and python3-docker-report are available for bullseye in the right versions
• Summarizing from irc, the real risk is the nginx config. Tests would need to be ran for:
  • publish a non-restricted image from build2001
  • publish a restricted image from deploy1002 (we test both authenticated and unauthenticated POST to a nonexistent upload path in httpbb, but a real build + push test would be better)
  • pull a non-restricted image without credentials from the public interface (already in httpbb)
  • pull a restricted image without credentials and see it fail (already in httpbb)
  • check the pipeline on gitlab still works

Why bullseye, this should be bookworm? docker-registry is packaged in Debian, so we can simply use bookworm and use the package from it. In fact, we are already using the bookworm package on the existing registry hosts (2.8.2+ds1-1)

If we jump right to bookworm, we need to copy the python3-docker-report package to bookworm.

A migration plan would look like:

1. Run httpbb /srv/deployment/httpbb-tests/docker-registry/*.yaml --host registry1004.eqiad.wmnet (one of the two passive eqiad nodes) and record results
2. Reimage registry1004 to bookworm
3. Run httpbb /srv/deployment/httpbb-tests/docker-registry/*.yaml --host registry1004.eqiad.wmnet again verifying results haven't changed
4. Run httpbb /srv/deployment/httpbb-tests/docker-registry/*.yaml --host registry2004.eqiad.wmnet (one of the two active codfw nodes) and record results
5. Depool registry2004 and reimage to bookworm
6. Run httpbb /srv/deployment/httpbb-tests/docker-registry/*.yaml --host registry2004.eqiad.wmnet again verifying results haven't changed
7. Pool registry2004 and depool registry2003
8. Verify image push works from build2001, deploy1002, and gitlab
9. Reimage registry2003 to bookworm and repool
10. Reimage registry1003 to bookworm

@JMeybohm could you check the httpbb tests are still relevant and returning the expected results?

Change #1050371 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] httpbb: Real for push has changed in nginx

https://gerrit.wikimedia.org/r/1050371

In T332016#9929672, @Clement_Goubert wrote:

@JMeybohm could you check the httpbb tests are still relevant and returning the expected results?

Almost. I've uploaded a patch to correct the one thing that seems to have changed.

The plan you laid unfortunately does not work. The httpbb tests, as they are now, do not work against a read-only registry (e.g. eqiad at the moment). But I would argue that we should be able to just add registry2005 (depooled) with bookworm, test against that and then decom one of the old ones (or create two new VMs and decom both of the old ones).

Change #1050371 merged by JMeybohm:

[operations/puppet@production] httpbb: Auth realm for pushing to docker registry has changed

https://gerrit.wikimedia.org/r/1050371