Improving liboqs' security and release workflow

[xuganyu96]

TL;DR

• Require a red-green commit pair for each security fix: the red commit contains a regression test that is verified to fail against the unpatched implementation, and the green commit contains the fix that makes the test pass.
• Starting with 0.17, experiment with supporting security patch releases and preparing releases on dedicated branches rather than on main. This means maintaining additional branches, especially during release preparation, but avoids the need to freeze main.

Commit history for security fixes

Until now, we have squash-merged pull requests—public and private—into main.
This approach keeps the main branch’s commit history concise while allowing
external contributors to organize their development history however they like.
For contributions unrelated to security advisories, I propose keeping this
workflow unchanged.

For security-sensitive contributions—specifically, fixes handled through GitHub
Security Advisories (GHSAs) and their associated pull requests—I propose an
additional validation and traceability requirement: each fix should be
represented by exactly two commits on each affected maintained branch.

The first commit, the red commit, contains a regression test that must be
verified to fail against the unpatched implementation because of the reported
issue, rather than an unrelated build or test failure. The second commit, the
green commit, contains the fix that makes the regression test pass. The
completed fix must also pass the applicable test suite.

The main benefit is a more auditable history of liboqs’ security fixes: each
regression test is demonstrated to detect the original issue before it is used
as evidence that the patch addresses that issue.

The trade-off is additional work during patch development. Before integration,
the development history will need to be organized into a red-green commit pair.
Both states must be revalidated after any history cleanup, rebase, or backport.

We will integrate security fixes without squashing the pair. To maintain a
linear history, the intended operation is to rebase the fix branch onto the
target branch, validate the resulting commits, and then fast-forward the
target branch to the completed green commit. We will not rebase the published
history of main or a release branch. See Git’s documentation on
rebasing.

Release-specific changes, such as version updates and release notes, may be
committed separately from the red-green pair.

Release branches and security patch releases

Starting with 0.17, I want to experiment with a branching release process and
support for security patch releases.

Each release line will have a branch named release/x.y, such as
release/0.17. Release candidates, the initial final release, and subsequent
patch releases will all be prepared on that branch rather than on main. Patch
numbers and release-candidate identifiers will appear in tags, not in the
release-branch name.

Each security patch release will contain one or more security fixes and the
necessary release metadata. It will increment the patch component of the
major.minor.patch version number. Release candidates will use a prerelease
suffix, such as 0.17.0-rc1. See the Semantic Versioning
specification.

For example:

1. Create the release branch. When we decide to prepare 0.17, select a base
   commit on main and create release/0.17.
2. Publish the first release candidate. Make the necessary
   release-preparation changes on release/0.17, tag the resulting commit as
   0.17.0-rc1, and publish the release candidate.
3. Gather feedback and finalize the release. Allow a feedback period—for
   example, one week. Apply any necessary corrections to release/0.17 and
   publish additional release candidates, such as 0.17.0-rc2, as needed. Once
   the release is ready, make any final release-metadata changes and tag the final
   release as 0.17.0. Generally applicable fixes made during release preparation
   should also be propagated to main.
4. Continue normal development independently. New features and unrelated
   changes will continue to land on main; they will not automatically be
   included in release/0.17. During release preparation, the release branch will
   accept release-blocking fixes and necessary release-preparation changes. After
   the final release, it will be maintained for security fixes and the changes
   needed to publish them.
5. Publish security patch releases from the supported release branch.
   Prepare each security fix privately against every affected maintained
   branch, including main and any upcoming release branch where applicable.
   Preserve and validate an appropriate red-green pair for each affected branch.
   For a fix to the released 0.17 line, publish the patched version from
   release/0.17 with the next patch tag, such as 0.17.1.

For embargoed vulnerabilities, public integration, publication of the patched
release, and publication of the advisory will be closely coordinated. The goal
is to have a patched release available when the advisory is published, without
exposing the fix substantially earlier through a public branch. GitHub’s
advisory
workflow
supports private patch development followed by release and disclosure.

This process should let us publish security fixes without including unrelated
development from main. Each advisory will identify the affected versions and
the first patched version for each relevant release line. Consistent
versioning, together with accurate advisory metadata, will help supported
security tooling identify affected users and available updates. See GitHub’s
guidance on writing security
advisories.

The process will also allow normal development to continue during release
preparation, avoiding a freeze of main that I find difficult to coordinate.

To limit the maintenance burden, we will support only one stable release line
at a time. Once 0.18.0 is released, support for the 0.17 line will end.

Normally, this means maintaining two long-lived branches: main and the
current supported release branch. During preparation of the next release, there
may temporarily be three: for example, main, release/0.17, and
release/0.18. This count excludes temporary development and security-fix
branches.

edit: passed through ChatGPT for suggested edits and fact checking

[baentsch]

fwiw, I completely agree on the improvement to the release process: Freezing main and doing a (necessarily) fast release cycle only works when you have dedicated release managers focusing on just this. Releasing off a separate(ly maintained) branch is the right way for an OSS project starved on very active committer-maintainers and interested in prioritizing feature-addition over procedures (release-freeze being such a thing). Coincidentally, oqsprovider already (factually, not yet firmly documented) proceeds that way as otherwise progress was stopped for too long.

The red-green fix commit history for security patches also conceptually makes sense. I wonder whether this is the right way for this project, though:

1. liboqs is not a production library demanding (or offering) such high-effort procedures: No-one is paying for this and it will cost maintainers time to enforce and "live" this that is missing elsewhere (reviewing and landing PRs, for example, contributing to an even longer PR backlog).
2. It makes contributing fixes harder: I'm very much concerned that this would lead to a longer backlog of vulnerability reports as the reporters may shy away from delivering patches themselves; this then would put this monkey on the back of a small set of maintainers, further reducing their time to do new work and/or review new features/PRs.

Thus, I'd at most suggest positing "red-green CI" as a desirable, but not mandatory feature into the internal security incident handling documentation.
If we really want (demand) this, I'd also demand this test as part of vulnerability disclosure. Advantages: a) Creating a red-green CI script then is something easily automated (in the worst case using an AI). b) Probably fewer security reports. Both reducing maintenance overhead.

[xuganyu96]

Thank you for the feedback. I agree with holding off the requirement for red-green commit until after I've tried it myself. On the other hand, I can start doing release branch as soon as 0.17 release.