ci: fixes for the sbom and secscan workflow, and trigger it on publishing

[tonyandrewmeyer]

Connect the publishing workflow to also trigger SBOM generation and secscan for the new artifacts. Also adds instructions to the release process to copy the resulting reports to the correct location.

Also fixes a few issues with the existing workflow:

• Explicitly install the libapt-pkg-dev package, which is required to install the Python apt package.
• Upload the reports into separate wheel and sdist locations, to avoid clashes.
• Remove the 2.23/7.23 versions -- those should be in the 2.23 branch.
• Remove an old configuration setting from the wheel config.

Also makes use of the new ability of sbomber to automatically detect the version number, so that the manifest files don't need to be updated for each release.

For ops 2.23, ops-scenario 7.23, and ops-tracing 2.23, new releases will continue to use manual processes.

[tonyandrewmeyer]

This still has two TODO items, but they're both trivial (and could be done in a follow-up PR if the merge and security approval isn't ready in time). Requesting a review in advance of that to hopefully get this included in the process for July's release, and address any issues found in review before those minor adjustments.

[benhoyt]

Oh, when discussing the other day whether to automate this I was thinking this was a once a cycle thing, or whenever the security team needed updates. I didn't realise it was every release. Do we need to do it every release, or just once a cycle (and ad-hoc other than that)?

[tonyandrewmeyer]

The policy says:

It is a requirement for each product and engineering group to complete vulnerability scanning, through Canonical's scanning range (hosted by Security Engineering), each cycle[1]
[1] Additional security scanning tools built into the CI/CD are highly encouraged, but at this point not required.

But also:

The minimum timeframe for rescanning should be monthly, but weekly or daily is much preferred. Infrastructure to automate this rescanning is recommended, to ensure reliability.

With the addition of the SSDLC ID Params, the doc says that the results are copied to a long term store. Nowhere in that doc does it say that you have to upload the results otherwise. I'll check with Ricardo about this when talking to him tomorrow -- perhaps we don't need to provide the results if we're using the params.

For SBOM, the policy says:

Coming soon, by the end of 25.04:
Automatic upload of SBOMs to Google Drive

However, I haven't seen any announcement of this. I can ask Ricardo about that too. It also doesn't specify anything about providing the results, other than saying that you should post on MM, but nothing about frequency.

The overall SSDLC policy says that both security scanning and SBOM generation are required every cycle. Most of it seems very designed to per-cycle releases like Ubuntu.

We do ~6 releases per cycle - it doesn't seem like a big ask to download and upload the results as part of that, in my opinion, although it would certainly be better if it was automated on the other side.

I do strongly feel that we should be doing the security scan for every release (even though I'm not particularly convinced it'll find anything that dependabot wouldn't), and generating an SBOM for every release (since the GitHub SBOM generator will only generate the current SBOM).

[benhoyt]

Sounds reasonable, thanks. Be good to hear more details from Ricardo on some of those items, and in the meantime I'm happy to progress with the manual download+upload (per monthly release).

[tonyandrewmeyer]

TL;DR: near-term we should be able to drop this requirement, for this cycle let's continue with the process as described here.

--

For SBOM, Ricardo has to check with some other people and hasn't been able to do that yet, so for the moment the ask is that we continue to upload it. Once a cycle is sufficient, but if we have it generated for each release then uploading that is nice. This is a short-term requirement, as there is definitely automation coming that will handle this for us.

For the security scan, they have this information (because our automation uses the ID params feature) so we don't need to store the results separately. However, our process bundles this with the SBOM so we'll end up doing it until we don't need the SBOM one any more.

There is a request to make sure that the security dashboard is updated with the secscan results once per cycle. I don't think that belongs in HACKING.md, though - there's automation coming that will produce a Jira SSDLC ticket for us with a DoD each cycle, and I think that's a better place to track once-per-cycle work.

[dimaqq]

Code looks good.

I'm not sure about the extra manual steps, but we can always start like this and improve later.

[dwilding]

I had a look through the workflow & configuration. Since this area is new to me, I focused mainly on the updated release process.

[dwilding]

Open the artifacts and verify that the security scan has not found any vulnerabilities

I haven't used artifacts like this before - is it self-explanatory how to verify the artifacts? From reading the instructions, my assumption is that they are text files, and there would be obvious warnings/errors if a vulnerability was found.

[tonyandrewmeyer]

You can download an example here if you'd like - I should have put that in the description.

They're HTML files, and hopefully look like this:

I don't have a failing one at hand, but it's pretty obviously not the "nothing found" result like the above.

I think it would be nice if the tool left a comment or scan result in the repo if it found something, but I don't think I have time to add that feature at the moment.

[tonyandrewmeyer]

This still has two TODO items, but they're both trivial (and could be done in a follow-up PR if the merge and security approval isn't ready in time). Requesting a review in advance of that to hopefully get this included in the process for July's release, and address any issues found in review before those minor adjustments.

The auto-detect version numbers feature is merged, so that's incorporated in here now. No answer from the security team about making sbomber public yet, so that should wait for a future PR (assuming approval is granted -- if not, then switching from my token to a prints-charming one).