To scream into the void: Yes, PyPi, someone was using those signatures. Distro package maintainers secured user supply chains with it!
I'm not looking forward to asking dozens of upstreams to host their signatures elsewhere (just stumbled across one case). Meanwhile reproducibility is now broken for those packages.
@dvzrv The number of signed sdist packages on PyPI is extremely low and PGP is not a great format for users to setup. In his Python Supply Chain Security talk, @di talks about the different threat vectors, and what improvements need to be made. https://youtu.be/VWWgkF-0cDQ?t=1014
@danyeaw@di None of that changes the fact that the change broke reproducibility on downstream packages and distribution packagers are now picking up the slack.
FWIW, when it comes to upstream OpenPGP signatures for sources: I'd rather have some instead of none. And I'd rather have a working chain of trust for as long as possible, than to have it being broken by the decision of a platform.
@dvzrv If 10 out of 400K packages are signed, then that is just technical debt that volunteers have to maintain and isn't helping supply chain security.
@danyeaw lol, what. Having a separately verifiable signature does increase the supply chain security, as one can establish a trust path.
Food for thought: The general public invisibility of signature files on the platform over the past years likely had an impact on it being used less.
Other than that: not sure which volunteers you are writing about, but I'm a volunteer and this breaking change burns up my free time and effectively makes me 2nd and triple guess working with Python 🤷
@dvzrv Another approach would have been for #PyPI to strengthen the infrastructure: associate accounts with OpenPGP keys, require and enforce a per-package list of authorized upload signing keys, keep a transparency log of authorized key changes, distribute per-package keyrings, etc.
It’s nothing fancy, ftp.gnu.org does some of that.
@dvzrv @sethmlarson this is an actual problem. It would have been better to remove the ability to upload new signatures, but as the signatures are part of the build artefacts they should never have been actually removed from pypi.
@Foxboron@dvzrv Apologies for the disruption this caused. I'm not sure that disabling uploading of new signatures would have been a better measure? It would still mean that new releases need to be waved through by downstream due to lack of signatures (unless lack of signature was acceptable before, I don't have full visibility into arch downstream)
@sethmlarson@Foxboron only preventing new uploads and having a much longer phase-out of the signature files would have been much preferred.
In the current situation this breaks too much at once (as the files can no longer be downloaded and we have to contact upstreams over how they are going to handle this).
tbqh, I'm contemplating an RFC to stop using package sources from #PyPi, as dealing with sdist tarballs there was often an uphill battle already to begin with (e.g. due to missing files, missing tests, etc.).
In a way removing the signatures was the last straw that broke the camel's back (for me at least). And yes, removing the files instead of just stopping to allow uploading them was not a good idea.
@dvzrv@Foxboron This seems reasonable to me. The fact that tests/configuration files are included in sdists at all is mostly incidental for many projects.
Future provenance attestation efforts will be focused on pinning a release back to a specific git ref, so that could be useful in the future as well for reproducibility.
Add comment