tests: Add end-to-end test for the examples#2948
Open
arpitjain099 wants to merge 1 commit into
Open
Conversation
Add an integration test that runs the repository, uploader and client examples against each other, the way a user runs them from the command line: * start examples/repository/repo as a live HTTP server * run examples/uploader/uploader to TOFU, claim a delegation and upload target files * run examples/client/client to TOFU and download the uploaded targets, which verifies them against the repository metadata The test uploads a second target after the client is already initialized and then downloads it, so the client has to update from the metadata version it trusts (N) to a newer version (N+M). This covers the still-open "examples" part of issue theupdateframework#2228; the tuf.repository interface tests requested there landed earlier in PR theupdateframework#2651. The test is hermetic: it points HOME at a temporary directory so the client and uploader trust stores do not touch the real home directory, runs the server on a free port, and polls for readiness instead of sleeping. Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Member
|
I'd really like to have this -- without it the examples are likely to break. From CI:
It's been so long since I touched tests that actually setup http servers that I don't have any useful advice but that issue does reproduce locally for me. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of the changes being introduced by the pull request:
This adds an end-to-end test that runs the
repository,uploaderandclientexamples against each other, which is the part of #2228 that was not covered by thetuf.repositoryinterface tests already merged in #2651.The new
tests/test_examples_e2e.pydrives the three examples the way a user runs them from the command line (see their READMEs):examples/repository/repois started as a live HTTP server on a free portexamples/uploader/uploaderdoes Trust-On-First-Use, claims a delegation, and uploads target files to that serverexamples/client/clientdoes Trust-On-First-Use and downloads the uploaded targetsBecause the client downloads through
ngclient.Updater, a successful download means the metadata chain and the target hash both verified, so the test asserts the downloaded content matches what the uploader published.It also covers the "client that has metadata version N can successfully update to N+M" case from the issue: after the client is initialized and has downloaded a first target, the uploader adds a second target (which publishes new targets, snapshot and timestamp metadata). The test checks that the snapshot version advanced and that the already-initialized client can then refresh and download the new target.
A few notes on how it stays deterministic and isolated:
from _simplerepo import SimpleRepository), so each example dir is put onPYTHONPATHPath.home(), soHOMEis pointed at a temporary directory; nothing touches the real home dir and every run starts from a clean repository1.root.json(reusingtests.utils.wait_for_serverfor the port) rather than sleepingThe examples themselves are not modified; this only wires the existing ones together as a test. Ran locally with
python -m unittest tests.test_examples_e2e(also green under the project's surrounding example tests), andruff check/ruff format/mypyare clean on the new file.I kept this scoped to the examples e2e slice. Whether the remaining idea in the issue (unit-testing
SimpleRepositorydirectly) is still wanted, and whether #2228 should stay open for it, is up to you.Addresses #2228
Please verify and check that the pull request fulfills the following
requirements: