With https://github.com/libp2p/test-plans/pull/121 merged, we should be able to use @master
directly. Still pointing to a concrete git hash.
This includes
6d1aed2ed5, thus
allowing interop tests to run from fork pull requests.
Always referencing the latest version of Rust for clippy creates problems during backporting as already fixed problems in master come up again during a backport.
Building inside the container allows Windows and MacOS users to also build this binary. Thanks to a new feature from docker, `--mount=type=cache`, rebuilding layers is fast without any additional hacks.
This patch adds a workflow that automatically publishes a docker image of our `ping` interop-test binaries on every release. Releases are different from Git tags. We publish a tag for each version of each crate but only a single release for each version of the `libp2p` crate.
Alternatively, this workflow can also be triggered manually:
1. Select the Git ref you want to run the workflow on. This should be the version of `libp2p` you want to publish the test binary for. For example, if you want to publish a version of the test binary for a hotfix release of a sub-crate, you would pick the respective tag of the hotfix release.
2. Choose the tag of the docker image.
To resolve https://github.com/libp2p/test-plans/issues/112, I am planning to create branches off current master that hardcodes the libp2p version in the test-binary to a particular one and then trigger this workflow for the respective branch.
We make the binary that should be copied into the container configurable via a build time `ARG`. This will allow us to reuse the same `Dockerfile` once we write more tests.
Additionally, this allows us to remove some of the directory switching and creation code. In order to not send the entire repository over to the docker daemon as a build context, we introduce a `.dockerignore` file that only allows select binary to be sent over.
The downside of this is that we need to extend this ignore file every time we add a new test. This shouldn't happen very often though and is easily discovered because the building of the docker container will fail.
This is much quicker to execute. To keep our CI reproducible, I opted to link to a specific version instead of the "latest" release. Updating to a new version is as easy as switching out the version number in the URL.
With the addition of more CI jobs, we are constantly running into API limits on setting up protoc. For all jobs that run on ubuntu, we can install it from `apt` instead.
On ubuntu 22.04, which is what `ubuntu-latest` points to, this installs `protoc v3.12.4`.
GitHub wraps the titles of commits if they are longer than 72 characters. See fbd4192e2a for example.
There is a convention that titles should be no more than 50 characters: https://cbea.ms/git-commit/#limit-50
This however makes crafting the message quite difficult, esp. with our use of conventional commit messages, thus limiting it to 72 seems more reasonable as that is where tooling (.e.g GitHub) seems to "break".
Currently, we have dependabot configured to automatically rebase PRs as soon as the base branches changes. This however causes two problems:
1. It unnecessarily consumes CI resources because dependabot rebases all open dependency PRs at once upon every merge of a PR into `master`.
2. It does not interact well with our strategy of dismissing reviews on updates to the PR: https://github.com/libp2p/rust-libp2p/pull/3226#event-8074236930
Overall, the rebasing of dependency PRs is a way to fix merge conflicts and to ensure that CI still passes even on the latest version of the base branch. The latter is already ensured by our merge queue and the former can be triggered manually with `@dependabot rebase`, thus disabling automatic rebasing is the better configuration for our setup.
Previously, some of the caches for these job runs were overlapping. By incorporating the relevant matrix variables into the cache key, every instance of this job gets its own cache.
Currently, we create a new cache for each workflow run for each crate. That ends up blowing the maximum allowed cache size of 10GB and GitHub deletes the least-recently used cache again. Effectively, this means we don't have any caching.
This patch introduces a cache factory workflow that only runs on master and always _saves_ a new cache. The CI workflow run for pull-requests on the other hand only restore these caches but don't save them.
Currently, a user can push code after it has been approved and the send-it label applied. We only want to merge code that we actually looked at. Use mergify to dismiss approvals in such circumstances.
This PR tweaks the installation of `cargo-semver-checks` to make it use the `--locked` flag.
Installing binaries with their locked dependency versions makes it less likely that you might run into issues caused by bugs in dependency libraries, since your installed dependency versions match the versions used in the binary's own test environment.
This is a recommendation that applies to most Rust binary tools. For example, here's cargo-nextest recommending the same: https://nexte.st/book/installing-from-source.html#installing-from-cratesio
We refactor our continuous integration workflow with the following goals in mind:
- Run as few jobs as possible
- Have the jobs finish as fast as possible
- Have the jobs redo as little work as possible
There are only so many jobs that GitHub Actions will run in parallel.
Thus, it makes sense to not create massive matrices but instead group
things together meaningfully.
The new `test` job will:
- Run once for each crate
- Ensure that the crate compiles on its specified MSRV
- Ensure that the tests pass
- Ensure that there are no semver violations
This is an improvement to before because we are running all of these
in parallel which speeds up execution and highlights more errors at
once. Previously, tests run later in the pipeline would not get run
at all until you make sure the "first" one passes.
We also previously did not verify the MSRV of each crate, making the
setting in the `Cargo.toml` rather pointless.
The new `cross` job supersedes the existing `wasm` job.
This is an improvement because we now also compile the crate for
windows and MacOS. Something that wasn't checked before.
We assume that checking MSRV and the tests under Linux is good enough.
Hence, this job only checks for compile-errors.
The new `feature_matrix` ensures we compile correctly with certain feature combinations.
`libp2p` exposes a fair few feature-flags. Some of the combinations
are worth checking independently. For the moment, this concerns only
the executor related transports together with the executor flags but
this list can easily be extended.
The new `clippy` job runs for `stable` and `beta` rust.
Clippy gets continuously extended with new lints. Up until now, we would only
learn about those as soon as a new version of Rust is released and CI would
run the new lints. This leads to unrelated failures in CI. Running clippy on with `beta`
Rust gives us a heads-up of 6 weeks before these lints land on stable.
Fixes#2951.
