We refactor our continuous integration workflow with the following goals in mind:
- Run as few jobs as possible
- Have the jobs finish as fast as possible
- Have the jobs redo as little work as possible
There are only so many jobs that GitHub Actions will run in parallel.
Thus, it makes sense to not create massive matrices but instead group
things together meaningfully.
The new `test` job will:
- Run once for each crate
- Ensure that the crate compiles on its specified MSRV
- Ensure that the tests pass
- Ensure that there are no semver violations
This is an improvement to before because we are running all of these
in parallel which speeds up execution and highlights more errors at
once. Previously, tests run later in the pipeline would not get run
at all until you make sure the "first" one passes.
We also previously did not verify the MSRV of each crate, making the
setting in the `Cargo.toml` rather pointless.
The new `cross` job supersedes the existing `wasm` job.
This is an improvement because we now also compile the crate for
windows and MacOS. Something that wasn't checked before.
We assume that checking MSRV and the tests under Linux is good enough.
Hence, this job only checks for compile-errors.
The new `feature_matrix` ensures we compile correctly with certain feature combinations.
`libp2p` exposes a fair few feature-flags. Some of the combinations
are worth checking independently. For the moment, this concerns only
the executor related transports together with the executor flags but
this list can easily be extended.
The new `clippy` job runs for `stable` and `beta` rust.
Clippy gets continuously extended with new lints. Up until now, we would only
learn about those as soon as a new version of Rust is released and CI would
run the new lints. This leads to unrelated failures in CI. Running clippy on with `beta`
Rust gives us a heads-up of 6 weeks before these lints land on stable.
Fixes#2951.
Previously, we would always run `IfWatcher`, even if we were only listening on a specific interface. This patch fixes this behaviour and aligns it with how `libp2p-quic` operates.
Previously, we had one callback for each kind of message that a `ConnectionHandler` would receive from either its `NetworkBehaviour` or the connection itself.
With this patch, we combine these functions, resulting in two callbacks:
- `on_behaviour_event`
- `on_connection_event`
Resolves#3080.
Currently, `libp2p-mdns` fails to compile unless we enable the `tokio` runtime feature flag. This is caused by missing `cfg` statements upstream. See https://github.com/bluejekyll/trust-dns/issues/1830.
Until this is fixed upstream, we temporarily enable the `tokio-runtime` feature of `trust-dns-proto`.
Previously, we would only track the metrics like the number of open connections. With this patch, we extend these metrics with a `protocols` label that contains a "protocol stack". A protocol stack is a multi-address with all variable parts removed. For example, `/ip4/127.0.0.1/tcp/1234` turns into `/ip4/tcp`.
Resolves https://github.com/libp2p/rust-libp2p/issues/2758.
Previously, the executor for connection tasks silently defaulted to a `futures::executor::ThreadPool`. This causes issues such as https://github.com/libp2p/rust-libp2p/issues/2230.
With this patch, we force the user to choose, which executor they want to run the connection tasks on which results in overall simpler API with less footguns.
Closes#3068.
Update to `if-watch` version 3.0.0 and pass through features, such that `libp2p-tcp/async-io` selects `if-watch/smol` and `libp2p-tcp/tokio` brings in `if-watch/tokio`.
The mDNS part is already done in #3096.
Various muxer implementations struggle to fulfill this test. In practice, it doesn't matter much because we always run `multistream-select` on top of a newly negotiated stream so we never end up actually reading from a stream that we have never written to.
Relevant discussion: https://github.com/kpp/rust-libp2p/pull/27#discussion_r1012128418
Co-authored-by: Demi Marie Obenour <demiobenour@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: David Craven <david@craven.ch>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Thomas Eizinger <thomas@eizinger.io>
Co-authored-by: elenaf9 <elena.frank@protonmail.com>
Co-authored-by: Marco Munizaga <marco@marcopolo.io>
Currently, our `NetworkBehaviour` derive macro depends on the `libp2p` crate to be in scope. This prevents standalone usage which forces us to depend on `libp2p` in all our tests where we want to derive a `NetworkBehaviour`.
This PR introduces a `prelude` option that - by default - points to `libp2p::swarm::derive_prelude`, a new module added to `libp2p_swarm`. With this config option, users of `libp2p_swarm` can now refer to the macro without depending on `libp2p`, breaking the circular dependency in our workspace. For consistency with the ecosystem, the macro is now also re-exported by `libp2p_swarm` instead of `libp2p` at the same position as the trait that it implements.
Lastly, we introduce an off-by-default `macros` feature flag that shrinks the dependency tree for users that don't need the derive macro.
* Refactor `InboundSubstreamState` to implement `Stream`
This allows us to use `stream::SelectAll` instead of iterating
through the states ourselves. It also allows us to fix an existing
TODO where we don't properly close a stream.
* Refactor `OutboundSubstreamState` to implement `Stream`
* Remove unnecessary keep_alive changes
The same thing is happening further down.
* Remove code duplication in answering requests
* Debug assert that we can answer every request
* Remove unreachable error case
Instead of taking the connection out of the map again, construct
the event to be returned with the data we already have available.
* Remove `Pool::get` and `PoolConnection`
These are effectively not used.
* Replace `iter_pending_info` with its only usage: `is_dialing`
* Add `is_for_same_remote_as` convenience function
* Remove `PendingConnection`
* Rename `PendingConnectionInfo` to `PendingConnection`
With the latter being gone, the name is now free.
* Merge `EstablishedConnectionInfo` and `EstablishedConnection`
This is a leftover from when `Pool` was still in `libp2p-core` and
one of them was a public API and the other one wasn't.
All of this is private to `libp2p-swarm` so we no longer need to
differentiate.
* Don't `pub use` out of `pub(crate)` modules
* ROADMAP: Remove project board for now
* ROADMAP: Encourage feedback
* ROADMAP: Remove TLS
* ROADMAP: Link to specs/ROADMAP.md
* ROADMAP: Prioritize QUIC beyond the first iteration
* ROADMAP: No longer mark QUIC as experimental
Co-authored-by: David Craven <david@craven.ch>
Co-authored-by: Demi Marie Obenour <demiobenour@gmail.com>
Co-authored-by: Elena Frank <elena.frank@protonmail.com>
Co-authored-by: Max Inden <mail@max-inden.de
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Roman Proskuryakov <r.proskuryakoff@gmail.com>
The items in this roadmap are ordered by importance where importance is defined as:
- Enable others to build with rust-libp2p before building components ourself.
- e.g.[cross behaviour communication](#cross-behaviour-communication) over[Kademlia client
mode](#kademlia-client-mode) where the former enables the latter
- Invest into tools that enable us to work more efficiently
- e.g.[Testground](#testground-abstraction) and [release process](#release-process)
- Better do one thing well than many things close to right.
- e.g. deprioritize [BitSwap implementation](#bitswap-implementation) and continue on
[QUIC](#quic-support) even once first ieration is merged. That does not imply that we won't
support community members implementing it.
- Improve existing components before introducing new ones.
- e.g.[Kademlia client mode](#kademlia-client-mode) before [WebTransport](#webtransport)
