We refactor our continuous integration workflow with the following goals in mind:
- Run as few jobs as possible
- Have the jobs finish as fast as possible
- Have the jobs redo as little work as possible
There are only so many jobs that GitHub Actions will run in parallel.
Thus, it makes sense to not create massive matrices but instead group
things together meaningfully.
The new `test` job will:
- Run once for each crate
- Ensure that the crate compiles on its specified MSRV
- Ensure that the tests pass
- Ensure that there are no semver violations
This is an improvement to before because we are running all of these
in parallel which speeds up execution and highlights more errors at
once. Previously, tests run later in the pipeline would not get run
at all until you make sure the "first" one passes.
We also previously did not verify the MSRV of each crate, making the
setting in the `Cargo.toml` rather pointless.
The new `cross` job supersedes the existing `wasm` job.
This is an improvement because we now also compile the crate for
windows and MacOS. Something that wasn't checked before.
We assume that checking MSRV and the tests under Linux is good enough.
Hence, this job only checks for compile-errors.
The new `feature_matrix` ensures we compile correctly with certain feature combinations.
`libp2p` exposes a fair few feature-flags. Some of the combinations
are worth checking independently. For the moment, this concerns only
the executor related transports together with the executor flags but
this list can easily be extended.
The new `clippy` job runs for `stable` and `beta` rust.
Clippy gets continuously extended with new lints. Up until now, we would only
learn about those as soon as a new version of Rust is released and CI would
run the new lints. This leads to unrelated failures in CI. Running clippy on with `beta`
Rust gives us a heads-up of 6 weeks before these lints land on stable.
Fixes#2951.
Previously, the executor for connection tasks silently defaulted to a `futures::executor::ThreadPool`. This causes issues such as https://github.com/libp2p/rust-libp2p/issues/2230.
With this patch, we force the user to choose, which executor they want to run the connection tasks on which results in overall simpler API with less footguns.
Closes#3068.
Previously, the `DummyConnectionHandler` offered a "keep alive" functionality,
i.e. it allowed users to set the value of what is returned from
`ConnectionHandler::keep_alive`. This handler is primarily used in tests or
`NetworkBehaviour`s that don't open any connections (like mDNS). In all of these
cases, it is statically known whether we want to keep connections alive. As
such, this functionality is better represented by a static
`KeepAliveConnectionHandler` that always returns `KeepAlive::Yes` and a
`DummyConnectionHandler` that always returns `KeepAlive::No`.
To follow the naming conventions described in
https://github.com/libp2p/rust-libp2p/issues/2217, we introduce a top-level
`keep_alive` and `dummy` behaviour in `libp2p-swarm` that contains both the
`NetworkBehaviour` and `ConnectionHandler` implementation for either case.
Remove default features. You need to enable required features
explicitly now. As a quick workaround, you may want to use the
new `full` feature which activates all features.
This commit removes the `Network` abstraction, thus managing `Listeners`
and the connection `Pool` in `Swarm` directly. This is done under the
assumption that noone uses the `Network` abstraction directly, but
instead everyone always uses it through `Swarm`. Both `Listeners` and
`Pool` are moved from `libp2p-core` into `libp2p-swarm`. Given that they
are no longer exposed via `Network`, they can be treated as an
implementation detail of `libp2p-swarm` and `Swarm`.
This change does not include any behavioural changes.
This change has the followin benefits:
- Removal of `NetworkEvent`, which was mostly an isomorphism of
`SwarmEvent`.
- Removal of the never-directly-used `Network` abstraction.
- Removal of now obsolete verbose `Peer` (`core/src/network/peer.rs`)
construct.
- Removal of `libp2p-core` `DialOpts`, which is a direct mapping of
`libp2p-swarm` `DialOpts`.
- Allowing breaking changes to the connection handling and `Swarm` API
interface without a breaking change in `libp2p-core` and thus a
without a breaking change in `/transport` protocols.
This change enables the following potential future changes:
- Removal of `NodeHandler` and `ConnectionHandler`. Thus allowing to
rename `ProtocolsHandler` into `ConnectionHandler`.
- Moving `NetworkBehaviour` and `ProtocolsHandler` into `libp2p-core`,
having `libp2p-xxx` protocol crates only depend on `libp2p-core` and
thus allowing general breaking changes to `Swarm` without breaking all
`libp2p-xxx` crates.
Enable a `NetworkBehaviour` or a user via `Swarm::dial` to override the
dial concurrency factor per dial. This is especially relevant in the
case of libp2p-autonat where one wants to probe addresses in sequence to
reduce the amount of work a remote peer can force onto the local node.
To enable the above, this commit also:
- Introduces `libp2p_core::DialOpts` mirroring `libp2p_swarm::DialOpts`.
Passed as an argument to `Network::dial`.
- Removes `Peer::dial` in favor of `Network::dial`.
- Simplifies `Swarm::dial_with_handler`.
The introduction of `libp2p_core::DialOpts` will be useful beyond this
feature, e.g. for https://github.com/libp2p/rust-libp2p/pull/2363.
In the long run I would like to move and merge `libp2p_core::Network`
and `libp2p_core::Pool` into `libp2p_swarm::Swarm` thus deduplicating
`libp2p_core::DialOpts` and `libp2p_swarm::DialOpts`.
Fixes#2385.
Concurrently dial address candidates within a single dial attempt.
Main motivation for this feature is to increase success rate on hole punching
(see https://github.com/libp2p/rust-libp2p/issues/1896#issuecomment-885894496
for details). Though, as a nice side effect, as one would expect, it does
improve connection establishment time.
Cleanups and fixes done along the way:
- Merge `pool.rs` and `manager.rs`.
- Instead of manually implementing state machines in `task.rs` use
`async/await`.
- Fix bug where `NetworkBehaviour::inject_connection_closed` is called without a
previous `NetworkBehaviour::inject_connection_established` (see
https://github.com/libp2p/rust-libp2p/issues/2242).
- Return handler to behaviour on incoming connection limit error. Missed in
https://github.com/libp2p/rust-libp2p/issues/2242.
Unless restricted by orphan rules, implementing `From` is superior
because it implies `Into` but leaves the choice to the user, which
one to use. Especially for errors, `From` is convenient because that
is what `?` builds on.
Co-authored-by: Max Inden <mail@max-inden.de>
- Change `PublicKey::into_protobuf_encoding` to
`PublicKey::to_protobuf_encoding`.
- Change `PublicKey::into_peer_id` to `PublicKey::to_peer_id`.
- Change `PeerId::from_public_key(PublicKey)` to
`PeerId::from_public_key(&PublicKey)`.
- Add `From<&PublicKey> for PeerId`.
Co-authored-by: Max Inden <mail@max-inden.de>
* Implement `/dnsaddr` support on `libp2p-dns`.
To that end, since resolving `/dnsaddr` addresses needs
"fully qualified" multiaddresses when dialing, i.e. those
that end with the `/p2p/...` protocol, we make sure that
dialing always uses such fully qualified addresses by
appending the `/p2p` protocol as necessary. As a side-effect,
this adds support for dialing peers via "fully qualified"
addresses, as an alternative to using a `PeerId` together
with a `Multiaddr` with or without the `/p2p` protocol.
* Adapt libp2p-relay.
* Update versions, changelogs and small cleanups.
Move transport upgrade protocols from `protocols/`
to `transports/`, such that only "application protocols"
that depend on `libp2p-swarm` remain in `protocols/`,
whereas there is no such dependency in `transports/`
outside of integration tests.
Tweak README and top-level CHANGELOG.
`futures-codec` has not been updated in the recent months. It still
depends on `bytes` `v0.5` preventing all downstream dependencies to
upgrade to `bytes` `v1.0`.
This commit replaces `futures_codec` in favor of `asynchronous-codec`
The latter is a fully upgraded fork of the former.
In addition this commit upgrades:
- bytes to v1
- unsigned-varint to v0.6.0
- prost to v0.7
* [multistream-select] Listener conformity for failed negotiations.
When `V1Lazy` is used and the listener does not support the
optimistic (and singular) proposal of the dialer, it currently
happens that dialer and listener get a different outcome of
the negotiation. The dialer eventually detects the failed
negotiation as soon as it tries to read from the stream, but
the listener either encounters an invalid message or unexpected
premature EOF, depending on the payload that the dialer sent
prematurely after its protocol proposal. In these cases the
listener must be lenient and fail the negotiation "normally",
i.e. not with a protocol violation or an I/O error.
* Update misc/multistream-select/src/tests.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Refine error handling.
Only be lenient with garbage or sudden EOF when reading
just after having sent a protocol rejection.
* Update misc/multistream-select/src/listener_select.rs
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Max Inden <mail@max-inden.de>
* Make the lazy variant interoperable.
The remaining optimisation for `V1Lazy` for a listener
in the negotiation, whereby the listener delays flushing
of the multistream version header, is hereby removed.
The remaining effect of `V1Lazy` is only on the side of
the dialer, which delays flushing of its singular
protocol proposal in order to send it together with
the first application data (or an attempt is made to
read from the negotiated stream, which similarly
triggers a flush of the protocol proposal). This
permits `V1Lazy` dialers to be interoperable with
`V1` listeners. The remaining theoretical pitfall whereby
application data gets misinterpreted as another protocol
proposal by a listener remains, however unlikely.
`V1` remains the default, but we may eventually risk
just making this lazy dialer flush a part of the default
`V1` implementation, removing the dedicated `V1Lazy`
version identifier.
* Update CHANGELOG
* Separate versions from mere header lines.
Every multistream-select version maps to a specific header line,
but there may be different variants of the same multistream-select
version using the same header line, i.e. the same wire protocol.
* Cleanup
* Update misc/multistream-select/CHANGELOG.md
* Fix ls response encoding/decoding.
Thereby remove the now unnecessary arbitrary protocol name
length limit. Since it an 'ls' response is always terminated
with a dedicated newline (and thus ends with two newlines),
an 'ls' response with a single protocol can be disambiguated
from a single protocol response by this additional newline.
* More commentary
* Update versions and changelogs.
* Resolve remaining conflict.
* Permit empty ls responses, as before.
Treat EOF error as [`NegotiationError::Failed`], not as
[`NegotiationError::ProtocolError`], allowing dropping or closing an I/O stream
as a permissible way to "gracefully" fail a negotiation.
This is e.g. important when a listener rejects a protocol with
[`Message::NotAvailable`] and the dialer does not have alternative protocols to
propose. Then the dialer will stop the negotiation and drop the corresponding
stream. As a listener this EOF should be interpreted as a failed negotiation.
* [multistream-select] Temp. disable "parallel" negotiation.
In order to later change the "ls" responses for spec-compliance.
* Update version.
* Update misc/multistream-select/CHANGELOG.md
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Max Inden <mail@max-inden.de>
