Previously we would increase a counter / gauge / histogram on each received identify information. These metrics are missleading, as e.g. they depend on the identify interval and don't represent the set of currently connected peers.
With this commit, identify information is tracked for the currently connected peers only. Instead of an increase on each received identify information, metrics represent the status quo (Gauge).
Example:
```
\# HELP libp2p_libp2p_identify_remote_protocols Number of connected nodes supporting a specific protocol, with "unrecognized" for each peer supporting one or more unrecognized protocols...
\# TYPE libp2p_libp2p_identify_remote_protocols gauge
libp2p_libp2p_identify_remote_protocols_total{protocol="/ipfs/id/push/1.0.0"} 1
libp2p_libp2p_identify_remote_protocols_total{protocol="/ipfs/id/1.0.0"} 1
libp2p_libp2p_identify_remote_protocols_total{protocol="/ipfs/ping/1.0.0"} 1
libp2p_libp2p_identify_remote_protocols_total{protocol="unrecognized"} 1
\# HELP libp2p_libp2p_identify_remote_listen_addresses Number of connected nodes advertising a specific listen address...
\# TYPE libp2p_libp2p_identify_remote_listen_addresses gauge
libp2p_libp2p_identify_remote_listen_addresses_total{listen_address="/ip4/tcp"} 1
libp2p_libp2p_identify_remote_listen_addresses_total{listen_address="/ip4/udp/quic"} 1
\# HELP libp2p_libp2p_identify_local_observed_addresses Number of connected nodes observing the local node at a specific address...
\# TYPE libp2p_libp2p_identify_local_observed_addresses gauge
libp2p_libp2p_identify_local_observed_addresses_total{observed_address="/ip4/tcp"} 1
```
Pull-Request: #3325.
Previously, the associated types on `NetworkBehaviour` and `ConnectionHandler` carried generic names like `InEvent` and `OutEvent`. These names are _correct_ in that `OutEvent`s are passed out and `InEvent`s are passed in but they don't help users understand how these types are used.
In theory, a `ConnectionHandler` could be used separately from `NetworkBehaviour`s but that is highly unlikely. Thus, we rename these associated types to indicate, where the message is going to be sent to:
- `NetworkBehaviour::OutEvent` is renamed to `ToSwarm`: It describes the message(s) a `NetworkBehaviour` can emit to the `Swarm`. The user is going to receive those in `SwarmEvent::Behaviour`.
- `ConnectionHandler::InEvent` is renamed to `FromBehaviour`: It describes the message(s) a `ConnectionHandler` can receive from its behaviour via `ConnectionHandler::on_swarm_event`. The `NetworkBehaviour` can send it via the `ToSwarm::NotifyHandler` command.
- `ConnectionHandler::OutEvent` is renamed to `ToBehaviour`: It describes the message(s) a `ConnectionHandler` can send back to the behaviour via the now also renamed `ConnectionHandlerEvent::NotifyBehaviour` (previously `ConnectionHandlerEvent::Custom`)
Resolves: #2854.
Pull-Request: #3848.
This patch tackles two things at once that are fairly intertwined:
1. There is no such thing as a "substream" in libp2p, the spec and other implementations only talk about "streams". We fix this by deprecating `NegotiatedSubstream`.
2. Previously, `NegotiatedSubstream` was a type alias that pointed to a type from `multistream-select`, effectively leaking the version of `multistream-select` to all dependencies of `libp2p-swarm`. We fix this by introducing a `Stream` newtype.
Resolves: #3759.
Related: #3748.
Pull-Request: #3912.
With this patch, implementations of `ConnectionHandler` (which are typically composed in a tree) can exchange information about the supported protocols of a remote with each other via `ConnectionHandlerEvent::ReportRemoteProtocols`. The provided `ProtocolSupport` enum can describe either additions or removals of the remote peer's protocols.
This information is aggregated in the connection and passed down to the `ConnectionHandler` via `ConnectionEvent::RemoteProtocolsChange`.
Similarly, if the listen protocols of a connection change, all `ConnectionHandler`s on the connection will be notified via `ConnectionEvent::LocalProtocolsChange`. This will allow us to eventually remove `PollParameters` from `NetworkBehaviour`.
This pattern allows protocols on a connection to communicate with each other. For example, protocols like identify can share the list of (supposedly) supported protocols by the remote with all other handlers. A protocol like kademlia can accurately add and remove a remote from its routing table as a result.
Resolves: #2680.
Related: #3124.
Pull-Request: #3651.
The currently provided `ConnectionHandlerUpgrErr` is very hard to use. Not only does it have a long name, it also features 3 levels of nesting which results in a lot of boilerplate. Last but not least, it exposes `multistream-select` as a dependency to all protocols.
We fix all of the above by renaming the type to `StreamUpgradeError` and flattening out its interface. Unrecoverable errors during protocol selection are hidden within the `Io` variant.
Related: #3759.
Pull-Request: #3882.
This patch removes all deprecated legacy code from `libp2p-noise` and attempts to collapse all the abstraction layers into something a lot simpler.
Pull-Request: #3511.
When an inbound stream upgrade fails, there isn't a whole lot we can do about that in the handler. In fact, for several errors, we wouldn't even know which specific handler to target, for example, `NegotiationFailed`. Similiarly, in case of an IO error during the upgrade, we don't know which handler the stream was eventually meant to be for.
Pull-Request: #3605.
Previously, a protocol could be any sequence of bytes as long as it started with `/`. Now, we directly parse a protocol as `String` which enforces it to be valid UTF8.
To notify users of this change, we delete the `ProtocolName` trait. The new requirement is that users need to provide a type that implements `AsRef<str>`.
We also add a `StreamProtocol` newtype in `libp2p-swarm` which provides an easy way for users to ensure their protocol strings are compliant. The newtype enforces that protocol strings start with `/`. `StreamProtocol` also implements `AsRef<str>`, meaning users can directly use it in their upgrades.
`multistream-select` by itself only changes marginally with this patch. The only thing we enforce in the type-system is that protocols must implement `AsRef<str>`.
Resolves: #2831.
Pull-Request: #3746.
Previously, we would specify the version and path of our workspace dependencies in each of our crates. This is error prone as https://github.com/libp2p/rust-libp2p/pull/3658#discussion_r1153278072 for example shows. Problems like these happened in the past too.
There is no need for us to ever depend on a earlier version than the most current one in our crates. It thus makes sense that we manage this version in a single place.
Cargo supports a feature called "workspace inheritance" which allows us to share a dependency declaration across a workspace and inherit it with `{ workspace = true }`.
We do this for all our workspace dependencies and for the MSRV.
Resolves#3787.
Pull-Request: #3715.
Previously, we only honored the `support_floodsub` setting when the user did not specify a custom protocol for gossipsub. This patch fixes this and allows users to advertise floodsub even when they use a custom protocol.
Pull-Request: #3837.
- Move helpers to the bottom
- Restructure tests to follow arrange-act-assert
- Only set necessary options on builder for each test
Pull-Request: #3836.
The `unreachable_pub` lint makes us aware of uses of `pub` that are not actually reachable from the crate root. This is considered good because it means reading a `pub` somewhere means it is actually public API. Some of our crates are quite large and keeping their entire API surface in your head is difficult.
We should strive for most items being `pub(crate)`. This lint helps us enforce that.
Pull-Request: #3735.
Previously, we closed the entire connection upon receiving too many upgrade errors. This is unnecessarily aggressive. For example, an upgrade error may be caused by the remote dropping a stream during the initial handshake which is completely isolated from other protocols running on the same connection.
Instead of closing the connection, set `KeepAlive::No`.
Related: #3591.
Resolves: #3690.
Pull-Request: #3625.
This PR renames some method names that don't follow Rust naming conventions or behave differently from what the name suggests:
- Enforce "try" prefix on all methods that return `Result`.
- Enforce "encode" method name for methods that return encoded bytes.
- Enforce "to_bytes" method name for methods that return raw bytes.
- Enforce "decode" method name for methods that convert encoded key.
- Enforce "from_bytes" method name for methods that convert raw bytes.
Pull-Request: #3775.
With https://github.com/libp2p/rust-libp2p/pull/3658, these crates depend on the `0.42.1` release to access the new `ToSwarm` type. With the currently specified version, a user could theoretically run into a compile error if they pin `libp2p-swarm` to `0.42.0` in their lockfile but update to the latest patch release of one of these crates.
Pull-Request: #3711.
Doesn't change any functionality but `pop` returns an `Option` whereas `remove` will panic on out-of-bounds. I am more comfortable with `pop` and a pattern match. Also, usage of `continue` allows us to not use an `else`.
Pull-Request: #3734.
Previously, we only mutably borrowed the `last_seq_no` in the current scope but did not modify the underlying number. This is because `u64` is copy and calling `wrapping_add` consumes `self` so the compiler just copied it. We introduce a new-type instead that is not `Copy`.
Additionally, `wrapping_add` and initializing with a random u64 might actually warp the number and thus not give us sequential numbers as intended in #3551. To solve this, we initialize with the current unix timestamp in nanoseconds. This allows a node to publish 1000000 messages a second and still not reuse sequence numbers even after a restart / re-initialization of the configuration. This is also what the go implementation does.
Resolves#3714.
Co-authored-by: Thomas Eizinger <thomas@eizinger.io>
Pull-Request: #3716.
The check for fanout peer inclusion in `test_join` does not check anything since `new_peers` is always empty.
12b785e94e/protocols/gossipsub/src/behaviour/tests.rs (L611)
I assume the intention was to fill the `new_peers` with the fanout peers. In this MR I do just that.
Pull-Request: #3628.
In earlier iterations of the design for generic connection management, we removed the `ConnectionId::new` constructor because it would have allowed users to create `ConnectionId`s that are already taken, thus breaking invariants that `NetworkBehaviour`s rely on. Later, we incorporated the creation of `ConnectionId` in `DialOpts` which mitigates this risk altogether.
Thus, it is reasonably safe to introduce a public, non-deprecated constructor for `ConnectionId` that can be used for tests.
Related https://github.com/libp2p/rust-libp2p/pull/3327#issuecomment-1469870307.
Pull-Request: #3652.
This modifies the gossipsub implementation to use monotonically increasing sequence numbers for signed messages (as dictated by the specification). There is a discussion about this in #3453. This change will make rust-libp2p gossipsub align with the go-implementation when messages are signed.
Messages will however still use randomized sequence numbers when messages are unsigned for security reasons (as discussed in the issue linked).
This shouldn't change any user-level API, only the seqno behavior. It is fully backwards compatible.
Resolves#3453.
Pull-Request: #3551.
This patch-set introduces `libp2p-swarm-test`. It provides utilities for quick and safe bootstrapping of tests for `NetworkBehaviour`s. The main design features are:
- Everything has timeouts
- APIs don't get in your way
- Minimal boilerplate
Closes#2884.
Pull-Request: #2888.
Instead of relying on `protoc` and buildscripts, we generate the bindings using `pb-rs` and version them within our codebase. This makes for a better IDE integration, a faster build and an easier use of `rust-libp2p` because we don't force the `protoc` dependency onto them.
Resolves#3024.
Pull-Request: #3312.
A large release with lots of changes I am looking forward to. Sorry for the long release cadence.
Anything folks would like to see included that is not yet in `master`? As usual I would like to only block on bug fixes.
Pull-Request: #3491.
Prefixing a variable with an underscore (`_`) in Rust indicates that it is not used. Through refactorings, it can sometimes happen that we do end up using such a variable. In this case, the underscore should be removed.
Clippy can help us with this.
Pull-Request: #3484.
Previously, a `ConnectionHandler` was immediately requested from the `NetworkBehaviour` as soon as a new dial was initiated or a new incoming connection accepted.
With this patch, we delay the creation of the handler until the connection is actually established and fully upgraded, i.e authenticated and multiplexed.
As a consequence, `NetworkBehaviour::new_handler` is now deprecated in favor of a new set of callbacks:
- `NetworkBehaviour::handle_pending_inbound_connection`
- `NetworkBehaviour::handle_pending_outbound_connection`
- `NetworkBehaviour::handle_established_inbound_connection`
- `NetworkBehaviour::handle_established_outbound_connection`
All callbacks are fallible, allowing the `NetworkBehaviour` to abort the connection either immediately or after it is fully established. All callbacks also receive a `ConnectionId` parameter which uniquely identifies the connection. For example, in case a `NetworkBehaviour` issues a dial via `NetworkBehaviourAction::Dial`, it can unambiguously detect this dial in these lifecycle callbacks via the `ConnectionId`.
Finally, `NetworkBehaviour::handle_pending_outbound_connection` also replaces `NetworkBehaviour::addresses_of_peer` by allowing the behaviour to return more addresses to be used for the dial.
Resolves#2824.
Pull-Request: #3254.
We create the `ConnectionId` for the new connection as part of `DialOpts`. This allows `NetworkBehaviour`s to accurately track state regarding their own dial attempts.
This patch is the main enabler of https://github.com/libp2p/rust-libp2p/pull/3254. Removing the `handler` field will allow us to deprecate the `NetworkBehaviour::new_handler` function in favor of four new ones that give more control over the connection lifecycle.
Most of this is trivial, apart from the rename of the `clippy::derive_hash_xor_eq` lint to `clippy::derived_hash_with_manual_eq`.
Instead of allowing that lint, we manually implement `PartialEq` and add a comment why the difference between the `PartialEq` and `Hash` implementations are okay.
