This PR adds some bandwidth improvements to gossipsub.
After a bit of inspection on live networks a number of improvements have been
made that can help reduce unnecessary bandwidth on gossipsub networks. This PR
introduces the following:
- A 1:1 tracking of all in-flight IWANT requests. This not only ensures that all
IWANT requests are answered and peers penalized accordingly, but gossipsub
will no no longer create multiple IWANT requests for multiple peers.
Previously, gossipsub sampled the in-flight IWANT requests in order to
penalize peers for not responding with a high probability that we would detect
non-responsive nodes. Futher, it was possible to re-request IWANT messages
that are already being requested causing added duplication in messages and
wasted unnecessary IWANT control messages. This PR shifts this logic to only
request message ids that we are not currently requesting from peers.
- Triangle routing naturally gives rise to unnecessary duplicates. Consider a
mesh of 4 peers that are interconnected. Peer 1 sends a new message to 2,3,4.
2 propagates to 3,4 and 3 propagates to 2,4 and 4 propagates to 2,3. In this
case 3 has received the message 3 times. If we keep track of peers that send
us messages, when publishing or forwarding we no longer send to peers that
have sent us a duplicate, we can eliminate one of the sends in the scenario
above. This only occurs when message validation is async however. This PR adds
this logic to remove some elements of triangle-routing duplicates.
Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Diva M <divma@protonmail.com>
* core: Mark "unused" field with "_"
We need to keep this marker type to ensure that the type continues
to be required to be pinned.
* tranports/noise: Derive `Default` for `Config`
`false` is the default for `bool`, we can derive this.
* protocols/request-response: Remove unused fields
These are already included the `RequestResponseMessage::Request`
variant.
* *: Allow clippy's large-enum-variant lint
Tackling these suggestions would require performance measurement
which we don't want to do at this stage.
Co-authored-by: Max Inden <mail@max-inden.de>
Don't report events of a connection to the `NetworkBehaviour`, if connection has
been established while the remote peer was banned. Among other guarantees this
upholds that `NetworkBehaviour::inject_event` is never called without a previous
`NetworkBehaviour::inject_connection_established` for said connection.
Co-authored-by: Max Inden <mail@max-inden.de>
Enable advanced dialing requests both on `Swarm` and via
`NetworkBehaviourAction`. Users can now trigger a dial with a specific
set of addresses, optionally extended via
`NetworkBehaviour::addresses_of_peer`. In addition the whole process is
now modelled in a type safe way via the builder pattern.
Example of a `NetworkBehaviour` requesting a dial to a specific peer
with a set of addresses additionally extended through
`NetworkBehaviour::addresses_of_peer`:
```rust
NetworkBehaviourAction::Dial {
opts: DialOpts::peer_id(peer_id)
.condition(PeerCondition::Always)
.addresses(addresses)
.extend_addresses_through_behaviour()
.build(),
handler,
}
```
Example of a user requesting a dial to an unknown peer with a single
address via `Swarm`:
```rust
swarm1.dial(
DialOpts::unknown_peer_id()
.address(addr2.clone())
.build()
)
```
Changes needed to get libp2p to run via `wasm32-unknown-unknown` in the browser
(both main thread and inside web workers).
Replaces wasm-timer with futures-timer and instant.
Co-authored-by: Oliver Wangler <oliver@wngr.de>
Concurrently dial address candidates within a single dial attempt.
Main motivation for this feature is to increase success rate on hole punching
(see https://github.com/libp2p/rust-libp2p/issues/1896#issuecomment-885894496
for details). Though, as a nice side effect, as one would expect, it does
improve connection establishment time.
Cleanups and fixes done along the way:
- Merge `pool.rs` and `manager.rs`.
- Instead of manually implementing state machines in `task.rs` use
`async/await`.
- Fix bug where `NetworkBehaviour::inject_connection_closed` is called without a
previous `NetworkBehaviour::inject_connection_established` (see
https://github.com/libp2p/rust-libp2p/issues/2242).
- Return handler to behaviour on incoming connection limit error. Missed in
https://github.com/libp2p/rust-libp2p/issues/2242.
Previously, peers that did not support gossipsub were removed from the
connection-id mappings.
This can cause the connection_id mappings to go out of sync with those managed
by the swarm. This PR corrects this and adds an extra event that can inform the
user that a peer that does not support the protocol has connected. The user can
then optionally handle peers that don't support the protocol.
Require `NetworkBehaviourAction::{DialPeer,DialAddress}` to contain a
`ProtocolsHandler`. This allows a behaviour to attach custom state to its
handler. The behaviour would no longer need to track this state separately
during connection establishment, thus reducing state required in a behaviour.
E.g. in the case of `libp2p-kad` the behaviour can include a `GetRecord` request
in its handler, or e.g. in the case of `libp2p-request-response` the behaviour
can include the first request in the handler.
Return `ProtocolsHandler` on connection error and close. This allows a behaviour
to extract its custom state previously included in the handler on connection
failure and connection closing. E.g. in the case of `libp2p-kad` the behaviour
could extract the attached `GetRecord` from the handler of the failed connection
and then start another connection attempt with a new handler with the same
`GetRecord` or bubble up an error to the user.
Co-authored-by: Thomas Eizinger <thomas@eizinger.io>
With f2905c07f1246c3c3fdc1cde95f7e9c5c1c9b01a the secp256k1 feature is
disabled by default. Instead of enabling it in the dev-dependency,
simply use ed25519.
Unless restricted by orphan rules, implementing `From` is superior
because it implies `Into` but leaves the choice to the user, which
one to use. Especially for errors, `From` is convenient because that
is what `?` builds on.
Co-authored-by: Max Inden <mail@max-inden.de>
This will allow capturing variables in these closures so that we can
make these functions aware of the forkId (necessary for altair).
Co-authored-by: Max Inden <mail@max-inden.de>
Not all implementations of `NetworkBehaviour` need all callbacks.
We've have been adding new callbacks with default implementations
for a while now. There is no reason the initial ones cannot also
be defaulted, thus making it easier create new implementations.
Co-authored-by: Max Inden <mail@max-inden.de>
- Change `PublicKey::into_protobuf_encoding` to
`PublicKey::to_protobuf_encoding`.
- Change `PublicKey::into_peer_id` to `PublicKey::to_peer_id`.
- Change `PeerId::from_public_key(PublicKey)` to
`PeerId::from_public_key(&PublicKey)`.
- Add `From<&PublicKey> for PeerId`.
Co-authored-by: Max Inden <mail@max-inden.de>
Add `ExpandedSwarm::disconnect_peer_id` and
`NetworkBehaviourAction::CloseConnection` to close connections to a specific
peer via an `ExpandedSwarm` or `NetworkBehaviour`.
Co-authored-by: Max Inden <mail@max-inden.de>
Change `Stream` implementation of `ExpandedSwarm` to return all
`SwarmEvents` instead of only the `NetworkBehaviour`'s events.
Remove `ExpandedSwarm::next_event`. Users can use `<ExpandedSwarm as
StreamExt>::next` instead.
Remove `ExpandedSwarm::next`. Users can use `<ExpandedSwarm as
StreamExt>::filter_map` instead.
Remove `Deref` and `DerefMut` implementations previously dereferencing
to the `NetworkBehaviour` on `Swarm`. Instead one can access the
`NetworkBehaviour` via `Swarm::behaviour` and `Swarm::behaviour_mut`.
Methods on `Swarm` can now be accessed directly, e.g. via
`my_swarm.local_peer_id()`.
Reasoning: Accessing the `NetworkBehaviour` of a `Swarm` through `Deref`
and `DerefMut` instead of a method call is an unnecessary complication,
especially for newcomers. In addition, `Swarm` is not a smart-pointer
and should thus not make use of `Deref` and `DerefMut`, see documentation
from the standard library below.
> Deref should only be implemented for smart pointers to avoid
confusion.
https://doc.rust-lang.org/std/ops/trait.Deref.html
* Implement `/dnsaddr` support on `libp2p-dns`.
To that end, since resolving `/dnsaddr` addresses needs
"fully qualified" multiaddresses when dialing, i.e. those
that end with the `/p2p/...` protocol, we make sure that
dialing always uses such fully qualified addresses by
appending the `/p2p` protocol as necessary. As a side-effect,
this adds support for dialing peers via "fully qualified"
addresses, as an alternative to using a `PeerId` together
with a `Multiaddr` with or without the `/p2p` protocol.
* Adapt libp2p-relay.
* Update versions, changelogs and small cleanups.
