This PR adds some bandwidth improvements to gossipsub.
After a bit of inspection on live networks a number of improvements have been
made that can help reduce unnecessary bandwidth on gossipsub networks. This PR
introduces the following:
- A 1:1 tracking of all in-flight IWANT requests. This not only ensures that all
IWANT requests are answered and peers penalized accordingly, but gossipsub
will no no longer create multiple IWANT requests for multiple peers.
Previously, gossipsub sampled the in-flight IWANT requests in order to
penalize peers for not responding with a high probability that we would detect
non-responsive nodes. Futher, it was possible to re-request IWANT messages
that are already being requested causing added duplication in messages and
wasted unnecessary IWANT control messages. This PR shifts this logic to only
request message ids that we are not currently requesting from peers.
- Triangle routing naturally gives rise to unnecessary duplicates. Consider a
mesh of 4 peers that are interconnected. Peer 1 sends a new message to 2,3,4.
2 propagates to 3,4 and 3 propagates to 2,4 and 4 propagates to 2,3. In this
case 3 has received the message 3 times. If we keep track of peers that send
us messages, when publishing or forwarding we no longer send to peers that
have sent us a duplicate, we can eliminate one of the sends in the scenario
above. This only occurs when message validation is async however. This PR adds
this logic to remove some elements of triangle-routing duplicates.
Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Diva M <divma@protonmail.com>
Add convenience function `is_relayed` to `ConnectedPoint`, determining
whether a connection is relayed based on whether it contains a
`Protocol::P2pCircuit`.
Implement `NetworkBehaviour` on `either::Either<L, R>` where both L
and R both implement `NetworkBehaviour`.
Add NetworkBehaviour derive tests for Either and Toggle
Co-authored-by: Max Inden <mail@max-inden.de>
* Adapt examples to async style loop
* Adapt async style loop for chat.rs
* Adapt async style loop for distributed-key-value-store.rs
* Adapt async style loop for gossibsub-chat.rs
* Adapt async style loop for ipfs-private.rs
* Adapt ping to use async
* Update tutorial crate to reflect new changes
Co-authored-by: Max Inden <mail@max-inden.de>
Propagate messages only to the target peers and not all connected peers.
Co-authored-by: Victor Ermolaev <victorermolaev@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
* core: Mark "unused" field with "_"
We need to keep this marker type to ensure that the type continues
to be required to be pinned.
* tranports/noise: Derive `Default` for `Config`
`false` is the default for `bool`, we can derive this.
* protocols/request-response: Remove unused fields
These are already included the `RequestResponseMessage::Request`
variant.
* *: Allow clippy's large-enum-variant lint
Tackling these suggestions would require performance measurement
which we don't want to do at this stage.
Co-authored-by: Max Inden <mail@max-inden.de>
`prost-build` is failing due to a missing `protoc` binary. Neither the OS
supplies one, nor can the bundled binaries be used. This commit downgrades the
OS used. The older OS is compatible with the bundled `protoc` binaries.
Don't report events of a connection to the `NetworkBehaviour`, if connection has
been established while the remote peer was banned. Among other guarantees this
upholds that `NetworkBehaviour::inject_event` is never called without a previous
`NetworkBehaviour::inject_connection_established` for said connection.
Co-authored-by: Max Inden <mail@max-inden.de>
With https://github.com/libp2p/specs/pull/368 the definition of the _peer name_
changed in the mDNS specification.
> peer-name is the case-insensitive unique identifier of the peer, and is less
> than 64 characters.
>
> As the this field doesn't carry any meaning, it is sufficient to ensure the
> uniqueness of this identifier. Peers SHOULD generate a random, lower-case
> alphanumeric string of least 32 characters in length when booting up their
> node. Peers SHOULD NOT use their Peer ID here because a future Peer ID could
> exceed the DNS label limit of 63 characters.
https://github.com/libp2p/specs/blob/master/discovery/mdns.md
This commit adjusts `libp2p-mdns` accordingly.
Also see https://github.com/libp2p/go-libp2p/pull/1222 for the corresponding
change on the Golang side.
Co-authored-by: Max Inden <mail@max-inden.de>
`NetworkBehaviour::inject_dial_failure` expects a reference for the error, thus
the error should not be cloned when passing it to the inner behaviours in the
`NetworkBehaviour` derivation.
Fixes Issue #2348.
Check multiaddr has valid peer id component or none at all. We don't want to
cache a multiaddr with a purposely wrong multiaddr. e.g. something that ends
with .../p2p/some-other-peer. While this should fail to dial because we [check
this before
dialing](https://github.com/libp2p/rust-libp2p/blob/master/core/src/connection/pool/concurrent_dial.rs#L144),
it's better to not cache this in the first place.
Enable advanced dialing requests both on `Swarm` and via
`NetworkBehaviourAction`. Users can now trigger a dial with a specific
set of addresses, optionally extended via
`NetworkBehaviour::addresses_of_peer`. In addition the whole process is
now modelled in a type safe way via the builder pattern.
Example of a `NetworkBehaviour` requesting a dial to a specific peer
with a set of addresses additionally extended through
`NetworkBehaviour::addresses_of_peer`:
```rust
NetworkBehaviourAction::Dial {
opts: DialOpts::peer_id(peer_id)
.condition(PeerCondition::Always)
.addresses(addresses)
.extend_addresses_through_behaviour()
.build(),
handler,
}
```
Example of a user requesting a dial to an unknown peer with a single
address via `Swarm`:
```rust
swarm1.dial(
DialOpts::unknown_peer_id()
.address(addr2.clone())
.build()
)
```
Rename `KademliaEvent::InboundRequestServed` to `KademliaEvent::InboundRequest` and move
`InboundPutRecordRequest` into `InboundRequest::PutRecord` and `InboundAddProviderRequest` into
`InboundRequest::AddProvider`.
Co-authored-by: supercmmetry <vishaals2000@gmail.com>
Changes needed to get libp2p to run via `wasm32-unknown-unknown` in the browser
(both main thread and inside web workers).
Replaces wasm-timer with futures-timer and instant.
Co-authored-by: Oliver Wangler <oliver@wngr.de>
Pass websocket CLOSE reason to the app as an
`Incoming::Closed(soketto::CloseReason)`.
This PR is a companion to https://github.com/paritytech/soketto/pull/31 and lets
applications know what the remote end claimed to be the reason for closing the
connection (if any).
`IncomingData` is now renamed to `Incoming` and the actual data (text or binary)
is moved into its own `Data` enum, which in turn allows `into_bytes()` and the
`AsRef` impl to apply more sanely to something that is actually data.
Co-authored-by: David Palm <dvdplm@gmail.com>
With https://github.com/libp2p/rust-libp2p/pull/2248 a connection task
`await`s sending an event to the behaviour before polling for new events
from the behaviour [1].
When `Swarm::poll` is unable to deliver an event to a connection task it
returns `Poll::Pending` even though (a) polling `Swarm::network` might be
able to make progress (`network_not_ready` being `false`) and (b) it
does not register a waker to be woken up [2].
In combination this can lead to a deadlock where a connection task waits
to send an event to the behaviour and `Swarm::poll` returns
`Poll::Pending` failing to send an event to the connection task, not
registering a waiker in order to be polled again.
With this commit `Swarm::poll` will only return `Poll::Pending`, when
failing to deliver an event to a connection task, if the network is
unable to make progress (i.e. `network_not_ready` being `true`).
In the long-run `Swarm::poll` should likely be redesigned, prioritizing
the behaviour over the network, given the former is the control plane
and the latter potentially yields new work from the outside.
[1]: ca1b7cf043/core/src/connection/pool/task.rs (L224-L232)
[2]: ca1b7cf043/swarm/src/lib.rs (L756-L783)
The test `max_established_incoming` starts two networks with a
configured connection limit, spawns up to `limit + 1` connections and
expects the last connection to be closed due to being over the limit.
The previous test implementation depended on both networks to handle
each connection in sequence, which is not always the case.
E.g. while network 2 might expect the last connection to close, network
1 might finish upgrading the last connection before the second to last
connection, thus expecting the second to last connection to close.
This commit drives network 1 and 2 in sequence and ensures both networks
are finished upgrading a connection before starting a new connection. In
addition it upgrade the test to use `quickcheck`.
