* [multistream-select] Fix panic with V1Lazy and add integration tests.
Fixes a panic when using the `V1Lazy` negotiation protocol,
a regression introduced in https://github.com/libp2p/rust-libp2p/pull/1484.
Thereby adds integration tests for a transport upgrade with both
`V1` and `V1Lazy` to the `multistream-select` crate to prevent
future regressions.
* Cleanup.
* Update changelog.
Also add warn if parsing of a packet fails, so problems with for example packet
sizes can more easily be detected.
Reasoning:
The previous 2048 bytes is already larger than the recommend size by
[rfc6762](https://tools.ietf.org/html/rfc6762#page-46), which recommends
a packet size that fits within the interface MTU.
Nevertheless the rfc also states, that if more data needs to be
transmitted one should rely on IP fragments, which would only pose
problems when interfacing with devices not understanding IP fragments.
The standard also imposes a strict upper limit of 9000 bytes.
Therefore raising `recv_buffer` to 4096 seems sensible, given that we
already exceed the recommend size and given that we are not restricting
the size when sending in any way. Meaning the implementation already
sends packets larger than recommended, just can't handle them on
reception.
* Update cuckoofilter requirement from 0.3.2 to 0.5.0
Updates the requirements on [cuckoofilter](https://github.com/axiomhq/rust-cuckoofilter) to permit the latest version.
- [Release notes](https://github.com/axiomhq/rust-cuckoofilter/releases)
- [Changelog](https://github.com/axiomhq/rust-cuckoofilter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axiomhq/rust-cuckoofilter/commits)
Signed-off-by: dependabot[bot] <support@github.com>
* protocols/floodsub: Warn when Cuckoofilter has to evict item
With Cuckoofilter `v0.4.0` `add` and `test_and_add` return an error when
an old element has to be evicted in favor of a new one to take its
place. The previous behavior was panicing.
With this commit the error is logged on `warn` level.
* *: Update changelogs and cargo tomls
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
* protocols/plaintext: Do not UVI encode goodput
Only prefix handshake messages with the message length in bytes as an
unsigned varint. Return a plain socket once handshaking succeeded.
* *: Bump minor version in changelogs and cargo tomls
* Restore `RequestResponse::throttled`.
In contrast to the existing "throttled" approach this PR adds back-
pressure to the protocol without requiring pre-existing knowledge
of all nodes about their limits. It adds small, CBOR-encoded headers
to the actual payload data. Extra credit messages communicate back
to the sender how many more requests it is allowed to send.
* Remove some noise.
* Resend credit grant after connection closed.
Should an error in some lower layer cause a connection to be closed,
our previously sent credit grant may not have reached the remote peer.
Therefore, pessimistically, a credit grant is resent whenever a
connection is closed. The remote ignores duplicate grants.
* Remove inbound/outbound tracking per peer.
* Send ACK as response to duplicate credit grants.
* Simplify.
* Fix grammar.
* Incorporate review feedback.
- Remove `ResponseSent` which was a leftover from previous attemps
and issue a credit grant immediately in `send_response`.
- Only resend credit grants after a connection is closed if we are
still connected to this peer.
* Move codec/header.rs to throttled/codec.rs.
* More review suggestions.
* Generalise `ProtocolWrapper` and use shorter prefix.
* Update protocols/request-response/src/lib.rs
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Update protocols/request-response/src/throttled.rs
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Update protocols/request-response/src/throttled.rs
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Minor comment changes.
* Limit max. header size to 8KiB
* Always construct initial limit with 1.
Since honest senders always assume a send budget of 1 and wait for
credit afterwards, setting the default limit to a higher value
can only become effective after informing the peer about it which
means leaving `max_recv` at 1 and setting `next_max` to the desired
value.
* Use LRU cache to keep previous peer infos.
Peers may send all their requests, reconnect and send again all their
requests, starting from a fresh budget. The LRU cache keeps the peer
information around and reuses it when the peer reconnects, continuing
with the previous remaining limit.
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Restore `RequestResponse::throttled`.
In contrast to the existing "throttled" approach this PR adds back-
pressure to the protocol without requiring pre-existing knowledge
of all nodes about their limits. It adds small, CBOR-encoded headers
to the actual payload data. Extra credit messages communicate back
to the sender how many more requests it is allowed to send.
* Remove some noise.
* Resend credit grant after connection closed.
Should an error in some lower layer cause a connection to be closed,
our previously sent credit grant may not have reached the remote peer.
Therefore, pessimistically, a credit grant is resent whenever a
connection is closed. The remote ignores duplicate grants.
* Remove inbound/outbound tracking per peer.
* Send ACK as response to duplicate credit grants.
* Simplify.
* Fix grammar.
* Incorporate review feedback.
- Remove `ResponseSent` which was a leftover from previous attemps
and issue a credit grant immediately in `send_response`.
- Only resend credit grants after a connection is closed if we are
still connected to this peer.
* Move codec/header.rs to throttled/codec.rs.
* More review suggestions.
* Generalise `ProtocolWrapper` and use shorter prefix.
* Update protocols/request-response/src/lib.rs
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Update protocols/request-response/src/throttled.rs
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Update protocols/request-response/src/throttled.rs
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Minor comment changes.
* Limit max. header size to 8KiB
* Always construct initial limit with 1.
Since honest senders always assume a send budget of 1 and wait for
credit afterwards, setting the default limit to a higher value
can only become effective after informing the peer about it which
means leaving `max_recv` at 1 and setting `next_max` to the desired
value.
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
In order to make use of the distances returned by `KBucketRef::range` in
a human readable format, one needs to be able to translate the 256 bit in
a `Distance` to a smaller space.
Compiling libp2p-kad for `--target wasm32-unknown-unknown` fails with
the cryptic error message `cannot infer type for type `usize``.
Explicitly converting to `usize` solves the issue.
Co-authored-by: Andronik Ordian <write@reusable.software>
* Store addresses of provider records.
So far, provider records are stored without their
addresses and the addresses of provider records are
obtained from the routing table on demand. This has
two shortcomings:
1. We can only return provider records whose provider
peers happen to currently be in the local routing table.
2. The local node never returns itself as a provider for
a key, even if it is indeed a provider.
These issues are addressed here by storing the addresses
together with the provider records, falling back to
addresses from the routing table only for backward-compatibility
with existing implementations of `RecordStore` using persistent
storage.
Resolves https://github.com/libp2p/rust-libp2p/issues/1526.
* Update protocols/kad/src/behaviour.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Remove negligible use of with_capacity.
* Update changelog.
Co-authored-by: Max Inden <mail@max-inden.de>
The codec impl did not check that it actually read any bytes in
`read_request` and `read_response`. The used `read_one` function
does not error on EOF either, so instead of signalling connection
loss the codec could produce empty `Ping` or `Pong` messages.
* Use a single exchange instead of two one_shots.
* Add `Throttled` to libp2p-request-response.
Wraps the existing `RequestResponse` behaviour and applies strict limits
to the number of inbound and outbound requests per peer.
The wrapper is opt-in and if not used, the protocol behaviour of
`RequestResponse` does not change. This PR also does not introduce
an extra protocol, hence the limits applied need to be known a priori
for all nodes which is not always possible or desirable. As mentioned
in #1687 I think that we should eventually augment the protocol with
metadata which allows a more dynamic exchange of requests and responses.
This PR also replaces the two oneshot channels with a single one from the
scambio crate which saves one allocation per request/response. If not
desirable because the crate has seen less testing the first commit could
be reverted.
* Fix rustdoc error.
* Remove some leftovers from development.
* Add docs to `NetworBehaviourAction::{map_in,map_out}`.
* Apply suggestions from code review
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Add `ping_protocol_throttled` test.
* Add another test.
* Revert "Use a single exchange instead of two one_shots."
This reverts commit e34e1297d411298f6c69e238aa6c96e0b795d989.
# Conflicts:
# protocols/request-response/Cargo.toml
# protocols/request-response/src/handler/protocol.rs
* Update CHANGELOG.
* Update CHANGELOG.
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Refactor the ping protocol.
Such that pings are sent over a single substream, as it is
done in other libp2p implementations. Note that, since each
peer sends its pings over a single, dedicated substream,
every peer that participates in the protocol has effectively
two open substreams.
* Cleanup
* Update ping changelog.
* Update protocols/ping/src/protocol.rs
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Max Inden <mail@max-inden.de>
With 826f513 a `StreamMuxer` can notify that the address of a remote
peer changed. This is needed to support the transport protocol QUIC as
remotes can change their IP addresses within the lifetime of a single
connection.
This commit implements the `NetworkBehaviour::inject_address_change`
handler to update the Kademlia routing table accordingly.
* Emit events for active connection close and fix `disconnect()`.
The `Network` does currently not emit events for actively
closed connections, e.g. via `EstablishedConnection::close`
or `ConnectedPeer::disconnect()`. As a result, when actively
closing connections, there will be `ConnectionEstablished`
events emitted without eventually a matching `ConnectionClosed`
event. This seems undesirable and has the consequence that
the `Swarm::ban_peer_id` feature in `libp2p-swarm` does not
result in appropriate calls to `NetworkBehaviour::inject_connection_closed`
and `NetworkBehaviour::inject_disconnected`. Furthermore,
the `disconnect()` functionality in `libp2p-core` is currently
broken as it leaves the `Pool` in an inconsistent state.
This commit does the following:
1. When connection background tasks are dropped
(i.e. removed from the `Manager`), they
always terminate immediately, without attempting
an orderly close of the connection.
2. An orderly close is sent to the background task
of a connection as a regular command. The
background task emits a `Closed` event
before terminating.
3. `Pool::disconnect()` removes all connection
tasks for the affected peer from the `Manager`,
i.e. without an orderly close, thereby also
fixing the discovered state inconsistency
due to not removing the corresponding entries
in the `Pool` itself after removing them from
the `Manager`.
4. A new test is added to `libp2p-swarm` that
exercises the ban/unban functionality and
places assertions on the number and order
of calls to the `NetworkBehaviour`. In that
context some new testing utilities have
been added to `libp2p-swarm`.
This addresses https://github.com/libp2p/rust-libp2p/issues/1584.
* Update swarm/src/lib.rs
Co-authored-by: Toralf Wittner <tw@dtex.org>
* Incorporate some review feedback.
* Adapt to changes in master.
* More verbose panic messages.
* Simplify
There is no need for a `StartClose` future.
* Fix doc links.
* Further small cleanup.
* Update CHANGELOGs and versions.
Co-authored-by: Toralf Wittner <tw@dtex.org>
This adds optional message signing and verification to the gossipsub protocol as
per the libp2p specifications.
In addition this commit:
- Removes the LruCache received cache and simply uses the memcache in it's
place.
- Send subscriptions to all peers
- Prevent invalid messages from being gossiped
- Send grafts when subscriptions are added to the mesh
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Rüdiger Klaehn <rklaehn@protonmail.com>
Co-authored-by: Rüdiger Klaehn <rklaehn@gmail.com>
* Add debug instances for MessageCache and GossipSub behaviour
* Manual impl of Debug for GossipsubMessage
* Add pretty printing of protocol_id in debug
* Use hex_fmt instead of hex
* Inline StringOrBytes helper struct
Since it is used only once here
* Limit data of gossipsub msg to 20 bytes
Otherwise they might become very large and useless for debugging
