* Disambiguate calls to NetworkBehaviour::inject_event
There is a gnarly edge-case with the custom-derive where rustc
cannot disambiguate the call if:
- The NetworkBehaviourEventProcess trait is imported
- We nest NetworkBehaviours that use the custom-derive
* Update misc/core-derive/src/lib.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Fix build and add CHANGELOG
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* feat: allow sent messages seen as subscribed
minor feature to allow mimicing the behaviour expected by ipfs api tests.
* refactor: rename per review comments
* refactor: rename Floodsub::options to config
* chore: update changelog
* Update CHANGELOG.md
Co-Authored-By: Max Inden <mail@max-inden.de>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
A node receiving a `GetRecord` request first checks whether it has the
given record. If it does have the record it does not return closer
nodes.
A node that knows the record for the given key is likely within a
neighborhood of nodes that know the record as well. In addition the node
likely knows its neighboorhood well.
When querying for a key with a quorum of 1 the above behavior of only
returning the record but not any close peers is fine. Once one queries
with a higher quorum having a node respond with the record as well as
close nodes is likely going to speed up the query, given that the
returned peers probably know the record as well.
* Update CHANGELOG.
* Update CHANGELOG.md
Co-Authored-By: Max Inden <mail@max-inden.de>
* More updates.
* Update format.
Co-authored-by: Max Inden <mail@max-inden.de>
* Pass the error to inject_listener_closed method
If there is an error when the listener closes, found in the
`NetworkEvent::ListenerClosed` `reason` field, we would like to pass it
on to the `inject_listener_closed()` method so that implementors of this
method have access to it.
Add an error parameter to `inject_listener_closed`. Convert the
`reason` field from a `Result` to an `Option` and if there is an error
pass `Some(error)` at the method call site.
* Pass 'reason' as a Result
* Finish change
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* mplex: Check for error and shutdown.
Issues #1504 and #1523 reported panics caused by polling the sink of
`secio::LenPrefixCodec` after it had entered its terminal state, i.e.
after it had previously encountered an error or was closed. According
to the reports this happened only when using mplex as a stream
multiplexer. It seems that because mplex always stores and keeps the
`Waker` when polling, a wakeup of any of those wakers will resume the
polling even for those cases where the previous poll did not return
`Poll::Pending` but resolved to a value.
To prevent polling after the connection was closed or an error
happened we check for those conditions prior to every poll.
* Keep error when operations fail.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* [libp2p-swarm] Make the multiple connections per peer first-class.
This commit makes the notion of multiple connections per peer
first-class in the API of libp2p-swarm, introducing the new
callbacks `inject_connection_established` and
`inject_connection_closed`. The `endpoint` parameter from
`inject_connected` and `inject_disconnected` is removed,
since the first connection to open may not be the last
connection to close, i.e. it cannot be guaranteed,
as was previously the case, that the endpoints passed
to these callbacks match up.
* Have identify track all addresses.
So that identify requests can be answered with the correct
observed address of the connection on which the request
arrives.
* Cleanup
* Cleanup
* Improve the `Peer` state API.
* Remove connection ID from `SwarmEvent::Dialing`.
* Mark `DialPeerCondition` non-exhaustive.
* Re-encapsulate `NetworkConfig`.
To retain the possibility of not re-exposing all
network configuration choices, thereby providing
a more convenient API on the \`SwarmBuilder\`.
* Rework Swarm::dial API.
* Update CHANGELOG.
* Doc formatting tweaks.
* core/src: Remove poll_broadcast connection notification mechanism
The `Network::poll_broadcast` function has not proven to be useful. This
commit removes the mechanism all the way down to the connection manager.
With `poll_broadcast` gone there is no mechanism left to send commands
to pending connections. Thereby command buffering for pending
connections is not needed anymore and is thus removed in this commit as
well.
* core/src/connection/manager.rs: Remove warning comment
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Given that the order of `PeerId`s within the `GetProvidersOk.providers`
set is irrelevant but duplication is at best confusing this commit makes
use of a `HashSet` instead of a `Vec` to return unique `PeerId`s only.
* Add `protocols_handler::multi` module.
An implementation of `ProtocolsHandler` that contains multiple other
`ProtocolsHandler`s indexed by some key type.
* Randomise start position of handler polling.
* Address some review concerns.
* Add `IntoMultiHandler`.
* Check protocol names for uniqueness.
* Changes due to review.
- No more `Debug` bound for the key type and more generic log messages.
- Additional comments.
- Imports instead of fully-qualified use.
- Renamed `DuplicateProtoname` to `DuplicateProtonameError`.
* Replace `HashMap` with `Vec` in `Upgrades`.
* Review suggestion.
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* [libp2p-swarm] Correct returned connections from notify_all.
If at least one connection was not ready (i.e. pending), only
those (pending) connections would be returned and considered on the next
iteration, whereas those which were ready should also remain
in the list of connections to notify on retry of `notify_all`.
* Simplify.
It seems unnecessary to use "poll all" -> "send all" semantics,
i.e. attempting an "atomic" broadcast. Rather, events send via
`notify_all` can be delivered as soon as possible, simplifying
the code further.
* Add addresses field for closing listeners
Add an addresses field to the ListenersEvent and the ListenerClosed to
hold the addresses of a listener that has just closed. When we return a
ListenerClosed network event loop over the addresses and call
inject_expired_listen_address on each one.
Fixes: #1482
* Use Vec instead of SmallVec
In order to not expose a third party dependency in our API use a `Vec`
type for the addresses list instead of a `SmallVec`.
* Do not clone for ListenersEvent::Closed
We would like to avoid clones where possible for efficiency reasons.
When returning a `ListenersEvent::Closed` we are already consuming the
listener (by way of a pin projection). We can therefore use a consuming
iterator instead of cloning.
Use `drain(..).collect()` instead of clone to consume the addresses when
returning a `ListenersEvent::Closed`.
* Expire addresses before listener
The listener and its addresses technically expire at the same time, but
since here we have to pick an order, it makes more sense that the
addresses expire first.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* protocols/kad: Do not attempt to store expired record in record store
`Kademlia::record_received` calculates the expiration time of a record
before inserting it into the record store. Instead of inserting the
record into the record store in any case, with this patch the record is
only inserted if it is not expired. If the record is expired a
`KademliaHandlerIn::Reset` for the given (sub) stream is triggered.
This would serve as a tiny defense mechanism against an attacker trying
to fill a node's record store with expired records before the record
store's clean up procedure removes the records.
* protocols/kad: Send regular ack when record discarded due to expiration
With this commit the remote receives a
[`KademliaHandlerIn::PutRecordRes`] even in the case where the record is
discarded due to being expired. Given that the remote sent the local
node a [`KademliaHandlerEvent::PutRecord`] request, the remote perceives
the local node as one node among the k closest nodes to the target.
Returning a [`KademliaHandlerIn::Reset`] instead of an
[`KademliaHandlerIn::PutRecordRes`] to have the remote try another node
would only result in the remote node to contact an even more distant
node. In addition returning [`KademliaHandlerIn::PutRecordRes`] does not
reveal any internal information to a possibly malicious remote node.
* protocols/kad/src/behaviour: Use `now` and reword expiration comment
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* protocols/kad: Add test to reproduce right shift overflow panic
* protocols/kad: Fix right shift overflow panic in record_received
Within `Behaviour::record_received` the exponentially decreasing
expiration based on the distance to the target for a record is
calculated as following:
1. Calculate the amount of nodes between us and the record key beyond
the k replication constant as `n`.
2. Shift the configured record time-to-live `n` times to the right to
calculate an exponentially decreasing expiration.
The configured record time-to-live is a u64. If `n` is larger or equal
to 64 the right shift will lead to an overflow which panics in debug
mode.
This patch uses a checked right shift instead, defaulting to 0 (`now +
0`) for the expiration on overflow.
* protocols/kad: Put attribute below comment
* protocols/kad: Extract shifting logic and rework test
Extract right shift into isolated function and replace complex
regression test with small isolated one.
* protocols/kad/src/behaviour: Refactor exp_decr_expiration
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Fix regression w.r.t. reporting of dial errors.
PR [1440] introduced a regression w.r.t. the reporting of
dial errors. In particular, if a connection attempt fails
due to an invalid remote peer ID, any remaining addresses
for the same peer would not be tried (intentional) but
the dial failure would not be reported to the behaviour,
causing e.g. libp2p-kad queries to potentially stall.
In hindsight, I figured it is better to preserve the
previous behaviour to still try alternative addresses
of the peer even on invalid peer ID errors on an earlier
address. In particular because in the context of libp2p-kad
it is not uncommon for peers to report localhost addresses
while the local node actually has e.g. an ipfs node running
on that address, obviously with a different peer ID, which
is the scenario causing frequent invalid peer ID (mismatch)
errors when running the ipfs-kad example.
This commit thus restores the previous behaviour w.r.t.
trying all remaining addresses on invalid peer ID errors
as well as making sure `inject_dial_error` is always
called when the last attempt failed.
[1440]: https://github.com/libp2p/rust-libp2p/pull/1440.
* Remove an fmt::Debug requirement.
* Allow multiple connections per peer in libp2p-core.
Instead of trying to enforce a single connection per peer,
which involves quite a bit of additional complexity e.g.
to prioritise simultaneously opened connections and can
have other undesirable consequences [1], we now
make multiple connections per peer a feature.
The gist of these changes is as follows:
The concept of a "node" with an implicit 1-1 correspondence
to a connection has been replaced with the "first-class"
concept of a "connection". The code from `src/nodes` has moved
(with varying degrees of modification) to `src/connection`.
A `HandledNode` has become a `Connection`, a `NodeHandler` a
`ConnectionHandler`, the `CollectionStream` was the basis for
the new `connection::Pool`, and so forth.
Conceptually, a `Network` contains a `connection::Pool` which
in turn internally employs the `connection::Manager` for
handling the background `connection::manager::Task`s, one
per connection, as before. These are all considered implementation
details. On the public API, `Peer`s are managed as before through
the `Network`, except now the API has changed with the shift of focus
to (potentially multiple) connections per peer. The `NetworkEvent`s have
accordingly also undergone changes.
The Swarm APIs remain largely unchanged, except for the fact that
`inject_replaced` is no longer called. It may now practically happen
that multiple `ProtocolsHandler`s are associated with a single
`NetworkBehaviour`, one per connection. If implementations of
`NetworkBehaviour` rely somehow on communicating with exactly
one `ProtocolsHandler`, this may cause issues, but it is unlikely.
[1]: https://github.com/paritytech/substrate/issues/4272
* Fix intra-rustdoc links.
* Update core/src/connection/pool.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Address some review feedback and fix doc links.
* Allow responses to be sent on the same connection.
* Remove unnecessary remainders of inject_replaced.
* Update swarm/src/behaviour.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update swarm/src/lib.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update core/src/connection/manager.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update core/src/connection/manager.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update core/src/connection/pool.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Incorporate more review feedback.
* Move module declaration below imports.
* Update core/src/connection/manager.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Update core/src/connection/manager.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Simplify as per review.
* Fix rustoc link.
* Add try_notify_handler and simplify.
* Relocate DialingConnection and DialingAttempt.
For better visibility constraints.
* Small cleanup.
* Small cleanup. More robust EstablishedConnectionIter.
* Clarify semantics of `DialingPeer::connect`.
* Don't call inject_disconnected on InvalidPeerId.
To preserve the previous behavior and ensure calls to
`inject_disconnected` are always paired with calls to
`inject_connected`.
* Provide public ConnectionId constructor.
Mainly needed for testing purposes, e.g. in substrate.
* Move the established connection limit check to the right place.
* Clean up connection error handling.
Separate connection errors into those occuring during
connection setup or upon rejecting a newly established
connection (the `PendingConnectionError`) and those
errors occurring on previously established connections,
i.e. for which a `ConnectionEstablished` event has
been emitted by the connection pool earlier.
* Revert change in log level and clarify an invariant.
* Remove inject_replaced entirely.
* Allow notifying all connection handlers.
Thereby simplify by introducing a new enum `NotifyHandler`,
used with a single constructor `NetworkBehaviourAction::NotifyHandler`.
* Finishing touches.
Small API simplifications and code deduplication.
Some more useful debug logging.
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
The API contract of Yamux is that the connection has to be polled
until `None` or some `Err(_)` is returned. Therefore while closing
we need to keep polling the connection. Only polling the control
channel will not have an effect but `close` will always return
`Poll::Pending` as the connection does not make progress.
The `QueryId` type should be exported as it is used in the
`NetworkBehaviour::ProtocolsHandler` type of `Kademlia`.
`Kademlia::protocol_name` is added for convenience.
