Previously, a protocol could be any sequence of bytes as long as it started with `/`. Now, we directly parse a protocol as `String` which enforces it to be valid UTF8.
To notify users of this change, we delete the `ProtocolName` trait. The new requirement is that users need to provide a type that implements `AsRef<str>`.
We also add a `StreamProtocol` newtype in `libp2p-swarm` which provides an easy way for users to ensure their protocol strings are compliant. The newtype enforces that protocol strings start with `/`. `StreamProtocol` also implements `AsRef<str>`, meaning users can directly use it in their upgrades.
`multistream-select` by itself only changes marginally with this patch. The only thing we enforce in the type-system is that protocols must implement `AsRef<str>`.
Resolves: #2831.
Pull-Request: #3746.
The `unreachable_pub` lint makes us aware of uses of `pub` that are not actually reachable from the crate root. This is considered good because it means reading a `pub` somewhere means it is actually public API. Some of our crates are quite large and keeping their entire API surface in your head is difficult.
We should strive for most items being `pub(crate)`. This lint helps us enforce that.
Pull-Request: #3735.
Most of this is trivial, apart from the rename of the `clippy::derive_hash_xor_eq` lint to `clippy::derived_hash_with_manual_eq`.
Instead of allowing that lint, we manually implement `PartialEq` and add a comment why the difference between the `PartialEq` and `Hash` implementations are okay.
As I do frequently, I corrected for the latest clippy warnings. This will make sure the CI won't complain in the future. We could automate this btw and maybe run the nightly version of clippy.
Instead of having a mix of `poll_event`, `poll_outbound` and `poll_close`, we
flatten the entire interface of `StreamMuxer` into 4 individual functions:
- `poll_inbound`
- `poll_outbound`
- `poll_address_change`
- `poll_close`
This design is closer to the design of other async traits like `AsyncRead` and
`AsyncWrite`. It also allows us to delete the `StreamMuxerEvent`.
This aligns the public API of the `libp2p-mplex` module with the one
from `libp2p-yamux`. This change has two benefits:
1. For standalone users of `libp2p-mplex`, the substreams itself are
now useful, similar to `libp2p-yamux` and don't necessarily need to
be polled via the `StreamMuxer`. The `StreamMuxer` only forwards to
the `Async{Read,Write}` implementations.
2. This will reduce the diff of #2648 because we can chunk the one
giant commit into smaller atomic ones.
`libp2p-core` provides the `StreamMuxer` abstraction so it can provide
functionality that abstracts over this trait.
We never use the `flush_all` function as part of our abstractions.
No one else is going to use it so we can remove it from the abstraction.
Implement the libp2p rendezvous protocol.
> A lightweight mechanism for generalized peer discovery. It can be used for
bootstrap purposes, real time peer discovery, application specific routing, and
so on.
Co-authored-by: rishflab <rishflab@hotmail.com>
Co-authored-by: Daniel Karzel <daniel@comit.network>
* Fix needless question mark operator
* Don't convert from u64 to u64
LocalStreamId is already a u64, no need to convert.
* Don't use `.into()` to convert to the same type
* Don't specify lifetime if it can be inferred
* Use `vec!` macro if we immediately push to it
This creates the vector with the appropriate capacity.
* Don't index array when taking a reference is enough
Co-authored-by: Max Inden <mail@max-inden.de>
* Implement `/dnsaddr` support on `libp2p-dns`.
To that end, since resolving `/dnsaddr` addresses needs
"fully qualified" multiaddresses when dialing, i.e. those
that end with the `/p2p/...` protocol, we make sure that
dialing always uses such fully qualified addresses by
appending the `/p2p` protocol as necessary. As a side-effect,
this adds support for dialing peers via "fully qualified"
addresses, as an alternative to using a `PeerId` together
with a `Multiaddr` with or without the `/p2p` protocol.
* Adapt libp2p-relay.
* Update versions, changelogs and small cleanups.
* Make clippy "happy".
Address all clippy complaints that are not purely stylistic (or even
have corner cases with false positives). Ignore all "style" and "pedantic" lints.
* Fix tests.
* Undo unnecessary API change.
`futures-codec` has not been updated in the recent months. It still
depends on `bytes` `v0.5` preventing all downstream dependencies to
upgrade to `bytes` `v1.0`.
This commit replaces `futures_codec` in favor of `asynchronous-codec`
The latter is a fully upgraded fork of the former.
In addition this commit upgrades:
- bytes to v1
- unsigned-varint to v0.6.0
- prost to v0.7
* [mplex] Benchmark different split_send_size configurations.
With both TCP and memory transports. As a result, change the
default `split_send_size` to 8KiB.
* Cleanup
* [mplex] Update CHANGELOG.
* Add throughput information.
* Streamline mplex and yamux configurations.
* For all configuration options that exist for both multiplexers
and have the same semantics, use the same names for the
configuration.
* Rename `Config` to `YamuxConfig` for consistentcy with
the majority of other protocols, e.g. `MplexConfig`, `PingConfig`,
`KademliaConfig`, etc.
* Completely hide `yamux` APIs within `libp2p-yamux`. This allows
to fully control the libp2p API and streamline it with other
muxer APIs, consciously choosing e.g. which configuration options
to make configurable in libp2p and which to fix to certain values.
It does also not necessarily prescribe new incompatible version bumps of
yamux for `libp2p-yamux`, as no `yamux` types are exposed. The cost
is some more duplication of configuration options in the API, as well
as the need to update `libp2p-yamux` if `yamux` introduces new
configuration options that `libp2p-yamux` wants to expose as well.
* Update CHANGELOGs.
* More granular execution of pending flushes.
Also replace fnv hashing with nohash-hasher.
* Don't forget the pending case.
* Simplify.
* Use AtomicU32 for connection IDs.
* Revert to random u64.
* Split the receive buffer per substream.
This split allows more efficient reading from the buffer
for a particular substream and to reset only the
offending substream if it reaches its buffer limit
with `MaxBufferBehaviour::ResetStream`. Previously
this was implemented as `MaxBufferBehaviour::CloseAll`
and resulted in the entire connection closing.
The buffer split should be advantageous whenever
not all substreams are read at the same pace and
some temporarily fall behind in consuming inbound
data frames.
* Tweak logging.
* Oops.
* Update muxers/mplex/src/io.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Rename field as per review suggestion.
* Adjust and clarify max-buffer-behaviour.
* Set max_buffer_len to 32.
Since the limit is now per substream and the default
`max_substreams` is `128`, this new limit retains the
previous overall resource bounds for the buffers.
* Expand tests and small cleanup.
Co-authored-by: Max Inden <mail@max-inden.de>
* Refactor Mplex.
Thereby addressing the following issues:
* Send a `Reset` frame when open substreams get dropped (313).
* Avoid stalls caused by a read operation on one substream
reading (and buffering) frames for another substream without
notifying the corresponding task. I.e. the tracked read-interest
must be scoped to a substream.
* Remove dropped substreams from the tracked set of open
substreams, to avoid artificially running into substream
limits.
* Update CHANGELOG.
* Refine behaviour of dropping substreams.
By taking the substream state into account. The refined
behaviour is modeled after the behaviour of Yamux.
* Tweak docs and recv buffer retention.
* Further small tweaks.
* Make the pending frames a FIFO queue.
* Take more care to avoid keeping read-wakers around
and to notify them when streams close.
* Prefer wake over unregister.
It is probably safer to always wake pending wakers.
* Update muxers/mplex/src/codec.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Update muxers/mplex/src/io.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Some review feedback and cosmetics.
* Update muxers/mplex/src/io.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Revise read control flow for clarity.
While seemingly duplicating some control flow between
`poll_next_strean` and `poll_read_stream`, the individual
control flow of each read operation is easier to follow.
* CI
* Rename Status::Ok to Status::Open.
* Rename pending_flush to pending_flush_open.
* Finishing touches.
* Tweak changelog.
Co-authored-by: Max Inden <mail@max-inden.de>
* Allow StreamMuxer to notify changes in the address
* Fix doc link
* Revert accidental rename
* Other accidental rename
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* mplex: Check for error and shutdown.
Issues #1504 and #1523 reported panics caused by polling the sink of
`secio::LenPrefixCodec` after it had entered its terminal state, i.e.
after it had previously encountered an error or was closed. According
to the reports this happened only when using mplex as a stream
multiplexer. It seems that because mplex always stores and keeps the
`Waker` when polling, a wakeup of any of those wakers will resume the
polling even for those cases where the previous poll did not return
`Poll::Pending` but resolved to a value.
To prevent polling after the connection was closed or an error
happened we check for those conditions prior to every poll.
* Keep error when operations fail.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* *: Remove usage of custom buffer initialization usage
With version `0.3.0-alpha.19` the futures-preview crate makes the
`AsyncRead::initializer` API unstable.
In order to improve interoperability with e.g. both a library depending
on alpha.18 as well as a library depending on alpha.19 and in order for
rust-libp2p to become stable again, this commit removes all usages of
the unstable `initializer` API.
* protocols/noise: Remove NoiseOutput Asyncread initializer
* transports/tcp: Remove TcpTransStream AsyncRead initializer
* *: Remove version pinning of futures-preview to 0.3.0-alpha.18
With version 0.3.0-alpha.19 the futures-preview crate makes the
AsyncRead::initializer API unstable. Given that the previous commits
removed usage of the initializer API, the version pinning is not needed
any longer.
* Remove tokio-codec dependency from multistream-select.
In preparation for the eventual switch from tokio to std futures.
Includes some initial refactoring in preparation for further work
in the context of https://github.com/libp2p/rust-libp2p/issues/659.
* Reduce default buffer sizes.
* Allow more than one frame to be buffered for sending.
* Doc tweaks.
* Remove superfluous (duplicated) Message types.
* Reduce roundtrips in multistream-select negotiation.
1. Enable 0-RTT: If the dialer only supports a single protocol, it can send
protocol data (e.g. the actual application request) together with
the multistream-select header and protocol proposal. Similarly,
if the listener supports a proposed protocol, it can send protocol
data (e.g. the actual application response) together with the
multistream-select header and protocol confirmation.
2. In general, the dialer "settles on" an expected protocol as soon
as it runs out of alternatives. Furthermore, both dialer and listener
do not immediately flush the final protocol confirmation, allowing it
to be sent together with application protocol data. Attempts to read
from the negotiated I/O stream implicitly flushes any pending data.
3. A clean / graceful shutdown of an I/O stream always completes protocol
negotiation.
The publich API of multistream-select changed slightly, requiring both
AsyncRead and AsyncWrite bounds for async reading and writing due to
the implicit buffering and "lazy" negotiation. The error types have
also been changed, but they were not previously fully exported.
Includes some general refactoring with simplifications and some more tests,
e.g. there was an edge case relating to a possible ambiguity when parsing
multistream-select protocol messages.
* Further missing commentary.
* Remove unused test dependency.
* Adjust commentary.
* Cleanup NegotiatedComplete::poll()
* Fix deflate protocol tests.
* Stabilise network_simult test.
The test implicitly relied on "slow" connection establishment
in order to have a sufficient probability of passing.
With the removal of roundtrips in multistream-select, it is now
more likely that within the up to 50ms duration between swarm1
and swarm2 dialing, the connection is already established, causing
the expectation of step == 1 to fail when receiving a Connected event,
since the step may then still be 0.
This commit aims to avoid these spurious errors by detecting runs
during which a connection is established "too quickly", repeating
the test run.
It still seems theoretically possible that, if connections are always
established "too quickly", the test runs forever. However, given that
the delta between swarm1 and swarm2 dialing is 0-50ms and that the
TCP transport is used, that seems probabilistically unlikely.
Nevertheless, the purpose of the artificial dialing delay between
swarm1 and swarm2 should be re-evaluated and possibly at least
the maximum delay further reduced.
* Complete negotiation between upgrades in libp2p-core.
While multistream-select, as a standalone library and providing
an API at the granularity of a single negotiation, supports
lazy negotiation (and in particular 0-RTT negotiation), in the
context of libp2p-core where any number of negotiations are
composed generically within the concept of composable "upgrades",
it is necessary to wait for protocol negotiation between upgrades
to complete.
* Clarify docs. Simplify listener upgrades.
Since reading from a Negotiated I/O stream implicitly flushes any pending
negotiation data, there is no pitfall involved in not waiting for completion.
Addresses https://github.com/libp2p/rust-libp2p/issues/1206 by always
registering the current task before calling poll_*_notify functions.
This is in the same spirit as the corresponding fix for yamux
in https://github.com/paritytech/yamux/pull/54.
Also adds missing registration of the current task in close()
and flush_all(), which have been observed to cause stalls
when trying to do a graceful connection shutdown / close.
* muxing: adds an error type to streammuxer
* Update examples/chat.rs
Co-Authored-By: montekki <fedor.sakharov@gmail.com>
* make the trait error type bound to io error
