Previously, a protocol could be any sequence of bytes as long as it started with `/`. Now, we directly parse a protocol as `String` which enforces it to be valid UTF8.
To notify users of this change, we delete the `ProtocolName` trait. The new requirement is that users need to provide a type that implements `AsRef<str>`.
We also add a `StreamProtocol` newtype in `libp2p-swarm` which provides an easy way for users to ensure their protocol strings are compliant. The newtype enforces that protocol strings start with `/`. `StreamProtocol` also implements `AsRef<str>`, meaning users can directly use it in their upgrades.
`multistream-select` by itself only changes marginally with this patch. The only thing we enforce in the type-system is that protocols must implement `AsRef<str>`.
Resolves: #2831.
Pull-Request: #3746.
Instead of having a mix of `poll_event`, `poll_outbound` and `poll_close`, we
flatten the entire interface of `StreamMuxer` into 4 individual functions:
- `poll_inbound`
- `poll_outbound`
- `poll_address_change`
- `poll_close`
This design is closer to the design of other async traits like `AsyncRead` and
`AsyncWrite`. It also allows us to delete the `StreamMuxerEvent`.
This aligns the public API of the `libp2p-mplex` module with the one
from `libp2p-yamux`. This change has two benefits:
1. For standalone users of `libp2p-mplex`, the substreams itself are
now useful, similar to `libp2p-yamux` and don't necessarily need to
be polled via the `StreamMuxer`. The `StreamMuxer` only forwards to
the `Async{Read,Write}` implementations.
2. This will reduce the diff of #2648 because we can chunk the one
giant commit into smaller atomic ones.
`libp2p-core` provides the `StreamMuxer` abstraction so it can provide
functionality that abstracts over this trait.
We never use the `flush_all` function as part of our abstractions.
No one else is going to use it so we can remove it from the abstraction.
* Make clippy "happy".
Address all clippy complaints that are not purely stylistic (or even
have corner cases with false positives). Ignore all "style" and "pedantic" lints.
* Fix tests.
* Undo unnecessary API change.
* Refactor Mplex.
Thereby addressing the following issues:
* Send a `Reset` frame when open substreams get dropped (313).
* Avoid stalls caused by a read operation on one substream
reading (and buffering) frames for another substream without
notifying the corresponding task. I.e. the tracked read-interest
must be scoped to a substream.
* Remove dropped substreams from the tracked set of open
substreams, to avoid artificially running into substream
limits.
* Update CHANGELOG.
* Refine behaviour of dropping substreams.
By taking the substream state into account. The refined
behaviour is modeled after the behaviour of Yamux.
* Tweak docs and recv buffer retention.
* Further small tweaks.
* Make the pending frames a FIFO queue.
* Take more care to avoid keeping read-wakers around
and to notify them when streams close.
* Prefer wake over unregister.
It is probably safer to always wake pending wakers.
* Update muxers/mplex/src/codec.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Update muxers/mplex/src/io.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Some review feedback and cosmetics.
* Update muxers/mplex/src/io.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Revise read control flow for clarity.
While seemingly duplicating some control flow between
`poll_next_strean` and `poll_read_stream`, the individual
control flow of each read operation is easier to follow.
* CI
* Rename Status::Ok to Status::Open.
* Rename pending_flush to pending_flush_open.
* Finishing touches.
* Tweak changelog.
Co-authored-by: Max Inden <mail@max-inden.de>
* Allow StreamMuxer to notify changes in the address
* Fix doc link
* Revert accidental rename
* Other accidental rename
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* mplex: Check for error and shutdown.
Issues #1504 and #1523 reported panics caused by polling the sink of
`secio::LenPrefixCodec` after it had entered its terminal state, i.e.
after it had previously encountered an error or was closed. According
to the reports this happened only when using mplex as a stream
multiplexer. It seems that because mplex always stores and keeps the
`Waker` when polling, a wakeup of any of those wakers will resume the
polling even for those cases where the previous poll did not return
`Poll::Pending` but resolved to a value.
To prevent polling after the connection was closed or an error
happened we check for those conditions prior to every poll.
* Keep error when operations fail.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* *: Remove usage of custom buffer initialization usage
With version `0.3.0-alpha.19` the futures-preview crate makes the
`AsyncRead::initializer` API unstable.
In order to improve interoperability with e.g. both a library depending
on alpha.18 as well as a library depending on alpha.19 and in order for
rust-libp2p to become stable again, this commit removes all usages of
the unstable `initializer` API.
* protocols/noise: Remove NoiseOutput Asyncread initializer
* transports/tcp: Remove TcpTransStream AsyncRead initializer
* *: Remove version pinning of futures-preview to 0.3.0-alpha.18
With version 0.3.0-alpha.19 the futures-preview crate makes the
AsyncRead::initializer API unstable. Given that the previous commits
removed usage of the initializer API, the version pinning is not needed
any longer.
* Remove tokio-codec dependency from multistream-select.
In preparation for the eventual switch from tokio to std futures.
Includes some initial refactoring in preparation for further work
in the context of https://github.com/libp2p/rust-libp2p/issues/659.
* Reduce default buffer sizes.
* Allow more than one frame to be buffered for sending.
* Doc tweaks.
* Remove superfluous (duplicated) Message types.
* Reduce roundtrips in multistream-select negotiation.
1. Enable 0-RTT: If the dialer only supports a single protocol, it can send
protocol data (e.g. the actual application request) together with
the multistream-select header and protocol proposal. Similarly,
if the listener supports a proposed protocol, it can send protocol
data (e.g. the actual application response) together with the
multistream-select header and protocol confirmation.
2. In general, the dialer "settles on" an expected protocol as soon
as it runs out of alternatives. Furthermore, both dialer and listener
do not immediately flush the final protocol confirmation, allowing it
to be sent together with application protocol data. Attempts to read
from the negotiated I/O stream implicitly flushes any pending data.
3. A clean / graceful shutdown of an I/O stream always completes protocol
negotiation.
The publich API of multistream-select changed slightly, requiring both
AsyncRead and AsyncWrite bounds for async reading and writing due to
the implicit buffering and "lazy" negotiation. The error types have
also been changed, but they were not previously fully exported.
Includes some general refactoring with simplifications and some more tests,
e.g. there was an edge case relating to a possible ambiguity when parsing
multistream-select protocol messages.
* Further missing commentary.
* Remove unused test dependency.
* Adjust commentary.
* Cleanup NegotiatedComplete::poll()
* Fix deflate protocol tests.
* Stabilise network_simult test.
The test implicitly relied on "slow" connection establishment
in order to have a sufficient probability of passing.
With the removal of roundtrips in multistream-select, it is now
more likely that within the up to 50ms duration between swarm1
and swarm2 dialing, the connection is already established, causing
the expectation of step == 1 to fail when receiving a Connected event,
since the step may then still be 0.
This commit aims to avoid these spurious errors by detecting runs
during which a connection is established "too quickly", repeating
the test run.
It still seems theoretically possible that, if connections are always
established "too quickly", the test runs forever. However, given that
the delta between swarm1 and swarm2 dialing is 0-50ms and that the
TCP transport is used, that seems probabilistically unlikely.
Nevertheless, the purpose of the artificial dialing delay between
swarm1 and swarm2 should be re-evaluated and possibly at least
the maximum delay further reduced.
* Complete negotiation between upgrades in libp2p-core.
While multistream-select, as a standalone library and providing
an API at the granularity of a single negotiation, supports
lazy negotiation (and in particular 0-RTT negotiation), in the
context of libp2p-core where any number of negotiations are
composed generically within the concept of composable "upgrades",
it is necessary to wait for protocol negotiation between upgrades
to complete.
* Clarify docs. Simplify listener upgrades.
Since reading from a Negotiated I/O stream implicitly flushes any pending
negotiation data, there is no pitfall involved in not waiting for completion.
Addresses https://github.com/libp2p/rust-libp2p/issues/1206 by always
registering the current task before calling poll_*_notify functions.
This is in the same spirit as the corresponding fix for yamux
in https://github.com/paritytech/yamux/pull/54.
Also adds missing registration of the current task in close()
and flush_all(), which have been observed to cause stalls
when trying to do a graceful connection shutdown / close.
* muxing: adds an error type to streammuxer
* Update examples/chat.rs
Co-Authored-By: montekki <fedor.sakharov@gmail.com>
* make the trait error type bound to io error
mplex gets a new flag `is_shutdown` to keep track of when `shutdown` has
been called. We need to make certain that after the shutdown no more
`Sink::poll_complete` or `Sink::start_send` calls are being made which
may result in a panic.
Update the `StreamMuxer` trait.
- `read_substream`, `write_substream` and `flush_substream` now return `Poll` instead of `Result`.
- A new `Shutdown` enum allows for half-closing of substreams and is used in `shutdown_substream`.
- `close_inbound` and `close_outbound` have been merged into `shutdown` which takes a `Shutdown` parameter to allow closing only one direction.
- Add a new `flush_all` method to allow flushing after a series of actions (e.g. multiple `shutdown_substream`).
W.r.t. flushing the general idea is that normal use drains buffers over time. Shutting down a substream does not imply flushing, so can be followed by `flush_substream` or (if multiple substreams are to be shut down) a single `flush_all`. Shutting down the muxer itself proceeds likewise, i.e. `shutdown` followed by `flush_all`.
* Rewrite the swarm
* Small improvement to Debug of ListenersStream
* Fix Swarm::Replaced never being produced
* Fix logic problem when reaching a node in swarm
* Small comment in swarm
* Add closed_multiaddr to Replaced event
* Add address to NodeClosed and NodeError
* Fix concerns
* Remove StreamMuxer::boxed
Retain all incoming buffer elements with different substream ID or equal
Endpoint. The latter was previously not considered which could result in
the removal of data for another substream with same ID but opposite
Endpoint.
