* Add support for PINGs and PONGs to websockets.
`Connection` (formerly `BytesConnection`) now supports more structured
`IncomingData` and `OutgoingData` which mirror the data types in
soketto (which are not exposed). This allows adding `Connection::ping`
and `Connection::pong` methods.
The non-framed websocket transport defines `BytesConnection` as a
wrapper around `Connection` and handles only binary data.
Mostly mechanical. Creating a `CommonTransport` yields an
`io::Result<_>` now since creating the `DnsConfig` may fail with an
`io::Error` when creating the `ThreadPool`.
The `DnsConfig` `Transport` impl had to be changed slightly:
(a) PR [[1311]] requires some `Send` bounds.
(b) The async block had to be changed to work around lifetime inference
issues which resulted in an "one type is more general than the other"
error.
[1311]: https://github.com/libp2p/rust-libp2p/pull/1311
* misc/mdns/service: Use async std with stack pinned futures
* misc/mdns: Define mdns broadcast address as lazy static
* misc/mdns: Drop future before borrowing their arguments again
* misc/mdns: Send queries on query socket, not socket
* misc/mdns: Use poll_next_unpin on query interval stream
* misc/mdns: Ensure underlying task is woken up on next interval tick
* misc/mdns: Wrap match expression in scope to drop future early
* misc/mdns: Adjust 'discovery_ourselves' test
* misc/mdns: Make query interval fire instantly on first tick
This is an optimization only important for short lived use cases, e.g.
unit tests. Instead of waiting for 20 seconds at first, the query
interval fires right away and thereby the service makes progress
instantly.
* misc/mdns: Adjust MdnsService documentation tests
* misc/mdns: Do not drop UDP socket send and reicv futures
Libp2p-mdns uses the async-std crate for network io. This crate only
offers async send and receive functions. In order to use this in non
async/await functions one needs to keep the future returned by the crate
functions around across `poll` invocations.
The future returned by the crate functions references the io resource.
Thus one has to keep both the io resource as well as the future
referencing it. This results in a self-referencing struct which is not
possible to create with safe Rust.
Instead, by having `MdnsService::next` (former `MdnsService::poll`) take
ownership of `self`, the Rust async magic takes care of the above (See
code comments for more details).
As a (negative) side effect, given that `MdnsService::next` takes
ownership of `self`, there is nothing to bind the lifetime of the
returned `MdnsPacket` to. With no better solution in mind, this patch
makes `MdnsPacket` static, not referencing the `MdnsService` receive
buffer.
* misc/mdns: Fix code comments and remove *if Free* TODO
* misc/mdns: Minor refactorings
* misc/mdns: Build responses in behaviour.rs directly
* misc/mdns: Move response ttl duration to constant
* misc/mdns: Remove optimization todo comment
* misc/mdns: Add query interval test
* misc/mdns: Move packet parsing into MdnPacket impl
* misc/mdns: Don't have receiving packets starve the query interval
When we 'await' on receiving a packet on the udp socket without
receiving a single packet we starve the remaining logic of the mdns
service, in this case the logic triggered on the receive interval.
* misc/mdns: Add debug_assert to MaybeBusyMdnsService check
* misc/mdns: Implement Debug for MaybeBusyMdnsService
* misc/mdns: Make ownership note a normal comment, not a doc comment
* misc/mdns: Have discovered_peers return an iterator
* protocols/plaintext: Move to stable futures and use unsigned varints
The plaintext 2.0 specification requires to use unsigned varints for
frame length delimiting instead of fixed 4 byte integer frame length
delimiting. This commit aligns the implementation with the
specification.
* protocols/secio: Fix doc comment BytesMut -> Vec<u8>
* protocols/plaintext: Add quick check smoke test
* protocols/plaintext: Rework imports and log levels
* protocols/plaintext: Use BytesMut instead of Vec<u8>
* protocols/plaintext: Use BoxFuture
Update yamux to development version.
For the boxed `futures::stream::Stream` we have to decide if we require
a `Send` bound or not. Since some upgrades may produce outputs which are
`!Send` we offer both upgrade versions.
* Implement Debug for (ed25519|secp256k1)::(Keypair|SecretKey) (#1285)
* Fix possible arithmetic overflow in libp2p-kad. (#1291)
When the number of active queries exceeds the (internal)
JOBS_MAX_QUERIES limit, which is only supposed to bound
the number of concurrent queries relating to background
jobs, an arithmetic overflow occurs. This is fixed by
using saturating subtraction.
* protocols/plaintext: Add example on how to upgrade with PlainTextConfig1 (#1286)
* [mdns] - Support for long mDNS names (Bug #1232) (#1287)
* Dead code -- commenting out with a note referencing future implementation
* Adding "std" feature so that cargo can build in other directories (notably `misc/mdns`, so that I could run these tests)
* Permitting `PeerID` to be built from an `Identity` multihash
* The length limit for DNS labels is 63 characters, as per RFC1035
* Allocates the vector with capacity for the service name plus additional QNAME encoding bytes
* Added support for encoding/decoding peer IDs with an encoded length greater than 63 characters
* Removing "std" from ring features
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Retaining MAX_INLINE_KEY_LENGTH with comment about future usage
* `segment_peer_id` consumes `peer_id` ... plus an early return for IDs that don't need to be segmented
* Fixing logic
* Bump most dependencies (#1268)
* Bump most dependencies
This actually builds 😊.
* Bump all dependencies
Includes the excellent work of @rschulman in #1265.
* Remove use of ed25519-dalek fork
* Monomorphize more dependencies
* Add compatibility hack for rand
Cargo allows a crate to depend on multiple versions of another, but
`cargo-web` panics in that situation. Use a wrapper crate to work
around the panic.
* Use @tomaka’s idea for using a newer `rand`
instead of my own ugly hack.
* Switch to Parity master
as its dependency-bumping PR has been merged.
* Update some depenendencies again
* Remove unwraps and `#[allow(deprecated)]`.
* Remove spurious changes to dependencies
Bumping minor or patch versions is not needed, and increases likelyhood
of merge conflicts.
* Remove some redundant Cargo.toml changes
* Replace a retry loop with an expect
`ed25519::SecretKey::from_bytes` will never fail for 32-byte inputs.
* Revert changes that don’t belong in this PR
* Remove using void to bypass ICE (#1295)
* Publish 0.13.0 (#1294)
* *: Remove usage of custom buffer initialization usage
With version `0.3.0-alpha.19` the futures-preview crate makes the
`AsyncRead::initializer` API unstable.
In order to improve interoperability with e.g. both a library depending
on alpha.18 as well as a library depending on alpha.19 and in order for
rust-libp2p to become stable again, this commit removes all usages of
the unstable `initializer` API.
* protocols/noise: Remove NoiseOutput Asyncread initializer
* transports/tcp: Remove TcpTransStream AsyncRead initializer
* *: Remove version pinning of futures-preview to 0.3.0-alpha.18
With version 0.3.0-alpha.19 the futures-preview crate makes the
AsyncRead::initializer API unstable. Given that the previous commits
removed usage of the initializer API, the version pinning is not needed
any longer.
* Configurable multistream-select protocol. Add V1Lazy variant. (#1245)
Make the multistream-select protocol (version) configurable
on transport upgrades as well as for individual substreams.
Add a "lazy" variant of multistream-select 1.0 that delays
sending of negotiation protocol frames as much as possible
but is only safe to use under additional assumptions that
go beyond what is required by the multistream-select v1
specification.
* Improve the code readability of the chat example (#1253)
* Add bridged chats (#1252)
* Try fix CI (#1261)
* Print Rust version on CI
* Don't print where not appropriate
* Change caching strategy
* Remove win32 build
* Remove win32 from list
* Update libsecp256k1 dep to 0.3.0 (#1258)
* Update libsecp256k1 dep to 0.3.0
* Sign now cannot fail
* Upgrade url and percent-encoding deps to 2.1.0 (#1267)
* Upgrade percent-encoding dep to 2.1.0
* Upgrade url dep to 2.1.0
* Revert CIPHERS set to null (#1273)
* Update dependency versions (#1265)
* Update versions of many dependencies
* Bump version of rand
* Updates for changed APIs in rand, ring, and webpki
* Replace references to `snow::Session`
`Session` no longer exists in `snow` but the replacement is two structs `HandshakeState` and `TransportState`
Something will have to be done to harmonize `NoiseOutput.session`
* Add precise type for UnparsedPublicKey
* Update data structures/functions to match new snow's API
* Delete diff.diff
Remove accidentally committed diff file
* Remove commented lines in identity/rsa.rs
* Bump libsecp256k1 to 0.3.1
* Implement /plaintext/2.0.0 (#1236)
* WIP
* plaintext/2.0.0
* Refactor protobuf related issues to compatible with the spec
* Rename: new PlainTextConfig -> PlainText2Config
* Keep plaintext/1.0.0 as PlainText1Config
* Config contains pubkey
* Rename: proposition -> exchange
* Add PeerId to Exchange
* Check the validity of the remote's `Exchange`
* Tweak
* Delete unused import
* Add debug log
* Delete unused field: public_key_encoded
* Delete unused field: local
* Delete unused field: exchange_bytes
* The inner instance should not be public
* identity::Publickey::Rsa is not available on wasm
* Delete PeerId from Config as it should be generated from the pubkey
* Catch up for #1240
* Tweak
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update protocols/plaintext/src/handshake.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Rename: pubkey -> local_public_key
* Delete unused error
* Rename: PeerIdValidationFailed -> InvalidPeerId
* Fix: HandShake -> Handshake
* Use bytes insteadof Publickey to avoid code duplication
* Replace with ProtobufError
* Merge HandshakeContext<()> into HandshakeContext<Local>
* Improve the peer ID validation to simplify the handshake
* Propagate Remote to allow extracting the PeerId from the Remote
* Collapse the same kind of errors into the variant
* [noise]: `sodiumoxide 0.2.5` (#1276)
Fixes https://github.com/RustSec/advisory-db/pull/192
* examples/ipfs-kad.rs: Remove outdated reference to `without_init` (#1280)
* CircleCI Test Fix (#1282)
* Disabling "Docker Layer Caching" because it breaks one of the circleci checks
* Bump to trigger CircleCI build
* unbump
* zeroize: Upgrade to v1.0 (#1284)
v1.0 final release is out. Release notes:
https://github.com/iqlusioninc/crates/pull/279
* *: Consolidate protobuf scripts and update to rust-protobuf 2.8.1 (#1275)
* *: Consolidate protobuf generation scripts
* *: Update to rust-protobuf 2.8.1
* *: Mark protobuf generated modules with '_proto'
* examples: Add distributed key value store (#1281)
* examples: Add distributed key value store
This commit adds a basic distributed key value store supporting GET and
PUT commands using Kademlia and mDNS.
* examples/distributed-key-value-store: Fix typo
* Simple Warning Cleanup (#1278)
* Cleaning up warnings - removing unused `use`
* Cleaning up warnings - unused tuple value
* Cleaning up warnings - removing dead code
* Cleaning up warnings - fixing deprecated name
* Cleaning up warnings - removing dead code
* Revert "Cleaning up warnings - removing dead code"
This reverts commit f18a765e4bf240b0ed9294ec3ae5dab5c186b801.
* Enable the std feature of ring (#1289)
* Configurable multistream-select protocol. Add V1Lazy variant. (#1245)
Make the multistream-select protocol (version) configurable
on transport upgrades as well as for individual substreams.
Add a "lazy" variant of multistream-select 1.0 that delays
sending of negotiation protocol frames as much as possible
but is only safe to use under additional assumptions that
go beyond what is required by the multistream-select v1
specification.
* Improve the code readability of the chat example (#1253)
* Add bridged chats (#1252)
* Try fix CI (#1261)
* Print Rust version on CI
* Don't print where not appropriate
* Change caching strategy
* Remove win32 build
* Remove win32 from list
* Update libsecp256k1 dep to 0.3.0 (#1258)
* Update libsecp256k1 dep to 0.3.0
* Sign now cannot fail
* Upgrade url and percent-encoding deps to 2.1.0 (#1267)
* Upgrade percent-encoding dep to 2.1.0
* Upgrade url dep to 2.1.0
* Fix more conflicts
* Revert CIPHERS set to null (#1273)
* protocols/noise: Fix obvious future errors
* protocol/noise: Make Handshake methods independent functions
* protocols/noise: Abstract T and C for handshake
* protocols/noise: Replace FutureResult with Result
* protocols/noise: Introduce recv_identity stub
* protocols/noise: Implement recv_identity stub
* protocols/noise: Change NoiseConfig::Future from Handshake to Result
* protocols/noise: Adjust to new Poll syntax
* protocols/noise: Return early on state creation failure
* protocols/noise: Add bounds Async{Write,Read} to initiator / respoder
* protocols/noise: Add Protocol trait bound for C in rt functions
* protocols/noise: Do io operations on state.io instead of state
* protocols/noise: Have upgrade_xxx return a pinned future
* protocols/noise: Have NoiseOutput::poll_read self be mutable
* protocols/noise: Make recv_identity buffers mutable
* protocols/noise: Fix warnings
* protocols/noise: Replace NoiseOutput io::Read impl with AsyncRead
* protocols/noise: Replace NoiseOutput io::Write impl with AsyncWrite
* protocols/noise: Adjust tests to new futures
* protocols/noise: Don't use {AsyncRead,AsyncWrite,TryStream}*Ext* bound
* protocols/noise: Don't use async_closure feature
* protocols/noise: use futures::ready! macro
* protocols/noise: Make NoiseOutput AsyncRead return unsafe NopInitializer
The previous implementation of AsyncRead for NoiseOutput would operate
on uninitialized buffers, given that it properly returned the number of
bytest that were written to the buffer. With this patch the current
implementation operates on uninitialized buffers as well by returning an
Initializer::nop() in AsyncRead::initializer.
* protocols/noise: Remove resolved TODO questions
* protocols/noise: Remove 'this = self' comment
Given that `let mut this = &mut *self` is not specific to a pinned self,
but follows the dereference coercion [1] happening at compile time when
trying to mutably borrow two distinct struct fields, this patch removes
the code comment.
[1]
```rust
let x = &mut self.deref_mut().x;
let y = &mut self.deref_mut().y; // error
// ---
let mut this = self.deref_mut();
let x = &mut this.x;
let y = &mut this.y; // ok
```
* Remove redundant nested futures.
* protocols/noise/Cargo: Update to futures preview 0.3.0-alpha.18
* protocols/noise: Improve formatting
* protocols/noise: Return pinned future on authenticated noise upgrade
* protocols/noise: Specify Output of Future embedded in Handshake directly
* *: Ensure Noise handshake futures are Send
* Revert "*: Ensure Noise handshake futures are Send"
This reverts commit 555c2df315e44f21ad39d4408445ce2cb84dd1a4.
* protocols/noise: Ensure NoiseConfig Future is Send
* protocols/noise: Use relative import path for {In,Out}boundUpgrade
* Rework the transport upgrade API.
ALthough transport upgrades must follow a specific pattern
in order fot the resulting transport to be usable with a
`Network` or `Swarm`, that pattern is currently not well
reflected in the transport upgrade API. Rather, transport
upgrades are rather laborious and involve non-trivial code
duplication.
This commit introduces a `transport::upgrade::Builder` that is
obtained from `Transport::upgrade`. The `Builder` encodes the
previously implicit rules for transport upgrades:
1. Authentication upgrades must happen first.
2. Any number of upgrades may follow.
3. A multiplexer upgrade must happen last.
Since multiplexing is the last (regular) transport upgrade (because
that upgrade yields a `StreamMuxer` which is no longer a `AsyncRead`
/ `AsyncWrite` resource, which the upgrade process is based on),
the upgrade starts with `Transport::upgrade` and ends with
`Builder::multiplex`, which drops back down to the `Transport`,
providing a fluent API.
Authentication and multiplexer upgrades must furthermore adhere
to a minimal contract w.r.t their outputs:
1. An authentication upgrade is given an (async) I/O resource `C`
and must produce a pair `(I, D)` where `I: ConnectionInfo` and
`D` is a new (async) I/O resource `D`.
2. A multiplexer upgrade is given an (async) I/O resource `C`
and must produce a `M: StreamMuxer`.
To that end, two changes to the `secio` and `noise` protocols have been
made:
1. The `secio` upgrade now outputs a pair of `(PeerId, SecioOutput)`.
The former implements `ConnectionInfo` and the latter `AsyncRead` /
`AsyncWrite`, fulfilling the `Builder` contract.
2. A new `NoiseAuthenticated` upgrade has been added that wraps around
any noise upgrade (i.e. `NoiseConfig`) and has an output of
`(PeerId, NoiseOutput)`, i.e. it checks if the `RemoteIdentity` from
the handshake output is an `IdentityKey`, failing if that is not the
case. This is the standard upgrade procedure one wants for integrating
noise with libp2p-core/swarm.
* Cleanup
* Add a new integration test.
* Add missing license.
* Refactor the Identify protocol.
Thereby updating the documentation. The low-level protocol
and handler modules are no longer exposed and some constructors
of the IdentifyEvent renamed.
* Update protocols/identify/src/protocol.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Replace `listen_addr` with `local_addr`.
In `ListenerUpgrade`, `ConnectedPoint` and other event types where we
were previously using the listen address we now report the local address
of an incoming connection. The reason being that it is difficult to get
the listen address right. In case clients want to know, which listener
produced an incoming connection upgrade they are advised to use the
`ListenerId` for such purposes.
* Update transports/tcp/src/lib.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Add listener ID and error event.
Report listener errors to client code so they are aware that an error
occurred within a listener. By default we continue to poll listeners
which produced an error, but clients can remove listeners by ID.
* tcp: Report errors.
Instead of silently waiting after errors we return all errors, but pause
after each error, before continuing.
* Add a test.
To ease testing, `Listener` is made generic and we test that no values
and errors are lost. Elapsed time between item generation is not
measured.
* Support the new methods in core-derive.
* Address review concerns.
* Remove `Display` impl of `ListenerId`.
* Add 'static bound to `on_listener_error` error.
* Remove tokio-codec dependency from multistream-select.
In preparation for the eventual switch from tokio to std futures.
Includes some initial refactoring in preparation for further work
in the context of https://github.com/libp2p/rust-libp2p/issues/659.
* Reduce default buffer sizes.
* Allow more than one frame to be buffered for sending.
* Doc tweaks.
* Remove superfluous (duplicated) Message types.
* Reduce roundtrips in multistream-select negotiation.
1. Enable 0-RTT: If the dialer only supports a single protocol, it can send
protocol data (e.g. the actual application request) together with
the multistream-select header and protocol proposal. Similarly,
if the listener supports a proposed protocol, it can send protocol
data (e.g. the actual application response) together with the
multistream-select header and protocol confirmation.
2. In general, the dialer "settles on" an expected protocol as soon
as it runs out of alternatives. Furthermore, both dialer and listener
do not immediately flush the final protocol confirmation, allowing it
to be sent together with application protocol data. Attempts to read
from the negotiated I/O stream implicitly flushes any pending data.
3. A clean / graceful shutdown of an I/O stream always completes protocol
negotiation.
The publich API of multistream-select changed slightly, requiring both
AsyncRead and AsyncWrite bounds for async reading and writing due to
the implicit buffering and "lazy" negotiation. The error types have
also been changed, but they were not previously fully exported.
Includes some general refactoring with simplifications and some more tests,
e.g. there was an edge case relating to a possible ambiguity when parsing
multistream-select protocol messages.
* Further missing commentary.
* Remove unused test dependency.
* Adjust commentary.
* Cleanup NegotiatedComplete::poll()
* Fix deflate protocol tests.
* Stabilise network_simult test.
The test implicitly relied on "slow" connection establishment
in order to have a sufficient probability of passing.
With the removal of roundtrips in multistream-select, it is now
more likely that within the up to 50ms duration between swarm1
and swarm2 dialing, the connection is already established, causing
the expectation of step == 1 to fail when receiving a Connected event,
since the step may then still be 0.
This commit aims to avoid these spurious errors by detecting runs
during which a connection is established "too quickly", repeating
the test run.
It still seems theoretically possible that, if connections are always
established "too quickly", the test runs forever. However, given that
the delta between swarm1 and swarm2 dialing is 0-50ms and that the
TCP transport is used, that seems probabilistically unlikely.
Nevertheless, the purpose of the artificial dialing delay between
swarm1 and swarm2 should be re-evaluated and possibly at least
the maximum delay further reduced.
* Complete negotiation between upgrades in libp2p-core.
While multistream-select, as a standalone library and providing
an API at the granularity of a single negotiation, supports
lazy negotiation (and in particular 0-RTT negotiation), in the
context of libp2p-core where any number of negotiations are
composed generically within the concept of composable "upgrades",
it is necessary to wait for protocol negotiation between upgrades
to complete.
* Clarify docs. Simplify listener upgrades.
Since reading from a Negotiated I/O stream implicitly flushes any pending
negotiation data, there is no pitfall involved in not waiting for completion.
