* Add listener ID and error event.
Report listener errors to client code so they are aware that an error
occurred within a listener. By default we continue to poll listeners
which produced an error, but clients can remove listeners by ID.
* tcp: Report errors.
Instead of silently waiting after errors we return all errors, but pause
after each error, before continuing.
* Add a test.
To ease testing, `Listener` is made generic and we test that no values
and errors are lost. Elapsed time between item generation is not
measured.
* Support the new methods in core-derive.
* Address review concerns.
* Remove `Display` impl of `ListenerId`.
* Add 'static bound to `on_listener_error` error.
* Remove tokio-codec dependency from multistream-select.
In preparation for the eventual switch from tokio to std futures.
Includes some initial refactoring in preparation for further work
in the context of https://github.com/libp2p/rust-libp2p/issues/659.
* Reduce default buffer sizes.
* Allow more than one frame to be buffered for sending.
* Doc tweaks.
* Remove superfluous (duplicated) Message types.
* Reduce roundtrips in multistream-select negotiation.
1. Enable 0-RTT: If the dialer only supports a single protocol, it can send
protocol data (e.g. the actual application request) together with
the multistream-select header and protocol proposal. Similarly,
if the listener supports a proposed protocol, it can send protocol
data (e.g. the actual application response) together with the
multistream-select header and protocol confirmation.
2. In general, the dialer "settles on" an expected protocol as soon
as it runs out of alternatives. Furthermore, both dialer and listener
do not immediately flush the final protocol confirmation, allowing it
to be sent together with application protocol data. Attempts to read
from the negotiated I/O stream implicitly flushes any pending data.
3. A clean / graceful shutdown of an I/O stream always completes protocol
negotiation.
The publich API of multistream-select changed slightly, requiring both
AsyncRead and AsyncWrite bounds for async reading and writing due to
the implicit buffering and "lazy" negotiation. The error types have
also been changed, but they were not previously fully exported.
Includes some general refactoring with simplifications and some more tests,
e.g. there was an edge case relating to a possible ambiguity when parsing
multistream-select protocol messages.
* Further missing commentary.
* Remove unused test dependency.
* Adjust commentary.
* Cleanup NegotiatedComplete::poll()
* Fix deflate protocol tests.
* Stabilise network_simult test.
The test implicitly relied on "slow" connection establishment
in order to have a sufficient probability of passing.
With the removal of roundtrips in multistream-select, it is now
more likely that within the up to 50ms duration between swarm1
and swarm2 dialing, the connection is already established, causing
the expectation of step == 1 to fail when receiving a Connected event,
since the step may then still be 0.
This commit aims to avoid these spurious errors by detecting runs
during which a connection is established "too quickly", repeating
the test run.
It still seems theoretically possible that, if connections are always
established "too quickly", the test runs forever. However, given that
the delta between swarm1 and swarm2 dialing is 0-50ms and that the
TCP transport is used, that seems probabilistically unlikely.
Nevertheless, the purpose of the artificial dialing delay between
swarm1 and swarm2 should be re-evaluated and possibly at least
the maximum delay further reduced.
* Complete negotiation between upgrades in libp2p-core.
While multistream-select, as a standalone library and providing
an API at the granularity of a single negotiation, supports
lazy negotiation (and in particular 0-RTT negotiation), in the
context of libp2p-core where any number of negotiations are
composed generically within the concept of composable "upgrades",
it is necessary to wait for protocol negotiation between upgrades
to complete.
* Clarify docs. Simplify listener upgrades.
Since reading from a Negotiated I/O stream implicitly flushes any pending
negotiation data, there is no pitfall involved in not waiting for completion.
* Remove tokio-codec dependency from multistream-select.
In preparation for the eventual switch from tokio to std futures.
Includes some initial refactoring in preparation for further work
in the context of https://github.com/libp2p/rust-libp2p/issues/659.
* Reduce default buffer sizes.
* Allow more than one frame to be buffered for sending.
* Doc tweaks.
* Remove superfluous (duplicated) Message types.
* core/src/translation: Add unit tests
* core/src/translation: Support dns4 and dns6
Add dns4 and dns6 as valid protocol replacements for the origin address
to construct external addresses of a given node.
* core/nodes/network: %s/nat_translation/address_translation/
When receiving an observed address on a tcp connection that we initiated, the
observed address contains our tcp dial port, not our tcp listen port. We know
which port we are listening on, thereby we can replace the port within the
observed address.
When receiving an observed address on a tcp connection that we did **not**
initiated, the observed address should contain our listening port. In case it
differs from our listening port there might be a NAT along the path.
With the above in mind, the function name `nat_translation` is misleading.
* Somewhat complete the implementation of Kademlia records.
This commit relates to [libp2p-146] and [libp2p-1089].
* All records expire (by default, configurable).
* Provider records are also stored in the RecordStore, and the RecordStore
API extended.
* Background jobs for periodic (re-)replication and (re-)publication
of records. Regular (value-)records are subject to re-replication and
re-publication as per standard Kademlia. Provider records are only
subject to re-publication.
* For standard Kademlia value lookups (quorum = 1), the record is cached
at the closest peer to the key that did not return the value, as per
standard Kademlia.
* Expiration times of regular (value-)records is computed exponentially
inversely proportional to the number of nodes between the local node
and the closest node known to the key (beyond the k closest), as per
standard Kademlia.
The protobuf messages are extended with two fields: `ttl` and `publisher`
in order to implement the different semantics of re-replication (by any
of the k closest peers to the key, not affecting expiry) and re-publication
(by the original publisher, resetting the expiry). This is not done yet in
other libp2p Kademlia implementations, see e.g. [libp2p-go-323]. The new protobuf fields
have been given somewhat unique identifiers to prevent future collision.
Similarly, periodic re-publication of provider records does not seem to
be done yet in other implementations, see e.g. [libp2p-js-98].
[libp2p-146]: https://github.com/libp2p/rust-libp2p/issues/146
[libp2p-1089]: https://github.com/libp2p/rust-libp2p/issues/1089
[libp2p-go-323]: https://github.com/libp2p/go-libp2p-kad-dht/issues/323
[libp2p-js-98]: https://github.com/libp2p/js-libp2p-kad-dht/issues/98
* Tweak kad-ipfs example.
* Add missing files.
* Ensure new delays are polled immediately.
To ensure task notification, since `NotReady` is returned right after.
* Fix ipfs-kad example and use wasm_timer.
* Small cleanup.
* Incorporate some feedback.
* Adjustments after rebase.
* Distinguish events further.
In order for a user to easily distinguish the result of e.g.
a `put_record` operation from the result of a later republication,
different event constructors are used. Furthermore, for now,
re-replication and "caching" of records (at the closest peer to
the key that did not return a value during a successful lookup)
do not yield events for now as they are less interesting.
* Speed up tests for CI.
* Small refinements and more documentation.
* Guard a node against overriding records for which it considers
itself to be the publisher.
* Document the jobs module more extensively.
* More inline docs around removal of "unreachable" addresses.
* Remove wildcard re-exports.
* Use NonZeroUsize for the constants.
* Re-add method lost on merge.
* Add missing 'pub'.
* Further increase the timeout in the ipfs-kad example.
* Readd log dependency to libp2p-kad.
* Simplify RecordStore API slightly.
* Some more commentary.
* Change Addresses::remove to return Result<(),()>.
Change the semantics of `Addresses::remove` so that the error case
is unambiguous, instead of the success case. Use the `Result` for
clearer semantics to that effect.
* Add some documentation to .
* Address some TODOs, refactor queries and public API.
The following left-over issues are addressed:
* The key for FIND_NODE requests is generalised to any Multihash,
instead of just peer IDs.
* All queries get a (configurable) timeout.
* Finishing queries as soon as enough results have been received is simplified
to avoid code duplication.
* No more panics in provider-API-related code paths. The provider API is
however still untested and (I think) still incomplete (e.g. expiration
of provider records).
* Numerous smaller TODOs encountered in the code.
The following public API changes / additions are made:
* Introduce a `KademliaConfig` with new configuration options for
the replication factor and query timeouts.
* Rename `find_node` to `get_closest_peers`.
* Rename `get_value` to `get_record` and `put_value` to `put_record`,
introducing a `Quorum` parameter for both functions, replacing the
existing `num_results` parameter with clearer semantics.
* Rename `add_providing` to `start_providing` and `remove_providing`
to `stop_providing`.
* Add a `bootstrap` function that implements a (almost) standard
Kademlia bootstrapping procedure.
* Rename `KademliaOut` to `KademliaEvent` with an updated list of
constructors (some renaming). All events that report query results
now report a `Result` to uniformly permit reporting of errors.
The following refactorings are made:
* Introduce some constants.
* Consolidate `query.rs` and `write.rs` behind a common query interface
to reduce duplication and facilitate better code reuse, introducing
the notion of a query peer iterator. `query/peers/closest.rs`
contains the code that was formerly in `query.rs`. `query/peers/fixed.rs` contains
a modified variant of `write.rs` (which is removed). The new `query.rs`
provides an interface for working with a collection of queries, taking
over some code from `behaviour.rs`.
* Reduce code duplication in tests and use the current_thread runtime for
polling swarms to avoid spurious errors in the test output due to aborted
connections when a test finishes prematurely (e.g. because a quorum of
results has been collected).
* Some additions / improvements to the existing tests.
* Fix test.
* Fix rebase.
* Tweak kad-ipfs example.
* Incorporate some feedback.
* Provide easy access and conversion to keys in error results.
multihash: Use `Bytes` instead of `Vec<u8>` internally.
To improve the efficiency of cloning multi-hashes (e.g. as the
representation of `PeerId`s), this PR replaces the `Vec<u8>`
representation with `Bytes`. The API is kept backwards
compatible and does not leak the representation type.
* Begin reimplementing the websocket transport.
* Add TLS support.
* Add support for redirects during handshake.
* Cosmetics.
* Remove unused error cases in tls module.
Left-overs from a previous implementation.
* No libp2p-websocket for wasm targets.
* Change tls::Config to make the server optional.
* Update transports/websocket/src/lib.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Duplicate config methods.
As per PR review feedback.
* Add parity_multiaddr::from_websockets_url
* Make from_websockets_url more genreic
* Oops, forgot the file
* Use the URL crate
* Add dns_and_port test
* Address review
* Add support for unix
* Bump to 0.7.0
* Update CHANGELOG.md
Co-Authored-By: tomaka <pierre.krieger1708@gmail.com>
* Update for #1078
* New version of multihash and multiaddr as well
The current implementation does not properly reserve enough space for
`Write::write_all` to succeed. The property test has been improved to
trigger that issue.
The functionality is available through `Multiaddr::replace`.
What we currently call "nat_traversal" is merley a replacement of an IP
address prefix in a `Multiaddr`, hence it can be done directly on
`Multiaddr` values instead of having to go through a `Transport`.
In addition this PR consolidates changes made to `Multiaddr` in
previous commits which resulted in lots of deprecations. It adds some
more (see below for the complete list of API changes) and removes all
deprecated functionality, requiring a minor version bump.
Here are the changes to `multiaddr` compared to the currently published
version:
1. Removed `into_bytes` (use `to_vec` instead).
2. Renamed `to_bytes` to `to_vec`.
3. Removed `from_bytes` (use the `TryFrom` impl instead).
4. Added `with_capacity`.
5. Added `len`.
6. Removed `as_slice` (use `AsRef` impl instead).
7. Removed `encapsulate` (use `push` or `with` instead).
8. Removed `decapsulate` (use `pop` instead).
9. Renamed `append` to `push`.
10. Added `with`.
11. Added `replace`.
12. Removed `ToMultiaddr` trait (use `TryFrom` instead).
* Consolidate keypairs in core.
Introduce the concept of a node's identity keypair in libp2p-core,
instead of only the public key:
* New module: libp2p_core::identity with submodules for the currently
supported key types. An identity::Keypair and identity::PublicKey
support the creation and verification of signatures. The public key
supports encoding/decoding according to the libp2p specs.
* The secio protocol is simplified as a result of moving code to libp2p-core.
* The noise protocol is slightly simplified by consolidating ed25519
keypairs in libp2p-core and using x25519-dalek for DH. Furthermore,
Ed25519 to X25519 keypair conversion is now complete and tested.
Generalise over the DH keys in the noise protocol.
Generalise over the DH keys and thus DH parameter in handshake patterns
of the Noise protocol, such that it is easy to support other DH schemes
in the future, e.g. X448.
* Address new review comments.
* Fix error being ignored in RwSinkStream
* Add test for partial reading
* Update misc/rw-stream-sink/src/lib.rs
Co-Authored-By: tomaka <pierre.krieger1708@gmail.com>
