* initial implementation of the records
* move to multihash keys
* correctly process query results
* comments and formatting
* correctly return closer_peers in query
* checking wrong peer id in test
* Apply suggestions from code review
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Fix changes from suggestions
* Send responses to PUT_VALUE requests
* Shortcut in get_value
* Update protocols/kad/src/behaviour.rs
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Revert "Update protocols/kad/src/behaviour.rs"
This reverts commit 579ce742a7f4c94587f1e1f0866d2a3a37418efb.
* Remove duplicate insertion
* Adds a record to a PUT_VALUE response
* Fix a racy put_value test
* Store value ourselves only if we are in K closest
* Abstract over storage
* Revert "Abstract over storage": bad take
This reverts commit eaebf5b6d915712eaf3b05929577fdf697f204d8.
* Abstract over records storage using hashmap as default
* Constructor for custom records
* New Record type and its traits
* Fix outdated storage name
* Fixes returning an event
* Change FindNodeReq key type to Multihash
* WriteState for a second stage of a PUT_VALUE request
* GET_VALUE should not have a record
* Refactor a match arm
* Add successes and failures counters to PutValueRes
* If value is found no need to return closer peers
* Remove a custo storage from tests
* Rename a test to get_value_not_found
* Adds a TODO to change FindNode request key to Multihash
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Move MemoryRecordStorage to record.rs
* Return a Cow-ed Record from get
* Fix incorrect GET_VALUE parsing
* Various fixes with review
* Fixes get_value_not_found
* Fix peerids names in test
* another fix
* PutValue correctly distributes values
* Simplify the test
* Check that results are actually the closest
* Reverts changes to tests
* Fix the test topology and checking the results
* Run put_value test ten times
* Adds a get_value test
* Apply suggestions from code review
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Make Record fields public
* Moves WriteState to write.rs
* A couple of minor fixes
* Another few fixes of review
* Simplify the put_value test
* Dont synchronously return an error from put_value
* Formatting fixes and comments
* Collect a bunch of results
* Take exactly as much elements as neede
* Check if the peer is still connected
* Adds a multiple GetValueResults results number test
* Unnecessary mut iterators in put_value
* Ask for num_results in get_value
* Dont allocate twice in get_value
* Dont count same errored peer multiple times
* Apply suggestions from code review
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Fix another review
* Apply suggestions from code review
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Bring back FromIterator and improve a panic message
* Update protocols/kad/src/behaviour.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Begin reimplementing the websocket transport.
* Add TLS support.
* Add support for redirects during handshake.
* Cosmetics.
* Remove unused error cases in tls module.
Left-overs from a previous implementation.
* No libp2p-websocket for wasm targets.
* Change tls::Config to make the server optional.
* Update transports/websocket/src/lib.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Duplicate config methods.
As per PR review feedback.
* Some improvements to the docs of NetworkBehaviour
* Apply suggestions from code review
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Map all 127.0.0.0/8 addresses to 127.0.0.1.
Since every local socket address in the 127.0.0.0/8 space is looped back
to 127.0.0.1/32 we should only have to report the later as the listen
address. For other addresses we still attempt to discover host addresses
when we encounter an unknown local address. We now also check that after
the host addresses have been reset that the address is now found,
otherwise we produce an error.
* Change listen address lookup.
Perform multiple steps:
1. Check for exact address match.
2. Else consider netmask and check for containment.
3. Else re-check host addresses and try 1 & 2 again.
4. Else report an error.
* Small fixes.
* Test and improve prefix_len.
* Simplify and inline the prefix_len logic.
* Fix status updates in KBuckets.
`KBucket::update` does currently not correctly update the `first_connected_pos`
before reinserting the node. This may result in connected nodes being considered
disconnected and thus eligible for replacement by a pending node if the bucket is
full and a new connected node is added.
Tests have been added for checking that `KBucket::update` preserves the status
and ordering of all other nodes in the bucket.
* Small test improvement.
Set an expectation for the new position, instead of taking the assigned position.
* Kademlia: Optimise iteration over closest entries.
The current implementation for finding the entries whose keys are closest
to some target key in the Kademlia routing table involves copying the
keys of all buckets into a new `Vec` which is then sorted based on the
distances to the target and turned into an iterator from which only a
small number of elements (by default 20) are drawn.
This commit introduces an iterator over buckets for finding the closest
keys to a target that visits the buckets in the optimal order, based on
the information contained in the distance bit-string representing the
distance between the local key and the target.
Correctness is tested against full-table scans.
Also included:
* Updated documentation.
* The `Entry` API was moved to the `kbucket::entry` sub-module for
ease of maintenance.
* The pending node handling has been slightly refactored in order to
bring code and documentation in agreement and clarify the semantics
a little.
* Rewrite pending node handling and add tests.
There are two issues with the current definition and use of Kademlia's
XOR metric:
1. The distance is currently equated with the bucket index, i.e.
`distance(a,b) - 1` is the index of the bucket into which either
peer is put by the other. The result is a metric that is not
unidirectional, as defined in the Kademlia paper and as implemented
in e.g. libp2p-go and libp2p-js, which is to interpret the result
of the XOR as an integer in its entirety.
2. The current `KBucketsPeerId` trait and its instances allow computing
distances between types with differing bit lengths as well as between
types that hash all inputs again (i.e. `KadHash`) and "plain" `PeerId`s
or `Multihash`es. This can result in computed distances that are either
incorrect as per the requirement of the libp2p specs that all distances
are to be computed from the XOR of the SHA256 of the input keys, or
even fall outside of the image of the metric used for the `KBucketsTable`.
In the latter case, such distances are not currently used as a bucket index
- they can only occur in the context of comparing distances for the purpose
of sorting peers - but that still seems undesirable.
These issues are addressed here as follows:
* Unidirectionality of the XOR metric is restored by keeping the "full"
integer representation of the bitwise XOR. The result is an XOR metric
as defined in the paper. This also opens the door to avoiding the
"full table scan" when searching for the keys closest to a given key -
the ideal order in which to visit the buckets can be computed with the
help of the distance bit string.
* As a simplification and to make it easy to "do the right thing", the
XOR metric is only defined on an opaque `kbucket::Key` type, partially
derived from the current `KadHash`. `KadHash` and `KBucketsPeerId`
are removed.
* Replace `secp256k1` crate with `libsecp256k1`.
Unfortunately we could not implement `AsRef<[u8]>` for `SecretKey`
as the crate does not provide a means to do so.
* Fix `DecodingError` invocation.
* Remove the cc for wasm
* Revert "Remove the cc for wasm"
This reverts commit 3a19db35e62931c6e9ffbff6c21f9b0d7ae5403a.
* Fix CircleCI build
* Add parity_multiaddr::from_websockets_url
* Make from_websockets_url more genreic
* Oops, forgot the file
* Use the URL crate
* Add dns_and_port test
* Address review
* Add support for unix
* Integrate use of identity keys into libp2p-noise.
In order to make libp2p-noise usable with a `Swarm`, which requires a
`Transport::Output` that is a pair of a peer ID and an implementation
of `StreamMuxer`, it is necessary to bridge the gap between static
DH public keys and public identity keys from which peer IDs are derived.
Because the DH static keys and the identity keys need not be
related, it is thus generally necessary that the public identity keys are
exchanged as part of the Noise handshake, which the Noise protocol
accomodates for through the use of handshake message payloads.
The implementation of the existing (IK, IX, XX) handshake patterns is thus
changed to send the public identity keys in the handshake payloads.
Additionally, to facilitate the use of any identity keypair with Noise
handshakes, the static DH public keys are signed using the identity
keypairs and the signatures sent alongside the public identity key
in handshake payloads, unless the static DH public key is "linked"
to the public identity key by other means, e.g. when an Ed25519 identity
keypair is (re)used as an X25519 keypair.
* libp2p-noise doesn't build for wasm.
Thus the development transport needs to be still constructed with secio
for transport security when building for wasm.
* Documentation tweaks.
* For consistency, avoid wildcard enum imports.
* For consistency, avoid wildcard enum imports.
* Slightly simplify io:🤝:State::finish.
* Simplify creation of 2-byte arrays.
* Remove unnecessary cast and obey 100 char line limit.
* Update protocols/noise/src/protocol.rs
Co-Authored-By: romanb <romanb@users.noreply.github.com>
* Address more review comments.
* Cosmetics
* Cosmetics
* Give authentic DH keypairs a distinct type.
This has a couple of advantages:
* Signing the DH public key only needs to happen once, before
creating a `NoiseConfig` for an authenticated handshake.
* The identity keypair only needs to be borrowed and can be
dropped if it is not used further outside of the Noise
protocol, since it is no longer needed during Noise handshakes.
* It is explicit in the construction of a `NoiseConfig` for
a handshake pattern, whether it operates with a plain `Keypair`
or a keypair that is authentic w.r.t. a public identity key
and future handshake patterns may be built with either.
* The function signatures for constructing `NoiseConfig`s for
handshake patterns are simplified and a few unnecessary trait
bounds removed.
* Post-merge corrections.
* Add note on experimental status of libp2p-noise.
Although not explicitly mentioned in the paper, it seems clear that
including an entry for the requesting peer in a FIND_NODE response
never gives useful information and just occupies a result slot that may
have been better filled with another peer that the requestor may not
know about.
There is one explicit mention that this is the desired behavior
in a somewhat dated design document of another p2p framework [1]:
"The recipient of a FIND_NODE should never return a triple containing
the nodeID of the requestor."
The same reasoning supposedly applies to the libp2p-specific `GET_PROVIDERS`
request.
[1] http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html#FIND_NODE
* Fix self-dialing in Kademlia.
Addresses https://github.com/libp2p/rust-libp2p/issues/341 which is the cause
for one of the observations made in https://github.com/libp2p/rust-libp2p/issues/1053.
However, the latter is not assumed to be fully addressed by these changes and
needs further investigation.
Currently, whenever a search for a key yields a response containing the initiating
peer as one of the closest peers known to the remote, the local node
would attempt to dial itself. That attempt is ignored by the Swarm, but
the Kademlia behaviour now believes it still has a query ongoing which is
always doomed to time out. That timeout delays successful completion of the query.
Hence, any query where a remote responds with the ID of the local node takes at
least as long as the `rpc_timeout` to complete, which possibly affects almost
all queries in smaller clusters where every node knows about every other.
This problem is fixed here by ensuring that Kademlia never tries to dial the local node.
Furthermore, `Discovered` events are no longer emitted for the local node
and it is not inserted into the `untrusted_addresses` from discovery, as described
in #341.
This commit also includes a change to the condition for freezing / terminating
a Kademlia query upon receiving a response. Specifically, the condition is
tightened such that it only applies if in addition to `parallelism`
consecutive responses that failed to yield a peer closer to the target, the
last response must also either not have reported any new peer or the
number of collected peers has already reached the number of desired results.
In effect, a Kademlia query now tries harder to actually return `k`
closest peers.
Tests have been refactored and expanded.
* Add another comment.
