Instead of creating unconstrained random number generators in quickcheck
tests to generate test data, have quickcheck provide a `Seed` to seed
those random number generators and thus make the test execution
deterministic / reproducible.
Make sure to decrease `num_waiting` when being notified of a peer
failure to allow an additional peer to be queried.
Given that `FixedPeersIter` is initialized with `replication_factor` by
`QueryPool` this bug will not surface today.
* protocols/kad/query/peers/closest: Consider K_VALUE nodes at init
By considering `K_VALUE` at `ClosestPeersIter` initialization, the initial peer
set length is independent of `num_results` and thus of the `replication_factor`.
* protocols/kad/src/behaviour/test: Enable building single nodes
Introduces the `build_node` function to build a single not connected
node. Along the way replace the notion of a `port_base` with returning
the actual `Multiaddr` of the node.
* protocols/kad/behaviour/test: Fix bootstrap test initialization
When looking for the closest node to a key, Kademlia considers
ALPHA_VALUE nodes to query at initialization. If `num_groups` is larger
than ALPHA_VALUE the remaining locally known nodes will not be
considered. Given that no other node is aware of them other than node 1,
they would be lost entirely. To prevent the above restrict `num_groups`
to be equal or smaller than ALPHA_VALUE.
* protocols/kad/behaviour/test: Fix put_record and get_provider
In the past, when trying to find the closest nodes to a key, Kademlia
would consider `num_result` amount of nodes to query out of all the
nodes it is aware of.
Both the `put_record` and the `get_provider` tests initialized their
swarms in the same way. The tests took the replication factor to use as
an input. The number of results to get was equal to the replication
factor. The amount of swarms to start was twice the replication factor.
Nodes would be grouped in two groups a replication factor nodes. The
first node would be aware of all of the nodes in the first group. The
last node of the first group would be aware of all the nodes in the
second group.
By coincidence (I assume) these numbers played together very well. At
initialization the first node would consider `num_results` amount of
peers (see first paragraph). It would then contact each of them. As the
first node is aware of the last node of the first group which in turn is
aware of all nodes in the second group, the first node would eventually
discover all nodes.
Recently the amount of nodes Kademlia considers at initialization when
looking for the nodes closest to a key was changed to only consider
ALPHA nodes.
With this in mind playing through the test setup above again would
result in (1) `replication_factor - ALPHA` nodes being entirely lost as
the first node would never consider them and (2) the first node probably
never contacting the last node out of the first group and thus not
discovering any nodes of the second group.
To keep the multi hop discovery in place while not basing ones test
setup on the lucky assumption of Kademlia considering replication factor
amount of nodes at initialization, this patch alters the two tests:
Build a fully connected set of nodes and one addition node (the first
node). Connect the first node to a single node of the fully connected
set (simulating a boot node). Continue as done previously.
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
When not making progress for `parallelism` time a `ClosestPeersIter`
becomes `State::Stalled`. When stalled an iterator is allowed to make
more parallel requests up to `num_results`. If `num_results` is smaller
than `parallelism` make sure to still allow up to `parallelism` requests
in-flight.
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
A node receiving a `GetRecord` request first checks whether it has the
given record. If it does have the record it does not return closer
nodes.
A node that knows the record for the given key is likely within a
neighborhood of nodes that know the record as well. In addition the node
likely knows its neighboorhood well.
When querying for a key with a quorum of 1 the above behavior of only
returning the record but not any close peers is fine. Once one queries
with a higher quorum having a node respond with the record as well as
close nodes is likely going to speed up the query, given that the
returned peers probably know the record as well.
* [libp2p-swarm] Make the multiple connections per peer first-class.
This commit makes the notion of multiple connections per peer
first-class in the API of libp2p-swarm, introducing the new
callbacks `inject_connection_established` and
`inject_connection_closed`. The `endpoint` parameter from
`inject_connected` and `inject_disconnected` is removed,
since the first connection to open may not be the last
connection to close, i.e. it cannot be guaranteed,
as was previously the case, that the endpoints passed
to these callbacks match up.
* Have identify track all addresses.
So that identify requests can be answered with the correct
observed address of the connection on which the request
arrives.
* Cleanup
* Cleanup
* Improve the `Peer` state API.
* Remove connection ID from `SwarmEvent::Dialing`.
* Mark `DialPeerCondition` non-exhaustive.
* Re-encapsulate `NetworkConfig`.
To retain the possibility of not re-exposing all
network configuration choices, thereby providing
a more convenient API on the \`SwarmBuilder\`.
* Rework Swarm::dial API.
* Update CHANGELOG.
* Doc formatting tweaks.
Given that the order of `PeerId`s within the `GetProvidersOk.providers`
set is irrelevant but duplication is at best confusing this commit makes
use of a `HashSet` instead of a `Vec` to return unique `PeerId`s only.
* protocols/kad: Do not attempt to store expired record in record store
`Kademlia::record_received` calculates the expiration time of a record
before inserting it into the record store. Instead of inserting the
record into the record store in any case, with this patch the record is
only inserted if it is not expired. If the record is expired a
`KademliaHandlerIn::Reset` for the given (sub) stream is triggered.
This would serve as a tiny defense mechanism against an attacker trying
to fill a node's record store with expired records before the record
store's clean up procedure removes the records.
* protocols/kad: Send regular ack when record discarded due to expiration
With this commit the remote receives a
[`KademliaHandlerIn::PutRecordRes`] even in the case where the record is
discarded due to being expired. Given that the remote sent the local
node a [`KademliaHandlerEvent::PutRecord`] request, the remote perceives
the local node as one node among the k closest nodes to the target.
Returning a [`KademliaHandlerIn::Reset`] instead of an
[`KademliaHandlerIn::PutRecordRes`] to have the remote try another node
would only result in the remote node to contact an even more distant
node. In addition returning [`KademliaHandlerIn::PutRecordRes`] does not
reveal any internal information to a possibly malicious remote node.
* protocols/kad/src/behaviour: Use `now` and reword expiration comment
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* protocols/kad: Add test to reproduce right shift overflow panic
* protocols/kad: Fix right shift overflow panic in record_received
Within `Behaviour::record_received` the exponentially decreasing
expiration based on the distance to the target for a record is
calculated as following:
1. Calculate the amount of nodes between us and the record key beyond
the k replication constant as `n`.
2. Shift the configured record time-to-live `n` times to the right to
calculate an exponentially decreasing expiration.
The configured record time-to-live is a u64. If `n` is larger or equal
to 64 the right shift will lead to an overflow which panics in debug
mode.
This patch uses a checked right shift instead, defaulting to 0 (`now +
0`) for the expiration on overflow.
* protocols/kad: Put attribute below comment
* protocols/kad: Extract shifting logic and rework test
Extract right shift into isolated function and replace complex
regression test with small isolated one.
* protocols/kad/src/behaviour: Refactor exp_decr_expiration
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* Allow multiple connections per peer in libp2p-core.
Instead of trying to enforce a single connection per peer,
which involves quite a bit of additional complexity e.g.
to prioritise simultaneously opened connections and can
have other undesirable consequences [1], we now
make multiple connections per peer a feature.
The gist of these changes is as follows:
The concept of a "node" with an implicit 1-1 correspondence
to a connection has been replaced with the "first-class"
concept of a "connection". The code from `src/nodes` has moved
(with varying degrees of modification) to `src/connection`.
A `HandledNode` has become a `Connection`, a `NodeHandler` a
`ConnectionHandler`, the `CollectionStream` was the basis for
the new `connection::Pool`, and so forth.
Conceptually, a `Network` contains a `connection::Pool` which
in turn internally employs the `connection::Manager` for
handling the background `connection::manager::Task`s, one
per connection, as before. These are all considered implementation
details. On the public API, `Peer`s are managed as before through
the `Network`, except now the API has changed with the shift of focus
to (potentially multiple) connections per peer. The `NetworkEvent`s have
accordingly also undergone changes.
The Swarm APIs remain largely unchanged, except for the fact that
`inject_replaced` is no longer called. It may now practically happen
that multiple `ProtocolsHandler`s are associated with a single
`NetworkBehaviour`, one per connection. If implementations of
`NetworkBehaviour` rely somehow on communicating with exactly
one `ProtocolsHandler`, this may cause issues, but it is unlikely.
[1]: https://github.com/paritytech/substrate/issues/4272
* Fix intra-rustdoc links.
* Update core/src/connection/pool.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Address some review feedback and fix doc links.
* Allow responses to be sent on the same connection.
* Remove unnecessary remainders of inject_replaced.
* Update swarm/src/behaviour.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update swarm/src/lib.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update core/src/connection/manager.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update core/src/connection/manager.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update core/src/connection/pool.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Incorporate more review feedback.
* Move module declaration below imports.
* Update core/src/connection/manager.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Update core/src/connection/manager.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Simplify as per review.
* Fix rustoc link.
* Add try_notify_handler and simplify.
* Relocate DialingConnection and DialingAttempt.
For better visibility constraints.
* Small cleanup.
* Small cleanup. More robust EstablishedConnectionIter.
* Clarify semantics of `DialingPeer::connect`.
* Don't call inject_disconnected on InvalidPeerId.
To preserve the previous behavior and ensure calls to
`inject_disconnected` are always paired with calls to
`inject_connected`.
* Provide public ConnectionId constructor.
Mainly needed for testing purposes, e.g. in substrate.
* Move the established connection limit check to the right place.
* Clean up connection error handling.
Separate connection errors into those occuring during
connection setup or upon rejecting a newly established
connection (the `PendingConnectionError`) and those
errors occurring on previously established connections,
i.e. for which a `ConnectionEstablished` event has
been emitted by the connection pool earlier.
* Revert change in log level and clarify an invariant.
* Remove inject_replaced entirely.
* Allow notifying all connection handlers.
Thereby simplify by introducing a new enum `NotifyHandler`,
used with a single constructor `NetworkBehaviourAction::NotifyHandler`.
* Finishing touches.
Small API simplifications and code deduplication.
Some more useful debug logging.
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
The `QueryId` type should be exported as it is used in the
`NetworkBehaviour::ProtocolsHandler` type of `Kademlia`.
`Kademlia::protocol_name` is added for convenience.
* Replace some remaining `AsRef` constraints for DHT keys with `Borrow`.
* Add a bit of debug/trace logging.
* Tiny refactoring and a debug assertion for the `bucket` module.
* Fix broken links in rustdoc
This fixes all of the rustdoc warnings on nightly.
* Check documentation intra-link
* Fix config
* Fix bad indent
* Make nightly explicit
* More links fixes
* Fix link broken after master merge
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
* Simplify trait bounds requirements
* More work
* Moar
* Finish
* Fix final tests
* More simplification
* Use separate traits for Inbound/Outbound
* Update gossipsub and remove warnings
* Add documentation to swarm
* Remove BoxSubstream
* Fix tests not compiling
* Fix stack overflow
* Address concerns
* For some reason my IDE ignored libp2p-kad
- Pin `futures_codec` to version 0.3.3 as later versions require
at least bytes-0.5 which he have not upgraded to yet.
- Replace `futures::executor::block_on` with `async_std::task::block_on`
where `async-std` is already a dependency to work around an issue with
`park`/`unpark` behaviour.
- Use the published version of `quicksink`.
* Implement Debug for (ed25519|secp256k1)::(Keypair|SecretKey) (#1285)
* Fix possible arithmetic overflow in libp2p-kad. (#1291)
When the number of active queries exceeds the (internal)
JOBS_MAX_QUERIES limit, which is only supposed to bound
the number of concurrent queries relating to background
jobs, an arithmetic overflow occurs. This is fixed by
using saturating subtraction.
* protocols/plaintext: Add example on how to upgrade with PlainTextConfig1 (#1286)
* [mdns] - Support for long mDNS names (Bug #1232) (#1287)
* Dead code -- commenting out with a note referencing future implementation
* Adding "std" feature so that cargo can build in other directories (notably `misc/mdns`, so that I could run these tests)
* Permitting `PeerID` to be built from an `Identity` multihash
* The length limit for DNS labels is 63 characters, as per RFC1035
* Allocates the vector with capacity for the service name plus additional QNAME encoding bytes
* Added support for encoding/decoding peer IDs with an encoded length greater than 63 characters
* Removing "std" from ring features
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Retaining MAX_INLINE_KEY_LENGTH with comment about future usage
* `segment_peer_id` consumes `peer_id` ... plus an early return for IDs that don't need to be segmented
* Fixing logic
* Bump most dependencies (#1268)
* Bump most dependencies
This actually builds 😊.
* Bump all dependencies
Includes the excellent work of @rschulman in #1265.
* Remove use of ed25519-dalek fork
* Monomorphize more dependencies
* Add compatibility hack for rand
Cargo allows a crate to depend on multiple versions of another, but
`cargo-web` panics in that situation. Use a wrapper crate to work
around the panic.
* Use @tomaka’s idea for using a newer `rand`
instead of my own ugly hack.
* Switch to Parity master
as its dependency-bumping PR has been merged.
* Update some depenendencies again
* Remove unwraps and `#[allow(deprecated)]`.
* Remove spurious changes to dependencies
Bumping minor or patch versions is not needed, and increases likelyhood
of merge conflicts.
* Remove some redundant Cargo.toml changes
* Replace a retry loop with an expect
`ed25519::SecretKey::from_bytes` will never fail for 32-byte inputs.
* Revert changes that don’t belong in this PR
* Remove using void to bypass ICE (#1295)
* Publish 0.13.0 (#1294)
* Bump most dependencies
This actually builds 😊.
* Bump all dependencies
Includes the excellent work of @rschulman in #1265.
* Remove use of ed25519-dalek fork
* Monomorphize more dependencies
* Add compatibility hack for rand
Cargo allows a crate to depend on multiple versions of another, but
`cargo-web` panics in that situation. Use a wrapper crate to work
around the panic.
* Use @tomaka’s idea for using a newer `rand`
instead of my own ugly hack.
* Switch to Parity master
as its dependency-bumping PR has been merged.
* Update some depenendencies again
* Remove unwraps and `#[allow(deprecated)]`.
* Remove spurious changes to dependencies
Bumping minor or patch versions is not needed, and increases likelyhood
of merge conflicts.
* Remove some redundant Cargo.toml changes
* Replace a retry loop with an expect
`ed25519::SecretKey::from_bytes` will never fail for 32-byte inputs.
* Revert changes that don’t belong in this PR
When the number of active queries exceeds the (internal)
JOBS_MAX_QUERIES limit, which is only supposed to bound
the number of concurrent queries relating to background
jobs, an arithmetic overflow occurs. This is fixed by
using saturating subtraction.
* Configurable multistream-select protocol. Add V1Lazy variant. (#1245)
Make the multistream-select protocol (version) configurable
on transport upgrades as well as for individual substreams.
Add a "lazy" variant of multistream-select 1.0 that delays
sending of negotiation protocol frames as much as possible
but is only safe to use under additional assumptions that
go beyond what is required by the multistream-select v1
specification.
* Improve the code readability of the chat example (#1253)
* Add bridged chats (#1252)
* Try fix CI (#1261)
* Print Rust version on CI
* Don't print where not appropriate
* Change caching strategy
* Remove win32 build
* Remove win32 from list
* Update libsecp256k1 dep to 0.3.0 (#1258)
* Update libsecp256k1 dep to 0.3.0
* Sign now cannot fail
* Upgrade url and percent-encoding deps to 2.1.0 (#1267)
* Upgrade percent-encoding dep to 2.1.0
* Upgrade url dep to 2.1.0
* Revert CIPHERS set to null (#1273)
* Update dependency versions (#1265)
* Update versions of many dependencies
* Bump version of rand
* Updates for changed APIs in rand, ring, and webpki
* Replace references to `snow::Session`
`Session` no longer exists in `snow` but the replacement is two structs `HandshakeState` and `TransportState`
Something will have to be done to harmonize `NoiseOutput.session`
* Add precise type for UnparsedPublicKey
* Update data structures/functions to match new snow's API
* Delete diff.diff
Remove accidentally committed diff file
* Remove commented lines in identity/rsa.rs
* Bump libsecp256k1 to 0.3.1
* Implement /plaintext/2.0.0 (#1236)
* WIP
* plaintext/2.0.0
* Refactor protobuf related issues to compatible with the spec
* Rename: new PlainTextConfig -> PlainText2Config
* Keep plaintext/1.0.0 as PlainText1Config
* Config contains pubkey
* Rename: proposition -> exchange
* Add PeerId to Exchange
* Check the validity of the remote's `Exchange`
* Tweak
* Delete unused import
* Add debug log
* Delete unused field: public_key_encoded
* Delete unused field: local
* Delete unused field: exchange_bytes
* The inner instance should not be public
* identity::Publickey::Rsa is not available on wasm
* Delete PeerId from Config as it should be generated from the pubkey
* Catch up for #1240
* Tweak
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update protocols/plaintext/src/handshake.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Update protocols/plaintext/src/error.rs
Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>
* Rename: pubkey -> local_public_key
* Delete unused error
* Rename: PeerIdValidationFailed -> InvalidPeerId
* Fix: HandShake -> Handshake
* Use bytes insteadof Publickey to avoid code duplication
* Replace with ProtobufError
* Merge HandshakeContext<()> into HandshakeContext<Local>
* Improve the peer ID validation to simplify the handshake
* Propagate Remote to allow extracting the PeerId from the Remote
* Collapse the same kind of errors into the variant
* [noise]: `sodiumoxide 0.2.5` (#1276)
Fixes https://github.com/RustSec/advisory-db/pull/192
* examples/ipfs-kad.rs: Remove outdated reference to `without_init` (#1280)
* CircleCI Test Fix (#1282)
* Disabling "Docker Layer Caching" because it breaks one of the circleci checks
* Bump to trigger CircleCI build
* unbump
* zeroize: Upgrade to v1.0 (#1284)
v1.0 final release is out. Release notes:
https://github.com/iqlusioninc/crates/pull/279
* *: Consolidate protobuf scripts and update to rust-protobuf 2.8.1 (#1275)
* *: Consolidate protobuf generation scripts
* *: Update to rust-protobuf 2.8.1
* *: Mark protobuf generated modules with '_proto'
* examples: Add distributed key value store (#1281)
* examples: Add distributed key value store
This commit adds a basic distributed key value store supporting GET and
PUT commands using Kademlia and mDNS.
* examples/distributed-key-value-store: Fix typo
* Simple Warning Cleanup (#1278)
* Cleaning up warnings - removing unused `use`
* Cleaning up warnings - unused tuple value
* Cleaning up warnings - removing dead code
* Cleaning up warnings - fixing deprecated name
* Cleaning up warnings - removing dead code
* Revert "Cleaning up warnings - removing dead code"
This reverts commit f18a765e4bf240b0ed9294ec3ae5dab5c186b801.
* Enable the std feature of ring (#1289)
