* Grow noise buffers dynamically.
Currently we allocate a buffer of 176 KiB for each noise state, i.e.
each connection. For connections which see only small data frames
this is wasteful. At the same time we limit the max. write buffer size
to 16 KiB to keep the total buffer size relatively small, which
results in smaller encrypted messages and also makes it less likely to
ever encounter the max. noise package size of 64 KiB in practice when
communicating with other nodes using the same implementation.
This PR repaces the static buffer allocation with a dynamic one. We
only reserve a small space for the authentication tag plus some extra
reserve and are able to buffer larger data frames before encrypting.
* Grow write buffer from offset.
As suggested by @mxinden, this prevents increasing the write buffer up
to MAX_WRITE_BUF_LEN.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Also implement `Deref` and `DerefMut` on `Config` to allow transparent
configuration of `yamux::Config`.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* hmm...
* Switch snow resolver to default
* Fix documentation
* Use the sha2 crate for sha512 hashing
* Use ring on native
* Use different features on different targets
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Add gossipsub and ping
* Implement swarm key parsing from environment
* WIP remove stuff
* WIP remove more stuff
* Use gossipsub instead of floodsub
* Make ipfs example work with or without swarm key
* Add support for /ipfs/Qm1234 multiaddrs
* Add documentation for ipfs example
* Rename example to ipfs-private
* Fix comments
* Move EitherTransport into either.rs
And prettify imports of ipfs-private example
* Sanitize multiaddr before parsing
...and remove the "ipfs" protocol from multiaddr
* Remove TSubstream type parameter
...so that it works with current master
* PR feedback
use source instead of cause
* Fix broken links in rustdoc
This fixes all of the rustdoc warnings on nightly.
* Check documentation intra-link
* Fix config
* Fix bad indent
* Make nightly explicit
* More links fixes
* Fix link broken after master merge
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
* Simplify trait bounds requirements
* More work
* Moar
* Finish
* Fix final tests
* More simplification
* Use separate traits for Inbound/Outbound
* Update gossipsub and remove warnings
* Add documentation to swarm
* Remove BoxSubstream
* Fix tests not compiling
* Fix stack overflow
* Address concerns
* For some reason my IDE ignored libp2p-kad
* Provides a debug instance for an RSA public key...
that is a bit easier on the eyes than the raw Vec<u8>
* Move to_hex into the fmt method
PR feedback
* Remove to_hex altogether
Instead of closing the connection on an error on the inbound substream
(both on `poll_next` as well as `poll_close`), one can set it to None
and depend on the remote to open an new one in case they have more data
to send.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Add peer id inlining for small public keys
* Apply @twittner suggestions
* Make PeerId compare equal accross hashes
* Fix mDNS
* Remove useless functions
* Add property test
Co-authored-by: Age Manning <Age@AgeManning.com>
* Add pnet protocol
copied from plaintext protocol, since that seems to be the closest match
* Minimalize the pnet protocol
* WIP private swarms with fixed key
* Different nonces for write and read
* Use per stream write buffer to avoid allocations
* Add parsing and formating of PSKs
* Directly call handshake
Also remove unneeded InboundUpgrade and OutboundUpgrade
* Add HandshakeError
* Add dedicated pnet example
* Add tests for PSK parsing and formatting
* Some more tests for the parsing, fail case
* Add fingerprint
To be able to check if a go-ipfs and rust-libp2p use the same key without
having to dump the actual key. Not sure if there is a spec for this anywhere,
but it is basically just copied from go-ipfs.
* Minimize dependencies and remove dead code
* Rename PSK to PreSharedKey and use pin_project
* Add crypt_writer
Basically a stripped down and modified version of async_std BufWriter that
also encrypts using the given cipher.
* cargo fmt
* Actually get rid of the Unpin requirement
* Rewrite flushing and remove written count from state
* Add docs for pnet/lib.rs
* Increase library version
* Remove pnet example
There will be a more elaborate and useful example in a different PR
* Return pending on pending...
also make doc text less ambiguous
* Add debug assertions to check invariants of poll_flush_buf
Also, clarify the invariants in the comments of that method
* Addressing #473 ... if I understood the ticket right, we want to pass through whatever the application provides as a topic identifier, leaving hashing (or not hashing) up to the application.
* Remove TopicDescriptor and use Topic newtype everywhere
* PR feedback
Use From<Topic> instead of Into<String>
Use impl Into<Topic> instead of Topic in public API
Co-authored-by: Peat Bakke <peat@peat.org>
* Create gossipsub crate - Basic template, borrowed from floodsub
* Add a GossipsubConfig struct and set up basic structures in the Gossipsub struct
* Begin implementation of join. Adds get_random_peers helper function and adds tests
* Implements gossipsub leave()
* Update publishMany to incorporate gossipsub mesh and fanout logic
* Use the gossipsub mesh for determining peer subscription
* Remove subscribed_topics field from the Gossipsub struct
* Rename gossipsubconfig to ProtocolConfig
* Implement the gossipsub control messages into the Codec's Encode/Decode and modifies GossipsubRpc
* Modify GossipsubActions to enums for succinctness.
* Modify the memcache to store Gossipsub messages
* Implement control message handling.
* Update control message handling to handle multiple messages.
* Handle received gossipsub messages using pre-built handlers.
* Remove excess connected peer hashmap
* Add extra peer mapping and consistent topic naming.
* Implement heartbeat, emit_gossip and send_graft_prune.
* Group logic in forwarding messages. Add messages to memcache.
* Add heartbeat timer and move location of helper function.
* Add gossipsub the libp2p workspace, makes layer structs public
* Add logging to gossipsub
- Adds the log crate and implements logging macros
- Specifies versions for external crates
* Add example chat for debugging purposes
* Implement #868 for gossipsub.
* Add rust documentation to gossipsub crate.
- Adds basic documentation, overview and examples to the gossipsub
crate.
* Re-introduce the initial heartbeat time config.
This commit also adds the inject_connected test.
* Add subscribe tests.
- Modifies `handle_received_subscriptions` to take a reference of
subscriptions
- Adds `test_subscribe`
- Adds `test_handle_received_subscriptions`
- Adds tests for the filter in `get_random_peers`
* Add Bug fixes and further testing for gossipsub.
- Corrects the tuple use of topic_hashes
- Corrects JOIN logic around fanout and adding peers to the mesh
- Adds test_unsubscribe
- Adds test_join
* Rename GossipsubMessage::msg_id -> id
* Add bug fix for handling disconnected peers.
* Implements (partially) #889 for Gossipsub.
* handle_iwant event count tests
* handle_ihave event count tests
* Move layer.rs tests into separate file.
* Implement clippy suggestions for gossipsub.
* Modify control message tests for specific types.
* Implement builder pattern for GossipsubConfig.
As suggested by @twittner - The builder pattern for building
GossipsubConfig struct is implemented.
* Package version updates as suggested by @twittner.
* Correct line lengths in gossipsub.
* Correct braces in found by @twittner.
* Implement @twittner's suggestions.
- Uses `HashSet` where applicable
- Update `FnvHashMap` to standard `HashMap`
- Uses `min` function in code simplification.
* Add NodeList struct to clarify topic_peers.
* Cleaner handling of messagelist
Co-Authored-By: AgeManning <Age@AgeManning.com>
* Cleaner handling of added peers.
Co-Authored-By: AgeManning <Age@AgeManning.com>
* handle_prune peer removed test
* basic grafting tests
* multiple topic grafting test
* Convert &vec to slice.
Co-Authored-By: AgeManning <Age@AgeManning.com>
* Convert to lazy insert.
Co-Authored-By: AgeManning <Age@AgeManning.com>
* Cleaner topic handling.
Co-Authored-By: AgeManning <Age@AgeManning.com>
* control pool piggybacking
using HashMap.drain() in control_pool_flush
going to squash this
* Add Debug derives to gossipsub and correct tests.
* changes from PR
squash this
all tests passing, but still some that need to be reconsidered
test reform
* Implements Arc for GossipsubRpc events
* Remove support for floodsub nodes
* Reconnected to disconnected peers, to mitigate timeout
* Use ReadOne WriteOne with configurable max gossip sizes
* Remove length delimination from RPC encoding
* Prevent peer duplication in mesh
* Allow oneshot handler's inactivity_timeout to be configurable
* Correct peer duplication in mesh bug
* Remove auto-reconnect to allow for user-level disconnects
* Single long-lived inbound/outbound streams to match go implementation
* Allow gossipsub topics to be optionally hashable
* Improves gossipsub stream handling
- Corrects the handler's keep alive.
- Correct the chat example.
- Instantly add peers to the mesh on subscription if the mesh is low.
* Allows message validation in gossipsub
* Replaces Cuckoofilter with LRUCache
The false positive rate was unacceptable for rejecting messages.
* Renames configuration parameter and corrects logic
* Removes peer from fanout on disconnection
* Add publish and fanout tests
* Apply @mxinden suggestions
* Resend message if outbound stream negotiated
- Downgrades log warnings
* Implement further reviewer suggestions
- Created associated functions to avoid unnecessary cloning
- Messages are rejected if their sequence numbers are not u64
- `GossipsbuConfigBuilder` has the same defaults as `GossipsubConfig`
- Miscellaneous typos
* Add MessageId type and remove unnecessary comments
* Add a return value to propagate_message function
* Adds user-customised gossipsub message ids
* Adds the message id to GossipsubEvent
* Implement Debug for GossipsubConfig
* protocols/gossipsub: Add basic smoke test
Implement a basic smoke test that:
1. Builds a fully connected graph of size N.
2. Subscribes each node to the same topic.
3. Publishes a single message.
4. Waits for all nodes to receive the above message.
N and the structure of the graph are reproducibly randomized via
Quickcheck.
* Corrections pointed out by @mxinden
* Add option to remove source id publishing
* protocols/gossipsub/tests/smoke: Remove unused variable
* Merge latest master
* protocols/gossipsub: Move to stable futures
* examples/gossipsub-chat.rs: Move to stable futures
* protocols/gossipsub/src/behaviour/tests: Update to stable futures
* protocols/gossipsub/tests: Update to stable futures
* protocols/gossipsub: Log substream errors
* protocols/gossipsub: Log outbound substream errors
* Remove rust-fmt formatting
* Shift to prost for protobuf compiling
* Use wasm_timer for wasm compatibility
Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Allow configuring the tasks executor
* Minor tweaks
* Add executor_fn
* Create ThreadsPool at the end if necessary
* Allow configuring the tasks executor
* Minor tweaks
* Add executor_fn
* Create ThreadsPool at the end if necessary
* WIP
* Don't depend on async-std and tokio in core
* Replace FutureObj with PinBoxFuture
* Some docs on Executor
* Fix tests
Co-authored-by: Toralf Wittner <tw@dtex.org>
* misc/mdns: Make MdnsService::new sync by using std::net::UdpSocket::bind
MdnsService uses an IP address to create a UDP socket. The address does
not need to be resolved. Therefore one can use std's UdpSocket::bind
instead of the async counterpart from async-std. As a result
MdnsService::new and MdnsService::silent don't need to be async.
* examples/mdns-passive-discovery: Don't await sync MdnsService::new
* misc/mdns/src/behaviour: Make Mdns::new sync
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* core/src/transport/memory: Add basic dialer listener test
Adds a basic test using two memory transport instances, one dialing the
other, sending a single message over the /wire/.
* core/src/nodes/network: Fix typo
* core/src/transport: Randomize listening port in unit test
* core/src/transport/memory: Update test to stable future
* core/src/transport/memory.rs: Use saturating_add for test port
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
* add docs about deriving NetworkBehaviour
* add commentary to the chat example
also minor stuff like reordering to match the struct elements, note why the listening address is
reported in poll.
* fix remove the warning on the example ignored field
* Apply suggestions from code review
Thanks for suggestions!
Co-Authored-By: Max Inden <mail@max-inden.de>
* chore cleanup added docs after suggestions
* fix print all listening addresses in examples/chat
not that there should be more than one but it makes sense to repeat the full example here.
* chore remove confusing writing on printing the listening addrs
* fix use intra-docs, spelling
Co-Authored-By: Roman S. Borschel <roman@parity.io>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
The current implementation defines the InboundUpgrade only for `R = ()`
so notably a `NoiseConfig` created by `NoiseConfig::ik_dialer` can not
be an `InboundUpgrade`.
Fixes#1378.
