1261 Commits

Author SHA1 Message Date
Pierre Krieger
bbed28b3ec
Make errors on listener non-fatal (#1427)
* Make errors on listener non-fatal

* Fix bad rename

* Some changes to trait bounds

* Fix noise tests

* Apply suggestions from code review

Co-Authored-By: Toralf Wittner <tw@dtex.org>

* Add reason for closure

* Fix intra-doc link

Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-02-13 12:05:45 +01:00
vms
e2ddab223d
add poll parameters to custom poll function (#1442)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-13 11:17:55 +01:00
Toralf Wittner
cdb733b479
yamux: Re-export WindowUpdateMode. (#1443)
Also implement `Deref` and `DerefMut` on `Config` to allow transparent
configuration of `yamux::Config`.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-13 10:57:29 +01:00
Roman Borschel
b872bd9030
Temporary canonical multihash in peer ID. (#1449)
* Temporary canonical multihash in peer ID.

* Reduce code duplication.

* Remove unnecessary impls.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-13 10:36:14 +01:00
Ashley
8238fd2100
Switch noise from the RingResolver to the DefaultResolver (#1439)
* hmm...

* Switch snow resolver to default

* Fix documentation

* Use the sha2 crate for sha512 hashing

* Use ring on native

* Use different features on different targets

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-12 21:44:50 +01:00
Toralf Wittner
afb1c877a3
multiaddr: Fix generation of arbitrary onion addr. (#1451)
Exclude port 0 which is not a valid port number for onion addresses.
2020-02-12 20:27:17 +01:00
Pierre Krieger
e855cd5915
Use SHA256 again for from_public_key (#1444) 2020-02-12 15:28:40 +01:00
Pierre Krieger
955352bfd2
Extract rw-stream-sink to its own repo (#1448) 2020-02-12 13:23:08 +01:00
Pierre Krieger
627e93ba17
Propagate rustdoc links check to GH actions (#1446)
* Propagate rustdoc links check to GH actions

* Fix WASM test

* Try fix WASM test again

* Try run tests on a real machine
2020-02-11 16:28:29 +01:00
Pierre Krieger
cbb3315822
Try to use GitHub actions (#1430) 2020-02-11 15:09:45 +01:00
Rüdiger Klaehn
d55871623c
Ipfs private swarms interop example (#1420)
* Add gossipsub and ping

* Implement swarm key parsing from environment

* WIP remove stuff

* WIP remove more stuff

* Use gossipsub instead of floodsub

* Make ipfs example work with or without swarm key

* Add support for /ipfs/Qm1234 multiaddrs

* Add documentation for ipfs example

* Rename example to ipfs-private

* Fix comments

* Move EitherTransport into either.rs

And prettify imports of ipfs-private example

* Sanitize multiaddr before parsing

...and remove the "ipfs" protocol from multiaddr

* Remove TSubstream type parameter

...so that it works with current master

* PR feedback

use source instead of cause
2020-02-10 18:35:51 +01:00
Pierre Krieger
fc4dec581e
Check documentation intra-link (#1432)
* Fix broken links in rustdoc

This fixes all of the rustdoc warnings on nightly.

* Check documentation intra-link

* Fix config

* Fix bad indent

* Make nightly explicit

* More links fixes

* Fix link broken after master merge

Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
2020-02-10 15:17:07 +01:00
Pierre Krieger
1eff4b9823
Simplify trait bounds on NetworkBehaviour (#1405)
* Simplify trait bounds requirements

* More work

* Moar

* Finish

* Fix final tests

* More simplification

* Use separate traits for Inbound/Outbound

* Update gossipsub and remove warnings

* Add documentation to swarm

* Remove BoxSubstream

* Fix tests not compiling

* Fix stack overflow

* Address concerns

* For some reason my IDE ignored libp2p-kad
2020-02-07 16:29:30 +01:00
Akihito Nakano
69852a580b
Add dnsaddr (#1356)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-06 13:54:50 +01:00
Shotaro Yamada
b200728443
Fix copy-paste error in doc string (#1428)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-06 11:17:05 +01:00
Rüdiger Klaehn
39794b333f
Provides a debug instance for an RSA public key... (#1421)
* Provides a debug instance for an RSA public key...

that is a bit easier on the eyes than the raw Vec<u8>

* Move to_hex into the fmt method

PR feedback

* Remove to_hex altogether
2020-02-05 19:50:13 +01:00
Shotaro Yamada
f2d7091376
Remove unused dependencies (#1429)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 17:04:04 +01:00
Max Inden
2b7c015aa5
protocols/gossipsub: Do not close connection when inbound fails (#1424)
Instead of closing the connection on an error on the inbound substream
(both on `poll_next` as well as `poll_close`), one can set it to None
and depend on the remote to open an new one in case they have more data
to send.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 15:31:50 +01:00
Ashley
c73a175963
Switch to futures-timer 3.0 (#1426) 2020-02-05 12:12:18 +01:00
Pierre Krieger
7c04f7e468
Bump multihash to 0.2.2 (#1425) 2020-02-04 15:18:20 +01:00
Stan Bondi
955b60796a
Added Onion3 support to multiaddr (#1354) 2020-02-04 13:40:57 +01:00
Piotr Gołąb
57fd677686
Update gossipsub-chat example docstring (#1419)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-04 11:45:06 +01:00
Qinxuan Chen
013b72791a
Add Blake2b256 and Blake2s128 (#1386)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-04 11:17:09 +01:00
Rüdiger Klaehn
dd0c5c37f6
Allow multiple pings over one connection (#1416)
* Allow multiple pings over one connection

This is so that pinging a rust-libp2p node from ipfs works even without passing -n 1

* Remove timeout
2020-02-04 10:28:00 +01:00
Toralf Wittner
44b5e3e176
Bump yamux version 0.4.1 (#1415) 2020-01-31 10:27:20 +01:00
Pierre Krieger
b964cacfa4
Finish PeerId inlining change (#1413)
* Add peer id inlining for small public keys

* Apply @twittner suggestions

* Make PeerId compare equal accross hashes

* Fix mDNS

* Remove useless functions

* Add property test

Co-authored-by: Age Manning <Age@AgeManning.com>
2020-01-29 11:33:19 +01:00
Rüdiger Klaehn
f5e7461cec Implement private networks (#1385)
* Add pnet protocol

copied from plaintext protocol, since that seems to be the closest match

* Minimalize the pnet protocol

* WIP private swarms with fixed key

* Different nonces for write and read

* Use per stream write buffer to avoid allocations

* Add parsing and formating of PSKs

* Directly call handshake

Also remove unneeded InboundUpgrade and OutboundUpgrade

* Add HandshakeError

* Add dedicated pnet example

* Add tests for PSK parsing and formatting

* Some more tests for the parsing, fail case

* Add fingerprint

To be able to check if a go-ipfs and rust-libp2p use the same key without
having to dump the actual key. Not sure if there is a spec for this anywhere,
but it is basically just copied from go-ipfs.

* Minimize dependencies and remove dead code

* Rename PSK to PreSharedKey and use pin_project

* Add crypt_writer

Basically a stripped down and modified version of async_std BufWriter that
also encrypts using the given cipher.

* cargo fmt

* Actually get rid of the Unpin requirement

* Rewrite flushing and remove written count from state

* Add docs for pnet/lib.rs

* Increase library version

* Remove pnet example

There will be a more elaborate and useful example in a different PR

* Return pending on pending...

also make doc text less ambiguous

* Add debug assertions to check invariants of poll_flush_buf

Also, clarify the invariants in the comments of that method
2020-01-28 13:22:09 +01:00
Pierre Krieger
4bf42d188f
Remove all mod.rs files (#1400) 2020-01-28 08:51:06 +01:00
Rüdiger Klaehn
3b50cbd1b8 Addressing #473 ... if I understood the ticket right, we want to pass… (#1395)
* Addressing #473 ... if I understood the ticket right, we want to pass through whatever the application provides as a topic identifier, leaving hashing (or not hashing) up to the application.

* Remove TopicDescriptor and use Topic newtype everywhere

* PR feedback

Use From<Topic> instead of Into<String>
Use impl Into<Topic> instead of Topic in public API

Co-authored-by: Peat Bakke <peat@peat.org>
2020-01-27 15:23:01 +01:00
Pierre Krieger
2ef7c40cda
Add metadata to gossipsub (#1409) v0.15.0 2020-01-24 17:57:54 +01:00
Pierre Krieger
f1f40956d6
Publish 0.15.0 (#1408)
* Publish 0.15.0

* Oops, script too efficient
2020-01-24 17:14:56 +01:00
Pierre Krieger
0a45f7310f
Support tokio in libp2p-tcp and libp2p-uds (#1402) 2020-01-24 16:40:48 +01:00
Age Manning
37c7d73b11 Gossipsub Protocol (#898)
* Create gossipsub crate - Basic template, borrowed from floodsub

* Add a GossipsubConfig struct and set up basic structures in the Gossipsub struct

* Begin implementation of join. Adds get_random_peers helper function and adds tests

* Implements gossipsub leave()

* Update publishMany to incorporate gossipsub mesh and fanout logic

* Use the gossipsub mesh for determining peer subscription

* Remove subscribed_topics field from the Gossipsub struct

* Rename gossipsubconfig to ProtocolConfig

* Implement the gossipsub control messages into the Codec's Encode/Decode and modifies GossipsubRpc

* Modify GossipsubActions to enums for succinctness.

* Modify the memcache to store Gossipsub messages

* Implement control message handling.

* Update control message handling to handle multiple messages.

* Handle received gossipsub messages using pre-built handlers.

* Remove excess connected peer hashmap

* Add extra peer mapping and consistent topic naming.

* Implement heartbeat, emit_gossip and send_graft_prune.

* Group logic in forwarding messages. Add messages to memcache.

* Add heartbeat timer and move location of helper function.

* Add gossipsub the libp2p workspace, makes layer structs public

* Add logging to gossipsub

- Adds the log crate and implements logging macros
- Specifies versions for external crates

* Add example chat for debugging purposes

* Implement #868 for gossipsub.

* Add rust documentation to gossipsub crate.

- Adds basic documentation, overview and examples to the gossipsub
crate.

* Re-introduce the initial heartbeat time config.

This commit also adds the inject_connected test.

* Add subscribe tests.

- Modifies `handle_received_subscriptions` to take a reference of
subscriptions
- Adds `test_subscribe`
- Adds `test_handle_received_subscriptions`
- Adds tests for the filter in `get_random_peers`

* Add Bug fixes and further testing for gossipsub.

- Corrects the tuple use of topic_hashes
- Corrects JOIN logic around fanout and adding peers to the mesh
- Adds test_unsubscribe
- Adds test_join

* Rename GossipsubMessage::msg_id -> id

* Add bug fix for handling disconnected peers.

* Implements (partially) #889 for Gossipsub.

* handle_iwant event count tests

* handle_ihave event count tests

* Move layer.rs tests into separate file.

* Implement clippy suggestions for gossipsub.

* Modify control message tests for specific types.

* Implement builder pattern for GossipsubConfig.

As suggested by @twittner - The builder pattern for building
GossipsubConfig struct is implemented.

* Package version updates as suggested by @twittner.

* Correct line lengths in gossipsub.

* Correct braces in  found by @twittner.

* Implement @twittner's suggestions.

- Uses `HashSet` where applicable
- Update `FnvHashMap` to standard `HashMap`
- Uses `min` function in code simplification.

* Add NodeList struct to clarify topic_peers.

* Cleaner handling of messagelist

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner handling of added peers.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* handle_prune peer removed test

* basic grafting tests

* multiple topic grafting test

* Convert &vec to slice.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Convert to lazy insert.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner topic handling.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* control pool piggybacking

using HashMap.drain() in control_pool_flush

going to squash this

* Add Debug derives to gossipsub and correct tests.

* changes from PR

squash this

all tests passing, but still some that need to be reconsidered

test reform

* Implements Arc for GossipsubRpc events

* Remove support for floodsub nodes

* Reconnected to disconnected peers, to mitigate timeout

* Use ReadOne WriteOne with configurable max gossip sizes

* Remove length delimination from RPC encoding

* Prevent peer duplication in mesh

* Allow oneshot handler's inactivity_timeout to be configurable

* Correct peer duplication in mesh bug

* Remove auto-reconnect to allow for user-level disconnects

* Single long-lived inbound/outbound streams to match go implementation

* Allow gossipsub topics to be optionally hashable

* Improves gossipsub stream handling

- Corrects the handler's keep alive.
- Correct the chat example.
- Instantly add peers to the mesh on subscription if the mesh is low.

* Allows message validation in gossipsub

* Replaces Cuckoofilter with LRUCache

The false positive rate was unacceptable for rejecting messages.

* Renames configuration parameter and corrects logic

* Removes peer from fanout on disconnection

* Add publish and fanout tests

* Apply @mxinden suggestions

* Resend message if outbound stream negotiated

- Downgrades log warnings

* Implement further reviewer suggestions

- Created associated functions to avoid unnecessary cloning
- Messages are rejected if their sequence numbers are not u64
- `GossipsbuConfigBuilder` has the same defaults as `GossipsubConfig`
- Miscellaneous typos

* Add MessageId type and remove unnecessary comments

* Add a return value to propagate_message function

* Adds user-customised gossipsub message ids

* Adds the message id to GossipsubEvent

* Implement Debug for GossipsubConfig

* protocols/gossipsub: Add basic smoke test

Implement a basic smoke test that:

1. Builds a fully connected graph of size N.

2. Subscribes each node to the same topic.

3. Publishes a single message.

4. Waits for all nodes to receive the above message.

N and the structure of the graph are reproducibly randomized via
Quickcheck.

* Corrections pointed out by @mxinden

* Add option to remove source id publishing

* protocols/gossipsub/tests/smoke: Remove unused variable

* Merge latest master

* protocols/gossipsub: Move to stable futures

* examples/gossipsub-chat.rs: Move to stable futures

* protocols/gossipsub/src/behaviour/tests: Update to stable futures

* protocols/gossipsub/tests: Update to stable futures

* protocols/gossipsub: Log substream errors

* protocols/gossipsub: Log outbound substream errors

* Remove rust-fmt formatting

* Shift to prost for protobuf compiling

* Use wasm_timer for wasm compatibility

Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-24 16:16:02 +01:00
Pierre Krieger
0cb3cd4262
Fix WouldBlock being returned in wasm-ext (#1407)
* Fix WouldBlock being returned

* Fix other WouldBlocks
2020-01-24 15:18:45 +01:00
Pierre Krieger
d8bafc1f0a
Force prost 0.6.1 (#1404) 2020-01-23 11:06:11 +01:00
Pierre Krieger
d6bf8a5aeb
Remove some Unpin requirements (#1403) 2020-01-23 10:44:52 +01:00
Volker Mische
3405fc8b32 Fix link to upgrade::Builder (#1401)
The auto-linking didn't work properly as there is a method called `upgrade`
within the transport module.
2020-01-22 15:06:54 +01:00
Pierre Krieger
7d0c80fce1
Executor + Send (#1399) 2020-01-20 15:22:47 +01:00
Pierre Krieger
f89683419a
Allow configuring the tasks executor (#1391)
* Allow configuring the tasks executor

* Minor tweaks

* Add executor_fn

* Create ThreadsPool at the end if necessary

* Allow configuring the tasks executor

* Minor tweaks

* Add executor_fn

* Create ThreadsPool at the end if necessary

* WIP

* Don't depend on async-std and tokio in core

* Replace FutureObj with PinBoxFuture

* Some docs on Executor

* Fix tests

Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-01-20 14:18:35 +01:00
Pierre Krieger
0ed684ee30
Don't set KeepAlive to Until all the time (#1398)
* Don't set KeepAlive to Until all the time

* Address concerns
2020-01-20 13:55:25 +01:00
Roman Borschel
c241014ddb
Log connection close in swarm. Simplify bounds. (#1393) 2020-01-16 13:17:39 +01:00
Pierre Krieger
89acb0d9f1
Allow building an upgrade from an async function (#1383)
* Allow building an upgrade from an async function

* Some styling
2020-01-15 14:57:02 +01:00
Max Inden
991f5af993 misc/mdns: Make MdnsService::new sync by using std::net::UdpSocket::bind (#1382)
* misc/mdns: Make MdnsService::new sync by using std::net::UdpSocket::bind

MdnsService uses an IP address to create a UDP socket. The address does
not need to be resolved. Therefore one can use std's UdpSocket::bind
instead of the async counterpart from async-std. As a result
MdnsService::new and MdnsService::silent don't need to be async.

* examples/mdns-passive-discovery: Don't await sync MdnsService::new

* misc/mdns/src/behaviour: Make Mdns::new sync

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-15 13:43:09 +01:00
Toralf Wittner
680c467f7e Replace protobuf crate with prost! (#1390)
* Replace protobuf crate with prost!

* Add copyright headers to build.rs files.

* kad: Fix error when mapping connection types.

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Fix more mapping mistakes.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-15 12:02:02 +01:00
Max Inden
9d2df148cd core/src/transport/memory: Add basic dialer listener test (#1283)
* core/src/transport/memory: Add basic dialer listener test

Adds a basic test using two memory transport instances, one dialing the
other, sending a single message over the /wire/.

* core/src/nodes/network: Fix typo

* core/src/transport: Randomize listening port in unit test

* core/src/transport/memory: Update test to stable future

* core/src/transport/memory.rs: Use saturating_add for test port

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-01-14 18:19:10 +01:00
Joonas Koivunen
7df3e74d79 Enhance NetworkBehaviour docs (#1362)
* add docs about deriving NetworkBehaviour

* add commentary to the chat example

also minor stuff like reordering to match the struct elements, note why the listening address is
reported in poll.

* fix remove the warning on the example ignored field

* Apply suggestions from code review

Thanks for suggestions!

Co-Authored-By: Max Inden <mail@max-inden.de>

* chore cleanup added docs after suggestions

* fix print all listening addresses in examples/chat

not that there should be more than one but it makes sense to repeat the full example here.

* chore remove confusing writing on printing the listening addrs

* fix use intra-docs, spelling

Co-Authored-By: Roman S. Borschel <roman@parity.io>

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-01-14 12:48:16 +01:00
Pierre Krieger
3f968cbf92
Remove some Unpin requirements on Futures (#1384)
* Remove lots of Unpin requirements

* Make Transport::and_then accept pinned futures

* Finish the PR

* Work on secio

* Fix BandwidthTransport

* Adjust ListenersStrema

* Fix nodes/tasks

* Fix nodes

* Various more fixes

* Fix yamux

* Fix Swarm

* Fix WebSockets

* Fix rw-stream-sink
2020-01-14 12:03:10 +01:00
Pierre Krieger
42a45e2630
Remove Negotiated from upgrade traits (#1388)
* Remove Negotiated from upgrade traits

* Remove import
2020-01-13 14:34:43 +01:00
Toralf Wittner
ff780b5bff
Make InboundUpgrade for IK pattern more generic. (#1379)
The current implementation defines the InboundUpgrade only for `R = ()`
so notably a `NoiseConfig` created by `NoiseConfig::ik_dialer` can not
be an `InboundUpgrade`.

Fixes #1378.
2020-01-10 15:09:36 +01:00
Qinxuan Chen
af464e18c5 Update some dependenices; Remove some useless dependencies (#1387)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-10 14:03:59 +01:00