1345 Commits

Author SHA1 Message Date
Shotaro Yamada
f2d7091376
Remove unused dependencies (#1429)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 17:04:04 +01:00
Max Inden
2b7c015aa5
protocols/gossipsub: Do not close connection when inbound fails (#1424)
Instead of closing the connection on an error on the inbound substream
(both on `poll_next` as well as `poll_close`), one can set it to None
and depend on the remote to open an new one in case they have more data
to send.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 15:31:50 +01:00
Ashley
c73a175963
Switch to futures-timer 3.0 (#1426) 2020-02-05 12:12:18 +01:00
Pierre Krieger
7c04f7e468
Bump multihash to 0.2.2 (#1425) 2020-02-04 15:18:20 +01:00
Stan Bondi
955b60796a
Added Onion3 support to multiaddr (#1354) 2020-02-04 13:40:57 +01:00
Piotr Gołąb
57fd677686
Update gossipsub-chat example docstring (#1419)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-04 11:45:06 +01:00
Qinxuan Chen
013b72791a
Add Blake2b256 and Blake2s128 (#1386)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-04 11:17:09 +01:00
Rüdiger Klaehn
dd0c5c37f6
Allow multiple pings over one connection (#1416)
* Allow multiple pings over one connection

This is so that pinging a rust-libp2p node from ipfs works even without passing -n 1

* Remove timeout
2020-02-04 10:28:00 +01:00
Toralf Wittner
44b5e3e176
Bump yamux version 0.4.1 (#1415) 2020-01-31 10:27:20 +01:00
Pierre Krieger
b964cacfa4
Finish PeerId inlining change (#1413)
* Add peer id inlining for small public keys

* Apply @twittner suggestions

* Make PeerId compare equal accross hashes

* Fix mDNS

* Remove useless functions

* Add property test

Co-authored-by: Age Manning <Age@AgeManning.com>
2020-01-29 11:33:19 +01:00
Rüdiger Klaehn
f5e7461cec Implement private networks (#1385)
* Add pnet protocol

copied from plaintext protocol, since that seems to be the closest match

* Minimalize the pnet protocol

* WIP private swarms with fixed key

* Different nonces for write and read

* Use per stream write buffer to avoid allocations

* Add parsing and formating of PSKs

* Directly call handshake

Also remove unneeded InboundUpgrade and OutboundUpgrade

* Add HandshakeError

* Add dedicated pnet example

* Add tests for PSK parsing and formatting

* Some more tests for the parsing, fail case

* Add fingerprint

To be able to check if a go-ipfs and rust-libp2p use the same key without
having to dump the actual key. Not sure if there is a spec for this anywhere,
but it is basically just copied from go-ipfs.

* Minimize dependencies and remove dead code

* Rename PSK to PreSharedKey and use pin_project

* Add crypt_writer

Basically a stripped down and modified version of async_std BufWriter that
also encrypts using the given cipher.

* cargo fmt

* Actually get rid of the Unpin requirement

* Rewrite flushing and remove written count from state

* Add docs for pnet/lib.rs

* Increase library version

* Remove pnet example

There will be a more elaborate and useful example in a different PR

* Return pending on pending...

also make doc text less ambiguous

* Add debug assertions to check invariants of poll_flush_buf

Also, clarify the invariants in the comments of that method
2020-01-28 13:22:09 +01:00
Pierre Krieger
4bf42d188f
Remove all mod.rs files (#1400) 2020-01-28 08:51:06 +01:00
Rüdiger Klaehn
3b50cbd1b8 Addressing #473 ... if I understood the ticket right, we want to pass… (#1395)
* Addressing #473 ... if I understood the ticket right, we want to pass through whatever the application provides as a topic identifier, leaving hashing (or not hashing) up to the application.

* Remove TopicDescriptor and use Topic newtype everywhere

* PR feedback

Use From<Topic> instead of Into<String>
Use impl Into<Topic> instead of Topic in public API

Co-authored-by: Peat Bakke <peat@peat.org>
2020-01-27 15:23:01 +01:00
Pierre Krieger
2ef7c40cda
Add metadata to gossipsub (#1409) v0.15.0 2020-01-24 17:57:54 +01:00
Pierre Krieger
f1f40956d6
Publish 0.15.0 (#1408)
* Publish 0.15.0

* Oops, script too efficient
2020-01-24 17:14:56 +01:00
Pierre Krieger
0a45f7310f
Support tokio in libp2p-tcp and libp2p-uds (#1402) 2020-01-24 16:40:48 +01:00
Age Manning
37c7d73b11 Gossipsub Protocol (#898)
* Create gossipsub crate - Basic template, borrowed from floodsub

* Add a GossipsubConfig struct and set up basic structures in the Gossipsub struct

* Begin implementation of join. Adds get_random_peers helper function and adds tests

* Implements gossipsub leave()

* Update publishMany to incorporate gossipsub mesh and fanout logic

* Use the gossipsub mesh for determining peer subscription

* Remove subscribed_topics field from the Gossipsub struct

* Rename gossipsubconfig to ProtocolConfig

* Implement the gossipsub control messages into the Codec's Encode/Decode and modifies GossipsubRpc

* Modify GossipsubActions to enums for succinctness.

* Modify the memcache to store Gossipsub messages

* Implement control message handling.

* Update control message handling to handle multiple messages.

* Handle received gossipsub messages using pre-built handlers.

* Remove excess connected peer hashmap

* Add extra peer mapping and consistent topic naming.

* Implement heartbeat, emit_gossip and send_graft_prune.

* Group logic in forwarding messages. Add messages to memcache.

* Add heartbeat timer and move location of helper function.

* Add gossipsub the libp2p workspace, makes layer structs public

* Add logging to gossipsub

- Adds the log crate and implements logging macros
- Specifies versions for external crates

* Add example chat for debugging purposes

* Implement #868 for gossipsub.

* Add rust documentation to gossipsub crate.

- Adds basic documentation, overview and examples to the gossipsub
crate.

* Re-introduce the initial heartbeat time config.

This commit also adds the inject_connected test.

* Add subscribe tests.

- Modifies `handle_received_subscriptions` to take a reference of
subscriptions
- Adds `test_subscribe`
- Adds `test_handle_received_subscriptions`
- Adds tests for the filter in `get_random_peers`

* Add Bug fixes and further testing for gossipsub.

- Corrects the tuple use of topic_hashes
- Corrects JOIN logic around fanout and adding peers to the mesh
- Adds test_unsubscribe
- Adds test_join

* Rename GossipsubMessage::msg_id -> id

* Add bug fix for handling disconnected peers.

* Implements (partially) #889 for Gossipsub.

* handle_iwant event count tests

* handle_ihave event count tests

* Move layer.rs tests into separate file.

* Implement clippy suggestions for gossipsub.

* Modify control message tests for specific types.

* Implement builder pattern for GossipsubConfig.

As suggested by @twittner - The builder pattern for building
GossipsubConfig struct is implemented.

* Package version updates as suggested by @twittner.

* Correct line lengths in gossipsub.

* Correct braces in  found by @twittner.

* Implement @twittner's suggestions.

- Uses `HashSet` where applicable
- Update `FnvHashMap` to standard `HashMap`
- Uses `min` function in code simplification.

* Add NodeList struct to clarify topic_peers.

* Cleaner handling of messagelist

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner handling of added peers.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* handle_prune peer removed test

* basic grafting tests

* multiple topic grafting test

* Convert &vec to slice.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Convert to lazy insert.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner topic handling.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* control pool piggybacking

using HashMap.drain() in control_pool_flush

going to squash this

* Add Debug derives to gossipsub and correct tests.

* changes from PR

squash this

all tests passing, but still some that need to be reconsidered

test reform

* Implements Arc for GossipsubRpc events

* Remove support for floodsub nodes

* Reconnected to disconnected peers, to mitigate timeout

* Use ReadOne WriteOne with configurable max gossip sizes

* Remove length delimination from RPC encoding

* Prevent peer duplication in mesh

* Allow oneshot handler's inactivity_timeout to be configurable

* Correct peer duplication in mesh bug

* Remove auto-reconnect to allow for user-level disconnects

* Single long-lived inbound/outbound streams to match go implementation

* Allow gossipsub topics to be optionally hashable

* Improves gossipsub stream handling

- Corrects the handler's keep alive.
- Correct the chat example.
- Instantly add peers to the mesh on subscription if the mesh is low.

* Allows message validation in gossipsub

* Replaces Cuckoofilter with LRUCache

The false positive rate was unacceptable for rejecting messages.

* Renames configuration parameter and corrects logic

* Removes peer from fanout on disconnection

* Add publish and fanout tests

* Apply @mxinden suggestions

* Resend message if outbound stream negotiated

- Downgrades log warnings

* Implement further reviewer suggestions

- Created associated functions to avoid unnecessary cloning
- Messages are rejected if their sequence numbers are not u64
- `GossipsbuConfigBuilder` has the same defaults as `GossipsubConfig`
- Miscellaneous typos

* Add MessageId type and remove unnecessary comments

* Add a return value to propagate_message function

* Adds user-customised gossipsub message ids

* Adds the message id to GossipsubEvent

* Implement Debug for GossipsubConfig

* protocols/gossipsub: Add basic smoke test

Implement a basic smoke test that:

1. Builds a fully connected graph of size N.

2. Subscribes each node to the same topic.

3. Publishes a single message.

4. Waits for all nodes to receive the above message.

N and the structure of the graph are reproducibly randomized via
Quickcheck.

* Corrections pointed out by @mxinden

* Add option to remove source id publishing

* protocols/gossipsub/tests/smoke: Remove unused variable

* Merge latest master

* protocols/gossipsub: Move to stable futures

* examples/gossipsub-chat.rs: Move to stable futures

* protocols/gossipsub/src/behaviour/tests: Update to stable futures

* protocols/gossipsub/tests: Update to stable futures

* protocols/gossipsub: Log substream errors

* protocols/gossipsub: Log outbound substream errors

* Remove rust-fmt formatting

* Shift to prost for protobuf compiling

* Use wasm_timer for wasm compatibility

Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-24 16:16:02 +01:00
Pierre Krieger
0cb3cd4262
Fix WouldBlock being returned in wasm-ext (#1407)
* Fix WouldBlock being returned

* Fix other WouldBlocks
2020-01-24 15:18:45 +01:00
Pierre Krieger
d8bafc1f0a
Force prost 0.6.1 (#1404) 2020-01-23 11:06:11 +01:00
Pierre Krieger
d6bf8a5aeb
Remove some Unpin requirements (#1403) 2020-01-23 10:44:52 +01:00
Volker Mische
3405fc8b32 Fix link to upgrade::Builder (#1401)
The auto-linking didn't work properly as there is a method called `upgrade`
within the transport module.
2020-01-22 15:06:54 +01:00
Pierre Krieger
7d0c80fce1
Executor + Send (#1399) 2020-01-20 15:22:47 +01:00
Pierre Krieger
f89683419a
Allow configuring the tasks executor (#1391)
* Allow configuring the tasks executor

* Minor tweaks

* Add executor_fn

* Create ThreadsPool at the end if necessary

* Allow configuring the tasks executor

* Minor tweaks

* Add executor_fn

* Create ThreadsPool at the end if necessary

* WIP

* Don't depend on async-std and tokio in core

* Replace FutureObj with PinBoxFuture

* Some docs on Executor

* Fix tests

Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-01-20 14:18:35 +01:00
Pierre Krieger
0ed684ee30
Don't set KeepAlive to Until all the time (#1398)
* Don't set KeepAlive to Until all the time

* Address concerns
2020-01-20 13:55:25 +01:00
Roman Borschel
c241014ddb
Log connection close in swarm. Simplify bounds. (#1393) 2020-01-16 13:17:39 +01:00
Pierre Krieger
89acb0d9f1
Allow building an upgrade from an async function (#1383)
* Allow building an upgrade from an async function

* Some styling
2020-01-15 14:57:02 +01:00
Max Inden
991f5af993 misc/mdns: Make MdnsService::new sync by using std::net::UdpSocket::bind (#1382)
* misc/mdns: Make MdnsService::new sync by using std::net::UdpSocket::bind

MdnsService uses an IP address to create a UDP socket. The address does
not need to be resolved. Therefore one can use std's UdpSocket::bind
instead of the async counterpart from async-std. As a result
MdnsService::new and MdnsService::silent don't need to be async.

* examples/mdns-passive-discovery: Don't await sync MdnsService::new

* misc/mdns/src/behaviour: Make Mdns::new sync

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-15 13:43:09 +01:00
Toralf Wittner
680c467f7e Replace protobuf crate with prost! (#1390)
* Replace protobuf crate with prost!

* Add copyright headers to build.rs files.

* kad: Fix error when mapping connection types.

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Fix more mapping mistakes.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-15 12:02:02 +01:00
Max Inden
9d2df148cd core/src/transport/memory: Add basic dialer listener test (#1283)
* core/src/transport/memory: Add basic dialer listener test

Adds a basic test using two memory transport instances, one dialing the
other, sending a single message over the /wire/.

* core/src/nodes/network: Fix typo

* core/src/transport: Randomize listening port in unit test

* core/src/transport/memory: Update test to stable future

* core/src/transport/memory.rs: Use saturating_add for test port

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-01-14 18:19:10 +01:00
Joonas Koivunen
7df3e74d79 Enhance NetworkBehaviour docs (#1362)
* add docs about deriving NetworkBehaviour

* add commentary to the chat example

also minor stuff like reordering to match the struct elements, note why the listening address is
reported in poll.

* fix remove the warning on the example ignored field

* Apply suggestions from code review

Thanks for suggestions!

Co-Authored-By: Max Inden <mail@max-inden.de>

* chore cleanup added docs after suggestions

* fix print all listening addresses in examples/chat

not that there should be more than one but it makes sense to repeat the full example here.

* chore remove confusing writing on printing the listening addrs

* fix use intra-docs, spelling

Co-Authored-By: Roman S. Borschel <roman@parity.io>

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-01-14 12:48:16 +01:00
Pierre Krieger
3f968cbf92
Remove some Unpin requirements on Futures (#1384)
* Remove lots of Unpin requirements

* Make Transport::and_then accept pinned futures

* Finish the PR

* Work on secio

* Fix BandwidthTransport

* Adjust ListenersStrema

* Fix nodes/tasks

* Fix nodes

* Various more fixes

* Fix yamux

* Fix Swarm

* Fix WebSockets

* Fix rw-stream-sink
2020-01-14 12:03:10 +01:00
Pierre Krieger
42a45e2630
Remove Negotiated from upgrade traits (#1388)
* Remove Negotiated from upgrade traits

* Remove import
2020-01-13 14:34:43 +01:00
Toralf Wittner
ff780b5bff
Make InboundUpgrade for IK pattern more generic. (#1379)
The current implementation defines the InboundUpgrade only for `R = ()`
so notably a `NoiseConfig` created by `NoiseConfig::ik_dialer` can not
be an `InboundUpgrade`.

Fixes #1378.
2020-01-10 15:09:36 +01:00
Qinxuan Chen
af464e18c5 Update some dependenices; Remove some useless dependencies (#1387)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-10 14:03:59 +01:00
Toralf Wittner
655609dfe0 dns: Resolve dns4 to IPv4 and dns6 to IPv6. (#1381) 2020-01-10 13:10:16 +01:00
Max Inden
b939aefdd6 protocols/plaintext: Adapt protobuf generation to consolidated scripts (#1309)
* protocols/plaintext: Adapt protobuf generation to consolidated scripts

* protocols/plaintext: Fix protobuf import

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-09 14:12:43 +01:00
Pierre Krieger
324f0dc326
Bump rw-stream-sink to 0.2 (#1377) v0.14.0-alpha.1 2020-01-07 15:54:39 +01:00
Pierre Krieger
84487cf904
Publish 0.14.0-alpha.1 (#1376)
* Publish 0.14.0-alpha.1

* multiaddr -> 0.7.0
2020-01-07 15:30:39 +01:00
Toralf Wittner
ab2eb7a51d multiaddr: Replace Bytes with Arc<Vec<u8>>. (#1370)
* multiaddr: Replace `Bytes` with `Arc<Vec<u8>>`.

* Annotate type of `io::Cursor`.

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Annotate type of `io::Cursor`.

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-07 13:06:16 +01:00
Akihito Nakano
d83619109b Add protocol string to Error::UnknownProtocolString (#1355)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-07 12:36:37 +01:00
Pierre Krieger
84b6a7d04d
Add Swarm::next_extended (#1374)
* Add Swarm::next_extended

* Fix ipfs-kad example

* Fix tests

* Renames
2020-01-07 11:57:00 +01:00
Pierre Krieger
56ca671376
Merge pull request #1328 from libp2p/stable-futures
Refactor the crate to use stable futures
2020-01-06 19:20:30 +01:00
Pierre Krieger
ced66a319c
Merge pull request #1373 from tomaka/loggin-stable-fut
Address review on stable-futures
2020-01-06 18:38:28 +01:00
Pierre Krieger
fd7c06ed23
Merge pull request #1375 from tomaka/another-review-fix2
Another review fix for stable-futures branch
2020-01-06 17:30:34 +01:00
Pierre Krieger
55d6925006
Terminology 2020-01-06 17:22:09 +01:00
Pierre Krieger
5e0f219555
Use a different fix 2020-01-06 17:20:44 +01:00
Pierre Krieger
1333d5b8d9
Another review fix for stable-futures branch 2020-01-06 17:13:39 +01:00
Pierre Krieger
be0f18d162
Update core/src/nodes/tasks/manager.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
2020-01-06 14:50:33 +01:00
Pierre Krieger
65a7de49b2
Address review on stable-futures 2020-01-06 14:08:02 +01:00
Pierre Krieger
84ebd20d01
Merge pull request #1372 from tomaka/another-review-fix
Another small review fix
2020-01-06 13:32:11 +01:00