395 Commits

Author SHA1 Message Date
Roman Borschel
b872bd9030
Temporary canonical multihash in peer ID. (#1449)
* Temporary canonical multihash in peer ID.

* Reduce code duplication.

* Remove unnecessary impls.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-13 10:36:14 +01:00
Ashley
8238fd2100
Switch noise from the RingResolver to the DefaultResolver (#1439)
* hmm...

* Switch snow resolver to default

* Fix documentation

* Use the sha2 crate for sha512 hashing

* Use ring on native

* Use different features on different targets

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-12 21:44:50 +01:00
Pierre Krieger
955352bfd2
Extract rw-stream-sink to its own repo (#1448) 2020-02-12 13:23:08 +01:00
Rüdiger Klaehn
d55871623c
Ipfs private swarms interop example (#1420)
* Add gossipsub and ping

* Implement swarm key parsing from environment

* WIP remove stuff

* WIP remove more stuff

* Use gossipsub instead of floodsub

* Make ipfs example work with or without swarm key

* Add support for /ipfs/Qm1234 multiaddrs

* Add documentation for ipfs example

* Rename example to ipfs-private

* Fix comments

* Move EitherTransport into either.rs

And prettify imports of ipfs-private example

* Sanitize multiaddr before parsing

...and remove the "ipfs" protocol from multiaddr

* Remove TSubstream type parameter

...so that it works with current master

* PR feedback

use source instead of cause
2020-02-10 18:35:51 +01:00
Pierre Krieger
fc4dec581e
Check documentation intra-link (#1432)
* Fix broken links in rustdoc

This fixes all of the rustdoc warnings on nightly.

* Check documentation intra-link

* Fix config

* Fix bad indent

* Make nightly explicit

* More links fixes

* Fix link broken after master merge

Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
2020-02-10 15:17:07 +01:00
Pierre Krieger
1eff4b9823
Simplify trait bounds on NetworkBehaviour (#1405)
* Simplify trait bounds requirements

* More work

* Moar

* Finish

* Fix final tests

* More simplification

* Use separate traits for Inbound/Outbound

* Update gossipsub and remove warnings

* Add documentation to swarm

* Remove BoxSubstream

* Fix tests not compiling

* Fix stack overflow

* Address concerns

* For some reason my IDE ignored libp2p-kad
2020-02-07 16:29:30 +01:00
Shotaro Yamada
b200728443
Fix copy-paste error in doc string (#1428)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-06 11:17:05 +01:00
Shotaro Yamada
f2d7091376
Remove unused dependencies (#1429)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 17:04:04 +01:00
Max Inden
2b7c015aa5
protocols/gossipsub: Do not close connection when inbound fails (#1424)
Instead of closing the connection on an error on the inbound substream
(both on `poll_next` as well as `poll_close`), one can set it to None
and depend on the remote to open an new one in case they have more data
to send.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 15:31:50 +01:00
Ashley
c73a175963
Switch to futures-timer 3.0 (#1426) 2020-02-05 12:12:18 +01:00
Rüdiger Klaehn
dd0c5c37f6
Allow multiple pings over one connection (#1416)
* Allow multiple pings over one connection

This is so that pinging a rust-libp2p node from ipfs works even without passing -n 1

* Remove timeout
2020-02-04 10:28:00 +01:00
Rüdiger Klaehn
f5e7461cec Implement private networks (#1385)
* Add pnet protocol

copied from plaintext protocol, since that seems to be the closest match

* Minimalize the pnet protocol

* WIP private swarms with fixed key

* Different nonces for write and read

* Use per stream write buffer to avoid allocations

* Add parsing and formating of PSKs

* Directly call handshake

Also remove unneeded InboundUpgrade and OutboundUpgrade

* Add HandshakeError

* Add dedicated pnet example

* Add tests for PSK parsing and formatting

* Some more tests for the parsing, fail case

* Add fingerprint

To be able to check if a go-ipfs and rust-libp2p use the same key without
having to dump the actual key. Not sure if there is a spec for this anywhere,
but it is basically just copied from go-ipfs.

* Minimize dependencies and remove dead code

* Rename PSK to PreSharedKey and use pin_project

* Add crypt_writer

Basically a stripped down and modified version of async_std BufWriter that
also encrypts using the given cipher.

* cargo fmt

* Actually get rid of the Unpin requirement

* Rewrite flushing and remove written count from state

* Add docs for pnet/lib.rs

* Increase library version

* Remove pnet example

There will be a more elaborate and useful example in a different PR

* Return pending on pending...

also make doc text less ambiguous

* Add debug assertions to check invariants of poll_flush_buf

Also, clarify the invariants in the comments of that method
2020-01-28 13:22:09 +01:00
Pierre Krieger
4bf42d188f
Remove all mod.rs files (#1400) 2020-01-28 08:51:06 +01:00
Rüdiger Klaehn
3b50cbd1b8 Addressing #473 ... if I understood the ticket right, we want to pass… (#1395)
* Addressing #473 ... if I understood the ticket right, we want to pass through whatever the application provides as a topic identifier, leaving hashing (or not hashing) up to the application.

* Remove TopicDescriptor and use Topic newtype everywhere

* PR feedback

Use From<Topic> instead of Into<String>
Use impl Into<Topic> instead of Topic in public API

Co-authored-by: Peat Bakke <peat@peat.org>
2020-01-27 15:23:01 +01:00
Pierre Krieger
2ef7c40cda
Add metadata to gossipsub (#1409) 2020-01-24 17:57:54 +01:00
Pierre Krieger
f1f40956d6
Publish 0.15.0 (#1408)
* Publish 0.15.0

* Oops, script too efficient
2020-01-24 17:14:56 +01:00
Age Manning
37c7d73b11 Gossipsub Protocol (#898)
* Create gossipsub crate - Basic template, borrowed from floodsub

* Add a GossipsubConfig struct and set up basic structures in the Gossipsub struct

* Begin implementation of join. Adds get_random_peers helper function and adds tests

* Implements gossipsub leave()

* Update publishMany to incorporate gossipsub mesh and fanout logic

* Use the gossipsub mesh for determining peer subscription

* Remove subscribed_topics field from the Gossipsub struct

* Rename gossipsubconfig to ProtocolConfig

* Implement the gossipsub control messages into the Codec's Encode/Decode and modifies GossipsubRpc

* Modify GossipsubActions to enums for succinctness.

* Modify the memcache to store Gossipsub messages

* Implement control message handling.

* Update control message handling to handle multiple messages.

* Handle received gossipsub messages using pre-built handlers.

* Remove excess connected peer hashmap

* Add extra peer mapping and consistent topic naming.

* Implement heartbeat, emit_gossip and send_graft_prune.

* Group logic in forwarding messages. Add messages to memcache.

* Add heartbeat timer and move location of helper function.

* Add gossipsub the libp2p workspace, makes layer structs public

* Add logging to gossipsub

- Adds the log crate and implements logging macros
- Specifies versions for external crates

* Add example chat for debugging purposes

* Implement #868 for gossipsub.

* Add rust documentation to gossipsub crate.

- Adds basic documentation, overview and examples to the gossipsub
crate.

* Re-introduce the initial heartbeat time config.

This commit also adds the inject_connected test.

* Add subscribe tests.

- Modifies `handle_received_subscriptions` to take a reference of
subscriptions
- Adds `test_subscribe`
- Adds `test_handle_received_subscriptions`
- Adds tests for the filter in `get_random_peers`

* Add Bug fixes and further testing for gossipsub.

- Corrects the tuple use of topic_hashes
- Corrects JOIN logic around fanout and adding peers to the mesh
- Adds test_unsubscribe
- Adds test_join

* Rename GossipsubMessage::msg_id -> id

* Add bug fix for handling disconnected peers.

* Implements (partially) #889 for Gossipsub.

* handle_iwant event count tests

* handle_ihave event count tests

* Move layer.rs tests into separate file.

* Implement clippy suggestions for gossipsub.

* Modify control message tests for specific types.

* Implement builder pattern for GossipsubConfig.

As suggested by @twittner - The builder pattern for building
GossipsubConfig struct is implemented.

* Package version updates as suggested by @twittner.

* Correct line lengths in gossipsub.

* Correct braces in  found by @twittner.

* Implement @twittner's suggestions.

- Uses `HashSet` where applicable
- Update `FnvHashMap` to standard `HashMap`
- Uses `min` function in code simplification.

* Add NodeList struct to clarify topic_peers.

* Cleaner handling of messagelist

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner handling of added peers.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* handle_prune peer removed test

* basic grafting tests

* multiple topic grafting test

* Convert &vec to slice.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Convert to lazy insert.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner topic handling.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* control pool piggybacking

using HashMap.drain() in control_pool_flush

going to squash this

* Add Debug derives to gossipsub and correct tests.

* changes from PR

squash this

all tests passing, but still some that need to be reconsidered

test reform

* Implements Arc for GossipsubRpc events

* Remove support for floodsub nodes

* Reconnected to disconnected peers, to mitigate timeout

* Use ReadOne WriteOne with configurable max gossip sizes

* Remove length delimination from RPC encoding

* Prevent peer duplication in mesh

* Allow oneshot handler's inactivity_timeout to be configurable

* Correct peer duplication in mesh bug

* Remove auto-reconnect to allow for user-level disconnects

* Single long-lived inbound/outbound streams to match go implementation

* Allow gossipsub topics to be optionally hashable

* Improves gossipsub stream handling

- Corrects the handler's keep alive.
- Correct the chat example.
- Instantly add peers to the mesh on subscription if the mesh is low.

* Allows message validation in gossipsub

* Replaces Cuckoofilter with LRUCache

The false positive rate was unacceptable for rejecting messages.

* Renames configuration parameter and corrects logic

* Removes peer from fanout on disconnection

* Add publish and fanout tests

* Apply @mxinden suggestions

* Resend message if outbound stream negotiated

- Downgrades log warnings

* Implement further reviewer suggestions

- Created associated functions to avoid unnecessary cloning
- Messages are rejected if their sequence numbers are not u64
- `GossipsbuConfigBuilder` has the same defaults as `GossipsubConfig`
- Miscellaneous typos

* Add MessageId type and remove unnecessary comments

* Add a return value to propagate_message function

* Adds user-customised gossipsub message ids

* Adds the message id to GossipsubEvent

* Implement Debug for GossipsubConfig

* protocols/gossipsub: Add basic smoke test

Implement a basic smoke test that:

1. Builds a fully connected graph of size N.

2. Subscribes each node to the same topic.

3. Publishes a single message.

4. Waits for all nodes to receive the above message.

N and the structure of the graph are reproducibly randomized via
Quickcheck.

* Corrections pointed out by @mxinden

* Add option to remove source id publishing

* protocols/gossipsub/tests/smoke: Remove unused variable

* Merge latest master

* protocols/gossipsub: Move to stable futures

* examples/gossipsub-chat.rs: Move to stable futures

* protocols/gossipsub/src/behaviour/tests: Update to stable futures

* protocols/gossipsub/tests: Update to stable futures

* protocols/gossipsub: Log substream errors

* protocols/gossipsub: Log outbound substream errors

* Remove rust-fmt formatting

* Shift to prost for protobuf compiling

* Use wasm_timer for wasm compatibility

Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-24 16:16:02 +01:00
Pierre Krieger
0cb3cd4262
Fix WouldBlock being returned in wasm-ext (#1407)
* Fix WouldBlock being returned

* Fix other WouldBlocks
2020-01-24 15:18:45 +01:00
Pierre Krieger
d8bafc1f0a
Force prost 0.6.1 (#1404) 2020-01-23 11:06:11 +01:00
Pierre Krieger
0ed684ee30
Don't set KeepAlive to Until all the time (#1398)
* Don't set KeepAlive to Until all the time

* Address concerns
2020-01-20 13:55:25 +01:00
Toralf Wittner
680c467f7e Replace protobuf crate with prost! (#1390)
* Replace protobuf crate with prost!

* Add copyright headers to build.rs files.

* kad: Fix error when mapping connection types.

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Fix more mapping mistakes.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-15 12:02:02 +01:00
Pierre Krieger
3f968cbf92
Remove some Unpin requirements on Futures (#1384)
* Remove lots of Unpin requirements

* Make Transport::and_then accept pinned futures

* Finish the PR

* Work on secio

* Fix BandwidthTransport

* Adjust ListenersStrema

* Fix nodes/tasks

* Fix nodes

* Various more fixes

* Fix yamux

* Fix Swarm

* Fix WebSockets

* Fix rw-stream-sink
2020-01-14 12:03:10 +01:00
Pierre Krieger
42a45e2630
Remove Negotiated from upgrade traits (#1388)
* Remove Negotiated from upgrade traits

* Remove import
2020-01-13 14:34:43 +01:00
Toralf Wittner
ff780b5bff
Make InboundUpgrade for IK pattern more generic. (#1379)
The current implementation defines the InboundUpgrade only for `R = ()`
so notably a `NoiseConfig` created by `NoiseConfig::ik_dialer` can not
be an `InboundUpgrade`.

Fixes #1378.
2020-01-10 15:09:36 +01:00
Qinxuan Chen
af464e18c5 Update some dependenices; Remove some useless dependencies (#1387)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-10 14:03:59 +01:00
Max Inden
b939aefdd6 protocols/plaintext: Adapt protobuf generation to consolidated scripts (#1309)
* protocols/plaintext: Adapt protobuf generation to consolidated scripts

* protocols/plaintext: Fix protobuf import

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-09 14:12:43 +01:00
Pierre Krieger
324f0dc326
Bump rw-stream-sink to 0.2 (#1377) 2020-01-07 15:54:39 +01:00
Pierre Krieger
84487cf904
Publish 0.14.0-alpha.1 (#1376)
* Publish 0.14.0-alpha.1

* multiaddr -> 0.7.0
2020-01-07 15:30:39 +01:00
Pierre Krieger
84b6a7d04d
Add Swarm::next_extended (#1374)
* Add Swarm::next_extended

* Fix ipfs-kad example

* Fix tests

* Renames
2020-01-07 11:57:00 +01:00
Pierre Krieger
17e90f0f2c
Merge master in stable-futures 2020-01-02 17:47:41 +01:00
Pierre Krieger
c6a303cc45
Comment 2020-01-02 16:04:00 +01:00
Pierre Krieger
455a5a92f4
Pin protobuf versin and publish 0.13.2 2020-01-02 15:29:43 +01:00
Toralf Wittner
d870c734ee Pin protobuf to version 2.8.1 2020-01-02 10:48:51 +01:00
Toralf Wittner
72f1018acd Update to unsigned-varint v0.3 2020-01-02 10:45:43 +01:00
Toralf Wittner
2bc8d9590d Update to bytes v0.5
Except for `multiaddr` which encapsulates its use of bytes v0.4 now.
2019-12-21 15:42:24 +01:00
Toralf Wittner
34a631d560 Fix deflate test.
Skip over empty messages or else the socket may not be connected by the
time `close` is called on it.
2019-12-20 10:30:32 +01:00
Toralf Wittner
f293594144 Several changes.
- Pin `futures_codec` to version 0.3.3 as later versions require
at least bytes-0.5 which he have not upgraded to yet.
- Replace `futures::executor::block_on` with `async_std::task::block_on`
where `async-std` is already a dependency to work around an issue with
`park`/`unpark` behaviour.
- Use the published version of `quicksink`.
2019-12-18 16:31:31 +01:00
Pierre Krieger
21f32da045
Merge remote-tracking branch 'upstream/master' into HEAD 2019-12-12 14:10:01 +01:00
Pierre Krieger
d0032702f4
Update protocols/noise/src/io.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
2019-12-11 17:42:24 +01:00
Pierre Krieger
073f073a77
Backport #1347 to master 2019-12-11 17:07:16 +01:00
Pierre Krieger
3f1cbaa3a8
Fix noise/io.rs not flushing the underlying stream 2019-12-11 17:02:32 +01:00
Toralf Wittner
481849e4f1
secio: Back to 4-byte BE length prefix. (#1337)
The secio spec states that each frame must be prefix with a 32-bit big
endian length prefix so we can not use an unsigned varint here.
2019-12-09 16:34:13 +01:00
Pierre Krieger
aa4c2898cb
Fix identify not compiling for wasm (#1326) 2019-11-29 13:42:46 +01:00
Pierre Krieger
98dac8d509
Merge master into stable-futures (#1325)
* Update parking_lot to v0.9 (#1300)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Publish 0.13.1 (#1304)

* Publish 0.13.1

* Update CHANGELOG.md

Co-Authored-By: Toralf Wittner <tw@dtex.org>

* Update some deps of core-derive (#1299)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2019-11-28 18:03:59 +01:00
Max Inden
26f58d20a8
protocols/kad: Fix tests + minor fix in mdns and noise (#1320)
* misc/mdns: Fix missleading error message

* protocols/noise: Remove unneeded tokio-io import

* protocols/kad: Update tests to use stable futures
2019-11-28 16:12:02 +01:00
Pierre Krieger
8be45f5318 Fix the identify tests (#1324)
* Fix identify tests

* Some clean-up
2019-11-26 14:47:49 +01:00
Pierre Krieger
e5b087d01f
Fix the WASM build with stable futures (#1322)
* Fix the WASM build with stable futures

* Fix duplicate dependencies error
2019-11-26 11:48:47 +01:00
Pierre Krieger
e083e82212 Fix tests of libp2p-ping (#1321) 2019-11-25 17:33:59 +01:00
Toralf Wittner
b7644722ee
Fix examples and update core-derive. (#1317) 2019-11-25 10:45:04 +01:00
Toralf Wittner
02c5f34fc0
Update more crates to futures-0.3 (#1312) 2019-11-19 11:18:16 +01:00