438 Commits

Author SHA1 Message Date
Shotaro Yamada
f2d7091376
Remove unused dependencies (#1429)
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 17:04:04 +01:00
Max Inden
2b7c015aa5
protocols/gossipsub: Do not close connection when inbound fails (#1424)
Instead of closing the connection on an error on the inbound substream
(both on `poll_next` as well as `poll_close`), one can set it to None
and depend on the remote to open an new one in case they have more data
to send.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-05 15:31:50 +01:00
Ashley
c73a175963
Switch to futures-timer 3.0 (#1426) 2020-02-05 12:12:18 +01:00
Rüdiger Klaehn
dd0c5c37f6
Allow multiple pings over one connection (#1416)
* Allow multiple pings over one connection

This is so that pinging a rust-libp2p node from ipfs works even without passing -n 1

* Remove timeout
2020-02-04 10:28:00 +01:00
Rüdiger Klaehn
f5e7461cec Implement private networks (#1385)
* Add pnet protocol

copied from plaintext protocol, since that seems to be the closest match

* Minimalize the pnet protocol

* WIP private swarms with fixed key

* Different nonces for write and read

* Use per stream write buffer to avoid allocations

* Add parsing and formating of PSKs

* Directly call handshake

Also remove unneeded InboundUpgrade and OutboundUpgrade

* Add HandshakeError

* Add dedicated pnet example

* Add tests for PSK parsing and formatting

* Some more tests for the parsing, fail case

* Add fingerprint

To be able to check if a go-ipfs and rust-libp2p use the same key without
having to dump the actual key. Not sure if there is a spec for this anywhere,
but it is basically just copied from go-ipfs.

* Minimize dependencies and remove dead code

* Rename PSK to PreSharedKey and use pin_project

* Add crypt_writer

Basically a stripped down and modified version of async_std BufWriter that
also encrypts using the given cipher.

* cargo fmt

* Actually get rid of the Unpin requirement

* Rewrite flushing and remove written count from state

* Add docs for pnet/lib.rs

* Increase library version

* Remove pnet example

There will be a more elaborate and useful example in a different PR

* Return pending on pending...

also make doc text less ambiguous

* Add debug assertions to check invariants of poll_flush_buf

Also, clarify the invariants in the comments of that method
2020-01-28 13:22:09 +01:00
Pierre Krieger
4bf42d188f
Remove all mod.rs files (#1400) 2020-01-28 08:51:06 +01:00
Rüdiger Klaehn
3b50cbd1b8 Addressing #473 ... if I understood the ticket right, we want to pass… (#1395)
* Addressing #473 ... if I understood the ticket right, we want to pass through whatever the application provides as a topic identifier, leaving hashing (or not hashing) up to the application.

* Remove TopicDescriptor and use Topic newtype everywhere

* PR feedback

Use From<Topic> instead of Into<String>
Use impl Into<Topic> instead of Topic in public API

Co-authored-by: Peat Bakke <peat@peat.org>
2020-01-27 15:23:01 +01:00
Pierre Krieger
2ef7c40cda
Add metadata to gossipsub (#1409) 2020-01-24 17:57:54 +01:00
Pierre Krieger
f1f40956d6
Publish 0.15.0 (#1408)
* Publish 0.15.0

* Oops, script too efficient
2020-01-24 17:14:56 +01:00
Age Manning
37c7d73b11 Gossipsub Protocol (#898)
* Create gossipsub crate - Basic template, borrowed from floodsub

* Add a GossipsubConfig struct and set up basic structures in the Gossipsub struct

* Begin implementation of join. Adds get_random_peers helper function and adds tests

* Implements gossipsub leave()

* Update publishMany to incorporate gossipsub mesh and fanout logic

* Use the gossipsub mesh for determining peer subscription

* Remove subscribed_topics field from the Gossipsub struct

* Rename gossipsubconfig to ProtocolConfig

* Implement the gossipsub control messages into the Codec's Encode/Decode and modifies GossipsubRpc

* Modify GossipsubActions to enums for succinctness.

* Modify the memcache to store Gossipsub messages

* Implement control message handling.

* Update control message handling to handle multiple messages.

* Handle received gossipsub messages using pre-built handlers.

* Remove excess connected peer hashmap

* Add extra peer mapping and consistent topic naming.

* Implement heartbeat, emit_gossip and send_graft_prune.

* Group logic in forwarding messages. Add messages to memcache.

* Add heartbeat timer and move location of helper function.

* Add gossipsub the libp2p workspace, makes layer structs public

* Add logging to gossipsub

- Adds the log crate and implements logging macros
- Specifies versions for external crates

* Add example chat for debugging purposes

* Implement #868 for gossipsub.

* Add rust documentation to gossipsub crate.

- Adds basic documentation, overview and examples to the gossipsub
crate.

* Re-introduce the initial heartbeat time config.

This commit also adds the inject_connected test.

* Add subscribe tests.

- Modifies `handle_received_subscriptions` to take a reference of
subscriptions
- Adds `test_subscribe`
- Adds `test_handle_received_subscriptions`
- Adds tests for the filter in `get_random_peers`

* Add Bug fixes and further testing for gossipsub.

- Corrects the tuple use of topic_hashes
- Corrects JOIN logic around fanout and adding peers to the mesh
- Adds test_unsubscribe
- Adds test_join

* Rename GossipsubMessage::msg_id -> id

* Add bug fix for handling disconnected peers.

* Implements (partially) #889 for Gossipsub.

* handle_iwant event count tests

* handle_ihave event count tests

* Move layer.rs tests into separate file.

* Implement clippy suggestions for gossipsub.

* Modify control message tests for specific types.

* Implement builder pattern for GossipsubConfig.

As suggested by @twittner - The builder pattern for building
GossipsubConfig struct is implemented.

* Package version updates as suggested by @twittner.

* Correct line lengths in gossipsub.

* Correct braces in  found by @twittner.

* Implement @twittner's suggestions.

- Uses `HashSet` where applicable
- Update `FnvHashMap` to standard `HashMap`
- Uses `min` function in code simplification.

* Add NodeList struct to clarify topic_peers.

* Cleaner handling of messagelist

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner handling of added peers.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* handle_prune peer removed test

* basic grafting tests

* multiple topic grafting test

* Convert &vec to slice.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Convert to lazy insert.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* Cleaner topic handling.

Co-Authored-By: AgeManning <Age@AgeManning.com>

* control pool piggybacking

using HashMap.drain() in control_pool_flush

going to squash this

* Add Debug derives to gossipsub and correct tests.

* changes from PR

squash this

all tests passing, but still some that need to be reconsidered

test reform

* Implements Arc for GossipsubRpc events

* Remove support for floodsub nodes

* Reconnected to disconnected peers, to mitigate timeout

* Use ReadOne WriteOne with configurable max gossip sizes

* Remove length delimination from RPC encoding

* Prevent peer duplication in mesh

* Allow oneshot handler's inactivity_timeout to be configurable

* Correct peer duplication in mesh bug

* Remove auto-reconnect to allow for user-level disconnects

* Single long-lived inbound/outbound streams to match go implementation

* Allow gossipsub topics to be optionally hashable

* Improves gossipsub stream handling

- Corrects the handler's keep alive.
- Correct the chat example.
- Instantly add peers to the mesh on subscription if the mesh is low.

* Allows message validation in gossipsub

* Replaces Cuckoofilter with LRUCache

The false positive rate was unacceptable for rejecting messages.

* Renames configuration parameter and corrects logic

* Removes peer from fanout on disconnection

* Add publish and fanout tests

* Apply @mxinden suggestions

* Resend message if outbound stream negotiated

- Downgrades log warnings

* Implement further reviewer suggestions

- Created associated functions to avoid unnecessary cloning
- Messages are rejected if their sequence numbers are not u64
- `GossipsbuConfigBuilder` has the same defaults as `GossipsubConfig`
- Miscellaneous typos

* Add MessageId type and remove unnecessary comments

* Add a return value to propagate_message function

* Adds user-customised gossipsub message ids

* Adds the message id to GossipsubEvent

* Implement Debug for GossipsubConfig

* protocols/gossipsub: Add basic smoke test

Implement a basic smoke test that:

1. Builds a fully connected graph of size N.

2. Subscribes each node to the same topic.

3. Publishes a single message.

4. Waits for all nodes to receive the above message.

N and the structure of the graph are reproducibly randomized via
Quickcheck.

* Corrections pointed out by @mxinden

* Add option to remove source id publishing

* protocols/gossipsub/tests/smoke: Remove unused variable

* Merge latest master

* protocols/gossipsub: Move to stable futures

* examples/gossipsub-chat.rs: Move to stable futures

* protocols/gossipsub/src/behaviour/tests: Update to stable futures

* protocols/gossipsub/tests: Update to stable futures

* protocols/gossipsub: Log substream errors

* protocols/gossipsub: Log outbound substream errors

* Remove rust-fmt formatting

* Shift to prost for protobuf compiling

* Use wasm_timer for wasm compatibility

Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-24 16:16:02 +01:00
Pierre Krieger
0cb3cd4262
Fix WouldBlock being returned in wasm-ext (#1407)
* Fix WouldBlock being returned

* Fix other WouldBlocks
2020-01-24 15:18:45 +01:00
Pierre Krieger
d8bafc1f0a
Force prost 0.6.1 (#1404) 2020-01-23 11:06:11 +01:00
Pierre Krieger
0ed684ee30
Don't set KeepAlive to Until all the time (#1398)
* Don't set KeepAlive to Until all the time

* Address concerns
2020-01-20 13:55:25 +01:00
Toralf Wittner
680c467f7e Replace protobuf crate with prost! (#1390)
* Replace protobuf crate with prost!

* Add copyright headers to build.rs files.

* kad: Fix error when mapping connection types.

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Fix more mapping mistakes.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-15 12:02:02 +01:00
Pierre Krieger
3f968cbf92
Remove some Unpin requirements on Futures (#1384)
* Remove lots of Unpin requirements

* Make Transport::and_then accept pinned futures

* Finish the PR

* Work on secio

* Fix BandwidthTransport

* Adjust ListenersStrema

* Fix nodes/tasks

* Fix nodes

* Various more fixes

* Fix yamux

* Fix Swarm

* Fix WebSockets

* Fix rw-stream-sink
2020-01-14 12:03:10 +01:00
Pierre Krieger
42a45e2630
Remove Negotiated from upgrade traits (#1388)
* Remove Negotiated from upgrade traits

* Remove import
2020-01-13 14:34:43 +01:00
Toralf Wittner
ff780b5bff
Make InboundUpgrade for IK pattern more generic. (#1379)
The current implementation defines the InboundUpgrade only for `R = ()`
so notably a `NoiseConfig` created by `NoiseConfig::ik_dialer` can not
be an `InboundUpgrade`.

Fixes #1378.
2020-01-10 15:09:36 +01:00
Qinxuan Chen
af464e18c5 Update some dependenices; Remove some useless dependencies (#1387)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-10 14:03:59 +01:00
Max Inden
b939aefdd6 protocols/plaintext: Adapt protobuf generation to consolidated scripts (#1309)
* protocols/plaintext: Adapt protobuf generation to consolidated scripts

* protocols/plaintext: Fix protobuf import

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-01-09 14:12:43 +01:00
Pierre Krieger
324f0dc326
Bump rw-stream-sink to 0.2 (#1377) 2020-01-07 15:54:39 +01:00
Pierre Krieger
84487cf904
Publish 0.14.0-alpha.1 (#1376)
* Publish 0.14.0-alpha.1

* multiaddr -> 0.7.0
2020-01-07 15:30:39 +01:00
Pierre Krieger
84b6a7d04d
Add Swarm::next_extended (#1374)
* Add Swarm::next_extended

* Fix ipfs-kad example

* Fix tests

* Renames
2020-01-07 11:57:00 +01:00
Pierre Krieger
17e90f0f2c
Merge master in stable-futures 2020-01-02 17:47:41 +01:00
Pierre Krieger
c6a303cc45
Comment 2020-01-02 16:04:00 +01:00
Pierre Krieger
455a5a92f4
Pin protobuf versin and publish 0.13.2 2020-01-02 15:29:43 +01:00
Toralf Wittner
d870c734ee Pin protobuf to version 2.8.1 2020-01-02 10:48:51 +01:00
Toralf Wittner
72f1018acd Update to unsigned-varint v0.3 2020-01-02 10:45:43 +01:00
Toralf Wittner
2bc8d9590d Update to bytes v0.5
Except for `multiaddr` which encapsulates its use of bytes v0.4 now.
2019-12-21 15:42:24 +01:00
Toralf Wittner
34a631d560 Fix deflate test.
Skip over empty messages or else the socket may not be connected by the
time `close` is called on it.
2019-12-20 10:30:32 +01:00
Toralf Wittner
f293594144 Several changes.
- Pin `futures_codec` to version 0.3.3 as later versions require
at least bytes-0.5 which he have not upgraded to yet.
- Replace `futures::executor::block_on` with `async_std::task::block_on`
where `async-std` is already a dependency to work around an issue with
`park`/`unpark` behaviour.
- Use the published version of `quicksink`.
2019-12-18 16:31:31 +01:00
Pierre Krieger
21f32da045
Merge remote-tracking branch 'upstream/master' into HEAD 2019-12-12 14:10:01 +01:00
Pierre Krieger
d0032702f4
Update protocols/noise/src/io.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
2019-12-11 17:42:24 +01:00
Pierre Krieger
073f073a77
Backport #1347 to master 2019-12-11 17:07:16 +01:00
Pierre Krieger
3f1cbaa3a8
Fix noise/io.rs not flushing the underlying stream 2019-12-11 17:02:32 +01:00
Toralf Wittner
481849e4f1
secio: Back to 4-byte BE length prefix. (#1337)
The secio spec states that each frame must be prefix with a 32-bit big
endian length prefix so we can not use an unsigned varint here.
2019-12-09 16:34:13 +01:00
Pierre Krieger
aa4c2898cb
Fix identify not compiling for wasm (#1326) 2019-11-29 13:42:46 +01:00
Pierre Krieger
98dac8d509
Merge master into stable-futures (#1325)
* Update parking_lot to v0.9 (#1300)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Publish 0.13.1 (#1304)

* Publish 0.13.1

* Update CHANGELOG.md

Co-Authored-By: Toralf Wittner <tw@dtex.org>

* Update some deps of core-derive (#1299)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2019-11-28 18:03:59 +01:00
Max Inden
26f58d20a8
protocols/kad: Fix tests + minor fix in mdns and noise (#1320)
* misc/mdns: Fix missleading error message

* protocols/noise: Remove unneeded tokio-io import

* protocols/kad: Update tests to use stable futures
2019-11-28 16:12:02 +01:00
Pierre Krieger
8be45f5318 Fix the identify tests (#1324)
* Fix identify tests

* Some clean-up
2019-11-26 14:47:49 +01:00
Pierre Krieger
e5b087d01f
Fix the WASM build with stable futures (#1322)
* Fix the WASM build with stable futures

* Fix duplicate dependencies error
2019-11-26 11:48:47 +01:00
Pierre Krieger
e083e82212 Fix tests of libp2p-ping (#1321) 2019-11-25 17:33:59 +01:00
Toralf Wittner
b7644722ee
Fix examples and update core-derive. (#1317) 2019-11-25 10:45:04 +01:00
Toralf Wittner
02c5f34fc0
Update more crates to futures-0.3 (#1312) 2019-11-19 11:18:16 +01:00
Max Inden
be73b90345 protocols/plaintext: Move to stable futures and use unsigned varints (#1306)
* protocols/plaintext: Move to stable futures and use unsigned varints

The plaintext 2.0 specification requires to use unsigned varints for
frame length delimiting instead of fixed 4 byte integer frame length
delimiting. This commit aligns the implementation with the
specification.

* protocols/secio: Fix doc comment BytesMut -> Vec<u8>

* protocols/plaintext: Add quick check smoke test

* protocols/plaintext: Rework imports and log levels

* protocols/plaintext: Use BytesMut instead of Vec<u8>

* protocols/plaintext: Use BoxFuture
2019-11-19 11:15:35 +01:00
Toralf Wittner
f85241dd36 Update core, tcp, secio and mplex to futures-0.3. (#1302)
* Update `rw-stream-sink` to futures-0.3.

* Update core, tcp, secio and mplex to futures-0.3.

On top of https://github.com/libp2p/rust-libp2p/pull/1301
2019-11-14 13:42:14 +01:00
Pierre Krieger
9e36ef41ac
Publish 0.13.1 (#1304)
* Publish 0.13.1

* Update CHANGELOG.md

Co-Authored-By: Toralf Wittner <tw@dtex.org>
2019-11-13 14:48:15 +01:00
Pierre Krieger
c1226b203a
Cherry-pick commits from master to stable-futures (#1296)
* Implement Debug for (ed25519|secp256k1)::(Keypair|SecretKey) (#1285)

* Fix possible arithmetic overflow in libp2p-kad. (#1291)

When the number of active queries exceeds the (internal)
JOBS_MAX_QUERIES limit, which is only supposed to bound
the number of concurrent queries relating to background
jobs, an arithmetic overflow occurs. This is fixed by
using saturating subtraction.

* protocols/plaintext: Add example on how to upgrade with PlainTextConfig1 (#1286)

* [mdns] - Support for long mDNS names (Bug #1232) (#1287)

* Dead code -- commenting out with a note referencing future implementation

* Adding "std" feature so that cargo can build in other directories (notably `misc/mdns`, so that I could run these tests)

* Permitting `PeerID` to be built from an `Identity` multihash

* The length limit for DNS labels is 63 characters, as per RFC1035

* Allocates the vector with capacity for the service name plus additional QNAME encoding bytes

* Added support for encoding/decoding peer IDs with an encoded length greater than 63 characters

* Removing "std" from ring features

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Retaining MAX_INLINE_KEY_LENGTH with comment about future usage

* `segment_peer_id` consumes `peer_id` ... plus an early return for IDs that don't need to be segmented

* Fixing logic

* Bump most dependencies (#1268)

* Bump most dependencies

This actually builds 😊.

* Bump all dependencies

Includes the excellent work of @rschulman in #1265.

* Remove use of ed25519-dalek fork

* Monomorphize more dependencies

* Add compatibility hack for rand

Cargo allows a crate to depend on multiple versions of another, but
`cargo-web` panics in that situation.  Use a wrapper crate to work
around the panic.

* Use @tomaka’s idea for using a newer `rand`

instead of my own ugly hack.

* Switch to Parity master

as its dependency-bumping PR has been merged.

* Update some depenendencies again

* Remove unwraps and `#[allow(deprecated)]`.

* Remove spurious changes to dependencies

Bumping minor or patch versions is not needed, and increases likelyhood
of merge conflicts.

* Remove some redundant Cargo.toml changes

* Replace a retry loop with an expect

`ed25519::SecretKey::from_bytes` will never fail for 32-byte inputs.

* Revert changes that don’t belong in this PR

* Remove using void to bypass ICE (#1295)

* Publish 0.13.0 (#1294)
2019-11-06 16:09:15 +01:00
Pierre Krieger
b3b9081f85
Publish 0.13.0 (#1294) 2019-11-06 09:37:22 +01:00
Pierre Krieger
96699a472a
Remove using void to bypass ICE (#1295) 2019-11-06 09:19:49 +01:00
Toralf Wittner
7eb4165d44 Use unsigned-varint and futures_codec from crates.io. (#1293) 2019-11-05 09:08:08 +01:00