diff --git a/README.md b/README.md index 98e1091b..f065c1cb 100644 --- a/README.md +++ b/README.md @@ -1,55 +1,24 @@ -# [WIP] Central repository for work on libp2p +# Central repository for work on libp2p -This repository is the central place for rust development of the -[libp2p](https://libp2p.io) spec. +This repository is the central place for Rust development of the [libp2p](https://libp2p.io) spec. -This readme along with many others will be more fleshed out the closer -the project gets to completion. Right now everything including the crate -organization is very much Work in Progress. +**This readme will be more fleshed out the closer the project gets to completion. +Right now everything including the crate organization is very much Work in Progress.** -## The main crate: libp2p +## Documentation This repository includes a facade crate named `libp2p`, which reexports the rest of the repository. -## General overview of the architecture +For documentation, you are encouraged to clone this repository or add `libp2p` as a dependency in +your Cargo.toml and run `cargo doc`. -Architecture of the other crates of this repository: +```toml +[dependencies] +libp2p = { git = "https://github.com/libp2p/rust-libp2p" } +``` -- `cid`: implements [CID (Content IDentifier)](https://github.com/ipld/cid): Self-describing content-addressed identifiers for distributed systems. -- `circular-buffer`: An optimized FIFO queue that allows safe access to the internal storage as a slice (i.e. not just element-by-element). This is useful for circular buffers of bytes. Since it uses `smallvec`'s `Array` trait it can only be backed by an array of static size, this may change in the future. -- `core`: Transport, protocol upgrade and swarm systems of libp2p. This crate contains all the core traits and mechanisms of the transport and swarm systems of libp2p. -- `datastore`: Utility library whose API provides a key-value storage with multiple possible backends. Used by `peerstore`. -- `dns`: this crate provides the type `DnsConfig` that allows one to resolve the `/dns4/` and `/dns6/` components of multiaddresses. -- `example`: Example usages of this library. -- `floodsub`: a flooding PubSub implementation for p2p messaging. -- `identify`: implementation of the `/ipfs/id/1.0.0` protocol that allows a node A to query another node B what information B knows about A. Implements the `ConnectionUpgrade` trait of `core`. Also includes the addresses B is listening on. -- `kad`: kademlia DHT implementation for peer routing -- `mplex`: Implements a binary stream [multiplex](https://github.com/maxogden/multiplex)er, streams multiple streams of binary data over a single binary stream. -- [`multiaddr`](https://github.com/multiformats/multiaddr): composable, future-proof, efficient and self-describing network addresses -- [`multihash`](https://github.com/multiformats/multihash): self identifying hashes; differentiating outputs from various well-established cryptographic hash functions; addressing size + encoding considerations; future-proofing use of hashes, and allowing multiple hash functions to coexist. -- `multistream-select`: used internally by libp2p to negotiate a protocol over a newly-established connection with a peer, or after a connection upgrade. -- `peerstore`: Generic storage for information about remote peers (their multiaddresses and their public key), with multiple possible backends. Each multiaddress also has a time-to-live. Used by `core`. -- `ping`: Implementation of the `ping` protocol (the exact protocol is specific to libp2p). Implements the `ConnectionUpgrade` trait of `core`. -- `ratelimit`: manages rate limiting with a connection. Also see [here](https://github.com/libp2p/specs/blob/master/8-implementations.md#811-swarm-dialer) for design. -- `relay`: Implements the [`/libp2p/circuit/relay/0.1.0` protocol](https://github.com/libp2p/specs/blob/master/relay/). It allows a source `A` to connect to a destination `B` via an intermediate relay node `R` to which `B` is already connected to. This is used as a last resort to make `B` reachable from other nodes when it would normally not be, e.g. due to certain NAT setups. -- `rw-stream-sink`: Utility library that makes it possible to wrap around a tokio `Stream + Sink` of bytes and implements `AsyncRead + AsyncWrite`. -- `secio`: Implementation of the `secio` protocol. Encrypts communications. Implements the `ConnectionUpgrade` trait of `core`. -- `tcp-transport`: Implementation of the `Transport` trait of `core` for TCP/IP. -- `uds`: Implementation of `Transport` for UNIX domain sockets. -- `varint`: encoding and decoding state machines for protobuf varints. -- `websocket`: Implementation of the `Transport` trait of `core` for Websockets. +## Notable users -## About the `impl Trait` syntax - -Right now a lot of code of this library uses `Box` or `Box` objects, or forces -`'static` lifetime bounds. - -This is caused by the lack of a stable `impl Trait` syntax in the Rust language. Once this syntax -is fully implemented and stabilized, it will be possible to change this code to use plain and -non-static objects instead of boxes. - -Progress for the `impl Trait` syntax can be tracked in [this issue of the Rust repository](https://github.com/rust-lang/rust/issues/34511). - -Once this syntax is stable in the nightly version, we will consider requiring the nightly version -of the compiler and switching to this syntax. +(open a pull request if you want your project to be added here) +- https://github.com/paritytech/polkadot diff --git a/core/README.md b/core/README.md deleted file mode 100644 index 28a7c23b..00000000 --- a/core/README.md +++ /dev/null @@ -1,178 +0,0 @@ -# libp2p-core - -Transport, protocol upgrade and swarm systems of *libp2p*. - -This crate contains all the core traits and mechanisms of the transport and swarm systems -of *libp2p*. - -# The `Transport` trait - -The main trait that this crate provides is `Transport`, which provides the `dial` and -`listen_on` methods and can be used to dial or listen on a multiaddress. The `swarm` crate -itself does not provide any concrete (i.e. non-dummy, non-adapter) implementation of this trait. -It is implemented on structs that are provided by external crates, such as `TcpConfig` from -`tcp-transport`, `UdpConfig`, or `WebsocketConfig` (note: as of the writing of this -documentation, the last two structs don't exist yet). - -Each implementation of `Transport` only supports *some* multiaddress protocols, for example -the `TcpConfig` struct only supports multiaddresses that look like `/ip*/*.*.*.*/tcp/*`. It is -possible to group two implementations of `Transport` with the `or_transport` method, in order -to obtain a single object that supports the protocols of both objects at once. This can be done -multiple times in a row in order to chain as many implementations as you want. - -``` -// TODO: right now only tcp-transport exists, we need to add an example for chaining -// multiple transports once that makes sense -``` - -## The `MuxedTransport` trait - -The `MuxedTransport` trait is an extension to the `Transport` trait, and is implemented on -transports that can receive incoming connections on streams that have been opened with `dial()`. - -The trait provides the `next_incoming()` method, which returns a future that will resolve to -the next substream that arrives from a dialed node. - -> **Note**: This trait is mainly implemented for transports that provide stream muxing -> capabilities, but it can also be implemented in a dummy way by returning an empty -> iterator. - -# Connection upgrades - -Once a socket has been opened with a remote through a `Transport`, it can be *upgraded*. This -consists in negotiating a protocol with the remote (through `multistream-select`), and applying -that protocol on the socket. - -A potential connection upgrade is represented with the `ConnectionUpgrade` trait. The trait -consists in a protocol name plus a method that turns the socket into an `Output` object whose -nature and type is specific to each upgrade. - -There exists three kinds of connection upgrades: middlewares, muxers, and actual protocols. - -## Middlewares - -Examples of middleware connection upgrades include `PlainTextConfig` (dummy upgrade) or -`SecioConfig` (encyption layer, provided by the `secio` crate). - -The output of a middleware connection upgrade implements the `AsyncRead` and `AsyncWrite` -traits, just like sockets do. - -A middleware can be applied on a transport by using the `with_upgrade` method of the -`Transport` trait. The return value of this method also implements the `Transport` trait, which -means that you can call `dial()` and `listen_on()` on it in order to directly obtain an -upgraded connection or a listener that will yield upgraded connections. Similarly, the -`next_incoming()` method will automatically apply the upgrade on both the dialer and the -listener. An error is produced if the remote doesn't support the protocol corresponding to the -connection upgrade. - -```rust -extern crate libp2p_core; -extern crate libp2p_tcp_transport; -extern crate tokio_current_thread; - -use libp2p_core::Transport; - -let tokio_core = tokio_core::reactor::Core::new().unwrap(); -let tcp_transport = libp2p_tcp_transport::TcpConfig::new(tokio_core.handle()); -let upgraded = tcp_transport.with_upgrade(libp2p_core::PlainTextConfig); - -// upgraded.dial(...) // automatically applies the plain text protocol on the socket -``` - -## Muxers - -The concept of *muxing* consists in using a single stream as if it was multiple substreams. - -If the output of the connection upgrade instead implements the `StreamMuxer` and `Clone` -traits, then you can turn the `UpgradedNode` struct into a `ConnectionReuse` struct by calling -`ConnectionReuse::from(upgraded_node)`. - -The `ConnectionReuse` struct then implements the `Transport` and `MuxedTransport` traits, and -can be used to dial or listen to multiaddresses, just like any other transport. The only -difference is that dialing a node will try to open a new substream on an existing connection -instead of opening a new one every time. - -> **Note**: Right now the `ConnectionReuse` struct is not fully implemented. - -`TODO: add an example once the multiplex pull request is merged` - -## Actual protocols - -*Actual protocols* work the same way as middlewares, except that their `Output` doesn't -implement the `AsyncRead` and `AsyncWrite` traits. This means that that the return value of -`with_upgrade` does **not** implement the `Transport` trait and thus cannot be used as a -transport. - -However the `UpgradedNode` struct returned by `with_upgrade` still provides methods named -`dial`, `listen_on`, and `next_incoming`, which will yield you a `Future` or a `Stream`, -which you can use to obtain the `Output`. This `Output` can then be used in a protocol-specific -way to use the protocol. - -```rust -extern crate futures; -extern crate libp2p_ping; -extern crate libp2p_core; -extern crate libp2p_tcp_transport; -extern crate tokio_current_thread; - -use futures::Future; -use libp2p_ping::Ping; -use libp2p_core::Transport; - -let mut core = tokio_core::reactor::Core::new().unwrap(); - -let ping_finished_future = libp2p_tcp_transport::TcpConfig::new() - // We have a `TcpConfig` struct that implements `Transport`, and apply a `Ping` upgrade on it. - .with_upgrade(Ping) - // TODO: right now the only available protocol is ping, but we want to replace it with - // something that is more simple to use - .dial("127.0.0.1:12345".parse::().unwrap()).unwrap_or_else(|_| panic!()) - .and_then(|((mut pinger, service), _)| { - pinger.ping().map_err(|_| panic!()).select(service).map_err(|_| panic!()) - }); - -// Runs until the ping arrives. -tokio_current_thread::block_on_all(ping_finished_future).unwrap(); -``` - -## Grouping protocols - -You can use the `.or_upgrade()` method to group multiple upgrades together. The return value -also implements the `ConnectionUpgrade` trait and will choose one of the protocols amongst the -ones supported. - -# Swarm - -Once you have created an object that implements the `Transport` trait, you can put it in a -*swarm*. This is done by calling the `swarm()` freestanding function with the transport -alongside with a function or a closure that will turn the output of the upgrade (usually an -actual protocol, as explained above) into a `Future` producing `()`. - -```rust -extern crate futures; -extern crate libp2p_ping; -extern crate libp2p_core; -extern crate libp2p_tcp_transport; -extern crate tokio_current_thread; - -use futures::Future; -use libp2p_ping::Ping; -use libp2p_core::Transport; - -let mut core = tokio_core::reactor::Core::new().unwrap(); - -let transport = libp2p_tcp_transport::TcpConfig::new() - .with_dummy_muxing(); - -let (swarm_controller, swarm_future) = libp2p_core::swarm(transport, Ping, |(mut pinger, service), client_addr| { - pinger.ping().map_err(|_| panic!()) - .select(service).map_err(|_| panic!()) - .map(|_| ()) -}); - -// The `swarm_controller` can then be used to do some operations. -swarm_controller.listen_on("/ip4/0.0.0.0/tcp/0".parse().unwrap()); - -// Runs until everything is finished. -tokio_current_thread::block_on_all(swarm_future).unwrap(); -``` diff --git a/core/src/lib.rs b/core/src/lib.rs index 46fc2812..c7956b98 100644 --- a/core/src/lib.rs +++ b/core/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! Transport, protocol upgrade and swarm systems of *libp2p*. //! //! This crate contains all the core traits and mechanisms of the transport and swarm systems diff --git a/core/src/transport/mod.rs b/core/src/transport/mod.rs index 814449f2..6b7ab637 100644 --- a/core/src/transport/mod.rs +++ b/core/src/transport/mod.rs @@ -116,7 +116,7 @@ pub trait Transport { /// implementation of `Transport` is only responsible for handling the protocols it supports. /// /// Returns `None` if nothing can be determined. This happens if this trait implementation - /// doesn't recognize the protocols, or if `server` and `observed` are related. + /// doesn't recognize the protocols, or if `server` and `observed` are unrelated. fn nat_traversal(&self, server: &Multiaddr, observed: &Multiaddr) -> Option; /// Applies a function on the output of the `Transport`. diff --git a/datastore/README.md b/datastore/README.md deleted file mode 100644 index e0783c35..00000000 --- a/datastore/README.md +++ /dev/null @@ -1,76 +0,0 @@ -General-purpose key-value storage. -The keys are strings, and the values are of any type you want. - -> **Note**: This crate is meant to be a utility for the implementation of other crates ; it -> does not directly participate in the stack of libp2p. - -This crate provides the `Datastore` trait, whose template parameter is the type of the value. -It is implemented on types that represent a key-value storage. -The only available implementation for now is `JsonFileDatastore`. - -# JSON file datastore - -The `JsonFileDatastore` can provide a key-value storage that loads and stores data in a single -JSON file. It is only available if the value implements the `Serialize`, `DeserializeOwned` -and `Clone` traits. - -The `JsonFileDatastore::new` method will attempt to load existing data from the path you pass -as parameter. This path is also where the data will be stored. The content of the store is -flushed on drop or if you call `flush()`. - -```rust -use datastore::Datastore; -use datastore::JsonFileDatastore; - -let datastore = JsonFileDatastore::>::new("/tmp/test.json").unwrap(); -datastore.put("foo".into(), vec![1, 2, 3]); -datastore.put("bar".into(), vec![0, 255, 127]); -assert_eq!(datastore.get("foo").unwrap(), &[1, 2, 3]); -datastore.flush().unwrap(); // optional -``` - -# Query - -In addition to simple operations such as `get` or `put`, the `Datastore` trait also provides -a way to perform queries on the key-value storage, using the `query` method. - -The struct returned by the `query` method implements the `Stream` trait from `futures`, -meaning that the result is asynchronous. - -> **Note**: For now the API of the `get` and `has` methods makes them potentially blocking -> operations, though the only available implementation doesn't block. The API of these -> methods may become asynchronous in the future if deemed necessary. - -```rust -extern crate datastore; -extern crate futures; - -use datastore::{Query, Order, Filter, FilterTy, FilterOp}; -use datastore::Datastore; -use datastore::JsonFileDatastore; -use futures::{Future, Stream}; - -let datastore = JsonFileDatastore::>::new("/tmp/test.json").unwrap(); -let query = datastore.query(Query { - // Only return the keys that start with this prefix. - prefix: "fo".into(), - // List of filters for the keys and/or values. - filters: vec![ - Filter { - ty: FilterTy::ValueCompare(&vec![6, 7, 8].into()), - operation: FilterOp::NotEqual, - }, - ], - // Order in which to sort the results. - orders: vec![Order::ByKeyDesc], - // Number of entries to skip at the beginning of the results (after sorting). - skip: 1, - // Limit to the number of entries to return (use `u64::max_value()` for no limit). - limit: 12, - // If true, don't load the values. For optimization purposes. - keys_only: false, -}); - -let results = query.collect().wait().unwrap(); -println!("{:?}", results); -``` \ No newline at end of file diff --git a/datastore/src/lib.rs b/datastore/src/lib.rs index 5dc18fa0..616f09f1 100644 --- a/datastore/src/lib.rs +++ b/datastore/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! General-purpose key-value storage. //! The keys are strings, and the values are of any type you want. //! diff --git a/dns/README.md b/dns/README.md deleted file mode 100644 index 42f95370..00000000 --- a/dns/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# libp2p-dns - -This crate provides the type `DnsConfig` that allows one to resolve the `/dns4/` and `/dns6/` -components of multiaddresses. - -## Usage - -In order to use this crate, create a `DnsConfig` with one of its constructors and pass it an -implementation of the `Transport` trait. - -Whenever we want to dial an address through the `DnsConfig` and that address contains a -`/dns4/` or `/dns6/` component, a DNS resolve will be performed and the component will be -replaced with respectively an `/ip4/` or an `/ip6/` component. diff --git a/dns/src/lib.rs b/dns/src/lib.rs index 92eceffc..24e7d4e9 100644 --- a/dns/src/lib.rs +++ b/dns/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! # libp2p-dns //! //! This crate provides the type `DnsConfig` that allows one to resolve the `/dns4/` and `/dns6/` diff --git a/libp2p/examples/README.md b/libp2p/examples/README.md deleted file mode 100644 index 28a90908..00000000 --- a/libp2p/examples/README.md +++ /dev/null @@ -1,10 +0,0 @@ -# How the keys were generated - -The keys used in the examples were generated like this: - -```sh -openssl genrsa -out private.pem 2048 -openssl rsa -in private.pem -outform DER -pubout -out public.der -openssl pkcs8 -in private.pem -topk8 -nocrypt -out private.pk8 -rm private.pem # optional -``` diff --git a/libp2p/src/lib.rs b/libp2p/src/lib.rs index ada0b3c1..aeade5b5 100644 --- a/libp2p/src/lib.rs +++ b/libp2p/src/lib.rs @@ -18,6 +18,118 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. +//! Libp2p is a peer-to-peer framework. +//! +//! # Major libp2p concepts +//! +//! Here is a list of all the major concepts of libp2p. +//! +//! ## Multiaddr +//! +//! A `Multiaddr` is a way to reach a node. Examples: +//! +//! * `/ip4/80.123.90.4/tcp/5432` +//! * `/ip6/[::1]/udp/10560` +//! * `/unix//path/to/socket` +//! +//! ## Transport +//! +//! `Transport` is a trait that represents an object capable of dialing multiaddresses or +//! listening on multiaddresses. The `Transport` produces an output which varies depending on the +//! object that implements the trait. +//! +//! Each implementation of `Transport` typically supports only some multiaddresses. For example +//! the `TcpConfig` type (which implements `Transport`) only supports multiaddresses of the format +//! `/ip4/.../tcp/...`. +//! +//! Example: +//! +//! ```rust +//! use libp2p::{Multiaddr, Transport, tcp::TcpConfig}; +//! let tcp_transport = TcpConfig::new(); +//! let addr: Multiaddr = "/ip4/98.97.96.95/tcp/20500".parse().expect("invalid multiaddr"); +//! let _outgoing_connec = tcp_transport.dial(addr); +//! // Note that `_outgoing_connec` is a `Future`, and therefore doesn't do anything by itself +//! // unless it is run through a tokio runtime. +//! ``` +//! +//! The easiest way to create a transport is to use the `CommonTransport` struct. This struct +//! provides support for the most common protocols. +//! +//! Example: +//! +//! ```rust +//! use libp2p::CommonTransport; +//! let _transport = CommonTransport::new(); +//! // _transport.dial(...); +//! ``` +//! +//! See the documentation of the `libp2p-core` crate for more details about transports. +//! +//! # Connection upgrades +//! +//! Once a connection has been opened with a remote through a `Transport`, it can be *upgraded*. +//! This consists in negotiating a protocol with the remote (through the `multistream-select` +//! protocol), and applying that protocol on the socket. +//! +//! Example upgrades: +//! +//! - Adding a security layer on top of the connection. +//! - Applying multiplexing, so that a connection can be split into multiple substreams. +//! - Negotiating a specific protocol, such as *ping* or *kademlia*. +//! +//! A potential connection upgrade is represented with the `ConnectionUpgrade` trait. The trait +//! consists in a protocol name plus a method that turns the socket into an `Output` object whose +//! nature and type is specific to each upgrade. For example, if you upgrade a connection with a +//! security layer, the output might contain an encrypted stream and the public key of the remote. +//! +//! You can combine a `Transport` with a compatible `ConnectionUpgrade` in order to obtain another +//! `Transport` that yields the output of the upgrade. +//! +//! Example: +//! +//! ```rust +//! use libp2p::{Transport, tcp::TcpConfig, secio::{SecioConfig, SecioKeyPair}}; +//! let tcp_transport = TcpConfig::new(); +//! let secio_upgrade = SecioConfig { +//! key: SecioKeyPair::ed25519_generated().unwrap(), +//! }; +//! let with_security = tcp_transport.with_upgrade(secio_upgrade); +//! // let _ = with_security.dial(...); +//! // `with_security` also implements the `Transport` trait, and all the connections opened +//! // through it will automatically negotiate the `secio` protocol. +//! ``` +//! +//! See the documentation of the `libp2p-core` crate for more details about upgrades. +//! +//! ## Swarm +//! +//! Once you have created an object that implements the `Transport` trait, you can put it in a +//! *swarm*. This is done by calling the `swarm()` freestanding function with the transport +//! alongside with a function or a closure that will turn the output of the upgrade (usually an +//! actual protocol, as explained above) into a `Future` producing `()`. +//! +//! See the documentation of the `libp2p-core` crate for more details about creating a swarm. +//! +//! # Using libp2p +//! +//! This section contains details about how to use libp2p in practice. +//! +//! The most simple way to use libp2p consists in the following steps: +//! +//! - Create a *base* implementation of `Transport` that combines all the protocols you want and +//! the upgrades you want, such as the security layer and multiplexing. +//! - Create structs that implement the `ConnectionUpgrade` trait for the protocols you want to +//! create, or use the protocols provided by the `libp2p` crate. +//! - Create a swarm that combines your base transport and all the upgrades and that handles the +//! behaviour that happens. +//! - Use this swarm to dial and listen. +//! +//! You probably also want to have some sort of nodes discovery mechanism, so that you +//! automatically connect to nodes of the network. The details of this haven't been fleshed out +//! in libp2p and will be written later. +//! + pub extern crate bytes; pub extern crate futures; #[cfg(not(target_os = "emscripten"))] diff --git a/mplex/README.md b/mplex/README.md deleted file mode 100644 index c0007bb1..00000000 --- a/mplex/README.md +++ /dev/null @@ -1,3 +0,0 @@ -# Multiplex - -A Rust implementation of [multiplex](https://github.com/maxogden/multiplex). diff --git a/multiaddr/README.md b/multiaddr/README.md deleted file mode 100644 index 56e5a80f..00000000 --- a/multiaddr/README.md +++ /dev/null @@ -1,63 +0,0 @@ -# rust-multiaddr - -[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io) -[![](https://img.shields.io/badge/project-multiformats-blue.svg?style=flat-square)](https://github.com/multiformats/multiformats) -[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](https://webchat.freenode.net/?channels=%23ipfs) -[![Travis CI](https://img.shields.io/travis/multiformats/rust-multiaddr.svg?style=flat-square&branch=master)](https://travis-ci.org/multiformats/rust-multiaddr) -[![codecov.io](https://img.shields.io/codecov/c/github/multiformats/rust-multiaddr.svg?style=flat-square&branch=master)](https://codecov.io/github/multiformats/rust-multiaddr?branch=master) -[![](https://img.shields.io/badge/rust-docs-blue.svg?style=flat-square)](https://docs.rs/crate/multiaddr) -[![crates.io](https://img.shields.io/badge/crates.io-v0.2.0-orange.svg?style=flat-square )](https://crates.io/crates/multiaddr) -[![](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) - - -> [multiaddr](https://github.com/multiformats/multiaddr) implementation in Rust. - -## Table of Contents - -- [Install](#install) -- [Usage](#usage) -- [Maintainers](#maintainers) -- [Contribute](#contribute) -- [License](#license) - -## Install - -First add this to your `Cargo.toml` - -```toml -[dependencies] -multiaddr = "*" -``` - -then run `cargo build`. - -## Usage - -```rust -extern crate multiaddr; - -use multiaddr::{Multiaddr, ToMultiaddr}; - -let address = "/ip4/127.0.0.1/udp/1234".parse::().unwrap(); -// or directly from a string -let other = "/ip4/127.0.0.1".to_multiaddr().unwrap(); - -assert_eq!(address.to_string(), "/ip4/127.0.0.1/udp/1234"); -assert_eq!(other.to_string(), "/ip4/127.0.0.1"); -``` - -## Maintainers - -Captain: [@dignifiedquire](https://github.com/dignifiedquire). - -## Contribute - -Contributions welcome. Please check out [the issues](https://github.com/multiformats/rust-multiaddr/issues). - -Check out our [contributing document](https://github.com/multiformats/multiformats/blob/master/contributing.md) for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). - -Small note: If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification. - -## License - -[MIT](LICENSE) © 2015-2017 Friedel Ziegelmeyer diff --git a/multihash/README.md b/multihash/README.md deleted file mode 100644 index 27dee716..00000000 --- a/multihash/README.md +++ /dev/null @@ -1,68 +0,0 @@ -# rust-multihash - -[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io) -[![](https://img.shields.io/badge/project-multiformats-blue.svg?style=flat-square)](https://github.com/multiformats/multiformats) -[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](https://webchat.freenode.net/?channels=%23ipfs) -[![Travis CI](https://img.shields.io/travis/multiformats/rust-multihash.svg?style=flat-square&branch=master)](https://travis-ci.org/multiformats/rust-multihash) -[![codecov.io](https://img.shields.io/codecov/c/github/multiformats/rust-multihash.svg?style=flat-square&branch=master)](https://codecov.io/github/multiformats/rust-multihash?branch=master) -[![](https://img.shields.io/badge/rust-docs-blue.svg?style=flat-square)](https://docs.rs/multihash/) -[![crates.io](https://img.shields.io/badge/crates.io-v0.4.0-orange.svg?style=flat-square )](https://crates.io/crates/multihash) -[![](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) - -> [multihash](https://github.com/multiformats/multihash) implementation in Rust. - -## Table of Contents - -- [Install](#install) -- [Usage](#usage) -- [Supported Hash Types](#supported-hash-types) -- [Dependencies](#dependencies) -- [Maintainers](#maintainers) -- [Contribute](#contribute) -- [License](#license) - -## Install - -First add this to your `Cargo.toml` - -```toml -[dependencies] -multihash = "*" -``` - -Then run `cargo build`. - -## Usage - -```rust -extern crate multihash; - -use multihash::{encode, decode, Hash}; - -let hash = encode(Hash::SHA2256, b"my hash").unwrap(); -let multi = decode(&hash).unwrap(); -``` - -## Supported Hash Types - -* `SHA1` -* `SHA2-256` -* `SHA2-512` -* `SHA3`/`Keccak` - -## Maintainers - -Captain: [@dignifiedquire](https://github.com/dignifiedquire). - -## Contribute - -Contributions welcome. Please check out [the issues](https://github.com/multiformats/rust-multihash/issues). - -Check out our [contributing document](https://github.com/multiformats/multiformats/blob/master/contributing.md) for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). - -Small note: If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification. - - -## License - -[MIT](LICENSE) © 2015-2017 Friedel Ziegelmayer diff --git a/multistream-select/README.md b/multistream-select/README.md deleted file mode 100644 index 1f179092..00000000 --- a/multistream-select/README.md +++ /dev/null @@ -1,92 +0,0 @@ -# Multistream-select - -This crate implements the `multistream-select` protocol, which is the protocol used by libp2p -to negotiate which protocol to use with the remote. - -> **Note**: This crate is used by the internals of *libp2p*, and it is not required to -> understand it in order to use *libp2p*. - -Whenever a new connection or a new multiplexed substream is opened, libp2p uses -`multistream-select` to negotiate with the remote which protocol to use. After a protocol has -been successfully negotiated, the stream (ie. the connection or the multiplexed substream) -immediately stops using `multistream-select` and starts using the negotiated protocol. - -## Protocol explanation - -The dialer has two options available: either request the list of protocols that the listener -supports, or suggest a protocol. If a protocol is suggested, the listener can either accept (by -answering with the same protocol name) or refuse the choice (by answering "not available"). - -## Examples - -For a dialer: - -```rust -extern crate bytes; -extern crate futures; -extern crate multistream_select; -extern crate tokio_current_thread; - -use bytes::Bytes; -use multistream_select::dialer_select_proto; -use futures::{Future, Sink, Stream}; -use tokio_core::net::TcpStream; -use tokio_core::reactor::Core; - -let mut core = Core::new().unwrap(); - -#[derive(Debug, Copy, Clone)] -enum MyProto { Echo, Hello } - -let client = TcpStream::connect(&"127.0.0.1:10333".parse().unwrap(), &core.handle()) - .from_err() - .and_then(move |connec| { - let protos = vec![ - (Bytes::from("/echo/1.0.0"), ::eq, MyProto::Echo), - (Bytes::from("/hello/2.5.0"), ::eq, MyProto::Hello), - ] - .into_iter(); - dialer_select_proto(connec, protos).map(|r| r.0) - }); - -let negotiated_protocol: MyProto = tokio_current_thread::block_on_all(client).expect("failed to find a protocol"); -println!("negotiated: {:?}", negotiated_protocol); -``` - -For a listener: - -```rust -extern crate bytes; -extern crate futures; -extern crate multistream_select; -extern crate tokio_current_thread; - -use bytes::Bytes; -use multistream_select::listener_select_proto; -use futures::{Future, Sink, Stream}; -use tokio_core::net::TcpListener; -use tokio_core::reactor::Core; - -let mut core = Core::new().unwrap(); - -#[derive(Debug, Copy, Clone)] -enum MyProto { Echo, Hello } - -let server = TcpListener::bind(&"127.0.0.1:0".parse().unwrap(), &core.handle()).unwrap() - .incoming() - .from_err() - .and_then(move |(connec, _)| { - let protos = vec![ - (Bytes::from("/echo/1.0.0"), ::eq, MyProto::Echo), - (Bytes::from("/hello/2.5.0"), ::eq, MyProto::Hello), - ] - .into_iter(); - listener_select_proto(connec, protos) - }) - .for_each(|(proto, _connec)| { - println!("new remote with {:?} negotiated", proto); - Ok(()) - }); - -tokio_current_thread::block_on_all(server).expect("failed to run server"); -``` diff --git a/multistream-select/src/lib.rs b/multistream-select/src/lib.rs index 89cd79b9..b747d46c 100644 --- a/multistream-select/src/lib.rs +++ b/multistream-select/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! # Multistream-select //! //! This crate implements the `multistream-select` protocol, which is the protocol used by libp2p diff --git a/peerstore/README.md b/peerstore/README.md deleted file mode 100644 index f5d3c9cb..00000000 --- a/peerstore/README.md +++ /dev/null @@ -1,48 +0,0 @@ -The `peerstore` crate allows one to store information about a peer. - -`peerstore` is a key-value database, where the keys are multihashes (which usually corresponds -to the hash of the public key of the peer, but that is not enforced by this crate) and the -values are the public key and a list of multiaddresses. Additionally, the multiaddresses stored -by the `peerstore` have a time-to-live after which they disappear. - -This crate consists of a generic `Peerstore` trait and the follow implementations: - -- `JsonPeerstore`: Stores the information in a single JSON file. -- `MemoryPeerstore`: Stores the information in memory. - -Note that the peerstore implementations do not consider information inside a peer store to be -critical. In case of an error (eg. corrupted file, disk error, etc.) they will prefer to lose -data rather than returning the error. - -# Example - -```rust -extern crate multiaddr; -extern crate libp2p_peerstore; - -use libp2p_peerstore::memory_peerstore::MemoryPeerstore; -use libp2p_peerstore::{Peerstore, PeerAccess}; -use multiaddr::Multiaddr; -use std::time::Duration; - -// In this example we use a `MemoryPeerstore`, but you can easily swap it for another backend. -let mut peerstore = MemoryPeerstore::empty(); -let peer_id = vec![1, 2, 3, 4]; - -// Let's write some information about a peer. -{ - // `peer_or_create` mutably borrows the peerstore, so we have to do it in a local scope. - let mut peer = peerstore.peer_or_create(&peer_id); - peer.set_pub_key(vec![60, 90, 120, 150]); - peer.add_addr("/ip4/10.11.12.13/tcp/20000".parse::().unwrap(), - Duration::from_millis(5000)); -} - -// Now let's load back the info. -{ - let mut peer = peerstore.peer(&peer_id).expect("peer doesn't exist in the peerstore"); - assert_eq!(peer.get_pub_key().unwrap(), &[60, 90, 120, 150]); - assert_eq!(peer.addrs().collect::>(), - &["/ip4/10.11.12.13/tcp/20000".parse::().unwrap()]); -} -``` diff --git a/ping/README.md b/ping/README.md deleted file mode 100644 index 6fa1dcfe..00000000 --- a/ping/README.md +++ /dev/null @@ -1,52 +0,0 @@ -Handles the `/ipfs/ping/1.0.0` protocol. This allows pinging a remote node and waiting for an -answer. - -# Usage - -Create a `Ping` struct, which implements the `ConnectionUpgrade` trait. When used as a -connection upgrade, it will produce a tuple of type `(Pinger, impl Future)` which -are named the *pinger* and the *ponger*. - -The *pinger* has a method named `ping` which will send a ping to the remote, while the *ponger* -is a future that will process the data received on the socket and will be signalled only when -the connection closes. - -# About timeouts - -For technical reasons, this crate doesn't handle timeouts. The action of pinging returns a -future that is signalled only when the remote answers. If the remote is not responsive, the -future will never be signalled. - -For implementation reasons, resources allocated for a ping are only ever fully reclaimed after -a pong has been received by the remote. Therefore if you repeatedly ping a non-responsive -remote you will end up using more and more memory (albeit the amount is very very small every -time), even if you destroy the future returned by `ping`. - -This is probably not a problem in practice, because the nature of the ping protocol is to -determine whether a remote is still alive, and any reasonable user of this crate will close -connections to non-responsive remotes (which would then de-allocate memory for the ping). - -# Example - -```rust -extern crate futures; -extern crate libp2p_ping; -extern crate libp2p_core; -extern crate libp2p_tcp_transport; -extern crate tokio_current_thread; - -use futures::Future; -use libp2p_ping::Ping; -use libp2p_core::Transport; - -let ping_finished_future = libp2p_tcp_transport::TcpConfig::new() - .with_upgrade(Ping) - .dial("127.0.0.1:12345".parse::().unwrap()).unwrap_or_else(|_| panic!()) - .and_then(|((mut pinger, service), _)| { - pinger.ping().map_err(|_| panic!()).select(service).map_err(|_| panic!()) - }); - -// Runs until the ping arrives. -tokio_current_thread::block_on_all(ping_finished_future).unwrap(); -``` - diff --git a/ping/src/lib.rs b/ping/src/lib.rs index 2ca484ec..adcd4e99 100644 --- a/ping/src/lib.rs +++ b/ping/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! Handles the `/ipfs/ping/1.0.0` protocol. This allows pinging a remote node and waiting for an //! answer. //! diff --git a/relay/README.md b/relay/README.md deleted file mode 100644 index e1487b30..00000000 --- a/relay/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Circuit Relay v0.1.0 - -Implements the `/libp2p/circuit/relay/0.1.0` protocol [[1][1]]. It allows a source `A` to connect -to a destination `B` via an intermediate relay node `R` to which `B` is already connected to. This -is used as a last resort to make `B` reachable from other nodes when it would normally not be, e.g. -due to certain NAT setups. - -[1]: https://github.com/libp2p/specs/blob/97b86236ce07ff35d9bff5c1ba60daa8879f8f03/relay/ - diff --git a/rw-stream-sink/README.md b/rw-stream-sink/README.md deleted file mode 100644 index 6d54ed54..00000000 --- a/rw-stream-sink/README.md +++ /dev/null @@ -1,8 +0,0 @@ -This crate provides the `RwStreamSink` type. It wraps around a `Stream + Sink` that produces -and accepts byte arrays, and implements `AsyncRead` and `AsyncWrite`. - -Each call to `write()` will send one packet on the sink. Calls to `read()` will read from -incoming packets. - -> **Note**: Although this crate is hosted in the libp2p repo, it is purely a utility crate and -> not at all specific to libp2p. diff --git a/rw-stream-sink/src/lib.rs b/rw-stream-sink/src/lib.rs index c59e2551..670084cb 100644 --- a/rw-stream-sink/src/lib.rs +++ b/rw-stream-sink/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! This crate provides the `RwStreamSink` type. It wraps around a `Stream + Sink` that produces //! and accepts byte arrays, and implements `AsyncRead` and `AsyncWrite`. //! diff --git a/secio/README.md b/secio/README.md deleted file mode 100644 index 3d8f3bf1..00000000 --- a/secio/README.md +++ /dev/null @@ -1,57 +0,0 @@ -The `secio` protocol is a middleware that will encrypt and decrypt communications going -through a socket (or anything that implements `AsyncRead + AsyncWrite`). - -# Connection upgrade - -The `SecioConfig` struct implements the `ConnectionUpgrade` trait. You can apply it over a -`Transport` by using the `with_upgrade` method. The returned object will also implement -`Transport` and will automatically apply the secio protocol over any connection that is opened -through it. - -```rust -extern crate futures; -extern crate tokio_current_thread; -extern crate tokio_io; -extern crate libp2p_core; -extern crate libp2p_secio; -extern crate libp2p_tcp_transport; - -use futures::Future; -use libp2p_secio::{SecioConfig, SecioKeyPair}; -use libp2p_core::{Multiaddr, Transport}; -use libp2p_tcp_transport::TcpConfig; -use tokio_core::reactor::Core; -use tokio_io::io::write_all; - -let mut core = Core::new().unwrap(); - -let transport = TcpConfig::new() - .with_upgrade({ - # let private_key = b""; - //let private_key = include_bytes!("test-rsa-private-key.pk8"); - # let public_key = vec![]; - //let public_key = include_bytes!("test-rsa-public-key.der").to_vec(); - SecioConfig { - // See the documentation of `SecioKeyPair`. - key: SecioKeyPair::rsa_from_pkcs8(private_key, public_key).unwrap(), - } - }); - -let future = transport.dial("/ip4/127.0.0.1/tcp/12345".parse::().unwrap()) - .unwrap_or_else(|_| panic!("Unable to dial node")) - .and_then(|(connection, _)| { - // Sends "hello world" on the connection, will be encrypted. - write_all(connection, "hello world") - }); - -tokio_current_thread::block_on_all(future).unwrap(); -``` - -# Manual usage - -> **Note**: You are encouraged to use `SecioConfig` as described above. - -You can add the `secio` layer over a socket by calling `SecioMiddleware::handshake()`. This -method will perform a handshake with the host, and return a future that corresponds to the -moment when the handshake succeeds or errored. On success, the future produces a -`SecioMiddleware` that implements `Sink` and `Stream` and can be used to send packets of data. diff --git a/tcp-transport/README.md b/tcp-transport/README.md deleted file mode 100644 index 7e902374..00000000 --- a/tcp-transport/README.md +++ /dev/null @@ -1,23 +0,0 @@ -# TCP transport - -Implementation of the libp2p `Transport` trait for TCP/IP. - -Uses [the *tokio* library](https://tokio.rs). - -## Usage - -Example: - -```rust -extern crate libp2p_tcp_transport; -extern crate tokio_current_thread; - -use libp2p_tcp_transport::TcpConfig; -use tokio_core::reactor::Core; - -let mut core = Core::new().unwrap(); -let tcp = TcpConfig::new(); -``` - -The `TcpConfig` structs implements the `Transport` trait of the `swarm` library. See the -documentation of `swarm` and of libp2p in general to learn how to use the `Transport` trait. \ No newline at end of file diff --git a/tcp-transport/src/lib.rs b/tcp-transport/src/lib.rs index 3388572e..a797b019 100644 --- a/tcp-transport/src/lib.rs +++ b/tcp-transport/src/lib.rs @@ -18,9 +18,6 @@ // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! Implementation of the libp2p `Transport` trait for TCP/IP. //! //! Uses [the *tokio* library](https://tokio.rs). diff --git a/websocket/README.md b/websocket/README.md deleted file mode 100644 index 9f24ca6b..00000000 --- a/websocket/README.md +++ /dev/null @@ -1,46 +0,0 @@ -Implementation of the libp2p `Transport` trait for Websockets. - -See the documentation of `swarm` and of libp2p in general to learn how to use the `Transport` -trait. - -This library is used in a different way depending on whether you are compiling for emscripten -or for a different operating system. - -# Emscripten - -On emscripten, you can create a `BrowserWsConfig` object with `BrowserWsConfig::new()`. It can -then be used as a transport. - -Listening on a websockets multiaddress isn't supported on emscripten. Dialing a multiaddress -which uses `ws` on top of TCP/IP will automatically use the `XMLHttpRequest` Javascript object. - -```rust -use libp2p_websocket::BrowserWsConfig; - -let ws_config = BrowserWsConfig::new(); -// let _ = ws_config.dial("/ip4/40.41.42.43/tcp/12345/ws".parse().unwrap()); -``` - -# Other operating systems - -On other operating systems, this library doesn't open any socket by itself. Instead it must be -plugged on top of another implementation of `Transport` such as TCP/IP. - -This underlying transport must be put inside a `WsConfig` object through the -`WsConfig::new()` function. - -```rust -extern crate libp2p_core; -extern crate libp2p_tcp_transport; -extern crate libp2p_websocket; -extern crate tokio_current_thread; - -use libp2p_core::{Multiaddr, Transport}; -use libp2p_tcp_transport::TcpConfig; -use libp2p_websocket::WsConfig; -use tokio_core::reactor::Core; - -let core = Core::new().unwrap(); -let ws_config = WsConfig::new(TcpConfig::new()); -let _ = ws_config.dial("/ip4/40.41.42.43/tcp/12345/ws".parse().unwrap()); -``` diff --git a/websocket/src/lib.rs b/websocket/src/lib.rs index c5ac4cf6..bc298664 100644 --- a/websocket/src/lib.rs +++ b/websocket/src/lib.rs @@ -20,9 +20,6 @@ #![recursion_limit = "512"] -// TODO: use this once stable ; for now we just copy-paste the content of the README.md -//#![doc(include = "../README.md")] - //! Implementation of the libp2p `Transport` trait for Websockets. //! //! See the documentation of `swarm` and of libp2p in general to learn how to use the `Transport`