369 lines
14 KiB
Rust
Raw Normal View History

// Copyright 2018 Parity Technologies (UK) Ltd.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
// DEALINGS IN THE SOFTWARE.
use crate::handler::{IdentifyHandler, IdentifyHandlerEvent};
use crate::protocol::{IdentifyInfo, ReplySubstream};
use futures::prelude::*;
use libp2p_core::{
ConnectedPoint,
Multiaddr,
PeerId,
PublicKey,
Multiple connections per peer (#1440) * Allow multiple connections per peer in libp2p-core. Instead of trying to enforce a single connection per peer, which involves quite a bit of additional complexity e.g. to prioritise simultaneously opened connections and can have other undesirable consequences [1], we now make multiple connections per peer a feature. The gist of these changes is as follows: The concept of a "node" with an implicit 1-1 correspondence to a connection has been replaced with the "first-class" concept of a "connection". The code from `src/nodes` has moved (with varying degrees of modification) to `src/connection`. A `HandledNode` has become a `Connection`, a `NodeHandler` a `ConnectionHandler`, the `CollectionStream` was the basis for the new `connection::Pool`, and so forth. Conceptually, a `Network` contains a `connection::Pool` which in turn internally employs the `connection::Manager` for handling the background `connection::manager::Task`s, one per connection, as before. These are all considered implementation details. On the public API, `Peer`s are managed as before through the `Network`, except now the API has changed with the shift of focus to (potentially multiple) connections per peer. The `NetworkEvent`s have accordingly also undergone changes. The Swarm APIs remain largely unchanged, except for the fact that `inject_replaced` is no longer called. It may now practically happen that multiple `ProtocolsHandler`s are associated with a single `NetworkBehaviour`, one per connection. If implementations of `NetworkBehaviour` rely somehow on communicating with exactly one `ProtocolsHandler`, this may cause issues, but it is unlikely. [1]: https://github.com/paritytech/substrate/issues/4272 * Fix intra-rustdoc links. * Update core/src/connection/pool.rs Co-Authored-By: Max Inden <mail@max-inden.de> * Address some review feedback and fix doc links. * Allow responses to be sent on the same connection. * Remove unnecessary remainders of inject_replaced. * Update swarm/src/behaviour.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update swarm/src/lib.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update core/src/connection/manager.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update core/src/connection/manager.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update core/src/connection/pool.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Incorporate more review feedback. * Move module declaration below imports. * Update core/src/connection/manager.rs Co-Authored-By: Toralf Wittner <tw@dtex.org> * Update core/src/connection/manager.rs Co-Authored-By: Toralf Wittner <tw@dtex.org> * Simplify as per review. * Fix rustoc link. * Add try_notify_handler and simplify. * Relocate DialingConnection and DialingAttempt. For better visibility constraints. * Small cleanup. * Small cleanup. More robust EstablishedConnectionIter. * Clarify semantics of `DialingPeer::connect`. * Don't call inject_disconnected on InvalidPeerId. To preserve the previous behavior and ensure calls to `inject_disconnected` are always paired with calls to `inject_connected`. * Provide public ConnectionId constructor. Mainly needed for testing purposes, e.g. in substrate. * Move the established connection limit check to the right place. * Clean up connection error handling. Separate connection errors into those occuring during connection setup or upon rejecting a newly established connection (the `PendingConnectionError`) and those errors occurring on previously established connections, i.e. for which a `ConnectionEstablished` event has been emitted by the connection pool earlier. * Revert change in log level and clarify an invariant. * Remove inject_replaced entirely. * Allow notifying all connection handlers. Thereby simplify by introducing a new enum `NotifyHandler`, used with a single constructor `NetworkBehaviourAction::NotifyHandler`. * Finishing touches. Small API simplifications and code deduplication. Some more useful debug logging. Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-03-04 13:49:25 +01:00
connection::ConnectionId,
upgrade::{ReadOneError, UpgradeError}
};
use libp2p_swarm::{
AddressScore,
NegotiatedSubstream,
NetworkBehaviour,
NetworkBehaviourAction,
PollParameters,
ProtocolsHandler,
ProtocolsHandlerUpgrErr
};
use std::{
collections::{HashMap, VecDeque},
io,
pin::Pin,
task::Context,
task::Poll
};
/// Network behaviour that automatically identifies nodes periodically, returns information
/// about them, and answers identify queries from other nodes.
///
/// All external addresses of the local node supposedly observed by remotes
/// are reported via [`NetworkBehaviourAction::ReportObservedAddr`] with a
/// [score](AddressScore) of `1`.
pub struct Identify {
/// Protocol version to send back to remotes.
protocol_version: String,
/// Agent version to send back to remotes.
agent_version: String,
/// The public key of the local node. To report on the wire.
local_public_key: PublicKey,
/// For each peer we're connected to, the observed address to send back to it.
observed_addresses: HashMap<PeerId, HashMap<ConnectionId, Multiaddr>>,
/// Pending replies to send.
pending_replies: VecDeque<Reply>,
/// Pending events to be emitted when polled.
Cherry-pick commits from master to stable-futures (#1296) * Implement Debug for (ed25519|secp256k1)::(Keypair|SecretKey) (#1285) * Fix possible arithmetic overflow in libp2p-kad. (#1291) When the number of active queries exceeds the (internal) JOBS_MAX_QUERIES limit, which is only supposed to bound the number of concurrent queries relating to background jobs, an arithmetic overflow occurs. This is fixed by using saturating subtraction. * protocols/plaintext: Add example on how to upgrade with PlainTextConfig1 (#1286) * [mdns] - Support for long mDNS names (Bug #1232) (#1287) * Dead code -- commenting out with a note referencing future implementation * Adding "std" feature so that cargo can build in other directories (notably `misc/mdns`, so that I could run these tests) * Permitting `PeerID` to be built from an `Identity` multihash * The length limit for DNS labels is 63 characters, as per RFC1035 * Allocates the vector with capacity for the service name plus additional QNAME encoding bytes * Added support for encoding/decoding peer IDs with an encoded length greater than 63 characters * Removing "std" from ring features Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Retaining MAX_INLINE_KEY_LENGTH with comment about future usage * `segment_peer_id` consumes `peer_id` ... plus an early return for IDs that don't need to be segmented * Fixing logic * Bump most dependencies (#1268) * Bump most dependencies This actually builds 😊. * Bump all dependencies Includes the excellent work of @rschulman in #1265. * Remove use of ed25519-dalek fork * Monomorphize more dependencies * Add compatibility hack for rand Cargo allows a crate to depend on multiple versions of another, but `cargo-web` panics in that situation. Use a wrapper crate to work around the panic. * Use @tomaka’s idea for using a newer `rand` instead of my own ugly hack. * Switch to Parity master as its dependency-bumping PR has been merged. * Update some depenendencies again * Remove unwraps and `#[allow(deprecated)]`. * Remove spurious changes to dependencies Bumping minor or patch versions is not needed, and increases likelyhood of merge conflicts. * Remove some redundant Cargo.toml changes * Replace a retry loop with an expect `ed25519::SecretKey::from_bytes` will never fail for 32-byte inputs. * Revert changes that don’t belong in this PR * Remove using void to bypass ICE (#1295) * Publish 0.13.0 (#1294)
2019-11-06 16:09:15 +01:00
events: VecDeque<NetworkBehaviourAction<(), IdentifyEvent>>,
}
/// A pending reply to an inbound identification request.
enum Reply {
/// The reply is queued for sending.
Queued {
peer: PeerId,
io: ReplySubstream<NegotiatedSubstream>,
observed: Multiaddr
},
/// The reply is being sent.
Sending {
peer: PeerId,
io: Pin<Box<dyn Future<Output = Result<(), io::Error>> + Send>>,
}
}
impl Identify {
/// Creates a new `Identify` network behaviour.
pub fn new(protocol_version: String, agent_version: String, local_public_key: PublicKey) -> Self {
Identify {
protocol_version,
agent_version,
local_public_key,
observed_addresses: HashMap::new(),
pending_replies: VecDeque::new(),
events: VecDeque::new(),
}
}
}
impl NetworkBehaviour for Identify {
type ProtocolsHandler = IdentifyHandler;
type OutEvent = IdentifyEvent;
fn new_handler(&mut self) -> Self::ProtocolsHandler {
IdentifyHandler::new()
}
fn addresses_of_peer(&mut self, _: &PeerId) -> Vec<Multiaddr> {
Vec::new()
}
fn inject_connected(&mut self, _: &PeerId) {
}
fn inject_connection_established(&mut self, peer_id: &PeerId, conn: &ConnectionId, endpoint: &ConnectedPoint) {
let addr = match endpoint {
ConnectedPoint::Dialer { address } => address.clone(),
ConnectedPoint::Listener { send_back_addr, .. } => send_back_addr.clone(),
};
self.observed_addresses.entry(*peer_id).or_default().insert(*conn, addr);
}
fn inject_connection_closed(&mut self, peer_id: &PeerId, conn: &ConnectionId, _: &ConnectedPoint) {
if let Some(addrs) = self.observed_addresses.get_mut(peer_id) {
addrs.remove(conn);
}
}
fn inject_disconnected(&mut self, peer_id: &PeerId) {
self.observed_addresses.remove(peer_id);
}
Multiple connections per peer (#1440) * Allow multiple connections per peer in libp2p-core. Instead of trying to enforce a single connection per peer, which involves quite a bit of additional complexity e.g. to prioritise simultaneously opened connections and can have other undesirable consequences [1], we now make multiple connections per peer a feature. The gist of these changes is as follows: The concept of a "node" with an implicit 1-1 correspondence to a connection has been replaced with the "first-class" concept of a "connection". The code from `src/nodes` has moved (with varying degrees of modification) to `src/connection`. A `HandledNode` has become a `Connection`, a `NodeHandler` a `ConnectionHandler`, the `CollectionStream` was the basis for the new `connection::Pool`, and so forth. Conceptually, a `Network` contains a `connection::Pool` which in turn internally employs the `connection::Manager` for handling the background `connection::manager::Task`s, one per connection, as before. These are all considered implementation details. On the public API, `Peer`s are managed as before through the `Network`, except now the API has changed with the shift of focus to (potentially multiple) connections per peer. The `NetworkEvent`s have accordingly also undergone changes. The Swarm APIs remain largely unchanged, except for the fact that `inject_replaced` is no longer called. It may now practically happen that multiple `ProtocolsHandler`s are associated with a single `NetworkBehaviour`, one per connection. If implementations of `NetworkBehaviour` rely somehow on communicating with exactly one `ProtocolsHandler`, this may cause issues, but it is unlikely. [1]: https://github.com/paritytech/substrate/issues/4272 * Fix intra-rustdoc links. * Update core/src/connection/pool.rs Co-Authored-By: Max Inden <mail@max-inden.de> * Address some review feedback and fix doc links. * Allow responses to be sent on the same connection. * Remove unnecessary remainders of inject_replaced. * Update swarm/src/behaviour.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update swarm/src/lib.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update core/src/connection/manager.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update core/src/connection/manager.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Update core/src/connection/pool.rs Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com> * Incorporate more review feedback. * Move module declaration below imports. * Update core/src/connection/manager.rs Co-Authored-By: Toralf Wittner <tw@dtex.org> * Update core/src/connection/manager.rs Co-Authored-By: Toralf Wittner <tw@dtex.org> * Simplify as per review. * Fix rustoc link. * Add try_notify_handler and simplify. * Relocate DialingConnection and DialingAttempt. For better visibility constraints. * Small cleanup. * Small cleanup. More robust EstablishedConnectionIter. * Clarify semantics of `DialingPeer::connect`. * Don't call inject_disconnected on InvalidPeerId. To preserve the previous behavior and ensure calls to `inject_disconnected` are always paired with calls to `inject_connected`. * Provide public ConnectionId constructor. Mainly needed for testing purposes, e.g. in substrate. * Move the established connection limit check to the right place. * Clean up connection error handling. Separate connection errors into those occuring during connection setup or upon rejecting a newly established connection (the `PendingConnectionError`) and those errors occurring on previously established connections, i.e. for which a `ConnectionEstablished` event has been emitted by the connection pool earlier. * Revert change in log level and clarify an invariant. * Remove inject_replaced entirely. * Allow notifying all connection handlers. Thereby simplify by introducing a new enum `NotifyHandler`, used with a single constructor `NetworkBehaviourAction::NotifyHandler`. * Finishing touches. Small API simplifications and code deduplication. Some more useful debug logging. Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-03-04 13:49:25 +01:00
fn inject_event(
&mut self,
peer_id: PeerId,
connection: ConnectionId,
event: <Self::ProtocolsHandler as ProtocolsHandler>::OutEvent,
) {
match event {
IdentifyHandlerEvent::Identified(remote) => {
self.events.push_back(
NetworkBehaviourAction::GenerateEvent(
IdentifyEvent::Received {
peer_id,
info: remote.info,
observed_addr: remote.observed_addr.clone(),
}));
self.events.push_back(
NetworkBehaviourAction::ReportObservedAddr {
address: remote.observed_addr,
score: AddressScore::Finite(1),
});
}
IdentifyHandlerEvent::Identify(sender) => {
let observed = self.observed_addresses.get(&peer_id)
.and_then(|addrs| addrs.get(&connection))
.expect("`inject_event` is only called with an established connection \
and `inject_connection_established` ensures there is an entry; qed");
self.pending_replies.push_back(
Reply::Queued {
peer: peer_id,
io: sender,
observed: observed.clone()
});
}
IdentifyHandlerEvent::IdentificationError(error) => {
self.events.push_back(
NetworkBehaviourAction::GenerateEvent(
IdentifyEvent::Error { peer_id, error }));
}
}
}
fn poll(
&mut self,
2020-07-27 20:27:33 +00:00
cx: &mut Context<'_>,
params: &mut impl PollParameters,
) -> Poll<
NetworkBehaviourAction<
<Self::ProtocolsHandler as ProtocolsHandler>::InEvent,
Self::OutEvent,
>,
> {
if let Some(event) = self.events.pop_front() {
return Poll::Ready(event);
}
if let Some(r) = self.pending_replies.pop_front() {
// The protocol names can be bytes, but the identify protocol except UTF-8 strings.
// There's not much we can do to solve this conflict except strip non-UTF-8 characters.
let protocols: Vec<_> = params
.supported_protocols()
.map(|p| String::from_utf8_lossy(&p).to_string())
.collect();
let mut listen_addrs: Vec<_> = params.external_addresses().map(|r| r.addr).collect();
listen_addrs.extend(params.listened_addresses());
let mut sending = 0;
let to_send = self.pending_replies.len() + 1;
let mut reply = Some(r);
loop {
match reply {
Some(Reply::Queued { peer, io, observed }) => {
let info = IdentifyInfo {
public_key: self.local_public_key.clone(),
protocol_version: self.protocol_version.clone(),
agent_version: self.agent_version.clone(),
listen_addrs: listen_addrs.clone(),
protocols: protocols.clone(),
};
let io = Box::pin(io.send(info, &observed));
reply = Some(Reply::Sending { peer, io });
}
Some(Reply::Sending { peer, mut io }) => {
sending += 1;
match Future::poll(Pin::new(&mut io), cx) {
Poll::Ready(Ok(())) => {
let event = IdentifyEvent::Sent { peer_id: peer };
return Poll::Ready(NetworkBehaviourAction::GenerateEvent(event));
},
Poll::Pending => {
self.pending_replies.push_back(Reply::Sending { peer, io });
if sending == to_send {
// All remaining futures are NotReady
break
} else {
reply = self.pending_replies.pop_front();
}
}
Poll::Ready(Err(err)) => {
let event = IdentifyEvent::Error {
peer_id: peer,
error: ProtocolsHandlerUpgrErr::Upgrade(UpgradeError::Apply(err.into()))
};
return Poll::Ready(NetworkBehaviourAction::GenerateEvent(event));
},
}
}
None => unreachable!()
}
}
}
Poll::Pending
}
}
/// Event emitted by the `Identify` behaviour.
#[derive(Debug)]
pub enum IdentifyEvent {
/// Identifying information has been received from a peer.
Received {
/// The peer that has been identified.
peer_id: PeerId,
/// The information provided by the peer.
info: IdentifyInfo,
/// The address observed by the peer for the local node.
observed_addr: Multiaddr,
},
/// Identifying information of the local node has been sent to a peer.
Sent {
/// The peer that the information has been sent to.
peer_id: PeerId,
},
/// Error while attempting to identify the remote.
Error {
/// The peer with whom the error originated.
peer_id: PeerId,
/// The error that occurred.
error: ProtocolsHandlerUpgrErr<ReadOneError>,
},
}
#[cfg(test)]
mod tests {
use crate::{Identify, IdentifyEvent};
use futures::{prelude::*, pin_mut};
use libp2p_core::{
identity,
PeerId,
muxing::StreamMuxerBox,
transport,
Rework the transport upgrade API. (#1240) * Rework the transport upgrade API. ALthough transport upgrades must follow a specific pattern in order fot the resulting transport to be usable with a `Network` or `Swarm`, that pattern is currently not well reflected in the transport upgrade API. Rather, transport upgrades are rather laborious and involve non-trivial code duplication. This commit introduces a `transport::upgrade::Builder` that is obtained from `Transport::upgrade`. The `Builder` encodes the previously implicit rules for transport upgrades: 1. Authentication upgrades must happen first. 2. Any number of upgrades may follow. 3. A multiplexer upgrade must happen last. Since multiplexing is the last (regular) transport upgrade (because that upgrade yields a `StreamMuxer` which is no longer a `AsyncRead` / `AsyncWrite` resource, which the upgrade process is based on), the upgrade starts with `Transport::upgrade` and ends with `Builder::multiplex`, which drops back down to the `Transport`, providing a fluent API. Authentication and multiplexer upgrades must furthermore adhere to a minimal contract w.r.t their outputs: 1. An authentication upgrade is given an (async) I/O resource `C` and must produce a pair `(I, D)` where `I: ConnectionInfo` and `D` is a new (async) I/O resource `D`. 2. A multiplexer upgrade is given an (async) I/O resource `C` and must produce a `M: StreamMuxer`. To that end, two changes to the `secio` and `noise` protocols have been made: 1. The `secio` upgrade now outputs a pair of `(PeerId, SecioOutput)`. The former implements `ConnectionInfo` and the latter `AsyncRead` / `AsyncWrite`, fulfilling the `Builder` contract. 2. A new `NoiseAuthenticated` upgrade has been added that wraps around any noise upgrade (i.e. `NoiseConfig`) and has an output of `(PeerId, NoiseOutput)`, i.e. it checks if the `RemoteIdentity` from the handshake output is an `IdentityKey`, failing if that is not the case. This is the standard upgrade procedure one wants for integrating noise with libp2p-core/swarm. * Cleanup * Add a new integration test. * Add missing license.
2019-09-10 15:42:45 +02:00
Transport,
upgrade
};
use libp2p_noise as noise;
use libp2p_tcp::TcpConfig;
use libp2p_swarm::{Swarm, SwarmEvent};
use libp2p_mplex::MplexConfig;
fn transport() -> (identity::PublicKey, transport::Boxed<(PeerId, StreamMuxerBox)>) {
let id_keys = identity::Keypair::generate_ed25519();
let noise_keys = noise::Keypair::<noise::X25519Spec>::new().into_authentic(&id_keys).unwrap();
let pubkey = id_keys.public();
let transport = TcpConfig::new()
.nodelay(true)
.upgrade(upgrade::Version::V1)
.authenticate(noise::NoiseConfig::xx(noise_keys).into_authenticated())
.multiplex(MplexConfig::new())
.boxed();
(pubkey, transport)
}
#[test]
fn periodic_id_works() {
let (mut swarm1, pubkey1) = {
let (pubkey, transport) = transport();
let protocol = Identify::new("a".to_string(), "b".to_string(), pubkey.clone());
let swarm = Swarm::new(transport, protocol, pubkey.clone().into_peer_id());
(swarm, pubkey)
};
let (mut swarm2, pubkey2) = {
let (pubkey, transport) = transport();
let protocol = Identify::new("c".to_string(), "d".to_string(), pubkey.clone());
let swarm = Swarm::new(transport, protocol, pubkey.clone().into_peer_id());
(swarm, pubkey)
};
Swarm::listen_on(&mut swarm1, "/ip4/127.0.0.1/tcp/0".parse().unwrap()).unwrap();
let listen_addr = async_std::task::block_on(async {
loop {
let swarm1_fut = swarm1.next_event();
pin_mut!(swarm1_fut);
match swarm1_fut.await {
SwarmEvent::NewListenAddr(addr) => return addr,
_ => {}
}
}
});
Swarm::dial_addr(&mut swarm2, listen_addr).unwrap();
// nb. Either swarm may receive the `Identified` event first, upon which
// it will permit the connection to be closed, as defined by
// `IdentifyHandler::connection_keep_alive`. Hence the test succeeds if
// either `Identified` event arrives correctly.
async_std::task::block_on(async move {
loop {
let swarm1_fut = swarm1.next();
pin_mut!(swarm1_fut);
let swarm2_fut = swarm2.next();
pin_mut!(swarm2_fut);
match future::select(swarm1_fut, swarm2_fut).await.factor_second().0 {
future::Either::Left(IdentifyEvent::Received { info, .. }) => {
assert_eq!(info.public_key, pubkey2);
assert_eq!(info.protocol_version, "c");
assert_eq!(info.agent_version, "d");
assert!(!info.protocols.is_empty());
assert!(info.listen_addrs.is_empty());
return;
}
future::Either::Right(IdentifyEvent::Received { info, .. }) => {
assert_eq!(info.public_key, pubkey1);
assert_eq!(info.protocol_version, "a");
assert_eq!(info.agent_version, "b");
assert!(!info.protocols.is_empty());
assert_eq!(info.listen_addrs.len(), 1);
return;
}
_ => {}
}
}
})
}
}