161 lines
5.3 KiB
Rust
Raw Normal View History

Add libp2p-request-response protocol. (#1596) * Add the libp2p-request-response protocol. This crate provides a generic implementation for request/response protocols, whereby each request is sent on a new substream. * Fix OneShotHandler usage in floodsub. * Custom ProtocolsHandler and multiple protocols. 1. Implement a custom ProtocolsHandler instead of using the OneShotHandler for better control and error handling. In particular, all request/response sending/receiving is kept in the substreams upgrades and thus the background task of a connection. 2. Support multiple protocols (usually protocol versions) with a single `RequestResponse` instance, with configurable inbound/outbound support. * Small doc clarification. * Remove unnecessary Sync bounds. * Remove redundant Clone constraint. * Update protocols/request-response/Cargo.toml Co-authored-by: Toralf Wittner <tw@dtex.org> * Update dev-dependencies. * Update Cargo.tomls. * Add changelog. * Remove Sync bound from RequestResponseCodec::Protocol. Apparently the compiler just needs some help with the scope of borrows, which is unfortunate. * Try async-trait. * Allow checking whether a ResponseChannel is still open. Also expand the commentary on `send_response` to indicate that responses may be discard if they come in too late. * Add `RequestResponse::is_pending`. As an analogue of `ResponseChannel::is_open` for outbound requests. * Revert now unnecessary changes to the OneShotHandler. Since `libp2p-request-response` is no longer using it. * Update CHANGELOG for libp2p-swarm. Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-06-29 17:08:40 +02:00
// Copyright 2020 Parity Technologies (UK) Ltd.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
// DEALINGS IN THE SOFTWARE.
//! The definition of a request/response protocol via inbound
//! and outbound substream upgrades. The inbound upgrade
//! receives a request and sends a response, whereas the
//! outbound upgrade send a request and receives a response.
use crate::RequestId;
use crate::codec::RequestResponseCodec;
Add `Throttled` to libp2p-request-response. (#1696) * Use a single exchange instead of two one_shots. * Add `Throttled` to libp2p-request-response. Wraps the existing `RequestResponse` behaviour and applies strict limits to the number of inbound and outbound requests per peer. The wrapper is opt-in and if not used, the protocol behaviour of `RequestResponse` does not change. This PR also does not introduce an extra protocol, hence the limits applied need to be known a priori for all nodes which is not always possible or desirable. As mentioned in #1687 I think that we should eventually augment the protocol with metadata which allows a more dynamic exchange of requests and responses. This PR also replaces the two oneshot channels with a single one from the scambio crate which saves one allocation per request/response. If not desirable because the crate has seen less testing the first commit could be reverted. * Fix rustdoc error. * Remove some leftovers from development. * Add docs to `NetworBehaviourAction::{map_in,map_out}`. * Apply suggestions from code review Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Add `ping_protocol_throttled` test. * Add another test. * Revert "Use a single exchange instead of two one_shots." This reverts commit e34e1297d411298f6c69e238aa6c96e0b795d989. # Conflicts: # protocols/request-response/Cargo.toml # protocols/request-response/src/handler/protocol.rs * Update CHANGELOG. * Update CHANGELOG. Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-08-12 16:04:54 +02:00
use futures::{channel::oneshot, future::BoxFuture, prelude::*};
use libp2p_core::upgrade::{InboundUpgrade, OutboundUpgrade, UpgradeInfo};
use libp2p_swarm::NegotiatedSubstream;
Add libp2p-request-response protocol. (#1596) * Add the libp2p-request-response protocol. This crate provides a generic implementation for request/response protocols, whereby each request is sent on a new substream. * Fix OneShotHandler usage in floodsub. * Custom ProtocolsHandler and multiple protocols. 1. Implement a custom ProtocolsHandler instead of using the OneShotHandler for better control and error handling. In particular, all request/response sending/receiving is kept in the substreams upgrades and thus the background task of a connection. 2. Support multiple protocols (usually protocol versions) with a single `RequestResponse` instance, with configurable inbound/outbound support. * Small doc clarification. * Remove unnecessary Sync bounds. * Remove redundant Clone constraint. * Update protocols/request-response/Cargo.toml Co-authored-by: Toralf Wittner <tw@dtex.org> * Update dev-dependencies. * Update Cargo.tomls. * Add changelog. * Remove Sync bound from RequestResponseCodec::Protocol. Apparently the compiler just needs some help with the scope of borrows, which is unfortunate. * Try async-trait. * Allow checking whether a ResponseChannel is still open. Also expand the commentary on `send_response` to indicate that responses may be discard if they come in too late. * Add `RequestResponse::is_pending`. As an analogue of `ResponseChannel::is_open` for outbound requests. * Revert now unnecessary changes to the OneShotHandler. Since `libp2p-request-response` is no longer using it. * Update CHANGELOG for libp2p-swarm. Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-06-29 17:08:40 +02:00
use smallvec::SmallVec;
use std::io;
/// The level of support for a particular protocol.
#[derive(Debug, Clone)]
pub enum ProtocolSupport {
/// The protocol is only supported for inbound requests.
Inbound,
/// The protocol is only supported for outbound requests.
Outbound,
/// The protocol is supported for inbound and outbound requests.
Full
}
impl ProtocolSupport {
/// Whether inbound requests are supported.
pub fn inbound(&self) -> bool {
match self {
ProtocolSupport::Inbound | ProtocolSupport::Full => true,
ProtocolSupport::Outbound => false,
}
}
/// Whether outbound requests are supported.
pub fn outbound(&self) -> bool {
match self {
ProtocolSupport::Outbound | ProtocolSupport::Full => true,
ProtocolSupport::Inbound => false,
}
}
}
/// Response substream upgrade protocol.
///
/// Receives a request and sends a response.
#[derive(Debug)]
pub struct ResponseProtocol<TCodec>
where
TCodec: RequestResponseCodec
{
pub(crate) codec: TCodec,
pub(crate) protocols: SmallVec<[TCodec::Protocol; 2]>,
Restore `RequestResponse::throttled`. (#1726) * Restore `RequestResponse::throttled`. In contrast to the existing "throttled" approach this PR adds back- pressure to the protocol without requiring pre-existing knowledge of all nodes about their limits. It adds small, CBOR-encoded headers to the actual payload data. Extra credit messages communicate back to the sender how many more requests it is allowed to send. * Remove some noise. * Resend credit grant after connection closed. Should an error in some lower layer cause a connection to be closed, our previously sent credit grant may not have reached the remote peer. Therefore, pessimistically, a credit grant is resent whenever a connection is closed. The remote ignores duplicate grants. * Remove inbound/outbound tracking per peer. * Send ACK as response to duplicate credit grants. * Simplify. * Fix grammar. * Incorporate review feedback. - Remove `ResponseSent` which was a leftover from previous attemps and issue a credit grant immediately in `send_response`. - Only resend credit grants after a connection is closed if we are still connected to this peer. * Move codec/header.rs to throttled/codec.rs. * More review suggestions. * Generalise `ProtocolWrapper` and use shorter prefix. * Update protocols/request-response/src/lib.rs Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Update protocols/request-response/src/throttled.rs Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Update protocols/request-response/src/throttled.rs Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Minor comment changes. * Limit max. header size to 8KiB * Always construct initial limit with 1. Since honest senders always assume a send budget of 1 and wait for credit afterwards, setting the default limit to a higher value can only become effective after informing the peer about it which means leaving `max_recv` at 1 and setting `next_max` to the desired value. Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-09-07 17:22:40 +02:00
pub(crate) request_sender: oneshot::Sender<(RequestId, TCodec::Request)>,
pub(crate) response_receiver: oneshot::Receiver<TCodec::Response>,
pub(crate) request_id: RequestId
Add libp2p-request-response protocol. (#1596) * Add the libp2p-request-response protocol. This crate provides a generic implementation for request/response protocols, whereby each request is sent on a new substream. * Fix OneShotHandler usage in floodsub. * Custom ProtocolsHandler and multiple protocols. 1. Implement a custom ProtocolsHandler instead of using the OneShotHandler for better control and error handling. In particular, all request/response sending/receiving is kept in the substreams upgrades and thus the background task of a connection. 2. Support multiple protocols (usually protocol versions) with a single `RequestResponse` instance, with configurable inbound/outbound support. * Small doc clarification. * Remove unnecessary Sync bounds. * Remove redundant Clone constraint. * Update protocols/request-response/Cargo.toml Co-authored-by: Toralf Wittner <tw@dtex.org> * Update dev-dependencies. * Update Cargo.tomls. * Add changelog. * Remove Sync bound from RequestResponseCodec::Protocol. Apparently the compiler just needs some help with the scope of borrows, which is unfortunate. * Try async-trait. * Allow checking whether a ResponseChannel is still open. Also expand the commentary on `send_response` to indicate that responses may be discard if they come in too late. * Add `RequestResponse::is_pending`. As an analogue of `ResponseChannel::is_open` for outbound requests. * Revert now unnecessary changes to the OneShotHandler. Since `libp2p-request-response` is no longer using it. * Update CHANGELOG for libp2p-swarm. Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-06-29 17:08:40 +02:00
}
impl<TCodec> UpgradeInfo for ResponseProtocol<TCodec>
where
TCodec: RequestResponseCodec
{
type Info = TCodec::Protocol;
type InfoIter = smallvec::IntoIter<[Self::Info; 2]>;
fn protocol_info(&self) -> Self::InfoIter {
self.protocols.clone().into_iter()
}
}
impl<TCodec> InboundUpgrade<NegotiatedSubstream> for ResponseProtocol<TCodec>
where
TCodec: RequestResponseCodec + Send + 'static,
{
type Output = ();
type Error = io::Error;
type Future = BoxFuture<'static, Result<Self::Output, Self::Error>>;
fn upgrade_inbound(mut self, mut io: NegotiatedSubstream, protocol: Self::Info) -> Self::Future {
async move {
let read = self.codec.read_request(&protocol, &mut io);
let request = read.await?;
Restore `RequestResponse::throttled`. (#1726) * Restore `RequestResponse::throttled`. In contrast to the existing "throttled" approach this PR adds back- pressure to the protocol without requiring pre-existing knowledge of all nodes about their limits. It adds small, CBOR-encoded headers to the actual payload data. Extra credit messages communicate back to the sender how many more requests it is allowed to send. * Remove some noise. * Resend credit grant after connection closed. Should an error in some lower layer cause a connection to be closed, our previously sent credit grant may not have reached the remote peer. Therefore, pessimistically, a credit grant is resent whenever a connection is closed. The remote ignores duplicate grants. * Remove inbound/outbound tracking per peer. * Send ACK as response to duplicate credit grants. * Simplify. * Fix grammar. * Incorporate review feedback. - Remove `ResponseSent` which was a leftover from previous attemps and issue a credit grant immediately in `send_response`. - Only resend credit grants after a connection is closed if we are still connected to this peer. * Move codec/header.rs to throttled/codec.rs. * More review suggestions. * Generalise `ProtocolWrapper` and use shorter prefix. * Update protocols/request-response/src/lib.rs Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Update protocols/request-response/src/throttled.rs Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Update protocols/request-response/src/throttled.rs Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Minor comment changes. * Limit max. header size to 8KiB * Always construct initial limit with 1. Since honest senders always assume a send budget of 1 and wait for credit afterwards, setting the default limit to a higher value can only become effective after informing the peer about it which means leaving `max_recv` at 1 and setting `next_max` to the desired value. Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-09-07 17:22:40 +02:00
if let Ok(()) = self.request_sender.send((self.request_id, request)) {
Add libp2p-request-response protocol. (#1596) * Add the libp2p-request-response protocol. This crate provides a generic implementation for request/response protocols, whereby each request is sent on a new substream. * Fix OneShotHandler usage in floodsub. * Custom ProtocolsHandler and multiple protocols. 1. Implement a custom ProtocolsHandler instead of using the OneShotHandler for better control and error handling. In particular, all request/response sending/receiving is kept in the substreams upgrades and thus the background task of a connection. 2. Support multiple protocols (usually protocol versions) with a single `RequestResponse` instance, with configurable inbound/outbound support. * Small doc clarification. * Remove unnecessary Sync bounds. * Remove redundant Clone constraint. * Update protocols/request-response/Cargo.toml Co-authored-by: Toralf Wittner <tw@dtex.org> * Update dev-dependencies. * Update Cargo.tomls. * Add changelog. * Remove Sync bound from RequestResponseCodec::Protocol. Apparently the compiler just needs some help with the scope of borrows, which is unfortunate. * Try async-trait. * Allow checking whether a ResponseChannel is still open. Also expand the commentary on `send_response` to indicate that responses may be discard if they come in too late. * Add `RequestResponse::is_pending`. As an analogue of `ResponseChannel::is_open` for outbound requests. * Revert now unnecessary changes to the OneShotHandler. Since `libp2p-request-response` is no longer using it. * Update CHANGELOG for libp2p-swarm. Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-06-29 17:08:40 +02:00
if let Ok(response) = self.response_receiver.await {
let write = self.codec.write_response(&protocol, &mut io, response);
write.await?;
}
}
io.close().await?;
Add libp2p-request-response protocol. (#1596) * Add the libp2p-request-response protocol. This crate provides a generic implementation for request/response protocols, whereby each request is sent on a new substream. * Fix OneShotHandler usage in floodsub. * Custom ProtocolsHandler and multiple protocols. 1. Implement a custom ProtocolsHandler instead of using the OneShotHandler for better control and error handling. In particular, all request/response sending/receiving is kept in the substreams upgrades and thus the background task of a connection. 2. Support multiple protocols (usually protocol versions) with a single `RequestResponse` instance, with configurable inbound/outbound support. * Small doc clarification. * Remove unnecessary Sync bounds. * Remove redundant Clone constraint. * Update protocols/request-response/Cargo.toml Co-authored-by: Toralf Wittner <tw@dtex.org> * Update dev-dependencies. * Update Cargo.tomls. * Add changelog. * Remove Sync bound from RequestResponseCodec::Protocol. Apparently the compiler just needs some help with the scope of borrows, which is unfortunate. * Try async-trait. * Allow checking whether a ResponseChannel is still open. Also expand the commentary on `send_response` to indicate that responses may be discard if they come in too late. * Add `RequestResponse::is_pending`. As an analogue of `ResponseChannel::is_open` for outbound requests. * Revert now unnecessary changes to the OneShotHandler. Since `libp2p-request-response` is no longer using it. * Update CHANGELOG for libp2p-swarm. Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-06-29 17:08:40 +02:00
Ok(())
}.boxed()
}
}
/// Request substream upgrade protocol.
///
/// Sends a request and receives a response.
#[derive(Debug, Clone)]
pub struct RequestProtocol<TCodec>
where
TCodec: RequestResponseCodec
{
pub(crate) codec: TCodec,
pub(crate) protocols: SmallVec<[TCodec::Protocol; 2]>,
pub(crate) request_id: RequestId,
pub(crate) request: TCodec::Request,
}
impl<TCodec> UpgradeInfo for RequestProtocol<TCodec>
where
TCodec: RequestResponseCodec
{
type Info = TCodec::Protocol;
type InfoIter = smallvec::IntoIter<[Self::Info; 2]>;
fn protocol_info(&self) -> Self::InfoIter {
self.protocols.clone().into_iter()
}
}
impl<TCodec> OutboundUpgrade<NegotiatedSubstream> for RequestProtocol<TCodec>
where
TCodec: RequestResponseCodec + Send + 'static,
{
type Output = TCodec::Response;
type Error = io::Error;
type Future = BoxFuture<'static, Result<Self::Output, Self::Error>>;
fn upgrade_outbound(mut self, mut io: NegotiatedSubstream, protocol: Self::Info) -> Self::Future {
async move {
let write = self.codec.write_request(&protocol, &mut io, self.request);
write.await?;
io.close().await?;
Add libp2p-request-response protocol. (#1596) * Add the libp2p-request-response protocol. This crate provides a generic implementation for request/response protocols, whereby each request is sent on a new substream. * Fix OneShotHandler usage in floodsub. * Custom ProtocolsHandler and multiple protocols. 1. Implement a custom ProtocolsHandler instead of using the OneShotHandler for better control and error handling. In particular, all request/response sending/receiving is kept in the substreams upgrades and thus the background task of a connection. 2. Support multiple protocols (usually protocol versions) with a single `RequestResponse` instance, with configurable inbound/outbound support. * Small doc clarification. * Remove unnecessary Sync bounds. * Remove redundant Clone constraint. * Update protocols/request-response/Cargo.toml Co-authored-by: Toralf Wittner <tw@dtex.org> * Update dev-dependencies. * Update Cargo.tomls. * Add changelog. * Remove Sync bound from RequestResponseCodec::Protocol. Apparently the compiler just needs some help with the scope of borrows, which is unfortunate. * Try async-trait. * Allow checking whether a ResponseChannel is still open. Also expand the commentary on `send_response` to indicate that responses may be discard if they come in too late. * Add `RequestResponse::is_pending`. As an analogue of `ResponseChannel::is_open` for outbound requests. * Revert now unnecessary changes to the OneShotHandler. Since `libp2p-request-response` is no longer using it. * Update CHANGELOG for libp2p-swarm. Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-06-29 17:08:40 +02:00
let read = self.codec.read_response(&protocol, &mut io);
let response = read.await?;
Ok(response)
}.boxed()
}
}