Commit Graph

82 Commits

Author SHA1 Message Date
ab59af4d46 refactor(gossipsub): revise symbol naming to follow conventions (#3303)
Changes regarding the  #2217
2023-01-27 04:44:04 +00:00
d1336a7d81 feat(swarm)!: introduce ListenError (#3375)
In case an error happens for an outgoing connection, `Pool` reports an `OutgoingConnectionError`. This one is mapped to a `DialError` and reported via `SwarmEvent::OutgoingConnectionError` and `FromSwarm::DialFailure`.

For incoming connections, we didn't quite do the same thing. For one, `SwarmEvent::IncomingConnectionError` directly contained a `PendingInboundConnectionError`. Two, `FromSwarm::ListenFailure` did not include an error at all.

With this patch, we now introduce a `ListenError` enum which we use in `SwarmEvent::IncomingConnectionError` and we pass a reference to it along in `FromSwarm::ListenFailure`.
2023-01-26 23:23:55 +00:00
14825c7ecb feat(swarm): remove unused DialError::ConnectionIo variant (#3374)
This variant is never constructed.
2023-01-26 08:10:08 +00:00
9f7145912a feat(swarm)!: report connections to our own PeerId in separate error (#3377)
Previously, inbound connections that happened to resolve to our own `PeerId` were reported as `WrongPeerId`. With this patch, we now report those in a dedicated `LocalPeerId` error.

Related: #3205.
2023-01-26 07:20:23 +00:00
90af08cb59 feat(swarm)!: remove unused PendingConnectionError::Io variant (#3373)
This variant is never constructed.
2023-01-26 05:26:54 +00:00
2621528639 chore(metrics): Upgrade to prometheus-client v0.19.0 (#3207) 2023-01-03 19:42:32 +00:00
9c96bbb54b refactor(relay): revise public API to follow naming convention (#3238)
Continues addressing https://github.com/libp2p/rust-libp2p/issues/2217.
2023-01-02 20:21:46 +00:00
5782a96af2 refactor(swarm)!: don't be generic over Transport (#3272)
Ever since we moved `Pool` into `libp2p-swarm`, we always use it with the same `Transport`: `Boxed`. It is thus unnecessary for us to be overly generic over what kind of `Transport` we are using. This allows us to remove a few type parameters from the implementation which overall simplifies things.

This is technically a breaking change because I am removing a type parameter from two exported type aliases:

- `PendingInboundConnectionError`
- `PendingOutboundConnectionError`

Those have always only be used with `std::io::Error` in our API but it is still a breaking change.
2022-12-23 00:13:34 +00:00
1765ae0395 feat(kademlia)!: use GATs on RecordStore trait (#3239)
Previously, we applied a lifetime onto the entire `RecordStore` to workaround Rust not having GATs. With Rust 1.65.0 we now have GATs so we can remove this workaround.

Related https://github.com/libp2p/rust-libp2p/issues/3240. Without this change, we would have to specify HRTB in various places.
2022-12-22 16:17:59 +00:00
929cbb4670 deps!: Update multiaddr & multihash to 0.17.0 (#3196) 2022-12-20 08:52:08 +00:00
93335b8818 refactor(dcutr): reshape public API to follow naming guidelines (#3214)
With this patch, the naming of types follows the guidelines discussed in #2217.
2022-12-20 05:03:40 +00:00
fb45ce3773 fix(metrics): Update connections_establishment_duration buckets (#3256)
Currently the `libp2p_swarm_connections_establishment_duration` metric has the following buckets:

```
// exponential_buckets(1e-3, 2., 10)

[0.001, 0.002, 0.004, 0.008, 0.016, 0.032, 0.064, 0.128, 0.256, 0.512]
```

It is unlikely that a connection is established in 1ms, even within a datacenter. It is very likely that libp2p connection establishment takes longer than 512ms over the internet.

This commit proposes the following buckets:

```
// exponential_buckets(0.01, 1.5, 20)

[0.01, 0.015, 0.0225, 0.03375, 0.050625, 0.0759375, 0.11390625, 0.170859375, 0.2562890625,
0.38443359375, 0.576650390625, 0.8649755859375, 1.29746337890625, 1.946195068359375,
2.9192926025390626, 4.378938903808594, 6.568408355712891, 9.852612533569337, 14.778918800354004,
22.168378200531006]
```

- Buckets start at 10ms.
- Reasonably high resolution in the sub-second area.
- Largest bucket at 22s, e.g. for a relayed connection.
- Unfortunately rather obscure numbers.
2022-12-17 20:49:09 +00:00
d7363a53d3 fix: Remove circular dependencies across workspace (#3023)
Circular dependencies are problematic in several ways:

- They result in cognitive overhead for developers, in trying to figure out what depends on what.
- They present `cargo` with limits in what order the crates can be compiled in.
- They invalidate build caches unnecessarily thus forcing `cargo` to rebuild certain crates.
- They cause problems with tooling such as `release-please`.

To actually break the circular dependencies, this patch inlines the uses of `development_transport` in the examples and tests for all sub-crates. This is only meant to be a short-term fix until https://github.com/libp2p/rust-libp2p/issues/3111 and https://github.com/libp2p/rust-libp2p/pull/2888 are fixed.

To ensure we don't accidentally reintroduce this dependency, we add a basic CI that queries `cargo metadata` using `jq`.

Resolves https://github.com/libp2p/rust-libp2p/issues/3053.
Fixes https://github.com/libp2p/rust-libp2p/issues/3223.
Related: https://github.com/libp2p/rust-libp2p/pull/2918#discussion_r976514245
Related: https://github.com/googleapis/release-please/issues/1662
2022-12-12 20:58:01 +00:00
f8f19baad0 time to establish connection (#3134)
Implementing #2745 , adding  a metric to break down time from connection pending to connection established, per protocol stack.

````
$curl -s http://127.0.0.1:42183/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Listener",protocols="/ip4/tcp"} 0.007
libp2p_swarm_connection_establishment_duration_count{role="Listener",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.008"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="+Inf"} 1

lbl@chomp:~lbl
$curl -s http://127.0.0.1:34283/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Dialer",protocols="/ip4/tcp"} 0.009
libp2p_swarm_connection_establishment_duration_count{role="Dialer",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.008"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="+Inf"} 1
````
2022-12-12 14:40:36 +00:00
7c10942058 chore: Bump libp2p-swarm dependents (#3225)
Follow-up to https://github.com/libp2p/rust-libp2p/pull/3170#issuecomment-1344847006
2022-12-12 11:30:23 +00:00
8cb79f4c79 chore(swarm): Remove deprecated functions (#3170)
Remove functions deprecated in 0.41.0.
2022-12-09 15:44:08 +01:00
c5f5b80c5e build(deps): Update env_logger requirement from 0.9.0 to 0.10.0 (#3166)
Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/env_logger/compare/v0.9.0...v0.10.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-02 18:06:09 +01:00
87308f0bc8 chore: Prepare rust-libp2p v0.50.0 release (#3163) 2022-11-25 09:37:55 +00:00
a99718162b protocols/kad: Improve options to efficiently retrieve (#2712) 2022-11-25 08:29:46 +00:00
0c85839dab .github/workflows: Refactor CI jobs (#3090)
We refactor our continuous integration workflow with the following goals in mind:

- Run as few jobs as possible
- Have the jobs finish as fast as possible
- Have the jobs redo as little work as possible

There are only so many jobs that GitHub Actions will run in parallel.
Thus, it makes sense to not create massive matrices but instead group
things together meaningfully.

The new `test` job will:

- Run once for each crate
- Ensure that the crate compiles on its specified MSRV
- Ensure that the tests pass
- Ensure that there are no semver violations

This is an improvement to before because we are running all of these
in parallel which speeds up execution and highlights more errors at
once. Previously, tests run later in the pipeline would not get run
at all until you make sure the "first" one passes.

We also previously did not verify the MSRV of each crate, making the
setting in the `Cargo.toml` rather pointless.

The new `cross` job supersedes the existing `wasm` job.

This is an improvement because we now also compile the crate for
windows and MacOS. Something that wasn't checked before.
We assume that checking MSRV and the tests under Linux is good enough.
Hence, this job only checks for compile-errors.

The new `feature_matrix` ensures we compile correctly with certain feature combinations.

`libp2p` exposes a fair few feature-flags. Some of the combinations
are worth checking independently. For the moment, this concerns only
the executor related transports together with the executor flags but
this list can easily be extended.

The new `clippy` job runs for `stable` and `beta` rust.

Clippy gets continuously extended with new lints. Up until now, we would only
learn about those as soon as a new version of Rust is released and CI would
run the new lints. This leads to unrelated failures in CI. Running clippy on with `beta`
Rust gives us a heads-up of 6 weeks before these lints land on stable.

Fixes #2951.
2022-11-18 11:04:16 +00:00
69efe63229 misc/metrics: Add protocols label to address-specific metrics (#2982)
Previously, we would only track the metrics like the number of open connections. With this patch, we extend these metrics with a `protocols` label that contains a "protocol stack". A protocol stack is a multi-address with all variable parts removed. For example, `/ip4/127.0.0.1/tcp/1234` turns into `/ip4/tcp`.

Resolves https://github.com/libp2p/rust-libp2p/issues/2758.
2022-11-15 20:45:14 +00:00
d5ea93dd71 feat(swarm): Make executor for connection tasks explicit (#3097)
Previously, the executor for connection tasks silently defaulted to a `futures::executor::ThreadPool`. This causes issues such as https://github.com/libp2p/rust-libp2p/issues/2230.

With this patch, we force the user to choose, which executor they want to run the connection tasks on which results in overall simpler API with less footguns.

Closes #3068.
2022-11-15 14:26:03 +00:00
afb777e937 swarm-derive: Add prelude configuration option to NetworkBehaviour macro (#3055)
Currently, our `NetworkBehaviour` derive macro depends on the `libp2p` crate to be in scope. This prevents standalone usage which forces us to depend on `libp2p` in all our tests where we want to derive a `NetworkBehaviour`.

This PR introduces a `prelude` option that - by default - points to `libp2p::swarm::derive_prelude`, a new module added to `libp2p_swarm`. With this config option, users of `libp2p_swarm` can now refer to the macro without depending on `libp2p`, breaking the circular dependency in our workspace. For consistency with the ecosystem, the macro is now also re-exported by `libp2p_swarm` instead of `libp2p` at the same position as the trait that it implements.

Lastly, we introduce an off-by-default `macros` feature flag that shrinks the dependency tree for users that don't need the derive macro.
2022-11-12 23:59:14 +00:00
71131e0622 *: Don't leak prost dependency in error types (#3058)
With the current design, a major version bump of `prost` leaks into
all consumers of `prost-codec`.
2022-11-02 12:02:21 +00:00
fcadc83aca *: Use auto_doc_cfg instead of doc(cfg) attributes (#2983)
Co-authored-by: João Oliveira <hello@jxs.pt>
Co-authored-by: Thomas Eizinger <thomas@eizinger.io>
2022-10-24 13:00:20 +11:00
d530e5112c *: Prepare v0.49.0 (#2931) 2022-10-14 15:30:16 +01:00
bdf9209824 swarm: Split off "keep alive" functionality from DummyConnectionHandler (#2859)
Previously, the `DummyConnectionHandler` offered a "keep alive" functionality,
i.e. it allowed users to set the value of what is returned from
`ConnectionHandler::keep_alive`. This handler is primarily used in tests or
`NetworkBehaviour`s that don't open any connections (like mDNS). In all of these
cases, it is statically known whether we want to keep connections alive. As
such, this functionality is better represented by a static
`KeepAliveConnectionHandler` that always returns `KeepAlive::Yes` and a
`DummyConnectionHandler` that always returns `KeepAlive::No`.

To follow the naming conventions described in
https://github.com/libp2p/rust-libp2p/issues/2217, we introduce a top-level
`keep_alive` and `dummy` behaviour in `libp2p-swarm` that contains both the
`NetworkBehaviour` and `ConnectionHandler` implementation for either case.
2022-10-05 17:50:11 +01:00
f13f9edeb4 protocols/identify: Fix bad changelog entries and manifest version (#2974)
Fix bad changelog entries and manifest version for libp2p-identify
2022-10-04 18:06:45 +01:00
a7a96e5502 protocols/identify: Revise symbol naming (#2927) 2022-10-04 11:17:31 +11:00
1da75b2b25 protocols/ping: Properly deprecate types with Ping prefix (#2937)
Co-authored-by: Elena Frank <elena.frank@protonmail.com>
Co-authored-by:  João Oliveira <hello@jxs.pt>
2022-10-01 00:19:34 +10:00
f6bb846c36 *: Remove default features from all crates (#2918)
Remove default features. You need to enable required features
explicitly now. As a quick workaround, you may want to use the
new `full` feature which activates all features.
2022-09-29 16:32:22 +01:00
b28ab2c6bc build(deps): Update lru to 0.8.0 (#2908)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
2022-09-27 11:39:10 +10:00
5906140d38 protocols/kad: Remove deprecated set_protocol_name() (#2866) 2022-09-15 15:30:32 +02:00
72bade1799 build(deps): Update env_logger to 0.9 and criterion to 0.4 (#2896) 2022-09-14 13:01:41 +10:00
83c67954e9 *: Prepare v0.48.0 (#2869) 2022-09-07 09:44:51 +02:00
8644c65a22 core/: Introduce rsa feature flag to avoid ring dependency (#2860)
- Introduce `rsa` feature flag to `libp2p-core`.
- Expose `rsa` feature in `libp2p`.
- Add `rsa` feature to `libp2p` `default`.
2022-09-07 08:16:22 +02:00
ca07ce4d64 swarm/behaviour: Remove deprecated NetworkBehaviourEventProcess (#2840)
Removes the `NetworkBehaviourEventProcess` and all its associated logic.

See deprecation pull request https://github.com/libp2p/rust-libp2p/pull/2784.

Find rational in https://github.com/libp2p/rust-libp2p/pull/2751.
2022-08-26 07:08:33 +02:00
4253080a43 *: Prepare v0.47.0 (#2830) 2022-08-22 05:14:04 +02:00
d2c50530e9 build(deps): Update prometheus-client requirement from 0.17.0 to 0.18.0 (#2822)
* build(deps): Update prometheus-client requirement from 0.17.0 to 0.18.0

Updates the requirements on [prometheus-client](https://github.com/prometheus/client_rust) to permit the latest version.
- [Release notes](https://github.com/prometheus/client_rust/releases)
- [Changelog](https://github.com/prometheus/client_rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_rust/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: prometheus-client
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-17 07:02:21 +02:00
c19a211dfd misc/metrics: fix clippy::assign-op-pattern (#2773) 2022-07-25 10:45:43 +02:00
51a847128c build(deps): Update prometheus-client requirement from 0.16.0 to 0.17.0 (#2761)
* build(deps): Update prometheus-client requirement from 0.16.0 to 0.17.0

Updates the requirements on [prometheus-client](https://github.com/prometheus/client_rust) to permit the latest version.
- [Release notes](https://github.com/prometheus/client_rust/releases)
- [Changelog](https://github.com/prometheus/client_rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_rust/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: prometheus-client
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-22 11:20:39 +02:00
1a553db596 core/muxing: Flatten StreamMuxer interface to poll_{inbound,outbound,address_change,close} (#2724)
Instead of having a mix of `poll_event`, `poll_outbound` and `poll_close`, we
flatten the entire interface of `StreamMuxer` into 4 individual functions:

- `poll_inbound`
- `poll_outbound`
- `poll_address_change`
- `poll_close`

This design is closer to the design of other async traits like `AsyncRead` and
`AsyncWrite`. It also allows us to delete the `StreamMuxerEvent`.
2022-07-18 05:20:11 +02:00
d4f8ec2d48 misc/metrics: Track # connected nodes supporting specific protocol (#2734)
* misc/metrics: Explicitly delegate event recording to each recorder

This allows delegating a single event to multiple `Recorder`s. That enables e.g. the
`identify::Metrics` `Recorder` to act both on `IdentifyEvent` and `SwarmEvent`. The latter enables
it to garbage collect per peer data on disconnects.

* protocols/dcutr: Expose PROTOCOL_NAME

* protocols/identify: Expose PROTOCOL_NAME and PUSH_PROTOCOL_NAME

* protocols/ping: Expose PROTOCOL_NAME

* protocols/relay: Expose HOP_PROTOCOL_NAME and STOP_PROTOCOL_NAME

* misc/metrics: Track # connected nodes supporting specific protocol

An example metric exposed with this patch:

```
libp2p_identify_protocols{protocol="/ipfs/ping/1.0.0"} 10
```

This implies that 10 of the currently connected nodes support the ping protocol.
2022-07-15 09:16:03 +02:00
d0da3a0973 swarm/: Set default dial concurrency factor to 8 (#2741) 2022-07-07 11:20:03 +02:00
7df6bae520 *: Prepare v0.46.0 (#2730) 2022-07-05 13:09:58 +02:00
eb490c08e9 core/muxing: Force StreamMuxer::Substream to implement Async{Read,Write} (#2707)
Co-authored-by: Max Inden <mail@max-inden.de>
2022-06-23 21:52:11 +10:00
cb3784a8b3 misc/metrics: Use correct event name for dcutr metrics (#2714) 2022-06-21 07:04:18 +02:00
6e1e314872 *: Prepare v0.45.0 (#2662) 2022-05-31 13:12:53 +02:00
686d6ff363 misc/metrics/README: Reference sample Grafana dashboard (#2655) 2022-05-20 09:27:03 +02:00
93fa86c263 build(deps): Update prometheus-client requirement from 0.15.0 to 0.16.0 (#2631)
* build(deps): Update prometheus-client requirement from 0.15.0 to 0.16.0

Updates the requirements on [prometheus-client](https://github.com/prometheus/client_rust) to permit the latest version.
- [Release notes](https://github.com/prometheus/client_rust/releases)
- [Changelog](https://github.com/prometheus/client_rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_rust/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: prometheus-client
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* *: Update changelogs

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
2022-05-06 14:57:40 +02:00