2028 Commits

Author SHA1 Message Date
Thomas Eizinger
7dae6502ed
ci: fix typo configuration of PR validation action (#3278) 2022-12-23 20:19:57 +00:00
Thomas Eizinger
5782a96af2
refactor(swarm)!: don't be generic over Transport (#3272)
Ever since we moved `Pool` into `libp2p-swarm`, we always use it with the same `Transport`: `Boxed`. It is thus unnecessary for us to be overly generic over what kind of `Transport` we are using. This allows us to remove a few type parameters from the implementation which overall simplifies things.

This is technically a breaking change because I am removing a type parameter from two exported type aliases:

- `PendingInboundConnectionError`
- `PendingOutboundConnectionError`

Those have always only be used with `std::io::Error` in our API but it is still a breaking change.
2022-12-23 00:13:34 +00:00
StemCll
aca3454c91
refactor(gossipsub): remove derive-builder dev-dependency (#3270)
Remove the `derive_builder` dev-dependency in gossipsub. We can manually implement the builder functionality on top of the `Default` instance of `InjectNodes`.

Resolved #3228.
2022-12-22 23:47:23 +00:00
Thomas Eizinger
d5f4acc6ed
refactor(swarm): express dial logic linearly (#3253)
Previously, the logic within `Swarm::dial` involved fairly convoluted `match` expressions. This patch refactors this function to use new utility functions introduced on `DialOpts` to handle one concern at a time.

This has the advantage that we are covering slightly more cases now. Because we are parsing the `PeerId` only once at the top, checks like banning will now also act on dials that specify the `PeerId` as part of the `/p2p` protocol.
2022-12-22 16:44:58 +00:00
Thomas Eizinger
1765ae0395
feat(kademlia)!: use GATs on RecordStore trait (#3239)
Previously, we applied a lifetime onto the entire `RecordStore` to workaround Rust not having GATs. With Rust 1.65.0 we now have GATs so we can remove this workaround.

Related https://github.com/libp2p/rust-libp2p/issues/3240. Without this change, we would have to specify HRTB in various places.
2022-12-22 16:17:59 +00:00
Thomas Eizinger
65ec5454ae
ci(dependabot): disable automatic rebase of PRs (#3266)
Currently, we have dependabot configured to automatically rebase PRs as soon as the base branches changes. This however causes two problems:

1. It unnecessarily consumes CI resources because dependabot rebases all open dependency PRs at once upon every merge of a PR into `master`.
2. It does not interact well with our strategy of dismissing reviews on updates to the PR: https://github.com/libp2p/rust-libp2p/pull/3226#event-8074236930

Overall, the rebasing of dependency PRs is a way to fix merge conflicts and to ensure that CI still passes even on the latest version of the base branch. The latter is already ensured by our merge queue and the former can be triggered manually with `@dependabot rebase`, thus disabling automatic rebasing is the better configuration for our setup.
2022-12-22 15:07:13 +00:00
dependabot[bot]
1b4624f74a
deps: Update base64 requirement from 0.13.0 to 0.20.0 (#3226) 2022-12-21 03:35:25 +00:00
Thomas Eizinger
13a59a38d3
ci(caching): split caches by matrix variables (#3257)
Previously, some of the caches for these job runs were overlapping. By incorporating the relevant matrix variables into the cache key, every instance of this job gets its own cache.
2022-12-20 10:24:39 +00:00
Pasha Podolsky
929cbb4670
deps!: Update multiaddr & multihash to 0.17.0 (#3196) 2022-12-20 08:52:08 +00:00
Thomas Eizinger
93335b8818
refactor(dcutr): reshape public API to follow naming guidelines (#3214)
With this patch, the naming of types follows the guidelines discussed in #2217.
2022-12-20 05:03:40 +00:00
dependabot[bot]
06aa694d0a
deps: Bump actions/upload-pages-artifact from 1.0.6 to 1.0.7 (#3259) 2022-12-20 03:57:26 +00:00
dependabot[bot]
ee775137f3
deps(noise): Update ed25519-compact requirement from 1.0.11 to 2.0.4 (#3232) 2022-12-20 01:48:15 +00:00
Thomas Eizinger
88fa8e66b8
refactor(gossipsub): don't store messages within Arc (#3243)
Currently, we store messages to be sent to the `ConnectionHandler` in an `Arc`. However, we never actually clone these messages as we can see with this patch, hence we remove this wrapping.

Related: https://github.com/libp2p/rust-libp2p/pull/3242
2022-12-20 00:59:25 +00:00
Max Inden
de61a74d33
fix(libp2p): Don't specify monorepo dev dependencies by version (#3261)
There is no need to specify dev dependencies from the monorepo by version. Versions can be inferred from the dependencies `Cargo.toml`. Specifying the version increases maintenance overhead, as they have to be bumped manually.
2022-12-19 23:41:07 +00:00
Thomas Eizinger
5458446205
feat(noise): deprecate non-compliant implementation (#3227)
For the last [two years](https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md#version-0190-2020-05-18), we have been carrying around a non-compliant implementation of the noise protocol for libp2p. It is time to follow through on the announcement in that changelog entry and deprecate it. Users should use the spec compliant implementation instead.

1. This will reduce the maintenance effort of our codebase.
2. We will improve compile-times because we no longer need to depend on `libsodium` to test cryptography that is only part of the non-compliant implementation.
3. It will simplify usage of `rust-libp2p` because users cannot accidentally choose the wrong implementation.
2022-12-19 13:28:07 +00:00
Nazar Mokrynskyi
fbd4192e2a
fix(kademlia): downgrade reusable stream dropping warning to debug log (#3234)
There is nothing wrong about being near the edge of the concurrency Kademlia allows. If there was an older stream about to be reused, it doesn't mean there was anything wrong to warn about.
2022-12-19 12:54:39 +00:00
Nick Loadholtes
2dd188e897
feat(floodsub): make use of prost-codec (#3224)
This patch addresses #2500 for the `libp2p-floodsub` crate.

For this PR the existing code was upgraded to use `Framed` with the `prost_codec::Codec` as the standard codec for handling the RPC message format serialization/deserialization.
2022-12-19 11:41:38 +00:00
Anton
76abab9e20
feat: log bandwidth on substream instead of socket level (#3180)
Previously, the `with_bandwidth_logging` extension to `Transport` would track the bytes sent and received on a socket level. This however only works in conjunction with `Transport` upgrades where a separate multiplexer is negotiated on top of a regular stream.

With QUIC and WebRTC landing, this no longer works as those `Transport`s bring their own multiplexing stack. To still allow for tracking of bandwidth, we refactor the `with_bandwidth_logging` extension to count the bytes send on all substreams opened through a `StreamMuxer`. This works, regardless of the underlying transport technology. It does omit certain layers. However, there isn't necessarily a "correct" layer to count bandwidth on because you can always go down another layer (IP, Ethernet, etc).

Closes #3157.
2022-12-19 07:26:19 +00:00
Hannes
160ddc58f2
docs: Add link to rustdocs for master to README (#3197)
I added a link to the master docs to the [README](README.md).
2022-12-19 06:34:29 +00:00
Max Inden
fb45ce3773
fix(metrics): Update connections_establishment_duration buckets (#3256)
Currently the `libp2p_swarm_connections_establishment_duration` metric has the following buckets:

```
// exponential_buckets(1e-3, 2., 10)

[0.001, 0.002, 0.004, 0.008, 0.016, 0.032, 0.064, 0.128, 0.256, 0.512]
```

It is unlikely that a connection is established in 1ms, even within a datacenter. It is very likely that libp2p connection establishment takes longer than 512ms over the internet.

This commit proposes the following buckets:

```
// exponential_buckets(0.01, 1.5, 20)

[0.01, 0.015, 0.0225, 0.03375, 0.050625, 0.0759375, 0.11390625, 0.170859375, 0.2562890625,
0.38443359375, 0.576650390625, 0.8649755859375, 1.29746337890625, 1.946195068359375,
2.9192926025390626, 4.378938903808594, 6.568408355712891, 9.852612533569337, 14.778918800354004,
22.168378200531006]
```

- Buckets start at 10ms.
- Reasonably high resolution in the sub-second area.
- Largest bucket at 22s, e.g. for a relayed connection.
- Unfortunately rather obscure numbers.
2022-12-17 20:49:09 +00:00
Thomas Eizinger
67c741e050
docs(README): fix invalid markdown link (#3250) 2022-12-17 17:57:49 +00:00
Thomas Eizinger
8c139f2d3b
chore: apply suggestions from beta clippy (#3251) 2022-12-17 02:01:45 +00:00
Thomas Eizinger
56398cbb8d
ci(cache): only save caches on master branch (#3249)
Previously, we would always save certain smaller caches like clippy etc on each PR. This still amounts to a lot of caches. With this patch, we conditionally only save them when the workflow is run on the master branch. This gives most pull requests a good base to work with to speed them up significantly without us keeping around a lot of caches and have GitHub invalidate them constantly.

On master, the condition evaluates to `true` which saves the cache: https://github.com/thomaseizinger/rust-libp2p/actions/runs/3699904381/jobs/6267774899#step:5:3
On pull-requests, the condition evaluates to `false` which skips saving it: https://github.com/thomaseizinger/rust-libp2p/actions/runs/3700055536/jobs/6268096357#step:4:3
2022-12-17 00:54:20 +00:00
Thomas Eizinger
e9d5a3014f
ci(cache): override rust version in cache factory for MSRV cache (#3248)
Previously, we forgot to override the default Rust version used to the newly installed one for the MSRV cache. This caused `swatinem/rust-cache` to compute the wrong cache key. See https://github.com/libp2p/rust-libp2p/actions/runs/3697491863/jobs/6262555434#step:5:21 for example.

This caused all pull-requests to not actually find this cache and rebuild everything from scratch: https://github.com/libp2p/rust-libp2p/actions/runs/3699757580/jobs/6267459677#step:6:21

With this patch, we are now considering the correct Rust version: https://github.com/thomaseizinger/rust-libp2p/actions/runs/3699821930/jobs/6267596081#step:5:21
2022-12-16 21:25:19 +00:00
Thomas Eizinger
e960aabefa
docs(pnet): use backticks for referencing types (#3252)
With the update to Rust 1.66, rustdoc is (rightfully) complaining about us writing bad HTML. `<u8>` is interpreted as an unclosed HTML tag. See https://github.com/libp2p/rust-libp2p/actions/runs/3708901592/jobs/6286942010#step:6:1380 for example.
2022-12-16 20:37:31 +00:00
Thomas Eizinger
f6f42968e2
refactor(autonat): buffer entire actions instead of events (#3241)
Previously, we used to buffer events separately and emit actions directly. That is unnecessary. We can have a single place where we return from the `poll` loop and shove all actions into the same buffer.
2022-12-15 16:22:55 +00:00
Max Inden
856e7516ef
fix(quic): Feature flag via async-std and not async-io (#3245)
Feature flag `RECEIVE_BUFFER_SIZE` via `async-std` and not `async-io` feature.
2022-12-15 15:42:22 +00:00
Roland Kuhn
c1e68b75c0
feat(core): Log protocol upgrades on trace (#3128)
Previously, every inbound or outbound upgrade generated a log at `debug` level, without information about the upgrade.
This commit changes it such that successful upgrades are logged at `trace` level (due to ubiquitous use of OneShot handlers) and that the negotiated protocol name is included in the message.
2022-12-14 18:04:50 +00:00
Thomas Eizinger
15ad7987f2
refactor(swarm): don't use task_event_buffer_size for pending connections (#3187)
The task for a pending connection only ever sends one event into this channel: Either a success or a failure. Cloning a sender adds one slot to the capacity of the channel. Hence, we can start this capacity at 0 and have the `cloning` of the `Sender` take care of properly increasing the capacity.
2022-12-14 16:50:06 +00:00
Hannes
d79c93abdb
chore: Implement latest clippy warnings (#3220)
As I do frequently, I corrected for the latest clippy warnings. This will make sure the CI won't complain in the future. We could automate this btw and maybe run the nightly version of clippy.
2022-12-14 15:45:04 +00:00
Thomas Eizinger
be3ec6c62b
refactor(swarm)!: deprecate PollParameters where possible (#3153)
This patch deprecates 3 out of 4 functions on `PollParameters`:

- `local_peer_id`
- `listened_addresses`
- `external_addresses`

The addresses can be obtained by inspecting the `FromSwarm` event. To make this easier, we introduce two utility structs in `libp2p-swarm`:

- `ExternalAddresses`
- `ListenAddresses`

A node's `PeerId` is always known to the caller, thus we can require them to pass it in.

Related: #3124.
2022-12-14 00:50:08 +00:00
Thomas Eizinger
5fe0dc44bd
ci(caching): make caching more effective (#3215)
Currently, we create a new cache for each workflow run for each crate. That ends up blowing the maximum allowed cache size of 10GB and GitHub deletes the least-recently used cache again. Effectively, this means we don't have any caching.

This patch introduces a cache factory workflow that only runs on master and always _saves_ a new cache. The CI workflow run for pull-requests on the other hand only restore these caches but don't save them.
2022-12-13 22:37:19 +00:00
João Oliveira
f80c7141ab
feat: move Identify I/O from NetworkBehaviour to ConnectionHandler (#3208)
Addresses #2885
2022-12-13 20:24:31 +00:00
Max Inden
c39d25ea08
chore(ROADMAP): Mark Kademlia efficient querying as done (#3233)
With https://github.com/libp2p/rust-libp2p/pull/2712 merged, this can be marked as done.
2022-12-13 12:57:08 +00:00
Thomas Eizinger
868c3805b8
ci: enforce PR titles to follow the conventional commit specification (#3204)
As per decision in https://github.com/libp2p/rust-libp2p/pull/3083, we want to use the conventional commits in our repository. To make sure this actually happens, this patch adds a workflow that validates PR titles to follow this convention. Together with our mergify configuration, these PR titles end up being the commit message title when squash-merging a pull request.

Related: https://github.com/libp2p/rust-libp2p/pull/3083
Related: https://github.com/libp2p/github-mgmt/pull/93
2022-12-13 12:37:15 +00:00
João Oliveira
f828db60cb
refactor(request-response): revise public API to follow naming convention (#3159) 2022-12-13 12:11:42 +00:00
Thomas Eizinger
cbf0a273cd
ci(mergify): dismiss approvals on push once send-it is applied (#3231)
Currently, a user can push code after it has been approved and the send-it label applied. We only want to merge code that we actually looked at. Use mergify to dismiss approvals in such circumstances.
2022-12-13 22:33:37 +11:00
Thomas Eizinger
bffe4153bc
fix(dcutr): rename error types to bypass cargo-semver-checks hickup (#3213)
`cargo semver-checks` is still missing features in regards to properly detecting renamed exports. To make our CI pass again, we remove the renamed export, replace it with type-aliases and deprecate them to point users types exported under a module which now follows the conventions set in #2217.
2022-12-13 04:14:57 +00:00
João Oliveira
7dedc351d7
fix: feature-gate symbols to resolve dead-code warning (#3175) 2022-12-13 01:33:00 +00:00
Thomas Eizinger
d7363a53d3
fix: Remove circular dependencies across workspace (#3023)
Circular dependencies are problematic in several ways:

- They result in cognitive overhead for developers, in trying to figure out what depends on what.
- They present `cargo` with limits in what order the crates can be compiled in.
- They invalidate build caches unnecessarily thus forcing `cargo` to rebuild certain crates.
- They cause problems with tooling such as `release-please`.

To actually break the circular dependencies, this patch inlines the uses of `development_transport` in the examples and tests for all sub-crates. This is only meant to be a short-term fix until https://github.com/libp2p/rust-libp2p/issues/3111 and https://github.com/libp2p/rust-libp2p/pull/2888 are fixed.

To ensure we don't accidentally reintroduce this dependency, we add a basic CI that queries `cargo metadata` using `jq`.

Resolves https://github.com/libp2p/rust-libp2p/issues/3053.
Fixes https://github.com/libp2p/rust-libp2p/issues/3223.
Related: https://github.com/libp2p/rust-libp2p/pull/2918#discussion_r976514245
Related: https://github.com/googleapis/release-please/issues/1662
2022-12-12 20:58:01 +00:00
John Turpish
f8f19baad0
time to establish connection (#3134)
Implementing #2745 , adding  a metric to break down time from connection pending to connection established, per protocol stack.

````
$curl -s http://127.0.0.1:42183/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Listener",protocols="/ip4/tcp"} 0.007
libp2p_swarm_connection_establishment_duration_count{role="Listener",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.008"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="+Inf"} 1

lbl@chomp:~lbl
$curl -s http://127.0.0.1:34283/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Dialer",protocols="/ip4/tcp"} 0.009
libp2p_swarm_connection_establishment_duration_count{role="Dialer",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.008"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="+Inf"} 1
````
2022-12-12 14:40:36 +00:00
Piotr Galar
63ffc7fb0f
docs: update security policy with private vulnerability reports info (#3168)
This PR updates the security policy to encourage users to file security vulnerability reports through https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability

The private vulnerability reports will show up here: https://github.com/libp2p/rust-libp2p/security/advisories?state=triage
The maintainers will receive GitHub notification about new private vulnerability reports.
2022-12-12 14:17:05 +00:00
dependabot[bot]
2c8a41c256
build(deps): Bump actions/upload-pages-artifact from 1.0.4 to 1.0.6 (#3219) 2022-12-12 12:22:48 +00:00
Max Inden
7c10942058
chore: Bump libp2p-swarm dependents (#3225)
Follow-up to https://github.com/libp2p/rust-libp2p/pull/3170#issuecomment-1344847006
2022-12-12 11:30:23 +00:00
Max Inden
4fe518de86
feat!: Remove simple::SimpleProtocol (#3191)
Due to lack of usage, remove the `simple::SimpleProtocol`. Users may look at `libp2p::core::upgrade`
for alternatives.
2022-12-09 18:56:30 +01:00
Hannes
8cb79f4c79
chore(swarm): Remove deprecated functions (#3170)
Remove functions deprecated in 0.41.0.
2022-12-09 15:44:08 +01:00
Thomas Eizinger
1c2712c1bc
ci: fix path to interop test plan composition file (#3203) 2022-12-07 12:47:30 +01:00
Thomas Eizinger
4da65089f8
fix(ci): avoid rate limits on GitHub API by supplying auth token (#3199)
Supply token to protoc install.
2022-12-07 09:44:50 +01:00
Justin Restivo
5755942d77
chore(README): Add HotShot as notable user (#3184) 2022-12-06 13:14:37 +01:00
Elena Frank
dfc5ec8e7d
feat(quic): Add draft-29 support (#3151)
Add support for QUIC draft-29 / the `quic` codepoint.  This enables both dialing and listening on `quic` addresses.
The motivation for adding support is to allow users to connect to old go-libp2p nodes that don't support the `quic-v1` codepoint.

**Per default support is disabled.**
2022-12-02 20:43:04 +01:00