2062 Commits

Author SHA1 Message Date
Nick Loadholtes
2dd188e897
feat(floodsub): make use of prost-codec (#3224)
This patch addresses #2500 for the `libp2p-floodsub` crate.

For this PR the existing code was upgraded to use `Framed` with the `prost_codec::Codec` as the standard codec for handling the RPC message format serialization/deserialization.
2022-12-19 11:41:38 +00:00
Anton
76abab9e20
feat: log bandwidth on substream instead of socket level (#3180)
Previously, the `with_bandwidth_logging` extension to `Transport` would track the bytes sent and received on a socket level. This however only works in conjunction with `Transport` upgrades where a separate multiplexer is negotiated on top of a regular stream.

With QUIC and WebRTC landing, this no longer works as those `Transport`s bring their own multiplexing stack. To still allow for tracking of bandwidth, we refactor the `with_bandwidth_logging` extension to count the bytes send on all substreams opened through a `StreamMuxer`. This works, regardless of the underlying transport technology. It does omit certain layers. However, there isn't necessarily a "correct" layer to count bandwidth on because you can always go down another layer (IP, Ethernet, etc).

Closes #3157.
2022-12-19 07:26:19 +00:00
Hannes
160ddc58f2
docs: Add link to rustdocs for master to README (#3197)
I added a link to the master docs to the [README](README.md).
2022-12-19 06:34:29 +00:00
Max Inden
fb45ce3773
fix(metrics): Update connections_establishment_duration buckets (#3256)
Currently the `libp2p_swarm_connections_establishment_duration` metric has the following buckets:

```
// exponential_buckets(1e-3, 2., 10)

[0.001, 0.002, 0.004, 0.008, 0.016, 0.032, 0.064, 0.128, 0.256, 0.512]
```

It is unlikely that a connection is established in 1ms, even within a datacenter. It is very likely that libp2p connection establishment takes longer than 512ms over the internet.

This commit proposes the following buckets:

```
// exponential_buckets(0.01, 1.5, 20)

[0.01, 0.015, 0.0225, 0.03375, 0.050625, 0.0759375, 0.11390625, 0.170859375, 0.2562890625,
0.38443359375, 0.576650390625, 0.8649755859375, 1.29746337890625, 1.946195068359375,
2.9192926025390626, 4.378938903808594, 6.568408355712891, 9.852612533569337, 14.778918800354004,
22.168378200531006]
```

- Buckets start at 10ms.
- Reasonably high resolution in the sub-second area.
- Largest bucket at 22s, e.g. for a relayed connection.
- Unfortunately rather obscure numbers.
2022-12-17 20:49:09 +00:00
Thomas Eizinger
67c741e050
docs(README): fix invalid markdown link (#3250) 2022-12-17 17:57:49 +00:00
Thomas Eizinger
8c139f2d3b
chore: apply suggestions from beta clippy (#3251) 2022-12-17 02:01:45 +00:00
Thomas Eizinger
56398cbb8d
ci(cache): only save caches on master branch (#3249)
Previously, we would always save certain smaller caches like clippy etc on each PR. This still amounts to a lot of caches. With this patch, we conditionally only save them when the workflow is run on the master branch. This gives most pull requests a good base to work with to speed them up significantly without us keeping around a lot of caches and have GitHub invalidate them constantly.

On master, the condition evaluates to `true` which saves the cache: https://github.com/thomaseizinger/rust-libp2p/actions/runs/3699904381/jobs/6267774899#step:5:3
On pull-requests, the condition evaluates to `false` which skips saving it: https://github.com/thomaseizinger/rust-libp2p/actions/runs/3700055536/jobs/6268096357#step:4:3
2022-12-17 00:54:20 +00:00
Thomas Eizinger
e9d5a3014f
ci(cache): override rust version in cache factory for MSRV cache (#3248)
Previously, we forgot to override the default Rust version used to the newly installed one for the MSRV cache. This caused `swatinem/rust-cache` to compute the wrong cache key. See https://github.com/libp2p/rust-libp2p/actions/runs/3697491863/jobs/6262555434#step:5:21 for example.

This caused all pull-requests to not actually find this cache and rebuild everything from scratch: https://github.com/libp2p/rust-libp2p/actions/runs/3699757580/jobs/6267459677#step:6:21

With this patch, we are now considering the correct Rust version: https://github.com/thomaseizinger/rust-libp2p/actions/runs/3699821930/jobs/6267596081#step:5:21
2022-12-16 21:25:19 +00:00
Thomas Eizinger
e960aabefa
docs(pnet): use backticks for referencing types (#3252)
With the update to Rust 1.66, rustdoc is (rightfully) complaining about us writing bad HTML. `<u8>` is interpreted as an unclosed HTML tag. See https://github.com/libp2p/rust-libp2p/actions/runs/3708901592/jobs/6286942010#step:6:1380 for example.
2022-12-16 20:37:31 +00:00
Thomas Eizinger
f6f42968e2
refactor(autonat): buffer entire actions instead of events (#3241)
Previously, we used to buffer events separately and emit actions directly. That is unnecessary. We can have a single place where we return from the `poll` loop and shove all actions into the same buffer.
2022-12-15 16:22:55 +00:00
Max Inden
856e7516ef
fix(quic): Feature flag via async-std and not async-io (#3245)
Feature flag `RECEIVE_BUFFER_SIZE` via `async-std` and not `async-io` feature.
2022-12-15 15:42:22 +00:00
Roland Kuhn
c1e68b75c0
feat(core): Log protocol upgrades on trace (#3128)
Previously, every inbound or outbound upgrade generated a log at `debug` level, without information about the upgrade.
This commit changes it such that successful upgrades are logged at `trace` level (due to ubiquitous use of OneShot handlers) and that the negotiated protocol name is included in the message.
2022-12-14 18:04:50 +00:00
Thomas Eizinger
15ad7987f2
refactor(swarm): don't use task_event_buffer_size for pending connections (#3187)
The task for a pending connection only ever sends one event into this channel: Either a success or a failure. Cloning a sender adds one slot to the capacity of the channel. Hence, we can start this capacity at 0 and have the `cloning` of the `Sender` take care of properly increasing the capacity.
2022-12-14 16:50:06 +00:00
Hannes
d79c93abdb
chore: Implement latest clippy warnings (#3220)
As I do frequently, I corrected for the latest clippy warnings. This will make sure the CI won't complain in the future. We could automate this btw and maybe run the nightly version of clippy.
2022-12-14 15:45:04 +00:00
Thomas Eizinger
be3ec6c62b
refactor(swarm)!: deprecate PollParameters where possible (#3153)
This patch deprecates 3 out of 4 functions on `PollParameters`:

- `local_peer_id`
- `listened_addresses`
- `external_addresses`

The addresses can be obtained by inspecting the `FromSwarm` event. To make this easier, we introduce two utility structs in `libp2p-swarm`:

- `ExternalAddresses`
- `ListenAddresses`

A node's `PeerId` is always known to the caller, thus we can require them to pass it in.

Related: #3124.
2022-12-14 00:50:08 +00:00
Thomas Eizinger
5fe0dc44bd
ci(caching): make caching more effective (#3215)
Currently, we create a new cache for each workflow run for each crate. That ends up blowing the maximum allowed cache size of 10GB and GitHub deletes the least-recently used cache again. Effectively, this means we don't have any caching.

This patch introduces a cache factory workflow that only runs on master and always _saves_ a new cache. The CI workflow run for pull-requests on the other hand only restore these caches but don't save them.
2022-12-13 22:37:19 +00:00
João Oliveira
f80c7141ab
feat: move Identify I/O from NetworkBehaviour to ConnectionHandler (#3208)
Addresses #2885
2022-12-13 20:24:31 +00:00
Max Inden
c39d25ea08
chore(ROADMAP): Mark Kademlia efficient querying as done (#3233)
With https://github.com/libp2p/rust-libp2p/pull/2712 merged, this can be marked as done.
2022-12-13 12:57:08 +00:00
Thomas Eizinger
868c3805b8
ci: enforce PR titles to follow the conventional commit specification (#3204)
As per decision in https://github.com/libp2p/rust-libp2p/pull/3083, we want to use the conventional commits in our repository. To make sure this actually happens, this patch adds a workflow that validates PR titles to follow this convention. Together with our mergify configuration, these PR titles end up being the commit message title when squash-merging a pull request.

Related: https://github.com/libp2p/rust-libp2p/pull/3083
Related: https://github.com/libp2p/github-mgmt/pull/93
2022-12-13 12:37:15 +00:00
João Oliveira
f828db60cb
refactor(request-response): revise public API to follow naming convention (#3159) 2022-12-13 12:11:42 +00:00
Thomas Eizinger
cbf0a273cd
ci(mergify): dismiss approvals on push once send-it is applied (#3231)
Currently, a user can push code after it has been approved and the send-it label applied. We only want to merge code that we actually looked at. Use mergify to dismiss approvals in such circumstances.
2022-12-13 22:33:37 +11:00
Thomas Eizinger
bffe4153bc
fix(dcutr): rename error types to bypass cargo-semver-checks hickup (#3213)
`cargo semver-checks` is still missing features in regards to properly detecting renamed exports. To make our CI pass again, we remove the renamed export, replace it with type-aliases and deprecate them to point users types exported under a module which now follows the conventions set in #2217.
2022-12-13 04:14:57 +00:00
João Oliveira
7dedc351d7
fix: feature-gate symbols to resolve dead-code warning (#3175) 2022-12-13 01:33:00 +00:00
Thomas Eizinger
d7363a53d3
fix: Remove circular dependencies across workspace (#3023)
Circular dependencies are problematic in several ways:

- They result in cognitive overhead for developers, in trying to figure out what depends on what.
- They present `cargo` with limits in what order the crates can be compiled in.
- They invalidate build caches unnecessarily thus forcing `cargo` to rebuild certain crates.
- They cause problems with tooling such as `release-please`.

To actually break the circular dependencies, this patch inlines the uses of `development_transport` in the examples and tests for all sub-crates. This is only meant to be a short-term fix until https://github.com/libp2p/rust-libp2p/issues/3111 and https://github.com/libp2p/rust-libp2p/pull/2888 are fixed.

To ensure we don't accidentally reintroduce this dependency, we add a basic CI that queries `cargo metadata` using `jq`.

Resolves https://github.com/libp2p/rust-libp2p/issues/3053.
Fixes https://github.com/libp2p/rust-libp2p/issues/3223.
Related: https://github.com/libp2p/rust-libp2p/pull/2918#discussion_r976514245
Related: https://github.com/googleapis/release-please/issues/1662
2022-12-12 20:58:01 +00:00
John Turpish
f8f19baad0
time to establish connection (#3134)
Implementing #2745 , adding  a metric to break down time from connection pending to connection established, per protocol stack.

````
$curl -s http://127.0.0.1:42183/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Listener",protocols="/ip4/tcp"} 0.007
libp2p_swarm_connection_establishment_duration_count{role="Listener",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.008"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="+Inf"} 1

lbl@chomp:~lbl
$curl -s http://127.0.0.1:34283/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Dialer",protocols="/ip4/tcp"} 0.009
libp2p_swarm_connection_establishment_duration_count{role="Dialer",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.008"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="+Inf"} 1
````
2022-12-12 14:40:36 +00:00
Piotr Galar
63ffc7fb0f
docs: update security policy with private vulnerability reports info (#3168)
This PR updates the security policy to encourage users to file security vulnerability reports through https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability

The private vulnerability reports will show up here: https://github.com/libp2p/rust-libp2p/security/advisories?state=triage
The maintainers will receive GitHub notification about new private vulnerability reports.
2022-12-12 14:17:05 +00:00
dependabot[bot]
2c8a41c256
build(deps): Bump actions/upload-pages-artifact from 1.0.4 to 1.0.6 (#3219) 2022-12-12 12:22:48 +00:00
Max Inden
7c10942058
chore: Bump libp2p-swarm dependents (#3225)
Follow-up to https://github.com/libp2p/rust-libp2p/pull/3170#issuecomment-1344847006
2022-12-12 11:30:23 +00:00
Max Inden
4fe518de86
feat!: Remove simple::SimpleProtocol (#3191)
Due to lack of usage, remove the `simple::SimpleProtocol`. Users may look at `libp2p::core::upgrade`
for alternatives.
2022-12-09 18:56:30 +01:00
Hannes
8cb79f4c79
chore(swarm): Remove deprecated functions (#3170)
Remove functions deprecated in 0.41.0.
2022-12-09 15:44:08 +01:00
Thomas Eizinger
1c2712c1bc
ci: fix path to interop test plan composition file (#3203) 2022-12-07 12:47:30 +01:00
Thomas Eizinger
4da65089f8
fix(ci): avoid rate limits on GitHub API by supplying auth token (#3199)
Supply token to protoc install.
2022-12-07 09:44:50 +01:00
Justin Restivo
5755942d77
chore(README): Add HotShot as notable user (#3184) 2022-12-06 13:14:37 +01:00
Elena Frank
dfc5ec8e7d
feat(quic): Add draft-29 support (#3151)
Add support for QUIC draft-29 / the `quic` codepoint.  This enables both dialing and listening on `quic` addresses.
The motivation for adding support is to allow users to connect to old go-libp2p nodes that don't support the `quic-v1` codepoint.

**Per default support is disabled.**
2022-12-02 20:43:04 +01:00
dependabot[bot]
c5f5b80c5e
build(deps): Update env_logger requirement from 0.9.0 to 0.10.0 (#3166)
Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/env_logger/compare/v0.9.0...v0.10.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-02 18:06:09 +01:00
Hannes
82a9e2cc74
feat: Host rustdoc of master on GitHub pages (#3126)
This PR hosts the documentation of the current master branch on GitHub Pages. The URL should be: https://libp2p.github.io/rust-libp2p
2022-12-02 15:40:35 +01:00
Max Inden
be0b62a78f
fix(derive): Release breaking change as v0.31.0 (#3178)
`libp2p-swarm-derive` released a breaking change (https://github.com/libp2p/rust-libp2p/pull/3011)
as a patch release `v0.30.2`.

This patch:

1. Prepares a new minor release of `libp2p-swarm-derive`.
2. Prepares a patch release for `libp2p-swarm` to use the minor release of `libp2p-swarm-derive`.

As a follow up we can yank the `libp2p-swarm-derive` `v0.30.2` release. In addition we might want to
release `libp2p-swarm-derive` `v0.30.3` containing all patches but https://github.com/libp2p/rust-libp2p/pull/3011.
2022-11-29 17:45:58 +01:00
Max Inden
5b4eab7baf
fix(tls): add description and repository to Cargo.toml (#3167) 2022-11-25 11:12:46 +01:00
Max Inden
87308f0bc8
chore: Prepare rust-libp2p v0.50.0 release (#3163) 2022-11-25 09:37:55 +00:00
Elena Frank
7b9611e130
fix(quic): allow closing stream after remote dropped it (#3164)
- Only send `STOP_SENDING` on a stream when dropping it if the remote did not finish the stream yet.
- Only call `quinn_proto::SendStream::finish` once. (A second call to it will always fail. Though I don't think this was the issue in #3144.)
- Add tests for reading and writing to streams after the remote dropped. Also adds a smoke test for backpressure.

Fixes #3144.
2022-11-25 09:05:50 +00:00
Friedel Ziegelmayer
a99718162b
protocols/kad: Improve options to efficiently retrieve (#2712) 2022-11-25 08:29:46 +00:00
Hannes
cff84f1897
feat(swarm): Add wasm-bindgen executor (#3115) 2022-11-24 22:12:35 +00:00
Max Inden
98336b7885
Revert "chore: Enforce conventional commit PR title" (#3165)
Revert "chore: Enforce conventional commit PR title (#3083)"

This reverts commit 611286be819b9bbca6acc793e420cafaea8eb0cd.
2022-11-24 21:26:33 +01:00
Thomas Eizinger
611286be81
chore: Enforce conventional commit PR title (#3083) 2022-11-23 20:32:27 +01:00
Hannes
76dba1773b
fix(swarm): Make executors pub and add convenience constructors for SwarmBuilder (#3155)
Addresses issues raised here: https://github.com/libp2p/rust-libp2p/pull/3097#discussion_r1026526312
2022-11-23 18:19:22 +00:00
Thomas Eizinger
2a5a5180e9
fix(kad): Don't wait for behaviour after receiving AddProvider (#3152)
Previously, we would erroneously always go into the `WaitingUser` (now called `WaitingBehaviour`) state after receiving a message on an inbound stream. However, the `AddProvider` message does not warrant a "response" from the behaviour. Thus, any incoming `AddProvider` message would result in a stale substream that would eventually be dropped as soon as more than 32 inbound streams have been opened.

With this patch, we inline the message handling into the upper match block and perform the correct state transition upon each message. For `AddProvider`, we go back into `WaitingMessage` because the spec mandates that we need to be ready to receive more messages on an inbound stream.

Fixes #3048.
2022-11-23 17:46:43 +00:00
Anton Kaliaev
babacc5555
fix(webrtc): Don't append /p2p to listen addresses (#3154)
This aligns with what other transports do.

Related: https://github.com/libp2p/rust-libp2p/pull/2622#discussion_r1024844957.
2022-11-23 16:33:36 +00:00
Michael G
b99a2137ac
docs(swarm): Reword Swarm top-level doc string (#3132) 2022-11-23 02:16:40 +00:00
Thomas Eizinger
2c96d644f9
feat: Better error reporting when features are disabled (#2972)
In case support for e.g. RSA keys is disabled at compile-time, we will now print a better error message. For example:

> Failed to dial Some(PeerId("QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt")): Failed to negotiate transport protocol(s): [(/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt): : Handshake failed: Handshake failed: Invalid public key: Key decoding error: RSA keys are unsupported)]

Fixes #2971.
2022-11-23 00:51:47 +00:00
Predrag Gruevski
9b182778e1
chore(ci): Add --locked flag to cargo install step (#3129)
This PR tweaks the installation of `cargo-semver-checks` to make it use the `--locked` flag.

Installing binaries with their locked dependency versions makes it less likely that you might run into issues caused by bugs in dependency libraries, since your installed dependency versions match the versions used in the binary's own test environment.

This is a recommendation that applies to most Rust binary tools. For example, here's cargo-nextest recommending the same: https://nexte.st/book/installing-from-source.html#installing-from-cratesio
2022-11-20 21:48:24 +00:00