We have a peerstore that keeps all data for all observed peers in memory with no eviction.
This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.
We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.
It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.
This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives. It can be configured with a MemoryDatastore if desired.
It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.
BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
The `AbortController` class is supported by browsers and node 14+ - we only support node 16+ (e.g. LTS+Current) so the `abort-controller` module isn't needed any more.
When we've previously seen loads of peers and stored them in the
datastore we'll try to dial them as part of starting the autodial
component.
If we or our peers have bad network connections this can make
starting a libp2p node take ages so don't wait for a round of auto
dialing before considering the component started.
Small refactor of the component stats - adds a `TrackedMap` which encapsulates updating the metrics and means we don't need to null guard on `this._metrics` everywhere.
If metrics are not enabled a regular `Map` is used.
We use timeout controllers to ensure we're not dialling peers forever but we can end up registering lots of listeners for the `abort` event when peers have a lot of addresses.
In node this means we see an unhelpful `MaxListenersExceededWarning` in the console warning of a potential memory leak.
Increase the max number of listeners on the signal to silence the warning.
Implements the idea from #1060 - allows us to get some insight into what's happening in a libp2p node out side of just bandwidth stats.
Configures a few default metrics if metrics are enabled - current connections, the state of the dial queue, etc.
Also makes the `Metrics` class not depend on the `ConnectionManager` class, otherwise we can't collect simple metrics from the connection manager class due to the circular dependency.
If the `Promise.race` throws, execution of the function is terminated so the pending dial target is never removed from the map and we leak memory.
This can happen when there are invalid multiaddrs or when a peer reports more dialable addresses than the threshold.
Instead wrap the `Promise.race` in a `try/finally` which will always remove the pending dial target in the event of success or failure.
Looks like this project stopped running the `test:node` npm script when it was migrated to gh actions.
Re-enable it and fix all the related test failures.
Do not abort all attempts to find peers when `findPeers` on one router throws synchronously
Co-authored-by: Robert Kiel <robert.kiel@hoprnet.io>
Co-authored-by: achingbrain <alex@achingbrain.net>
Sometimes you encounter peers with lots of addresses. When this happens
you can attach more than 10x event listeners to the abort signal we
use to abort all the dials - this causes node to print a warning
which is misleading.
This PR increases the default number of listeners on the signal.
Fixes#900
