feat: async peerstore backed by datastores (#1058)

We have a peerstore that keeps all data for all observed peers in memory with no eviction.

This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.

We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.

It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.

This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives.  It can be configured with a MemoryDatastore if desired.

It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.

BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
This commit is contained in:
Alex Potsides
2022-01-20 12:03:35 +00:00
committed by GitHub
parent 0a4dc54d08
commit 978eb3676f
94 changed files with 3263 additions and 4039 deletions

View File

@ -1,4 +1,3 @@
{
"presets": ["@babel/preset-env"],
"plugins": ["syntax-async-functions","transform-regenerator"]
}

View File

@ -14,12 +14,11 @@
"author": "",
"license": "ISC",
"dependencies": {
"@babel/preset-env": "^7.13.0",
"@chainsafe/libp2p-noise": "^5.0.2",
"libp2p": "../../",
"libp2p-bootstrap": "^0.13.0",
"libp2p-bootstrap": "^0.14.0",
"libp2p-mplex": "^0.10.4",
"@chainsafe/libp2p-noise": "^4.1.0",
"libp2p-webrtc-star": "^0.23.0",
"libp2p-webrtc-star": "^0.25.0",
"libp2p-websockets": "^0.16.1"
},
"devDependencies": {