Updates all deps needed to support passing lists of byte arrays where they have been created from multiple input buffers.
When reading multiplexed data, all messages arrive in length-prefixed buffers, which means the first few bytes tell the consumer how many bytes long next chunk will be.
One length prefixed chunk can be delivered in several payloads from the underlying network transport. The first payload can also include the length prefix and some or all of the data, so we stitch these together in a `Uint8ArrayList` to avoid having to concatenate `Uint8Array`s together.
Previously once we'd received enough bytes to satisfy the length prefix we'd concatenate the bytes together, but this is a potentially expensive operation where transports have small message sizes so instead just pass the `Uint8ArrayList` to the consumer and let them decide wether to concatenate or not as some consumers will be smart enough to operate on lists of `Uint8Array`s instead of always requiring a contiguous block of memory.
BREAKING CHANGE: Streams are now `Duplex<Uint8ArrayList, Uint8ArrayList | Uint8Array>`
* feat: limit protocol streams per-connection
Uses the `maxInboundStreams` and `maxOutboundStreams` of the `registrar.handle`
opts to limit the number of concurrent streams open on each connection
on a per-protocol basis.
Both values default to 1 so some tuning will be necessary to set
appropriate values for some protocols.
* chore: make error codes consistent
* chore: fix up examples
Converts this module to typescript.
- Ecosystem modules renamed from (e.g.) `libp2p-tcp` to `@libp2p/tcp`
- Ecosystem module now have named exports
- Configuration has been updated, now pass instances of modules instead of classes:
- Some configuration keys have been renamed to make them more descriptive. `transport` -> `transports`, `connEncryption` -> `connectionEncryption`. In general where we pass multiple things, the key is now plural, e.g. `streamMuxer` -> `streamMuxers`, `contentRouting` -> `contentRouters`, etc. Where we are configuring a singleton the config key is singular, e.g. `connProtector` -> `connectionProtector` etc.
- Properties of the `modules` config key have been moved to the root
- Properties of the `config` config key have been moved to the root
```js
// before
import Libp2p from 'libp2p'
import TCP from 'libp2p-tcp'
await Libp2p.create({
modules: {
transport: [
TCP
],
}
config: {
transport: {
[TCP.tag]: {
foo: 'bar'
}
},
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
}
}
})
```
```js
// after
import { createLibp2p } from 'libp2p'
import { TCP } from '@libp2p/tcp'
await createLibp2p({
transports: [
new TCP({ foo: 'bar' })
],
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
}
})
```
- Use of `enabled` flag has been reduced - previously you could pass a module but disable it with config. Now if you don't want a feature, just don't pass an implementation. Eg:
```js
// before
await Libp2p.create({
modules: {
transport: [
TCP
],
pubsub: Gossipsub
},
config: {
pubsub: {
enabled: false
}
}
})
```
```js
// after
await createLibp2p({
transports: [
new TCP()
]
})
```
- `.multiaddrs` renamed to `.getMultiaddrs()` because it's not a property accessor, work is done by that method to calculate announce addresses, observed addresses, etc
- `/p2p/${peerId}` is now appended to all addresses returned by `.getMultiaddrs()` so they can be used opaquely (every consumer has to append the peer ID to the address to actually use it otherwise). If you need low-level unadulterated addresses, call methods on the address manager.
BREAKING CHANGE: types are no longer hand crafted, this module is now ESM only
We have a peerstore that keeps all data for all observed peers in memory with no eviction.
This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.
We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.
It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.
This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives. It can be configured with a MemoryDatastore if desired.
It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.
BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
* chore: examples not using secio
* chore(docs): remove unused dep
* chore(docs): remove reference of secio in setup
* chore(docs): replace circuit secio reference with noise
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
* chore: update examples to 0.28 api
* chore: use libp2p-noise in examples
* chore: examples using multiaddrs property of libp2p
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
* docs: update language around secio in crypto example
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
https://github.com/multiformats/js-multiaddr/pull/76 changed the
default protocol from ipfs to p2p.
js-multiaddr is a transitive dependency of peer-info, so in order
to get this change, we had to bump the version of peer-info.
* fix: revert ipfs -> p2p change for some tests
As per PR feedback. Needed for backwards-compatibility.
* docs: update chat example and add info to its readme
* docs: update echo example
* docs: update libp2p in browser example
* docs: update pubsub example
* docs: update peer and content routing examples
* docs: update discovery mechanisms example
* docs: update encrypted comms example
* docs: update protocol and stream muxing example
* feat: add config validation
* test: update CI configs, use only node 8
* feat: new super simplified API
* feat: append peer id to multiaddr if not there
* [WIP] Awesome DHT (#86)
* feat: integrate dht
* better interfaces
* docs: add documentation for peerRouting, contentRouting, dht
* fix: take in passed datastore
* fix: update usage of _getPeerInfo
* fix: getPeerInfo
* docs: update docs
* moar
feat: correctly handle p2p-circuit addrs when creating a peer info object
refactor: rework config options
* feat: adding circuit relaying
* feat: rework circuit relay for protobufs
* feat: circuit loading and tests
* fix: clean up _getPeerInfo to work with /p2p-circuit
* wip: tests cleaup
* test: clean up
* wip
* fix: bringing back test reworks and new aegir
* test: group tests
* test: clean up
* test: adjust test
* fix: use getPeerId to determine if the ipfs fragment is missing
* feat: adding circuit relaying
* feat: circuit loading and tests
* test: clean up
* wip
* feat: upgrade to latest aegir
* fix: removing unused tests
* feat: cleanup tests
* fix: create node defautl options
* chore: upgrade swarm to latest version
* fix: updated aegir and adjust timeouts
* feat: more timeouts
* chore: updating deps
* fix: circle ci builds
* test: timeouts
