Compare commits

...

16 Commits

Author SHA1 Message Date
ed2dbd9bea fix: return empty array when no multiaddrs are known
Returning `undefined` makes the address length check in [dialler/index.js](https://github.com/libp2p/js-libp2p/blob/master/src/dialer/index.js#L73-L75)
fail with `cannot read property length of undefined` so the change
here is to always return an array, but it might be empty if we don't
know any multiaddrs for the given peer.
2020-07-20 10:51:29 +01:00
856b38de67 chore: add migration guide template (#711) 2020-07-16 18:14:02 +02:00
798d7b73c1 chore: release version v0.28.7 2020-07-14 19:13:55 +02:00
f2d0d8b51d chore: update contributors 2020-07-14 19:13:54 +02:00
999c1b7740 fix: retimer reschedule does not work as interval (#710)
* fix: retimer reschedule does not work as interval

* chore: apply suggestions from code review

Co-authored-by: Jacob Heun <jacobheun@gmail.com>

Co-authored-by: Jacob Heun <jacobheun@gmail.com>
2020-07-14 19:11:29 +02:00
99ff361a55 chore: release version v0.28.6 2020-07-14 16:13:00 +02:00
6115f8b680 chore: update contributors 2020-07-14 16:12:59 +02:00
9ccab40fc8 fix: not dial all known peers in parallel on startup (#698)
* fix: not dial all known peers on startup

* feat: connection manager should proactively connect to peers from peerStore

* chore: increase bundle size

* fix: do connMgr proactive dial on an interval

* chore: address review

* chore: use retimer reschedule

* chore: address review

* fix: use minConnections in default config

* chore: minPeers to minConnections everywhere
2020-07-14 16:05:26 +02:00
619e5dd73c chore: release version v0.28.5 2020-07-10 17:26:21 +02:00
0fd23f6a5f chore: update contributors 2020-07-10 17:26:21 +02:00
5a84dd56d0 fix: pass libp2p to the dht (#700) 2020-07-10 17:16:45 +02:00
7b05d6922b docs: add dht discovery documentation reference (#697)
Extended the description with a link for how to configure the libp2p-kad-dht for peer discovery.
2020-07-08 15:03:56 +02:00
08a4fad80b chore: add modules to update in 0.28.x migration (#695) 2020-07-08 14:12:40 +02:00
51da8874d8 test: add pubsub reconnect test (#693)
* test: add pubsub reconnect test

* chore: dep bump

* chore: remove temp pubsub dep

Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>

Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
2020-07-07 18:31:51 +02:00
5cd8c19567 docs(fix): update link to ipfs config files (#690) 2020-07-07 12:56:17 +02:00
db766eaca9 chore: update circuit readme (#689)
* chore: update circuit readme

* chore: address review
2020-07-07 12:55:52 +02:00
19 changed files with 458 additions and 133 deletions

View File

@ -45,7 +45,7 @@ const after = async () => {
}
module.exports = {
bundlesize: { maxSize: '200kB' },
bundlesize: { maxSize: '202kB' },
hooks: {
pre: before,
post: after

View File

@ -1,3 +1,33 @@
<a name="0.28.7"></a>
## [0.28.7](https://github.com/libp2p/js-libp2p/compare/v0.28.6...v0.28.7) (2020-07-14)
### Bug Fixes
* retimer reschedule does not work as interval ([#710](https://github.com/libp2p/js-libp2p/issues/710)) ([999c1b7](https://github.com/libp2p/js-libp2p/commit/999c1b7))
<a name="0.28.6"></a>
## [0.28.6](https://github.com/libp2p/js-libp2p/compare/v0.28.5...v0.28.6) (2020-07-14)
### Bug Fixes
* not dial all known peers in parallel on startup ([#698](https://github.com/libp2p/js-libp2p/issues/698)) ([9ccab40](https://github.com/libp2p/js-libp2p/commit/9ccab40))
<a name="0.28.5"></a>
## [0.28.5](https://github.com/libp2p/js-libp2p/compare/v0.28.4...v0.28.5) (2020-07-10)
### Bug Fixes
* pass libp2p to the dht ([#700](https://github.com/libp2p/js-libp2p/issues/700)) ([5a84dd5](https://github.com/libp2p/js-libp2p/commit/5a84dd5))
<a name="0.28.4"></a>
## [0.28.4](https://github.com/libp2p/js-libp2p/compare/v0.28.3...v0.28.4) (2020-07-03)

45
MIGRATION_TEMPLATE.md Normal file
View File

@ -0,0 +1,45 @@
<!--Specify versions for migration below-->
# Migrating to libp2p@__
A migration guide for refactoring your application code from libp2p v__ to v__.
## Table of Contents
- [API](#api)
- [Module Updates](#module-updates)
## API
<!--Describe breaking APIs with examples for Before and After
Example:
### Peer Discovery
__Describe__
**Before**
```js
```
**After**
```js
```
-->
## Module Updates
With this release you should update the following libp2p modules if you are relying on them:
<!--Specify module versions in JSON for migration below.
It's recommended to check package.json changes for this:
`git diff <release> <prev> -- package.json`
-->
```json
```

View File

@ -270,7 +270,7 @@ const node = await Libp2p.create({
},
config: {
peerDiscovery: {
autoDial: true, // Auto connect to discovered peers (limited by ConnectionManager minPeers)
autoDial: true, // Auto connect to discovered peers (limited by ConnectionManager minConnections)
// The `tag` property will be searched when creating the instance of your Peer Discovery service.
// The associated object, will be passed to the service when it is instantiated.
[MulticastDNS.tag]: {
@ -649,8 +649,8 @@ const node = await Libp2p.create({
As libp2p is designed to be a modular networking library, its usage will vary based on individual project needs. We've included links to some existing project configurations for your reference, in case you wish to replicate their configuration:
- [libp2p-ipfs-nodejs](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/libp2p-nodejs.js) - libp2p configuration used by js-ipfs when running in Node.js
- [libp2p-ipfs-browser](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/libp2p-browser.js) - libp2p configuration used by js-ipfs when running in a Browser (that supports WebRTC)
- [libp2p-ipfs-nodejs](https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/src/core/runtime/libp2p-nodejs.js) - libp2p configuration used by js-ipfs when running in Node.js
- [libp2p-ipfs-browser](https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/src/core/runtime/libp2p-browser.js) - libp2p configuration used by js-ipfs when running in a Browser (that supports WebRTC)
If you have developed a project using `js-libp2p`, please consider submitting your configuration to this list so that it can be found easily by other users.

View File

@ -217,7 +217,7 @@ const node = await Libp2p.create({
},
config: {
peerDiscovery: {
autoDial: true, // Auto connect to discovered peers (limited by ConnectionManager minPeers)
autoDial: true, // Auto connect to discovered peers (limited by ConnectionManager minConnections)
// The `tag` property will be searched when creating the instance of your Peer Discovery service.
// The associated object, will be passed to the service when it is instantiated.
[Bootstrap.tag]: {

View File

@ -10,6 +10,7 @@ A migration guide for refactoring your application code from libp2p v0.27.x to v
- [API Implications](#api-implications)
- [Connection Manager and Registrar](#connection-manager-and-registrar)
- [Events](#events)
- [Module Updates](#module-updates)
## PeerStore API
@ -321,6 +322,21 @@ libp2p.on('peer:discovery', (peerId) => {
})
```
## Module Updates
With `libp2p@0.28` you should update the following libp2p modules if you are relying on them:
```json
"libp2p-bootstrap": "^0.11.0",
"libp2p-delegated-content-routing": "^0.5.0",
"libp2p-delegated-peer-routing": "^0.5.0",
"libp2p-floodsub": "^0.21.0",
"libp2p-gossipsub": "^0.4.0",
"libp2p-kad-dht": "^0.19.1",
"libp2p-mdns": "^0.14.1",
"libp2p-webrtc-star": "^0.18.0"
```
[connection]: https://github.com/libp2p/js-interfaces/tree/master/src/connection
[multiaddr]: https://github.com/multiformats/js-multiaddr
[peer-id]: https://github.com/libp2p/js-peer-id

View File

@ -170,5 +170,5 @@ Discovered: QmRcXXhtG8vTqwVBRonKWtV4ovDoC1Fe56WYtcrw694eiJ
There are plenty more Peer Discovery Mechanisms out there, you can:
- Find one in [libp2p-webrtc-star](https://github.com/libp2p/js-libp2p-webrtc-star). Yes, a transport with discovery capabilities! This happens because WebRTC requires a rendezvous point for peers to exchange [SDP](https://tools.ietf.org/html/rfc4317) offer, which means we have one or more points that can introduce peers to each other. Think of it as MulticastDNS for the Web, as in MulticastDNS only works in LAN.
- Any DHT will offer you a discovery capability. You can simple _random-walk_ the routing tables to find other peers to connect to.
- Any DHT will offer you a discovery capability. You can simple _random-walk_ the routing tables to find other peers to connect to. For example [libp2p-kad-dht](https://github.com/libp2p/js-libp2p-kad-dht) can be used for peer discovery. An example how to configure it to enable random walks can be found [here](https://github.com/libp2p/js-libp2p/blob/v0.28.4/doc/CONFIGURATION.md#customizing-dht).
- You can create your own Discovery service, a registry, a list, a radio beacon, you name it!

View File

@ -1,6 +1,6 @@
{
"name": "libp2p",
"version": "0.28.4",
"version": "0.28.7",
"description": "JavaScript implementation of libp2p, a modular peer to peer network stack",
"leadMaintainer": "Jacob Heun <jacobheun@gmail.com>",
"main": "src/index.js",
@ -102,7 +102,7 @@
"libp2p-delegated-content-routing": "^0.5.0",
"libp2p-delegated-peer-routing": "^0.5.0",
"libp2p-floodsub": "^0.21.0",
"libp2p-gossipsub": "^0.4.0",
"libp2p-gossipsub": "^0.4.6",
"libp2p-kad-dht": "^0.19.1",
"libp2p-mdns": "^0.14.1",
"libp2p-mplex": "^0.9.5",
@ -131,37 +131,38 @@
"Alex Potsides <alex@achingbrain.net>",
"Maciej Krüger <mkg20001@gmail.com>",
"Hugo Dias <mail@hugodias.me>",
"dirkmc <dirkmdev@gmail.com>",
"Volker Mische <volker.mische@gmail.com>",
"dirkmc <dirkmdev@gmail.com>",
"Richard Littauer <richard.littauer@gmail.com>",
"Elven <mon.samuel@qq.com>",
"Andrew Nesbitt <andrewnez@gmail.com>",
"Giovanni T. Parra <fiatjaf@gmail.com>",
"Ryan Bell <ryan@piing.net>",
"Thomas Eizinger <thomas@eizinger.io>",
"Ryan Bell <ryan@piing.net>",
"Giovanni T. Parra <fiatjaf@gmail.com>",
"Andrew Nesbitt <andrewnez@gmail.com>",
"ᴠɪᴄᴛᴏʀ ʙᴊᴇʟᴋʜᴏʟᴍ <victorbjelkholm@gmail.com>",
"Didrik Nordström <didrik@betamos.se>",
"Francis Gulotta <wizard@roborooter.com>",
"Julien Bouquillon <contact@revolunet.com>",
"Kevin Kwok <antimatter15@gmail.com>",
"Florian-Merle <florian.david.merle@gmail.com>",
"Nuno Nogueira <nunofmn@gmail.com>",
"Fei Liu <liu.feiwood@gmail.com>",
"RasmusErik Voel Jensen <github@solsort.com>",
"robertkiel <robert.kiel@validitylabs.org>",
"Soeren <nikorpoulsen@gmail.com>",
"Sönke Hahn <soenkehahn@gmail.com>",
"Elven <mon.samuel@qq.com>",
"Didrik Nordström <didrik.nordstrom@gmail.com>",
"Tiago Alves <alvesjtiago@gmail.com>",
"Dmitriy Ryajov <dryajov@gmail.com>",
"Diogo Silva <fsdiogo@gmail.com>",
"Yusef Napora <yusef@napora.org>",
"Zane Starr <zcstarr@gmail.com>",
"Daijiro Wachi <daijiro.wachi@gmail.com>",
"ebinks <elizabethjbinks@gmail.com>",
"Chris Bratlien <chrisbratlien@gmail.com>",
"isan_rivkin <isanrivkin@gmail.com>",
"robertkiel <robert.kiel@validitylabs.org>",
"RasmusErik Voel Jensen <github@solsort.com>",
"Bernd Strehl <bernd.strehl@gmail.com>",
"Chris Bratlien <chrisbratlien@gmail.com>",
"Daijiro Wachi <daijiro.wachi@gmail.com>",
"Diogo Silva <fsdiogo@gmail.com>",
"Dmitriy Ryajov <dryajov@gmail.com>",
"Fei Liu <liu.feiwood@gmail.com>",
"Florian-Merle <florian.david.merle@gmail.com>",
"Francis Gulotta <wizard@roborooter.com>",
"Henrique Dias <hacdias@gmail.com>",
"Irakli Gozalishvili <rfobic@gmail.com>",
"Joel Gustafson <joelg@mit.edu>"
"Joel Gustafson <joelg@mit.edu>",
"Julien Bouquillon <contact@revolunet.com>",
"Kevin Kwok <antimatter15@gmail.com>",
"Nuno Nogueira <nunofmn@gmail.com>",
"Soeren <nikorpoulsen@gmail.com>",
"Sönke Hahn <soenkehahn@gmail.com>"
]
}

View File

@ -1,12 +1,20 @@
# js-libp2p-circuit
> Node.js implementation of the Circuit module that libp2p uses, which implements the [interface-connection](https://github.com/libp2p/js-interfaces/tree/master/src/connection) interface for dial/listen.
> Node.js implementation of the Circuit module that libp2p uses, which implements the [interface-connection](https://github.com/libp2p/js-libp2p-interfaces/tree/master/src/connection) interface for dial/listen.
**Note**: git history prior to merging into js-libp2p can be found in the original repository, https://github.com/libp2p/js-libp2p-circuit.
`libp2p-circuit` implements the circuit-relay mechanism that allows nodes that don't speak the same protocol to communicate using a third _relay_ node.
`libp2p-circuit` implements the circuit-relay mechanism that allows nodes that don't speak the same protocol to communicate using a third _relay_ node. You can read more about this in its [spec](https://github.com/libp2p/specs/tree/master/relay).
This module uses [pull-streams](https://pull-stream.github.io) for all stream based interfaces.
## Table of Contents
- [js-libp2p-circuit](#js-libp2p-circuit)
- [Why?](#why)
- [libp2p-circuit and IPFS](#libp2p-circuit-and-ipfs)
- [Table of Contents](#table-of-contents)
- [Usage](#usage)
- [API](#api)
- [Implementation rational](#implementation-rational)
### Why?
@ -16,77 +24,42 @@ The use of circuit-relaying is not limited to routing traffic between browser no
- routing traffic between private nets and circumventing NAT layers
- route mangling for better privacy (matreshka/shallot dialing).
It's also possible to use it for clients that implement exotic transports such as devices that only have bluetooth radios to be reachable over bluetooth enabled relays and become full p2p nodes.
It's also possible to use it for clients that implement exotic transports such as devices that only have bluetooth radios to be reachable over bluetooth enabled relays and become full p2p nodes.
### libp2p-circuit and IPFS
Prior to `libp2p-circuit` there was a rift in the IPFS network, were IPFS nodes could only access content from nodes that speak the same protocol, for example TCP only nodes could only dial to other TCP only nodes, same for any other protocol combination. In practice, this limitation was most visible in JS-IPFS browser nodes, since they can only dial out but not be dialed in over WebRTC or WebSockets, hence any content that the browser node held was not reachable by the rest of the network even through it was announced on the DHT. Non browser IPFS nodes would usually speak more than one protocol such as TCP, WebSockets and/or WebRTC, this made the problem less severe outside of the browser. `libp2p-circuit` solves this problem completely, as long as there are `relay nodes` capable of routing traffic between those nodes their content should be available to the rest of the IPFS network.
## Table of Contents
- [js-libp2p-circuit](#js-libp2p-circuit)
- [Why?](#why)
- [libp2p-circuit and IPFS](#libp2p-circuit-and-ipfs)
- [Table of Contents](#table-of-contents)
- [Usage](#usage)
- [Example](#example)
- [Create dialer/listener](#create-dialerlistener)
- [Create `relay`](#create-relay)
- [API](#api)
- [Implementation rational](#implementation-rational)
## Usage
### Example
Libp2p circuit configuration can be seen at [Setup with Relay](../../doc/CONFIGURATION.md#setup-with-relay).
#### Create dialer/listener
Once you have a circuit relay node running, you can configure other nodes to use it as a relay as follows:
```js
const Circuit = require('libp2p-circuit')
const multiaddr = require('multiaddr')
const pull = require('pull-stream')
const Libp2p = require('libp2p')
const TCP = require('libp2p-tcp')
const MPLEX = require('libp2p-mplex')
const SECIO = require('libp2p-secio')
const mh1 = multiaddr('/p2p-circuit/p2p/QmHash') // dial /p2p/QmHash over any circuit
const relayAddr = ...
const circuit = new Circuit(swarmInstance, options) // pass swarm instance and options
const listener = circuit.createListener(mh1, (connection) => {
console.log('new connection opened')
pull(
pull.values(['hello']),
socket
)
const node = await Libp2p.create({
addresses: {
listen: [multiaddr(`${relayAddr}/p2p-circuit`)]
},
modules: {
transport: [TCP],
streamMuxer: [MPLEX],
connEncryption: [SECIO]
},
config: {
relay: { // Circuit Relay options (this config is part of libp2p core configurations)
enabled: true // Allows you to dial and accept relayed connections. Does not make you a relay.
}
}
})
listener.listen(() => {
console.log('listening')
pull(
circuit.dial(mh1),
pull.log,
pull.onEnd(() => {
circuit.close()
})
)
})
```
Outputs:
```sh
listening
new connection opened
hello
```
#### Create `relay`
```js
const Relay = require('libp2p-circuit').Relay
const relay = new Relay(options)
relay.mount(swarmInstance) // start relaying traffic
```
## API
@ -101,7 +74,7 @@ Both for dialing and listening.
### Implementation rational
This module is not a transport, however it implements `interface-transport` interface in order to allow circuit to be plugged with `libp2p-swarm`. The rational behind it is that, `libp2p-circuit` has a dial and listen flow, which fits nicely with other transports, moreover, it requires the _raw_ connection to be encrypted and muxed just as a regular transport's connection does. All in all, `interface-transport` ended up being the correct level of abstraction for circuit, as well as allowed us to reuse existing integration points in `libp2p-swarm` and `libp2p` without adding any ad-hoc logic. All parts of `interface-transport` are used, including `.getAddr` which returns a list of `/p2p-circuit` addresses that circuit is currently listening.
This module is not a transport, however it implements `interface-transport` interface in order to allow circuit to be plugged with `libp2p`. The rational behind it is that, `libp2p-circuit` has a dial and listen flow, which fits nicely with other transports, moreover, it requires the _raw_ connection to be encrypted and muxed just as a regular transport's connection does. All in all, `interface-transport` ended up being the correct level of abstraction for circuit, as well as allowed us to reuse existing integration points in `libp2p` and `libp2p` without adding any ad-hoc logic. All parts of `interface-transport` are used, including `.getAddr` which returns a list of `/p2p-circuit` addresses that circuit is currently listening.
```
libp2p libp2p-circuit (transport)
@ -109,13 +82,13 @@ libp2p
| +---------------------------------+ | | |
| | | | | +------------------+ |
| | | | circuit-relay listens for the HOP | | | |
| | libp2p-swarm <------------------------------------------------| circuit-relay | |
| | libp2p <------------------------------------------------| circuit-relay | |
| | | | message to handle incomming relay | | | |
| | | | requests from other nodes | +------------------+ |
| +---------------------------------+ | | |
| ^ ^ ^ ^ ^ ^ | | +------------------+ |
| | | | | | | | | | +-------------+ | |
| | | | | | | | dialer uses libp2p-swarm to dial | | | | | |
| | | | | | | | dialer uses libp2p to dial | | | | | |
| | | | +----------------------------------------------------------------------> dialer | | |
| | | transports | | to a circuit-relay node using the | | | | | |
| | | | | | | HOP message | | +-------------+ | |

View File

@ -12,7 +12,7 @@ const DefaultConfig = {
noAnnounce: []
},
connectionManager: {
minPeers: 25
minConnections: 25
},
transportManager: {
faultTolerance: FaultTolerance.FATAL_ALL

View File

@ -1,9 +1,12 @@
'use strict'
const debug = require('debug')
const log = debug('libp2p:connection-manager')
log.error = debug('libp2p:connection-manager:error')
const errcode = require('err-code')
const mergeOptions = require('merge-options')
const LatencyMonitor = require('./latency-monitor')
const debug = require('debug')('libp2p:connection-manager')
const retimer = require('retimer')
const { EventEmitter } = require('events')
@ -22,6 +25,7 @@ const defaultOptions = {
maxReceivedData: Infinity,
maxEventLoopDelay: Infinity,
pollInterval: 2000,
autoDialInterval: 10000,
movingAverageInterval: 60000,
defaultPeerValue: 1
}
@ -45,6 +49,8 @@ class ConnectionManager extends EventEmitter {
* @param {Number} options.pollInterval How often, in milliseconds, metrics and latency should be checked. Default=2000
* @param {Number} options.movingAverageInterval How often, in milliseconds, to compute averages. Default=60000
* @param {Number} options.defaultPeerValue The value of the peer. Default=1
* @param {boolean} options.autoDial Should preemptively guarantee connections are above the low watermark. Default=true
* @param {Number} options.autoDialInterval How often, in milliseconds, it should preemptively guarantee connections are above the low watermark. Default=10000
*/
constructor (libp2p, options) {
super()
@ -57,7 +63,7 @@ class ConnectionManager extends EventEmitter {
throw errcode(new Error('Connection Manager maxConnections must be greater than minConnections'), ERR_INVALID_PARAMETERS)
}
debug('options: %j', this._options)
log('options: %j', this._options)
this._libp2p = libp2p
@ -73,8 +79,11 @@ class ConnectionManager extends EventEmitter {
*/
this.connections = new Map()
this._started = false
this._timer = null
this._autoDialTimeout = null
this._checkMetrics = this._checkMetrics.bind(this)
this._autoDial = this._autoDial.bind(this)
}
/**
@ -101,7 +110,11 @@ class ConnectionManager extends EventEmitter {
})
this._onLatencyMeasure = this._onLatencyMeasure.bind(this)
this._latencyMonitor.on('data', this._onLatencyMeasure)
debug('started')
this._started = true
log('started')
this._options.autoDial && this._autoDial()
}
/**
@ -109,11 +122,13 @@ class ConnectionManager extends EventEmitter {
* @async
*/
async stop () {
this._autoDialTimeout && this._autoDialTimeout.clear()
this._timer && this._timer.clear()
this._latencyMonitor && this._latencyMonitor.removeListener('data', this._onLatencyMeasure)
this._started = false
await this._close()
debug('stopped')
log('stopped')
}
/**
@ -157,13 +172,13 @@ class ConnectionManager extends EventEmitter {
_checkMetrics () {
const movingAverages = this._libp2p.metrics.global.movingAverages
const received = movingAverages.dataReceived[this._options.movingAverageInterval].movingAverage()
this._checkLimit('maxReceivedData', received)
this._checkMaxLimit('maxReceivedData', received)
const sent = movingAverages.dataSent[this._options.movingAverageInterval].movingAverage()
this._checkLimit('maxSentData', sent)
this._checkMaxLimit('maxSentData', sent)
const total = received + sent
this._checkLimit('maxData', total)
debug('metrics update', total)
this._timer.reschedule(this._options.pollInterval)
this._checkMaxLimit('maxData', total)
log('metrics update', total)
this._timer = retimer(this._checkMetrics, this._options.pollInterval)
}
/**
@ -188,7 +203,7 @@ class ConnectionManager extends EventEmitter {
this._peerValues.set(peerIdStr, this._options.defaultPeerValue)
}
this._checkLimit('maxConnections', this.size)
this._checkMaxLimit('maxConnections', this.size)
}
/**
@ -248,7 +263,7 @@ class ConnectionManager extends EventEmitter {
* @param {*} summary The LatencyMonitor summary
*/
_onLatencyMeasure (summary) {
this._checkLimit('maxEventLoopDelay', summary.avgMs)
this._checkMaxLimit('maxEventLoopDelay', summary.avgMs)
}
/**
@ -257,15 +272,61 @@ class ConnectionManager extends EventEmitter {
* @param {string} name The name of the field to check limits for
* @param {number} value The current value of the field
*/
_checkLimit (name, value) {
_checkMaxLimit (name, value) {
const limit = this._options[name]
debug('checking limit of %s. current value: %d of %d', name, value, limit)
log('checking limit of %s. current value: %d of %d', name, value, limit)
if (value > limit) {
debug('%s: limit exceeded: %s, %d', this._peerId, name, value)
log('%s: limit exceeded: %s, %d', this._peerId, name, value)
this._maybeDisconnectOne()
}
}
/**
* Proactively tries to connect to known peers stored in the PeerStore.
* It will keep the number of connections below the upper limit and sort
* the peers to connect based on wether we know their keys and protocols.
* @async
* @private
*/
async _autoDial () {
const minConnections = this._options.minConnections
// Already has enough connections
if (this.size >= minConnections) {
this._autoDialTimeout = retimer(this._autoDial, this._options.autoDialInterval)
return
}
// Sort peers on wether we know protocols of public keys for them
const peers = Array.from(this._libp2p.peerStore.peers.values())
.sort((a, b) => {
if (b.protocols && b.protocols.length && (!a.protocols || !a.protocols.length)) {
return 1
} else if (b.id.pubKey && !a.id.pubKey) {
return 1
}
return -1
})
for (let i = 0; i < peers.length && this.size < minConnections; i++) {
if (!this.get(peers[i].id)) {
log('connecting to a peerStore stored peer %s', peers[i].id.toB58String())
try {
await this._libp2p.dialer.connectToPeer(peers[i].id)
// Connection Manager was stopped
if (!this._started) {
return
}
} catch (err) {
log.error('could not connect to peerStore stored peer', err)
}
}
}
this._autoDialTimeout = retimer(this._autoDial, this._options.autoDialInterval)
}
/**
* If we have more connections than our maximum, close a connection
* to the lowest valued peer.
@ -274,12 +335,12 @@ class ConnectionManager extends EventEmitter {
_maybeDisconnectOne () {
if (this._options.minConnections < this.connections.size) {
const peerValues = Array.from(this._peerValues).sort(byPeerValue)
debug('%s: sorted peer values: %j', this._peerId, peerValues)
log('%s: sorted peer values: %j', this._peerId, peerValues)
const disconnectPeer = peerValues[0]
if (disconnectPeer) {
const peerId = disconnectPeer[0]
debug('%s: lowest value peer is %s', this._peerId, peerId)
debug('%s: closing a connection to %j', this._peerId, peerId)
log('%s: lowest value peer is %s', this._peerId, peerId)
log('%s: closing a connection to %j', this._peerId, peerId)
for (const connections of this.connections.values()) {
if (connections[0].remotePeer.toB58String() === peerId) {
connections[0].close()

View File

@ -65,7 +65,13 @@ class Libp2p extends EventEmitter {
this._discovery = new Map() // Discovery service instances/references
// Create the Connection Manager
this.connectionManager = new ConnectionManager(this, this._options.connectionManager)
if (this._options.connectionManager.minPeers) { // Remove in 0.29
this._options.connectionManager.minConnections = this._options.connectionManager.minPeers
}
this.connectionManager = new ConnectionManager(this, {
autoDial: this._config.peerDiscovery.autoDial,
...this._options.connectionManager
})
// Create Metrics
if (this._options.metrics.enabled) {
@ -166,6 +172,7 @@ class Libp2p extends EventEmitter {
if (this._modules.dht) {
const DHT = this._modules.dht
this._dht = new DHT({
libp2p: this,
dialer: this.dialer,
peerId: this.peerId,
peerStore: this.peerStore,
@ -459,19 +466,19 @@ class Libp2p extends EventEmitter {
async _onDidStart () {
this._isStarted = true
this.connectionManager.start()
this.peerStore.on('peer', peerId => {
this.emit('peer:discovery', peerId)
this._maybeConnect(peerId)
})
// Once we start, emit and dial any peers we may have already discovered
// Once we start, emit any peers we may have already discovered
// TODO: this should be removed, as we already discovered these peers in the past
for (const peer of this.peerStore.peers.values()) {
this.emit('peer:discovery', peer.id)
this._maybeConnect(peer.id)
}
this.connectionManager.start()
// Peer discovery
await this._setupPeerDiscovery()
}
@ -495,15 +502,15 @@ class Libp2p extends EventEmitter {
/**
* Will dial to the given `peerId` if the current number of
* connected peers is less than the configured `ConnectionManager`
* minPeers.
* minConnections.
* @private
* @param {PeerId} peerId
*/
async _maybeConnect (peerId) {
// If auto dialing is on and we have no connection to the peer, check if we should dial
if (this._config.peerDiscovery.autoDial === true && !this.connectionManager.get(peerId)) {
const minPeers = this._options.connectionManager.minPeers || 0
if (minPeers > this.connectionManager.size) {
const minConnections = this._options.connectionManager.minConnections || 0
if (minConnections > this.connectionManager.size) {
log('connecting to discovered peer %s', peerId.toB58String())
try {
await this.dialer.connectToPeer(peerId)

View File

@ -128,11 +128,7 @@ class Stats extends EventEmitter {
* @returns {void}
*/
_resetComputeTimeout () {
if (this._timeout) {
this._timeout.reschedule(this._nextTimeout())
} else {
this._timeout = retimer(this._update, this._nextTimeout())
}
this._timeout = retimer(this._update, this._nextTimeout())
}
/**

View File

@ -179,7 +179,7 @@ class AddressBook extends Book {
const record = this.data.get(peerId.toB58String())
if (!record) {
return undefined
return []
}
return record.map((address) => {

View File

@ -7,6 +7,9 @@ chai.use(require('chai-as-promised'))
const { expect } = chai
const sinon = require('sinon')
const delay = require('delay')
const pWaitFor = require('p-wait-for')
const peerUtils = require('../utils/creators/peer')
const mockConnection = require('../utils/mockConnection')
const baseOptions = require('../utils/base-options.browser')
@ -112,4 +115,148 @@ describe('libp2p.connections', () => {
await libp2p.stop()
await remoteLibp2p.stop()
})
describe('proactive connections', () => {
let nodes = []
beforeEach(async () => {
nodes = await peerUtils.createPeer({
number: 2,
config: {
addresses: {
listen: ['/ip4/127.0.0.1/tcp/0/ws']
}
}
})
})
afterEach(async () => {
await Promise.all(nodes.map((node) => node.stop()))
sinon.reset()
})
it('should connect to all the peers stored in the PeerStore, if their number is below minConnections', async () => {
const [libp2p] = await peerUtils.createPeer({
fixture: false,
started: false,
config: {
addresses: {
listen: ['/ip4/127.0.0.1/tcp/0/ws']
},
connectionManager: {
minConnections: 3
}
}
})
// Populate PeerStore before starting
libp2p.peerStore.addressBook.set(nodes[0].peerId, nodes[0].multiaddrs)
libp2p.peerStore.addressBook.set(nodes[1].peerId, nodes[1].multiaddrs)
await libp2p.start()
// Wait for peers to connect
await pWaitFor(() => libp2p.connectionManager.size === 2)
await libp2p.stop()
})
it('should connect to all the peers stored in the PeerStore until reaching the minConnections', async () => {
const minConnections = 1
const [libp2p] = await peerUtils.createPeer({
fixture: false,
started: false,
config: {
addresses: {
listen: ['/ip4/127.0.0.1/tcp/0/ws']
},
connectionManager: {
minConnections
}
}
})
// Populate PeerStore before starting
libp2p.peerStore.addressBook.set(nodes[0].peerId, nodes[0].multiaddrs)
libp2p.peerStore.addressBook.set(nodes[1].peerId, nodes[1].multiaddrs)
await libp2p.start()
// Wait for peer to connect
await pWaitFor(() => libp2p.connectionManager.size === minConnections)
// Wait more time to guarantee no other connection happened
await delay(200)
expect(libp2p.connectionManager.size).to.eql(minConnections)
await libp2p.stop()
})
it('should connect to all the peers stored in the PeerStore until reaching the minConnections sorted', async () => {
const minConnections = 1
const [libp2p] = await peerUtils.createPeer({
fixture: false,
started: false,
config: {
addresses: {
listen: ['/ip4/127.0.0.1/tcp/0/ws']
},
connectionManager: {
minConnections
}
}
})
// Populate PeerStore before starting
libp2p.peerStore.addressBook.set(nodes[0].peerId, nodes[0].multiaddrs)
libp2p.peerStore.addressBook.set(nodes[1].peerId, nodes[1].multiaddrs)
libp2p.peerStore.protoBook.set(nodes[1].peerId, ['/protocol-min-conns'])
await libp2p.start()
// Wait for peer to connect
await pWaitFor(() => libp2p.connectionManager.size === minConnections)
// Should have connected to the peer with protocols
expect(libp2p.connectionManager.get(nodes[0].peerId)).to.not.exist()
expect(libp2p.connectionManager.get(nodes[1].peerId)).to.exist()
await libp2p.stop()
})
it('should connect to peers in the PeerStore when a peer disconnected', async () => {
const minConnections = 1
const autoDialInterval = 1000
const [libp2p] = await peerUtils.createPeer({
fixture: false,
config: {
addresses: {
listen: ['/ip4/127.0.0.1/tcp/0/ws']
},
connectionManager: {
minConnections,
autoDialInterval
}
}
})
// Populate PeerStore after starting (discovery)
libp2p.peerStore.addressBook.set(nodes[0].peerId, nodes[0].multiaddrs)
// Wait for peer to connect
const conn = await libp2p.dial(nodes[0].peerId)
expect(libp2p.connectionManager.get(nodes[0].peerId)).to.exist()
await conn.close()
// Closed
await pWaitFor(() => libp2p.connectionManager.size === 0)
// Connected
await pWaitFor(() => libp2p.connectionManager.size === 1)
expect(libp2p.connectionManager.get(nodes[0].peerId)).to.exist()
await libp2p.stop()
})
})
})

View File

@ -58,7 +58,8 @@ describe('Connection Manager', () => {
config: {
modules: baseOptions.modules,
connectionManager: {
maxConnections: max
maxConnections: max,
minConnections: 2
}
},
started: false
@ -96,7 +97,8 @@ describe('Connection Manager', () => {
config: {
modules: baseOptions.modules,
connectionManager: {
maxConnections: max
maxConnections: max,
minConnections: 0
}
},
started: false

View File

@ -31,10 +31,13 @@ describe('peer discovery', () => {
sinon.reset()
})
it('should dial know peers on startup', async () => {
it('should dial know peers on startup below the minConnections watermark', async () => {
libp2p = new Libp2p({
...baseOptions,
peerId
peerId,
connectionManager: {
minConnections: 2
}
})
libp2p.peerStore.addressBook.set(remotePeerId, [multiaddr('/ip4/165.1.1.1/tcp/80')])

View File

@ -323,10 +323,10 @@ describe('addressBook', () => {
throw new Error('invalid peerId should throw error')
})
it('returns undefined if no multiaddrs are known for the provided peer', () => {
it('returns empty array if no multiaddrs are known for the provided peer', () => {
const addresses = ab.getMultiaddrsForPeer(peerId)
expect(addresses).to.not.exist()
expect(addresses).to.be.empty()
})
it('returns the multiaddrs stored', () => {

View File

@ -190,12 +190,18 @@ describe('Pubsub subsystem operates correctly', () => {
})
})
describe('pubsub started after disconnect and connect', () => {
describe('pubsub with intermittent connections', () => {
beforeEach(async () => {
libp2p = await create(mergeOptions(subsystemOptions, {
peerId,
addresses: {
listen: [listenAddr]
},
config: {
pubsub: {
enabled: true,
emitSelf: false
}
}
}))
@ -206,7 +212,8 @@ describe('Pubsub subsystem operates correctly', () => {
},
config: {
pubsub: {
enabled: true
enabled: true,
emitSelf: false
}
}
}))
@ -226,7 +233,7 @@ describe('Pubsub subsystem operates correctly', () => {
sinon.restore()
})
it('should receive pubsub messages', async () => {
it('should receive pubsub messages after a node restart', async () => {
const topic = 'test-topic'
const data = 'hey!'
const libp2pId = libp2p.peerId.toB58String()
@ -275,5 +282,42 @@ describe('Pubsub subsystem operates correctly', () => {
await defer2.promise
})
it('should handle quick reconnects with a delayed disconnect', async () => {
// Subscribe on both
const handlerSpy = sinon.spy()
const topic = 'reconnect-channel'
await Promise.all([
libp2p.pubsub.subscribe(topic, handlerSpy),
remoteLibp2p.pubsub.subscribe(topic, handlerSpy)
])
// Create two connections to the remote peer
const originalConnection = await libp2p.dialer.connectToPeer(remoteLibp2p.peerId)
// second connection
await libp2p.dialer.connectToPeer(remoteLibp2p.peerId)
expect(libp2p.connections.get(remoteLibp2p.peerId.toB58String())).to.have.length(2)
// Wait for subscriptions to occur
await pWaitFor(() => {
return libp2p.pubsub.getSubscribers(topic).includes(remoteLibp2p.peerId.toB58String()) &&
remoteLibp2p.pubsub.getSubscribers(topic).includes(libp2p.peerId.toB58String())
})
// Verify messages go both ways
libp2p.pubsub.publish(topic, 'message1')
remoteLibp2p.pubsub.publish(topic, 'message2')
await pWaitFor(() => handlerSpy.callCount === 2)
expect(handlerSpy.args.map(([message]) => message.data.toString())).to.include.members(['message1', 'message2'])
// Disconnect the first connection (this acts as a delayed reconnect)
await originalConnection.close()
// Verify messages go both ways after the disconnect
handlerSpy.resetHistory()
libp2p.pubsub.publish(topic, 'message3')
remoteLibp2p.pubsub.publish(topic, 'message4')
await pWaitFor(() => handlerSpy.callCount === 2)
expect(handlerSpy.args.map(([message]) => message.data.toString())).to.include.members(['message3', 'message4'])
})
})
})