Compare commits

...

28 Commits

Author SHA1 Message Date
Jacob Heun
619e5dd73c chore: release version v0.28.5 2020-07-10 17:26:21 +02:00
Jacob Heun
0fd23f6a5f chore: update contributors 2020-07-10 17:26:21 +02:00
Jacob Heun
5a84dd56d0 fix: pass libp2p to the dht (#700) 2020-07-10 17:16:45 +02:00
Bernd Strehl
7b05d6922b docs: add dht discovery documentation reference (#697)
Extended the description with a link for how to configure the libp2p-kad-dht for peer discovery.
2020-07-08 15:03:56 +02:00
Vasco Santos
08a4fad80b chore: add modules to update in 0.28.x migration (#695) 2020-07-08 14:12:40 +02:00
Jacob Heun
51da8874d8 test: add pubsub reconnect test (#693)
* test: add pubsub reconnect test

* chore: dep bump

* chore: remove temp pubsub dep

Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>

Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
2020-07-07 18:31:51 +02:00
Jacob Heun
5cd8c19567 docs(fix): update link to ipfs config files (#690) 2020-07-07 12:56:17 +02:00
Vasco Santos
db766eaca9 chore: update circuit readme (#689)
* chore: update circuit readme

* chore: address review
2020-07-07 12:55:52 +02:00
Jacob Heun
a1308d640d chore: release version v0.28.4 2020-07-03 16:14:47 +02:00
Jacob Heun
19e7254c3d chore: update contributors 2020-07-03 16:14:47 +02:00
Vasco Santos
f4898eb883 chore: add test for pubsub on reconnect (#691)
* chore: add test for pubsub on reconnect

* chore: update interface dep

Co-authored-by: Jacob Heun <jacobheun@gmail.com>
2020-07-03 16:08:37 +02:00
Francis Gulotta
9bec719fce docs(fix): link to streaming-iterables' repo (#688) 2020-07-02 13:34:01 +02:00
Vasco Santos
b664c0aafd chore: libp2p in the browser using webrtc servers (#684) 2020-06-29 16:31:07 +02:00
Jacob Heun
b524bbf627 chore: release version v0.28.3 2020-06-18 15:40:38 +02:00
Jacob Heun
07bd269fb0 chore: update contributors 2020-06-18 15:40:38 +02:00
Jacob Heun
a8219e61a0 fix: catch pipe errors (#678)
* fix: catch pipe errors

There were some pipe errors not being caught. This can result in unhandled exceptions being thrown

* fix: catch pipe errors in identify push handler
2020-06-18 15:33:08 +02:00
Vasco Santos
a1590acc8b chore: add libp2p docs website to release checklist 2020-06-18 11:48:31 +02:00
Jacob Heun
a0cf83c640 chore: release version v0.28.2 2020-06-15 12:48:25 +02:00
Jacob Heun
b56cdda0ef chore: update contributors 2020-06-15 12:48:25 +02:00
Jacob Heun
b621fbdfdc revert: "fix: throw if no conn encryption module provided (#665)"
This reverts commit c038550fad.
2020-06-15 12:45:49 +02:00
Jacob Heun
24dd1d22c5 chore: release version v0.28.1 2020-06-12 16:48:30 +02:00
Jacob Heun
a1a1213bdc chore: update contributors 2020-06-12 16:48:29 +02:00
Cayman
90a9d93968 chore: refactor ConnectionManager#get 2020-06-11 16:21:51 +02:00
Cayman
8f680e20e9 feat: add ConnectionManager#getAll 2020-06-11 16:21:51 +02:00
Cayman
afafd08943 chore: fix discv5 npm link
chore: update package name for discv5
2020-06-11 13:03:31 +02:00
Vasco Santos
c038550fad fix: throw if no conn encryption module provided (#665)
* fix: throw if no conn encryption module provided

* chore: address review

* chore: apply suggestions from code review

Co-authored-by: Jacob Heun <jacobheun@gmail.com>

Co-authored-by: Jacob Heun <jacobheun@gmail.com>
2020-06-11 12:49:32 +02:00
Alex Potsides
d60a3215d0 chore: update interface datastore (#663)
* chore: update interface datastore

Updates to v1.x.x to not have multiple versions of this module in the ipfs browser bundle.

* fix: let batch commits complete before continuing tests

Batch commits are async but the tests weren't waiting for them to complete,
mainly because they are triggered by events.

There's no way that I can see of waiting for the batch commit to finish so
I've added delays to the tests. Not great but a start.

* chore: use error log

* test: wait for commit spies to complete

* chore: bump interface-datastore

Co-authored-by: Jacob Heun <jacobheun@gmail.com>
2020-06-10 23:39:40 +02:00
Vasco Santos
1e51295150 chore: use interop release 2020-06-08 12:20:48 +02:00
22 changed files with 375 additions and 144 deletions

View File

@@ -1,3 +1,53 @@
<a name="0.28.5"></a>
## [0.28.5](https://github.com/libp2p/js-libp2p/compare/v0.28.4...v0.28.5) (2020-07-10)
### Bug Fixes
* pass libp2p to the dht ([#700](https://github.com/libp2p/js-libp2p/issues/700)) ([5a84dd5](https://github.com/libp2p/js-libp2p/commit/5a84dd5))
<a name="0.28.4"></a>
## [0.28.4](https://github.com/libp2p/js-libp2p/compare/v0.28.3...v0.28.4) (2020-07-03)
<a name="0.28.3"></a>
## [0.28.3](https://github.com/libp2p/js-libp2p/compare/v0.28.2...v0.28.3) (2020-06-18)
### Bug Fixes
* catch pipe errors ([#678](https://github.com/libp2p/js-libp2p/issues/678)) ([a8219e6](https://github.com/libp2p/js-libp2p/commit/a8219e6))
<a name="0.28.2"></a>
## [0.28.2](https://github.com/libp2p/js-libp2p/compare/v0.28.1...v0.28.2) (2020-06-15)
### Reverts
* "fix: throw if no conn encryption module provided ([#665](https://github.com/libp2p/js-libp2p/issues/665))" ([b621fbd](https://github.com/libp2p/js-libp2p/commit/b621fbd))
<a name="0.28.1"></a>
## [0.28.1](https://github.com/libp2p/js-libp2p/compare/v0.28.0...v0.28.1) (2020-06-12)
### Bug Fixes
* throw if no conn encryption module provided ([#665](https://github.com/libp2p/js-libp2p/issues/665)) ([c038550](https://github.com/libp2p/js-libp2p/commit/c038550))
### Features
* add ConnectionManager#getAll ([8f680e2](https://github.com/libp2p/js-libp2p/commit/8f680e2))
<a name="0.28.0"></a>
# [0.28.0](https://github.com/libp2p/js-libp2p/compare/v0.28.0-rc.0...v0.28.0) (2020-06-05)

View File

@@ -155,7 +155,7 @@ List of packages currently in existence for libp2p
| [`libp2p-kad-dht`](//github.com/libp2p/js-libp2p-kad-dht) | [![npm](https://img.shields.io/npm/v/libp2p-kad-dht.svg?maxAge=86400&style=flat-square)](//github.com/libp2p/js-libp2p-kad-dht/releases) | [![Deps](https://david-dm.org/libp2p/js-libp2p-kad-dht.svg?style=flat-square)](https://david-dm.org/libp2p/js-libp2p-kad-dht) | [![Travis CI](https://flat.badgen.net/travis/libp2p/js-libp2p-kad-dht/master)](https://travis-ci.com/libp2p/js-libp2p-kad-dht) | [![codecov](https://codecov.io/gh/libp2p/js-libp2p-kad-dht/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-libp2p-kad-dht) | [Vasco Santos](mailto:vasco.santos@moxy.studio) |
| [`libp2p-mdns`](//github.com/libp2p/js-libp2p-mdns) | [![npm](https://img.shields.io/npm/v/libp2p-mdns.svg?maxAge=86400&style=flat-square)](//github.com/libp2p/js-libp2p-mdns/releases) | [![Deps](https://david-dm.org/libp2p/js-libp2p-mdns.svg?style=flat-square)](https://david-dm.org/libp2p/js-libp2p-mdns) | [![Travis CI](https://flat.badgen.net/travis/libp2p/js-libp2p-mdns/master)](https://travis-ci.com/libp2p/js-libp2p-mdns) | [![codecov](https://codecov.io/gh/libp2p/js-libp2p-mdns/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-libp2p-mdns) | [Jacob Heun](mailto:jacobheun@gmail.com) |
| [`libp2p-webrtc-star`](//github.com/libp2p/js-libp2p-webrtc-star) | [![npm](https://img.shields.io/npm/v/libp2p-webrtc-star.svg?maxAge=86400&style=flat-square)](//github.com/libp2p/js-libp2p-webrtc-star/releases) | [![Deps](https://david-dm.org/libp2p/js-libp2p-webrtc-star.svg?style=flat-square)](https://david-dm.org/libp2p/js-libp2p-webrtc-star) | [![Travis CI](https://flat.badgen.net/travis/libp2p/js-libp2p-webrtc-star/master)](https://travis-ci.com/libp2p/js-libp2p-webrtc-star) | [![codecov](https://codecov.io/gh/libp2p/js-libp2p-webrtc-star/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-libp2p-webrtc-star) | [Vasco Santos](mailto:vasco.santos@moxy.studio) |
| [`discv5`](//github.com/ChainSafe/discv5) | [![npm](https://img.shields.io/npm/v/discv5.svg?maxAge=86400&style=flat-square)](//github.com/ChainSafe/discv5/releases) | [![Deps](https://david-dm.org/ChainSafe/discv5.svg?style=flat-square)](https://david-dm.org/ChainSafe/discv5) | [![Travis CI](https://flat.badgen.net/travis/ChainSafe/discv5/master)](https://travis-ci.com/ChainSafe/discv5) | [![codecov](https://codecov.io/gh/ChainSafe/discv5/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/ChainSafe/discv5) | [Cayman Nava](mailto:caymannava@gmail.com) |
| [`@chainsafe/discv5`](//github.com/ChainSafe/discv5) | [![npm](https://img.shields.io/npm/v/@chainsafe/discv5.svg?maxAge=86400&style=flat-square)](//github.com/ChainSafe/discv5/releases) | [![Deps](https://david-dm.org/ChainSafe/discv5.svg?style=flat-square)](https://david-dm.org/ChainSafe/discv5) | [![Travis CI](https://flat.badgen.net/travis/ChainSafe/discv5/master)](https://travis-ci.com/ChainSafe/discv5) | [![codecov](https://codecov.io/gh/ChainSafe/discv5/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/ChainSafe/discv5) | [Cayman Nava](mailto:caymannava@gmail.com) |
| **content routing** |
| [`libp2p-delegated-content-routing`](//github.com/libp2p/js-libp2p-delegated-content-routing) | [![npm](https://img.shields.io/npm/v/libp2p-delegated-content-routing.svg?maxAge=86400&style=flat-square)](//github.com/libp2p/js-libp2p-delegated-content-routing/releases) | [![Deps](https://david-dm.org/libp2p/js-libp2p-delegated-content-routing.svg?style=flat-square)](https://david-dm.org/libp2p/js-libp2p-delegated-content-routing) | [![Travis CI](https://flat.badgen.net/travis/libp2p/js-libp2p-delegated-content-routing/master)](https://travis-ci.com/libp2p/js-libp2p-delegated-content-routing) | [![codecov](https://codecov.io/gh/libp2p/js-libp2p-delegated-content-routing/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-libp2p-delegated-content-routing) | [Jacob Heun](mailto:jacobheun@gmail.com) |
| [`libp2p-kad-dht`](//github.com/libp2p/js-libp2p-kad-dht) | [![npm](https://img.shields.io/npm/v/libp2p-kad-dht.svg?maxAge=86400&style=flat-square)](//github.com/libp2p/js-libp2p-kad-dht/releases) | [![Deps](https://david-dm.org/libp2p/js-libp2p-kad-dht.svg?style=flat-square)](https://david-dm.org/libp2p/js-libp2p-kad-dht) | [![Travis CI](https://flat.badgen.net/travis/libp2p/js-libp2p-kad-dht/master)](https://travis-ci.com/libp2p/js-libp2p-kad-dht) | [![codecov](https://codecov.io/gh/libp2p/js-libp2p-kad-dht/branch/master/graph/badge.svg?style=flat-square)](https://codecov.io/gh/libp2p/js-libp2p-kad-dht) | [Vasco Santos](mailto:vasco.santos@moxy.studio) |

View File

@@ -26,6 +26,7 @@
- Documentation
- [ ] Ensure that README.md is up to date
- [ ] Ensure that all the examples run
- [ ] Ensure that [libp2p/docs](https://github.com/libp2p/docs) is updated
- Communication
- [ ] Create the release issue
- [ ] Take a snapshot between of everyone that has contributed to this release (including its subdeps in IPFS, libp2p, IPLD and multiformats) using [`name-your-contributors`](https://www.npmjs.com/package/name-your-contributors). Generate a nice markdown list with [this script](https://gist.github.com/jacobheun/d2ff479ca991733c13cdcf688a1317e5)

View File

@@ -649,8 +649,8 @@ const node = await Libp2p.create({
As libp2p is designed to be a modular networking library, its usage will vary based on individual project needs. We've included links to some existing project configurations for your reference, in case you wish to replicate their configuration:
- [libp2p-ipfs-nodejs](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/libp2p-nodejs.js) - libp2p configuration used by js-ipfs when running in Node.js
- [libp2p-ipfs-browser](https://github.com/ipfs/js-ipfs/tree/master/src/core/runtime/libp2p-browser.js) - libp2p configuration used by js-ipfs when running in a Browser (that supports WebRTC)
- [libp2p-ipfs-nodejs](https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/src/core/runtime/libp2p-nodejs.js) - libp2p configuration used by js-ipfs when running in Node.js
- [libp2p-ipfs-browser](https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs/src/core/runtime/libp2p-browser.js) - libp2p configuration used by js-ipfs when running in a Browser (that supports WebRTC)
If you have developed a project using `js-libp2p`, please consider submitting your configuration to this list so that it can be found easily by other users.

View File

@@ -161,4 +161,4 @@ const duplex = {
[it-pipe]: https://github.com/alanshaw/it-pipe
[it-pushable]: https://github.com/alanshaw/it-pushable
[it-reader]: https://github.com/alanshaw/it-reader
[streaming-iterables]: https://github.com/bustle/streaming-iterables
[streaming-iterables]: https://github.com/reconbot/streaming-iterables

View File

@@ -10,6 +10,7 @@ A migration guide for refactoring your application code from libp2p v0.27.x to v
- [API Implications](#api-implications)
- [Connection Manager and Registrar](#connection-manager-and-registrar)
- [Events](#events)
- [Module Updates](#module-updates)
## PeerStore API
@@ -321,6 +322,21 @@ libp2p.on('peer:discovery', (peerId) => {
})
```
## Module Updates
With `libp2p@0.28` you should update the following libp2p modules if you are relying on them:
```json
"libp2p-bootstrap": "^0.11.0",
"libp2p-delegated-content-routing": "^0.5.0",
"libp2p-delegated-peer-routing": "^0.5.0",
"libp2p-floodsub": "^0.21.0",
"libp2p-gossipsub": "^0.4.0",
"libp2p-kad-dht": "^0.19.1",
"libp2p-mdns": "^0.14.1",
"libp2p-webrtc-star": "^0.18.0"
```
[connection]: https://github.com/libp2p/js-interfaces/tree/master/src/connection
[multiaddr]: https://github.com/multiformats/js-multiaddr
[peer-id]: https://github.com/libp2p/js-peer-id

View File

@@ -170,5 +170,5 @@ Discovered: QmRcXXhtG8vTqwVBRonKWtV4ovDoC1Fe56WYtcrw694eiJ
There are plenty more Peer Discovery Mechanisms out there, you can:
- Find one in [libp2p-webrtc-star](https://github.com/libp2p/js-libp2p-webrtc-star). Yes, a transport with discovery capabilities! This happens because WebRTC requires a rendezvous point for peers to exchange [SDP](https://tools.ietf.org/html/rfc4317) offer, which means we have one or more points that can introduce peers to each other. Think of it as MulticastDNS for the Web, as in MulticastDNS only works in LAN.
- Any DHT will offer you a discovery capability. You can simple _random-walk_ the routing tables to find other peers to connect to.
- Any DHT will offer you a discovery capability. You can simple _random-walk_ the routing tables to find other peers to connect to. For example [libp2p-kad-dht](https://github.com/libp2p/js-libp2p-kad-dht) can be used for peer discovery. An example how to configure it to enable random walks can be found [here](https://github.com/libp2p/js-libp2p/blob/v0.28.4/doc/CONFIGURATION.md#customizing-dht).
- You can create your own Discovery service, a registry, a list, a radio beacon, you name it!

View File

@@ -11,26 +11,9 @@ cd ./examples/libp2p-in-the-browser
npm install
```
## Signaling Server
This example uses the `libp2p-webrtc-star` module, which enables libp2p browser nodes to establish direct connections to one another via a central signaling server. For this example, we are using the signaling server that ships with `libp2p-webrtc-star`.
You can start the server by running `npm run server`. This will start a signaling server locally on port `9090`. If you'd like to run a signaling server outside of this example, you can see instructions on how to do so in the [`libp2p-webrtc-star` README](https://github.com/libp2p/js-libp2p-webrtc-star).
When you run the server, you should see output that looks something like this:
```log
$ npm run server
> libp2p-in-browser@1.0.0 server
> star-signal
Listening on: http://0.0.0.0:9090
```
## Running the examples
Once you have started the signaling server, you can run the Parcel server.
Start by running the Parcel server:
```
npm start
@@ -53,3 +36,11 @@ This will compile the code and start a server listening on port [http://localhos
Now, if you open a second browser tab to `http://localhost:1234`, you should discover your node from the previous tab. This is due to the fact that the `libp2p-webrtc-star` transport also acts as a Peer Discovery interface. Your node will be notified of any peer that connects to the same signaling server you are connected to. Once libp2p discovers this new peer, it will attempt to establish a direct WebRTC connection.
**Note**: In the example we assign libp2p to `window.libp2p`, in case you would like to play around with the API directly in the browser. You can of course make changes to `index.js` and Parcel will automatically rebuild and reload the browser tabs.
## Going to production?
This example uses public `libp2p-webrtc-star` servers. These servers should be used for experimenting and demos, they **MUST** not be used in production as there is no guarantee on availability.
You can see how to deploy your own signaling server in [libp2p/js-libp2p-webrtc-star/DEPLOYMENT.md](https://github.com/libp2p/js-libp2p-webrtc-star/blob/master/DEPLOYMENT.md).
Once you have your own server running, you should add its listen address in your libp2p node configuration.

View File

@@ -14,7 +14,10 @@ document.addEventListener('DOMContentLoaded', async () => {
// Add the signaling server address, along with our PeerId to our multiaddrs list
// libp2p will automatically attempt to dial to the signaling server so that it can
// receive inbound connections from other peers
listen: ['/ip4/0.0.0.0/tcp/9090/wss/p2p-webrtc-star']
listen: [
'/dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star',
'/dns4/wrtc-star2.sjc.dwebops.pub/tcp/443/wss/p2p-webrtc-star'
]
},
modules: {
transport: [Websockets, WebRTCStar],

View File

@@ -8,8 +8,7 @@
],
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "parcel index.html",
"server": "star-signal"
"start": "parcel index.html"
},
"keywords": [],
"author": "",

View File

@@ -33,7 +33,7 @@
["libp2p/js-libp2p-kad-dht", "libp2p-kad-dht"],
["libp2p/js-libp2p-mdns", "libp2p-mdns"],
["libp2p/js-libp2p-webrtc-star", "libp2p-webrtc-star"],
["ChainSafe/discv5", "discv5"],
["ChainSafe/discv5", "@chainsafe/discv5"],
"content routing",
["libp2p/js-libp2p-delegated-content-routing", "libp2p-delegated-content-routing"],

View File

@@ -1,6 +1,6 @@
{
"name": "libp2p",
"version": "0.28.0",
"version": "0.28.5",
"description": "JavaScript implementation of libp2p, a modular peer to peer network stack",
"leadMaintainer": "Jacob Heun <jacobheun@gmail.com>",
"main": "src/index.js",
@@ -50,7 +50,7 @@
"err-code": "^2.0.0",
"events": "^3.1.0",
"hashlru": "^2.3.0",
"interface-datastore": "^0.8.3",
"interface-datastore": "^1.0.4",
"ipfs-utils": "^2.2.0",
"it-all": "^1.0.1",
"it-buffer": "^0.1.2",
@@ -59,7 +59,7 @@
"it-pipe": "^1.1.0",
"it-protocol-buffers": "^0.2.0",
"libp2p-crypto": "^0.17.6",
"libp2p-interfaces": "^0.3.0",
"libp2p-interfaces": "^0.3.1",
"libp2p-utils": "^0.1.2",
"mafmt": "^7.0.0",
"merge-options": "^2.0.0",
@@ -92,7 +92,7 @@
"datastore-level": "^1.1.0",
"delay": "^4.3.0",
"dirty-chai": "^2.0.1",
"interop-libp2p": "libp2p/interop#chore/update-libp2p-daemon-with-peerstore",
"interop-libp2p": "^0.1.0",
"ipfs-http-client": "^44.0.0",
"it-concat": "^1.0.0",
"it-pair": "^1.0.0",
@@ -102,7 +102,7 @@
"libp2p-delegated-content-routing": "^0.5.0",
"libp2p-delegated-peer-routing": "^0.5.0",
"libp2p-floodsub": "^0.21.0",
"libp2p-gossipsub": "^0.4.0",
"libp2p-gossipsub": "^0.4.6",
"libp2p-kad-dht": "^0.19.1",
"libp2p-mdns": "^0.14.1",
"libp2p-mplex": "^0.9.5",
@@ -125,42 +125,44 @@
"Jacob Heun <jacobheun@gmail.com>",
"Vasco Santos <vasco.santos@moxy.studio>",
"Alan Shaw <alan@tableflip.io>",
"Cayman <caymannava@gmail.com>",
"Pedro Teixeira <i@pgte.me>",
"Friedel Ziegelmayer <dignifiedquire@gmail.com>",
"Alex Potsides <alex@achingbrain.net>",
"Maciej Krüger <mkg20001@gmail.com>",
"Hugo Dias <mail@hugodias.me>",
"Cayman <caymannava@gmail.com>",
"dirkmc <dirkmdev@gmail.com>",
"Volker Mische <volker.mische@gmail.com>",
"dirkmc <dirkmdev@gmail.com>",
"Richard Littauer <richard.littauer@gmail.com>",
"Elven <mon.samuel@qq.com>",
"Andrew Nesbitt <andrewnez@gmail.com>",
"Giovanni T. Parra <fiatjaf@gmail.com>",
"Ryan Bell <ryan@piing.net>",
"Thomas Eizinger <thomas@eizinger.io>",
"Ryan Bell <ryan@piing.net>",
"Giovanni T. Parra <fiatjaf@gmail.com>",
"Andrew Nesbitt <andrewnez@gmail.com>",
"ᴠɪᴄᴛᴏʀ ʙᴊᴇʟᴋʜᴏʟᴍ <victorbjelkholm@gmail.com>",
"Didrik Nordström <didrik@betamos.se>",
"Elven <mon.samuel@qq.com>",
"Didrik Nordström <didrik.nordstrom@gmail.com>",
"Tiago Alves <alvesjtiago@gmail.com>",
"Yusef Napora <yusef@napora.org>",
"Zane Starr <zcstarr@gmail.com>",
"ebinks <elizabethjbinks@gmail.com>",
"isan_rivkin <isanrivkin@gmail.com>",
"robertkiel <robert.kiel@validitylabs.org>",
"RasmusErik Voel Jensen <github@solsort.com>",
"Bernd Strehl <bernd.strehl@gmail.com>",
"Chris Bratlien <chrisbratlien@gmail.com>",
"Daijiro Wachi <daijiro.wachi@gmail.com>",
"Diogo Silva <fsdiogo@gmail.com>",
"Dmitriy Ryajov <dryajov@gmail.com>",
"Fei Liu <liu.feiwood@gmail.com>",
"Florian-Merle <florian.david.merle@gmail.com>",
"Francis Gulotta <wizard@roborooter.com>",
"Henrique Dias <hacdias@gmail.com>",
"Irakli Gozalishvili <rfobic@gmail.com>",
"Joel Gustafson <joelg@mit.edu>",
"Julien Bouquillon <contact@revolunet.com>",
"Kevin Kwok <antimatter15@gmail.com>",
"Florian-Merle <florian.david.merle@gmail.com>",
"Nuno Nogueira <nunofmn@gmail.com>",
"Fei Liu <liu.feiwood@gmail.com>",
"RasmusErik Voel Jensen <github@solsort.com>",
"robertkiel <robert.kiel@validitylabs.org>",
"Soeren <nikorpoulsen@gmail.com>",
"Sönke Hahn <soenkehahn@gmail.com>",
"Tiago Alves <alvesjtiago@gmail.com>",
"Dmitriy Ryajov <dryajov@gmail.com>",
"Diogo Silva <fsdiogo@gmail.com>",
"Yusef Napora <yusef@napora.org>",
"Zane Starr <zcstarr@gmail.com>",
"Daijiro Wachi <daijiro.wachi@gmail.com>",
"ebinks <elizabethjbinks@gmail.com>",
"Chris Bratlien <chrisbratlien@gmail.com>",
"isan_rivkin <isanrivkin@gmail.com>",
"Henrique Dias <hacdias@gmail.com>",
"Irakli Gozalishvili <rfobic@gmail.com>"
"Sönke Hahn <soenkehahn@gmail.com>"
]
}

View File

@@ -1,12 +1,20 @@
# js-libp2p-circuit
> Node.js implementation of the Circuit module that libp2p uses, which implements the [interface-connection](https://github.com/libp2p/js-interfaces/tree/master/src/connection) interface for dial/listen.
> Node.js implementation of the Circuit module that libp2p uses, which implements the [interface-connection](https://github.com/libp2p/js-libp2p-interfaces/tree/master/src/connection) interface for dial/listen.
**Note**: git history prior to merging into js-libp2p can be found in the original repository, https://github.com/libp2p/js-libp2p-circuit.
`libp2p-circuit` implements the circuit-relay mechanism that allows nodes that don't speak the same protocol to communicate using a third _relay_ node.
`libp2p-circuit` implements the circuit-relay mechanism that allows nodes that don't speak the same protocol to communicate using a third _relay_ node. You can read more about this in its [spec](https://github.com/libp2p/specs/tree/master/relay).
This module uses [pull-streams](https://pull-stream.github.io) for all stream based interfaces.
## Table of Contents
- [js-libp2p-circuit](#js-libp2p-circuit)
- [Why?](#why)
- [libp2p-circuit and IPFS](#libp2p-circuit-and-ipfs)
- [Table of Contents](#table-of-contents)
- [Usage](#usage)
- [API](#api)
- [Implementation rational](#implementation-rational)
### Why?
@@ -16,77 +24,42 @@ The use of circuit-relaying is not limited to routing traffic between browser no
- routing traffic between private nets and circumventing NAT layers
- route mangling for better privacy (matreshka/shallot dialing).
It's also possible to use it for clients that implement exotic transports such as devices that only have bluetooth radios to be reachable over bluetooth enabled relays and become full p2p nodes.
It's also possible to use it for clients that implement exotic transports such as devices that only have bluetooth radios to be reachable over bluetooth enabled relays and become full p2p nodes.
### libp2p-circuit and IPFS
Prior to `libp2p-circuit` there was a rift in the IPFS network, were IPFS nodes could only access content from nodes that speak the same protocol, for example TCP only nodes could only dial to other TCP only nodes, same for any other protocol combination. In practice, this limitation was most visible in JS-IPFS browser nodes, since they can only dial out but not be dialed in over WebRTC or WebSockets, hence any content that the browser node held was not reachable by the rest of the network even through it was announced on the DHT. Non browser IPFS nodes would usually speak more than one protocol such as TCP, WebSockets and/or WebRTC, this made the problem less severe outside of the browser. `libp2p-circuit` solves this problem completely, as long as there are `relay nodes` capable of routing traffic between those nodes their content should be available to the rest of the IPFS network.
## Table of Contents
- [js-libp2p-circuit](#js-libp2p-circuit)
- [Why?](#why)
- [libp2p-circuit and IPFS](#libp2p-circuit-and-ipfs)
- [Table of Contents](#table-of-contents)
- [Usage](#usage)
- [Example](#example)
- [Create dialer/listener](#create-dialerlistener)
- [Create `relay`](#create-relay)
- [API](#api)
- [Implementation rational](#implementation-rational)
## Usage
### Example
Libp2p circuit configuration can be seen at [Setup with Relay](../../doc/CONFIGURATION.md#setup-with-relay).
#### Create dialer/listener
Once you have a circuit relay node running, you can configure other nodes to use it as a relay as follows:
```js
const Circuit = require('libp2p-circuit')
const multiaddr = require('multiaddr')
const pull = require('pull-stream')
const Libp2p = require('libp2p')
const TCP = require('libp2p-tcp')
const MPLEX = require('libp2p-mplex')
const SECIO = require('libp2p-secio')
const mh1 = multiaddr('/p2p-circuit/p2p/QmHash') // dial /p2p/QmHash over any circuit
const relayAddr = ...
const circuit = new Circuit(swarmInstance, options) // pass swarm instance and options
const listener = circuit.createListener(mh1, (connection) => {
console.log('new connection opened')
pull(
pull.values(['hello']),
socket
)
const node = await Libp2p.create({
addresses: {
listen: [multiaddr(`${relayAddr}/p2p-circuit`)]
},
modules: {
transport: [TCP],
streamMuxer: [MPLEX],
connEncryption: [SECIO]
},
config: {
relay: { // Circuit Relay options (this config is part of libp2p core configurations)
enabled: true // Allows you to dial and accept relayed connections. Does not make you a relay.
}
}
})
listener.listen(() => {
console.log('listening')
pull(
circuit.dial(mh1),
pull.log,
pull.onEnd(() => {
circuit.close()
})
)
})
```
Outputs:
```sh
listening
new connection opened
hello
```
#### Create `relay`
```js
const Relay = require('libp2p-circuit').Relay
const relay = new Relay(options)
relay.mount(swarmInstance) // start relaying traffic
```
## API
@@ -101,7 +74,7 @@ Both for dialing and listening.
### Implementation rational
This module is not a transport, however it implements `interface-transport` interface in order to allow circuit to be plugged with `libp2p-swarm`. The rational behind it is that, `libp2p-circuit` has a dial and listen flow, which fits nicely with other transports, moreover, it requires the _raw_ connection to be encrypted and muxed just as a regular transport's connection does. All in all, `interface-transport` ended up being the correct level of abstraction for circuit, as well as allowed us to reuse existing integration points in `libp2p-swarm` and `libp2p` without adding any ad-hoc logic. All parts of `interface-transport` are used, including `.getAddr` which returns a list of `/p2p-circuit` addresses that circuit is currently listening.
This module is not a transport, however it implements `interface-transport` interface in order to allow circuit to be plugged with `libp2p`. The rational behind it is that, `libp2p-circuit` has a dial and listen flow, which fits nicely with other transports, moreover, it requires the _raw_ connection to be encrypted and muxed just as a regular transport's connection does. All in all, `interface-transport` ended up being the correct level of abstraction for circuit, as well as allowed us to reuse existing integration points in `libp2p` and `libp2p` without adding any ad-hoc logic. All parts of `interface-transport` are used, including `.getAddr` which returns a list of `/p2p-circuit` addresses that circuit is currently listening.
```
libp2p libp2p-circuit (transport)
@@ -109,13 +82,13 @@ libp2p
| +---------------------------------+ | | |
| | | | | +------------------+ |
| | | | circuit-relay listens for the HOP | | | |
| | libp2p-swarm <------------------------------------------------| circuit-relay | |
| | libp2p <------------------------------------------------| circuit-relay | |
| | | | message to handle incomming relay | | | |
| | | | requests from other nodes | +------------------+ |
| +---------------------------------+ | | |
| ^ ^ ^ ^ ^ ^ | | +------------------+ |
| | | | | | | | | | +-------------+ | |
| | | | | | | | dialer uses libp2p-swarm to dial | | | | | |
| | | | | | | | dialer uses libp2p to dial | | | | | |
| | | | +----------------------------------------------------------------------> dialer | | |
| | | transports | | to a circuit-relay node using the | | | | | |
| | | | | | | HOP message | | +-------------+ | |

View File

@@ -215,6 +215,19 @@ class ConnectionManager extends EventEmitter {
* @returns {Connection}
*/
get (peerId) {
const connections = this.getAll(peerId)
if (connections.length) {
return connections[0]
}
return null
}
/**
* Get all open connections with a peer.
* @param {PeerId} peerId
* @returns {Array<Connection>}
*/
getAll (peerId) {
if (!PeerId.isPeerId(peerId)) {
throw errcode(new Error('peerId must be an instance of peer-id'), ERR_INVALID_PARAMETERS)
}
@@ -222,11 +235,11 @@ class ConnectionManager extends EventEmitter {
const id = peerId.toB58String()
const connections = this.connections.get(id)
// Return the first, open connection
// Return all open connections
if (connections) {
return connections.find(connection => connection.stat.status === 'open')
return connections.filter(connection => connection.stat.status === 'open')
}
return null
return []
}
/**

View File

@@ -208,7 +208,7 @@ class IdentifyService {
* @param {*} options.stream
* @param {Connection} options.connection
*/
_handleIdentify ({ connection, stream }) {
async _handleIdentify ({ connection, stream }) {
let publicKey = Buffer.alloc(0)
if (this.peerId.pubKey) {
publicKey = this.peerId.pubKey.bytes
@@ -223,12 +223,16 @@ class IdentifyService {
protocols: Array.from(this._protocols.keys())
})
pipe(
[message],
lp.encode(),
stream,
consume
)
try {
await pipe(
[message],
lp.encode(),
stream,
consume
)
} catch (err) {
log.error('could not respond to identify request', err)
}
}
/**
@@ -239,17 +243,16 @@ class IdentifyService {
* @param {Connection} options.connection
*/
async _handlePush ({ connection, stream }) {
const [data] = await pipe(
[],
stream,
lp.decode(),
take(1),
toBuffer,
collect
)
let message
try {
const [data] = await pipe(
[],
stream,
lp.decode(),
take(1),
toBuffer,
collect
)
message = Message.decode(data)
} catch (err) {
return log.error('received invalid message', err)

View File

@@ -166,6 +166,7 @@ class Libp2p extends EventEmitter {
if (this._modules.dht) {
const DHT = this._modules.dht
this._dht = new DHT({
libp2p: this,
dialer: this.dialer,
peerId: this.peerId,
peerStore: this.peerStore,

View File

@@ -215,7 +215,7 @@ class Metrics {
const _sink = stream.sink
stream.sink = source => {
pipe(
return pipe(
source,
tap(chunk => metrics._onMessage({
remotePeer,

View File

@@ -96,7 +96,9 @@ class PersistentPeerStore extends PeerStore {
if (this._dirtyPeers.size >= this.threshold) {
// Commit current data
this._commitData()
this._commitData().catch(err => {
log.error('error committing data', err)
})
}
}
@@ -120,7 +122,9 @@ class PersistentPeerStore extends PeerStore {
if (this._dirtyPeers.size >= this.threshold) {
// Commit current data
this._commitData()
this._commitData().catch(err => {
log.error('error committing data', err)
})
}
}

View File

@@ -17,7 +17,7 @@ const handshake = require('it-handshake')
const { NONCE_LENGTH } = require('./key-generator')
const debug = require('debug')
const log = debug('libp2p:pnet')
log.err = debug('libp2p:pnet:err')
log.error = debug('libp2p:pnet:err')
/**
* Takes a Private Shared Key (psk) and provides a `protect` method
@@ -69,7 +69,7 @@ class Protector {
// Decrypt all inbound traffic
createUnboxStream(remoteNonce, this.psk),
external
)
).catch(log.error)
return internal
}

View File

@@ -258,7 +258,7 @@ class Upgrader {
}
// Pipe all data through the muxer
pipe(upgradedConn, muxer, upgradedConn)
pipe(upgradedConn, muxer, upgradedConn).catch(log.error)
}
const _timeline = maConn.timeline

View File

@@ -5,7 +5,6 @@ const chai = require('chai')
chai.use(require('dirty-chai'))
const { expect } = chai
const sinon = require('sinon')
const PeerStore = require('../../src/peer-store/persistent')
const multiaddr = require('multiaddr')
const { MemoryDatastore } = require('interface-datastore')
@@ -62,6 +61,7 @@ describe('Persisted PeerStore', () => {
const protocols = ['/ping/1.0.0']
const spyDirty = sinon.spy(peerStore, '_addDirtyPeer')
const spyDs = sinon.spy(datastore, 'batch')
const commitSpy = sinon.spy(peerStore, '_commitData')
await peerStore.start()
@@ -71,12 +71,18 @@ describe('Persisted PeerStore', () => {
expect(spyDirty).to.have.property('callCount', 1) // Address
expect(spyDs).to.have.property('callCount', 1)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// ProtoBook
peerStore.protoBook.set(peer, protocols)
expect(spyDirty).to.have.property('callCount', 2) // Protocol
expect(spyDs).to.have.property('callCount', 2)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// Should have three peer records stored in the datastore
const queryParams = {
prefix: '/peers/'
@@ -98,6 +104,7 @@ describe('Persisted PeerStore', () => {
it('should load content to the peerStore when restart but not put in datastore again', async () => {
const spyDs = sinon.spy(datastore, 'batch')
const peers = await peerUtils.createPeerId({ number: 2 })
const commitSpy = sinon.spy(peerStore, '_commitData')
const multiaddrs = [
multiaddr('/ip4/156.10.1.22/tcp/1000'),
multiaddr('/ip4/156.10.1.23/tcp/1000')
@@ -110,17 +117,29 @@ describe('Persisted PeerStore', () => {
peerStore.addressBook.set(peers[0], [multiaddrs[0]])
peerStore.addressBook.set(peers[1], [multiaddrs[1]])
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// KeyBook
peerStore.keyBook.set(peers[0], peers[0].pubKey)
peerStore.keyBook.set(peers[1], peers[1].pubKey)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// ProtoBook
peerStore.protoBook.set(peers[0], protocols)
peerStore.protoBook.set(peers[1], protocols)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// MetadataBook
peerStore.metadataBook.set(peers[0], 'location', Buffer.from('earth'))
// let batch commit complete
await Promise.all(commitSpy.returnValues)
expect(spyDs).to.have.property('callCount', 7) // 2 Address + 2 Key + 2 Proto + 1 Metadata
expect(peerStore.peers.size).to.equal(2)
@@ -148,6 +167,7 @@ describe('Persisted PeerStore', () => {
const [peer] = await peerUtils.createPeerId()
const multiaddrs = [multiaddr('/ip4/156.10.1.22/tcp/1000')]
const protocols = ['/ping/1.0.0']
const commitSpy = sinon.spy(peerStore, '_commitData')
await peerStore.start()
@@ -158,6 +178,9 @@ describe('Persisted PeerStore', () => {
// MetadataBook
peerStore.metadataBook.set(peer, 'location', Buffer.from('earth'))
// let batch commit complete
await Promise.all(commitSpy.returnValues)
const spyDs = sinon.spy(datastore, 'batch')
const spyAddressBook = sinon.spy(peerStore.addressBook, 'delete')
const spyKeyBook = sinon.spy(peerStore.keyBook, 'delete')
@@ -166,6 +189,10 @@ describe('Persisted PeerStore', () => {
// Delete from PeerStore
peerStore.delete(peer)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
await peerStore.stop()
expect(spyAddressBook).to.have.property('callCount', 1)
@@ -197,6 +224,7 @@ describe('Persisted PeerStore', () => {
const spyDirty = sinon.spy(peerStore, '_addDirtyPeer')
const spyDirtyMetadata = sinon.spy(peerStore, '_addDirtyPeerMetadata')
const spyDs = sinon.spy(datastore, 'batch')
const commitSpy = sinon.spy(peerStore, '_commitData')
const peers = await peerUtils.createPeerId({ number: 2 })
@@ -213,9 +241,15 @@ describe('Persisted PeerStore', () => {
peerStore.protoBook.set(peers[0], protocols)
peerStore.metadataBook.set(peers[0], 'location', Buffer.from('earth'))
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// Remove data from the same Peer
peerStore.addressBook.delete(peers[0])
// let batch commit complete
await Promise.all(commitSpy.returnValues)
expect(spyDirty).to.have.property('callCount', 3) // 2 AddrBook ops, 1 ProtoBook op
expect(spyDirtyMetadata).to.have.property('callCount', 1) // 1 MetadataBook op
expect(peerStore._dirtyPeers.size).to.equal(1)
@@ -231,6 +265,9 @@ describe('Persisted PeerStore', () => {
// Add data for second book
peerStore.addressBook.set(peers[1], multiaddrs)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
expect(spyDirty).to.have.property('callCount', 4)
expect(spyDirtyMetadata).to.have.property('callCount', 1)
expect(spyDs).to.have.property('callCount', 1)
@@ -333,6 +370,7 @@ describe('libp2p.peerStore (Persisted)', () => {
})
it('should load content to the peerStore when a new node is started with the same datastore', async () => {
const commitSpy = sinon.spy(libp2p.peerStore, '_commitData')
const peers = await peerUtils.createPeerId({ number: 2 })
const multiaddrs = [
multiaddr('/ip4/156.10.1.22/tcp/1000'),
@@ -346,10 +384,16 @@ describe('libp2p.peerStore (Persisted)', () => {
libp2p.peerStore.addressBook.set(peers[0], [multiaddrs[0]])
libp2p.peerStore.addressBook.set(peers[1], [multiaddrs[1]])
// let batch commit complete
await Promise.all(commitSpy.returnValues)
// ProtoBook
libp2p.peerStore.protoBook.set(peers[0], protocols)
libp2p.peerStore.protoBook.set(peers[1], protocols)
// let batch commit complete
await Promise.all(commitSpy.returnValues)
expect(libp2p.peerStore.peers.size).to.equal(2)
await libp2p.stop()

View File

@@ -189,4 +189,135 @@ describe('Pubsub subsystem operates correctly', () => {
await defer.promise
})
})
describe('pubsub with intermittent connections', () => {
beforeEach(async () => {
libp2p = await create(mergeOptions(subsystemOptions, {
peerId,
addresses: {
listen: [listenAddr]
},
config: {
pubsub: {
enabled: true,
emitSelf: false
}
}
}))
remoteLibp2p = await create(mergeOptions(subsystemOptions, {
peerId: remotePeerId,
addresses: {
listen: [remoteListenAddr]
},
config: {
pubsub: {
enabled: true,
emitSelf: false
}
}
}))
await libp2p.start()
await remoteLibp2p.start()
libp2p.peerStore.addressBook.set(remotePeerId, remoteLibp2p.multiaddrs)
})
afterEach(() => Promise.all([
libp2p && libp2p.stop(),
remoteLibp2p && remoteLibp2p.stop()
]))
afterEach(() => {
sinon.restore()
})
it('should receive pubsub messages after a node restart', async () => {
const topic = 'test-topic'
const data = 'hey!'
const libp2pId = libp2p.peerId.toB58String()
let counter = 0
const defer1 = pDefer()
const defer2 = pDefer()
const handler = (msg) => {
expect(msg.data.toString()).to.equal(data)
counter++
counter === 1 ? defer1.resolve() : defer2.resolve()
}
await libp2p.dial(remotePeerId)
let subscribedTopics = libp2p.pubsub.getTopics()
expect(subscribedTopics).to.not.include(topic)
libp2p.pubsub.subscribe(topic, handler)
subscribedTopics = libp2p.pubsub.getTopics()
expect(subscribedTopics).to.include(topic)
// wait for remoteLibp2p to know about libp2p subscription
await pWaitFor(() => {
const subscribedPeers = remoteLibp2p.pubsub.getSubscribers(topic)
return subscribedPeers.includes(libp2pId)
})
remoteLibp2p.pubsub.publish(topic, data)
await defer1.promise
await remoteLibp2p.stop()
await remoteLibp2p.start()
libp2p.peerStore.addressBook.set(remotePeerId, remoteLibp2p.multiaddrs)
await libp2p.dial(remotePeerId)
// wait for remoteLibp2p to know about libp2p subscription
await pWaitFor(() => {
const subscribedPeers = remoteLibp2p.pubsub.getSubscribers(topic)
return subscribedPeers.includes(libp2pId)
})
remoteLibp2p.pubsub.publish(topic, data)
await defer2.promise
})
it('should handle quick reconnects with a delayed disconnect', async () => {
// Subscribe on both
const handlerSpy = sinon.spy()
const topic = 'reconnect-channel'
await Promise.all([
libp2p.pubsub.subscribe(topic, handlerSpy),
remoteLibp2p.pubsub.subscribe(topic, handlerSpy)
])
// Create two connections to the remote peer
const originalConnection = await libp2p.dialer.connectToPeer(remoteLibp2p.peerId)
// second connection
await libp2p.dialer.connectToPeer(remoteLibp2p.peerId)
expect(libp2p.connections.get(remoteLibp2p.peerId.toB58String())).to.have.length(2)
// Wait for subscriptions to occur
await pWaitFor(() => {
return libp2p.pubsub.getSubscribers(topic).includes(remoteLibp2p.peerId.toB58String()) &&
remoteLibp2p.pubsub.getSubscribers(topic).includes(libp2p.peerId.toB58String())
})
// Verify messages go both ways
libp2p.pubsub.publish(topic, 'message1')
remoteLibp2p.pubsub.publish(topic, 'message2')
await pWaitFor(() => handlerSpy.callCount === 2)
expect(handlerSpy.args.map(([message]) => message.data.toString())).to.include.members(['message1', 'message2'])
// Disconnect the first connection (this acts as a delayed reconnect)
await originalConnection.close()
// Verify messages go both ways after the disconnect
handlerSpy.resetHistory()
libp2p.pubsub.publish(topic, 'message3')
remoteLibp2p.pubsub.publish(topic, 'message4')
await pWaitFor(() => handlerSpy.callCount === 2)
expect(handlerSpy.args.map(([message]) => message.data.toString())).to.include.members(['message3', 'message4'])
})
})
})