Compare commits

..

3 Commits

Author SHA1 Message Date
14d09970ca chore: release version v0.7.0 2020-11-03 18:26:50 +01:00
c98c58e824 chore: update contributors 2020-11-03 18:26:49 +01:00
946b046440 feat: pubsub: add global signature policy (#66)
BREAKING CHANGE:
`signMessages` and `strictSigning` pubsub configuration options replaced
with a `globalSignaturePolicy` option
2020-11-03 18:22:03 +01:00
11 changed files with 237 additions and 138 deletions

View File

@ -1,3 +1,22 @@
<a name="0.7.0"></a>
# [0.7.0](https://github.com/libp2p/js-interfaces/compare/v0.5.2...v0.7.0) (2020-11-03)
### Features
* pubsub: add global signature policy ([#66](https://github.com/libp2p/js-interfaces/issues/66)) ([946b046](https://github.com/libp2p/js-interfaces/commit/946b046))
* update pubsub getMsgId return type to Uint8Array ([#65](https://github.com/libp2p/js-interfaces/issues/65)) ([e148443](https://github.com/libp2p/js-interfaces/commit/e148443))
### BREAKING CHANGES
* `signMessages` and `strictSigning` pubsub configuration options replaced
with a `globalSignaturePolicy` option
* new getMsgId return type is not backwards compatible with prior `string`
return type.
<a name="0.6.0"></a>
# [0.6.0](https://github.com/libp2p/js-interfaces/compare/v0.5.2...v0.6.0) (2020-10-05)

View File

@ -1,6 +1,6 @@
{
"name": "libp2p-interfaces",
"version": "0.6.0",
"version": "0.7.0",
"description": "Interfaces for JS Libp2p",
"leadMaintainer": "Jacob Heun <jacobheun@gmail.com>",
"main": "src/index.js",
@ -53,8 +53,10 @@
"it-pipe": "^1.1.0",
"it-pushable": "^1.4.0",
"libp2p-crypto": "^0.18.0",
"libp2p-tcp": "^0.15.0",
"multiaddr": "^8.0.0",
"multibase": "^3.0.0",
"multihashes": "^3.0.1",
"p-defer": "^3.0.0",
"p-limit": "^2.3.0",
"p-wait-for": "^3.1.0",

View File

@ -209,8 +209,6 @@ class Connection {
* @return {Promise<void>}
*/
async close () {
this.streams.map(s => s.close && s.close())
if (this.stat.status === Status.CLOSED) {
return
}

View File

@ -11,6 +11,9 @@ Table of Contents
* [Extend interface](#extend-interface)
* [Example](#example)
* [API](#api)
* [Constructor](#constructor)
* [new Pubsub(options)](#new-pubsuboptions)
* [Parameters](#parameters)
* [Start](#start)
* [pubsub.start()](#pubsubstart)
* [Returns](#returns)
@ -19,24 +22,24 @@ Table of Contents
* [Returns](#returns-1)
* [Publish](#publish)
* [pubsub.publish(topics, message)](#pubsubpublishtopics-message)
* [Parameters](#parameters)
* [Parameters](#parameters-1)
* [Returns](#returns-2)
* [Subscribe](#subscribe)
* [pubsub.subscribe(topic)](#pubsubsubscribetopic)
* [Parameters](#parameters-1)
* [Parameters](#parameters-2)
* [Unsubscribe](#unsubscribe)
* [pubsub.unsubscribe(topic)](#pubsubunsubscribetopic)
* [Parameters](#parameters-2)
* [Parameters](#parameters-3)
* [Get Topics](#get-topics)
* [pubsub.getTopics()](#pubsubgettopics)
* [Returns](#returns-3)
* [Get Peers Subscribed to a topic](#get-peers-subscribed-to-a-topic)
* [pubsub.getSubscribers(topic)](#pubsubgetsubscriberstopic)
* [Parameters](#parameters-3)
* [Parameters](#parameters-4)
* [Returns](#returns-4)
* [Validate](#validate)
* [pubsub.validate(message)](#pubsubvalidatemessage)
* [Parameters](#parameters-4)
* [Parameters](#parameters-5)
* [Returns](#returns-5)
* [Test suite usage](#test-suite-usage)
@ -49,7 +52,7 @@ You can check the following implementations as examples for building your own pu
## Interface usage
`interface-pubsub` abstracts the implementation protocol registration within `libp2p` and takes care of all the protocol connections and streams, as well as the subscription management. This way, a pubsub implementation can focus on its message routing algorithm, instead of also needing to create the setup for it.
`interface-pubsub` abstracts the implementation protocol registration within `libp2p` and takes care of all the protocol connections and streams, as well as the subscription management and the features describe in the libp2p [pubsub specs](https://github.com/libp2p/specs/tree/master/pubsub). This way, a pubsub implementation can focus on its message routing algorithm, instead of also needing to create the setup for it.
### Extend interface
@ -74,7 +77,7 @@ All the remaining functions **MUST NOT** be overwritten.
The following example aims to show how to create your pubsub implementation extending this base protocol. The pubsub implementation will handle the subscriptions logic.
```JavaScript
const Pubsub = require('libp2p-pubsub')
const Pubsub = require('libp2p-interfaces/src/pubsub')
class PubsubImplementation extends Pubsub {
constructor({ libp2p, options })
@ -82,8 +85,7 @@ class PubsubImplementation extends Pubsub {
debugName: 'libp2p:pubsub',
multicodecs: '/pubsub-implementation/1.0.0',
libp2p,
signMessages: options.signMessages,
strictSigning: options.strictSigning
globalSigningPolicy: options.globalSigningPolicy
})
}
@ -98,6 +100,23 @@ class PubsubImplementation extends Pubsub {
The interface aims to specify a common interface that all pubsub router implementation should follow. A pubsub router implementation should extend the [EventEmitter](https://nodejs.org/api/events.html#events_class_eventemitter). When peers receive pubsub messages, these messages will be emitted by the event emitter where the `eventName` will be the `topic` associated with the message.
### Constructor
The base class constructor configures the pubsub instance for use with a libp2p instance. It includes settings for logging, signature policies, etc.
#### `new Pubsub({options})`
##### Parameters
| Name | Type | Description | Default |
|------|------|-------------|---------|
| options.libp2p | `Libp2p` | libp2p instance | required, no default |
| options.debugName | `string` | log namespace | required, no default |
| options.multicodecs | `string \| Array<string>` | protocol identifier(s) | required, no default |
| options.globalSignaturePolicy | `'StrictSign' \| 'StrictNoSign'` | signature policy to be globally applied | `'StrictSign'` |
| options.canRelayMessage | `boolean` | if can relay messages if not subscribed | `false` |
| options.emitSelf | `boolean` | if `publish` should emit to self, if subscribed | `false` |
### Start
Starts the pubsub subsystem. The protocol will be registered to `libp2p`, which will result in pubsub being notified when peers who support the protocol connect/disconnect to `libp2p`.
@ -185,7 +204,7 @@ Get a list of the [PeerId](https://github.com/libp2p/js-peer-id) strings that ar
### Validate
Validates the signature of a message.
Validates a message according to the signature policy and topic-specific validation function.
#### `pubsub.validate(message)`

View File

@ -1,6 +1,46 @@
'use strict'
exports.codes = {
/**
* Signature policy is invalid
*/
ERR_INVALID_SIGNATURE_POLICY: 'ERR_INVALID_SIGNATURE_POLICY',
/**
* Signature policy is unhandled
*/
ERR_UNHANDLED_SIGNATURE_POLICY: 'ERR_UNHANDLED_SIGNATURE_POLICY',
// Strict signing codes
/**
* Message expected to have a `signature`, but doesn't
*/
ERR_MISSING_SIGNATURE: 'ERR_MISSING_SIGNATURE',
ERR_INVALID_SIGNATURE: 'ERR_INVALID_SIGNATURE'
/**
* Message expected to have a `seqno`, but doesn't
*/
ERR_MISSING_SEQNO: 'ERR_MISSING_SEQNO',
/**
* Message `signature` is invalid
*/
ERR_INVALID_SIGNATURE: 'ERR_INVALID_SIGNATURE',
// Strict no-signing codes
/**
* Message expected to not have a `from`, but does
*/
ERR_UNEXPECTED_FROM: 'ERR_UNEXPECTED_FROM',
/**
* Message expected to not have a `signature`, but does
*/
ERR_UNEXPECTED_SIGNATURE: 'ERR_UNEXPECTED_SIGNATURE',
/**
* Message expected to not have a `key`, but does
*/
ERR_UNEXPECTED_KEY: 'ERR_UNEXPECTED_KEY',
/**
* Message expected to not have a `seqno`, but does
*/
ERR_UNEXPECTED_SEQNO: 'ERR_UNEXPECTED_SEQNO'
}

View File

@ -13,6 +13,7 @@ const { codes } = require('./errors')
*/
const message = require('./message')
const PeerStreams = require('./peer-streams')
const { SignaturePolicy } = require('./signature-policy')
const utils = require('./utils')
const {
@ -44,8 +45,7 @@ class PubsubBaseProtocol extends EventEmitter {
* @param {String} props.debugName log namespace
* @param {Array<string>|string} props.multicodecs protocol identificers to connect
* @param {Libp2p} props.libp2p
* @param {boolean} [props.signMessages = true] if messages should be signed
* @param {boolean} [props.strictSigning = true] if message signing should be required
* @param {SignaturePolicy} [props.globalSignaturePolicy = SignaturePolicy.StrictSign] defines how signatures should be handled
* @param {boolean} [props.canRelayMessage = false] if can relay messages not subscribed
* @param {boolean} [props.emitSelf = false] if publish should emit to self, if subscribed
* @abstract
@ -54,8 +54,7 @@ class PubsubBaseProtocol extends EventEmitter {
debugName,
multicodecs,
libp2p,
signMessages = true,
strictSigning = true,
globalSignaturePolicy = SignaturePolicy.StrictSign,
canRelayMessage = false,
emitSelf = false
}) {
@ -109,14 +108,17 @@ class PubsubBaseProtocol extends EventEmitter {
*/
this.peers = new Map()
// Message signing
this.signMessages = signMessages
// validate signature policy
if (!SignaturePolicy[globalSignaturePolicy]) {
throw errcode(new Error('Invalid global signature policy'), codes.ERR_INVALID_SIGUATURE_POLICY)
}
/**
* If message signing should be required for incoming messages
* @type {boolean}
* The signature policy to follow by default
*
* @type {SignaturePolicy}
*/
this.strictSigning = strictSigning
this.globalSignaturePolicy = globalSignaturePolicy
/**
* If router can relay received messages, even if not subscribed
@ -440,7 +442,15 @@ class PubsubBaseProtocol extends EventEmitter {
* @returns {Uint8Array} message id as bytes
*/
getMsgId (msg) {
return utils.msgId(msg.from, msg.seqno)
const signaturePolicy = this.globalSignaturePolicy
switch (signaturePolicy) {
case SignaturePolicy.StrictSign:
return utils.msgId(msg.from, msg.seqno)
case SignaturePolicy.StrictNoSign:
return utils.noSignMsgId(msg.data)
default:
throw errcode(new Error('Cannot get message id: unhandled signature policy: ' + signaturePolicy), codes.ERR_UNHANDLED_SIGNATURE_POLICY)
}
}
/**
@ -511,16 +521,36 @@ class PubsubBaseProtocol extends EventEmitter {
* @returns {Promise<void>}
*/
async validate (message) { // eslint-disable-line require-await
// If strict signing is on and we have no signature, abort
if (this.strictSigning && !message.signature) {
throw errcode(new Error('Signing required and no signature was present'), codes.ERR_MISSING_SIGNATURE)
const signaturePolicy = this.globalSignaturePolicy
switch (signaturePolicy) {
case SignaturePolicy.StrictNoSign:
if (message.from) {
throw errcode(new Error('StrictNoSigning: from should not be present'), codes.ERR_UNEXPECTED_FROM)
}
if (message.signature) {
throw errcode(new Error('StrictNoSigning: signature should not be present'), codes.ERR_UNEXPECTED_SIGNATURE)
}
if (message.key) {
throw errcode(new Error('StrictNoSigning: key should not be present'), codes.ERR_UNEXPECTED_KEY)
}
if (message.seqno) {
throw errcode(new Error('StrictNoSigning: seqno should not be present'), codes.ERR_UNEXPECTED_SEQNO)
}
break
case SignaturePolicy.StrictSign:
if (!message.signature) {
throw errcode(new Error('StrictSigning: Signing required and no signature was present'), codes.ERR_MISSING_SIGNATURE)
}
if (!message.seqno) {
throw errcode(new Error('StrictSigning: Signing required and no seqno was present'), codes.ERR_MISSING_SEQNO)
}
if (!(await verifySignature(message))) {
throw errcode(new Error('StrictSigning: Invalid message signature'), codes.ERR_INVALID_SIGNATURE)
}
break
default:
throw errcode(new Error('Cannot validate message: unhandled signature policy: ' + signaturePolicy), codes.ERR_UNHANDLED_SIGNATURE_POLICY)
}
// Check the message signature if present
if (message.signature && !(await verifySignature(message))) {
throw errcode(new Error('Invalid message signature'), codes.ERR_INVALID_SIGNATURE)
}
for (const topic of message.topicIDs) {
const validatorFn = this.topicValidators.get(topic)
if (!validatorFn) {
@ -538,11 +568,16 @@ class PubsubBaseProtocol extends EventEmitter {
* @returns {Promise<Message>}
*/
_buildMessage (message) {
const msg = utils.normalizeOutRpcMessage(message)
if (this.signMessages) {
return signMessage(this.peerId, msg)
} else {
return message
const signaturePolicy = this.globalSignaturePolicy
switch (signaturePolicy) {
case SignaturePolicy.StrictSign:
message.from = this.peerId.toB58String()
message.seqno = utils.randomSeqno()
return signMessage(this.peerId, utils.normalizeOutRpcMessage(message))
case SignaturePolicy.StrictNoSign:
return message
default:
throw errcode(new Error('Cannot build message: unhandled signature policy: ' + signaturePolicy), codes.ERR_UNHANDLED_SIGNATURE_POLICY)
}
}
@ -586,13 +621,11 @@ class PubsubBaseProtocol extends EventEmitter {
const from = this.peerId.toB58String()
let msgObject = {
receivedFrom: from,
from: from,
data: message,
seqno: utils.randomSeqno(),
topicIDs: [topic]
}
// ensure that any operations performed on the message will include the signature
// ensure that the message follows the signature policy
const outMsg = await this._buildMessage(msgObject)
msgObject = utils.normalizeInRpcMessage(outMsg)
@ -666,3 +699,4 @@ class PubsubBaseProtocol extends EventEmitter {
module.exports = PubsubBaseProtocol
module.exports.message = message
module.exports.utils = utils
module.exports.SignaturePolicy = SignaturePolicy

View File

@ -0,0 +1,28 @@
'use strict'
/**
* Enum for Signature Policy
* Details how message signatures are produced/consumed
*/
exports.SignaturePolicy = {
/**
* On the producing side:
* * Build messages with the signature, key (from may be enough for certain inlineable public key types), from and seqno fields.
*
* On the consuming side:
* * Enforce the fields to be present, reject otherwise.
* * Propagate only if the fields are valid and signature can be verified, reject otherwise.
*/
StrictSign: 'StrictSign',
/**
* On the producing side:
* * Build messages without the signature, key, from and seqno fields.
* * The corresponding protobuf key-value pairs are absent from the marshalled message, not just empty.
*
* On the consuming side:
* * Enforce the fields to be absent, reject otherwise.
* * Propagate only if the fields are absent, reject otherwise.
* * A message_id function will not be able to use the above fields, and should instead rely on the data field. A commonplace strategy is to calculate a hash.
*/
StrictNoSign: 'StrictNoSign'
}

View File

@ -4,6 +4,7 @@ const randomBytes = require('libp2p-crypto/src/random-bytes')
const uint8ArrayToString = require('uint8arrays/to-string')
const uint8ArrayFromString = require('uint8arrays/from-string')
const PeerId = require('peer-id')
const multihash = require('multihashes')
exports = module.exports
/**
@ -32,6 +33,15 @@ exports.msgId = (from, seqno) => {
return msgId
}
/**
* Generate a message id, based on message `data`.
*
* @param {Uint8Array} data
* @returns {Uint8Array}
* @private
*/
exports.noSignMsgId = (data) => multihash.encode(data, 'sha2')
/**
* Check if any member of the first set is also a member
* of the second set.

View File

@ -2,17 +2,17 @@
/* eslint max-nested-callbacks: ["error", 8] */
'use strict'
const chai = require('chai')
const expect = chai.expect
chai.use(require('dirty-chai'))
const pair = require('it-pair/duplex')
const pipe = require('it-pipe')
const { consume } = require('streaming-iterables')
const Tcp = require('libp2p-tcp')
const multiaddr = require('multiaddr')
const abortable = require('abortable-iterator')
const AbortController = require('abort-controller')
const uint8arrayFromString = require('uint8arrays/from-string')
const mh = multiaddr('/ip4/127.0.0.1/tcp/0')
function pause (ms) {
return new Promise(resolve => setTimeout(resolve, ms))
}
@ -38,31 +38,33 @@ module.exports = (common) => {
Muxer = await common.setup()
})
it('closing underlying socket closes streams', async () => {
it('closing underlying socket closes streams (tcp)', async () => {
const mockConn = muxer => ({
newStream: (...args) => muxer.newStream(...args)
})
const mockUpgrade = maConn => {
const mockUpgrade = () => maConn => {
const muxer = new Muxer(stream => pipe(stream, stream))
pipe(maConn, muxer, maConn)
return mockConn(muxer)
}
const [local, remote] = pair()
const controller = new AbortController()
const abortableRemote = abortable.duplex(remote, controller.signal, {
returnOnAbort: true
const mockUpgrader = () => ({
upgradeInbound: mockUpgrade(),
upgradeOutbound: mockUpgrade()
})
mockUpgrade(abortableRemote)
const dialerConn = mockUpgrade(local)
const tcp = new Tcp({ upgrader: mockUpgrader() })
const tcpListener = tcp.createListener()
await tcpListener.listen(mh)
const dialerConn = await tcp.dial(tcpListener.getAddrs()[0])
const s1 = await dialerConn.newStream()
const s2 = await dialerConn.newStream()
// close the remote in a bit
setTimeout(() => controller.abort(), 50)
// close the listener in a bit
setTimeout(() => tcpListener.close(), 50)
const s1Result = pipe(infiniteRandom, s1, consume)
const s2Result = pipe(infiniteRandom, s2, consume)
@ -113,69 +115,5 @@ module.exports = (common) => {
// These should now all resolve without error
await Promise.all(streamResults)
})
it('can close a stream for writing', (done) => {
const p = pair()
const dialer = new Muxer()
const data = [randomBuffer(), randomBuffer()]
const listener = new Muxer(async stream => {
// Immediate close for write
await stream.closeWrite()
const results = await pipe(stream, async (source) => {
const data = []
for await (const chunk of source) {
data.push(chunk.slice())
}
return data
})
expect(results).to.eql(data)
try {
await stream.sink([randomBuffer()])
} catch (err) {
expect(err).to.exist()
return done()
}
expect.fail('should not support writing to closed writer')
})
pipe(p[0], dialer, p[0])
pipe(p[1], listener, p[1])
const stream = dialer.newStream()
stream.sink(data)
})
it('can close a stream for reading', (done) => {
const p = pair()
const dialer = new Muxer()
const data = [randomBuffer(), randomBuffer()]
const listener = new Muxer(async stream => {
const results = await pipe(stream, async (source) => {
const data = []
for await (const chunk of source) {
data.push(chunk.slice())
}
return data
})
expect(results).to.eql(data)
done()
})
pipe(p[0], dialer, p[0])
pipe(p[1], listener, p[1])
const stream = dialer.newStream()
stream.closeRead()
// Source should be done
;(async () => {
expect(await stream.source.next()).to.eql({ done: true })
stream.sink(data)
})()
})
})
}

View File

@ -5,7 +5,7 @@ const { expect } = require('aegir/utils/chai')
const sinon = require('sinon')
const PubsubBaseImpl = require('../../src/pubsub')
const { randomSeqno } = require('../../src/pubsub/utils')
const { SignaturePolicy } = require('../../src/pubsub/signature-policy')
const {
createPeerId,
mockRegistrar
@ -34,9 +34,7 @@ describe('pubsub base messages', () => {
it('_buildMessage normalizes and signs messages', async () => {
const message = {
receivedFrom: peerId.id,
from: peerId.id,
data: 'hello',
seqno: randomSeqno(),
topicIDs: ['test-topic']
}
@ -44,27 +42,46 @@ describe('pubsub base messages', () => {
expect(pubsub.validate(signedMessage)).to.not.be.rejected()
})
it('validate with strict signing off will validate a present signature', async () => {
it('validate with StrictNoSign will reject a message with from, signature, key, seqno present', async () => {
const message = {
receivedFrom: peerId.id,
from: peerId.id,
data: 'hello',
seqno: randomSeqno(),
topicIDs: ['test-topic']
}
sinon.stub(pubsub, 'strictSigning').value(false)
sinon.stub(pubsub, 'globalSignaturePolicy').value(SignaturePolicy.StrictSign)
const signedMessage = await pubsub._buildMessage(message)
sinon.stub(pubsub, 'globalSignaturePolicy').value(SignaturePolicy.StrictNoSign)
await expect(pubsub.validate(signedMessage)).to.be.rejected()
delete signedMessage.from
await expect(pubsub.validate(signedMessage)).to.be.rejected()
delete signedMessage.signature
await expect(pubsub.validate(signedMessage)).to.be.rejected()
delete signedMessage.key
await expect(pubsub.validate(signedMessage)).to.be.rejected()
delete signedMessage.seqno
await expect(pubsub.validate(signedMessage)).to.not.be.rejected()
})
it('validate with StrictNoSign will validate a message without a signature, key, and seqno', async () => {
const message = {
receivedFrom: peerId.id,
data: 'hello',
topicIDs: ['test-topic']
}
sinon.stub(pubsub, 'globalSignaturePolicy').value(SignaturePolicy.StrictNoSign)
const signedMessage = await pubsub._buildMessage(message)
expect(pubsub.validate(signedMessage)).to.not.be.rejected()
})
it('validate with strict signing requires a signature', async () => {
it('validate with StrictSign requires a signature', async () => {
const message = {
receivedFrom: peerId.id,
from: peerId.id,
data: 'hello',
seqno: randomSeqno(),
topicIDs: ['test-topic']
}

View File

@ -10,8 +10,8 @@ const PeerId = require('peer-id')
const uint8ArrayEquals = require('uint8arrays/equals')
const uint8ArrayFromString = require('uint8arrays/from-string')
const { utils } = require('../../src/pubsub')
const PeerStreams = require('../../src/pubsub/peer-streams')
const { SignaturePolicy } = require('../../src/pubsub/signature-policy')
const {
createPeerId,
@ -30,6 +30,8 @@ describe('topic validators', () => {
pubsub = new PubsubImplementation(protocol, {
peerId: peerId,
registrar: mockRegistrar
}, {
globalSignaturePolicy: SignaturePolicy.StrictNoSign
})
pubsub.start()
@ -42,8 +44,6 @@ describe('topic validators', () => {
it('should filter messages by topic validator', async () => {
// use _publish.callCount() to see if a message is valid or not
sinon.spy(pubsub, '_publish')
// Disable strict signing
sinon.stub(pubsub, 'strictSigning').value(false)
sinon.stub(pubsub.peers, 'get').returns({})
const filteredTopic = 't'
const peer = new PeerStreams({ id: await PeerId.create() })
@ -59,9 +59,7 @@ describe('topic validators', () => {
const validRpc = {
subscriptions: [],
msgs: [{
from: peer.id.toBytes(),
data: uint8ArrayFromString('a message'),
seqno: utils.randomSeqno(),
topicIDs: [filteredTopic]
}]
}
@ -76,9 +74,7 @@ describe('topic validators', () => {
const invalidRpc = {
subscriptions: [],
msgs: [{
from: peer.id.toBytes(),
data: uint8ArrayFromString('a different message'),
seqno: utils.randomSeqno(),
topicIDs: [filteredTopic]
}]
}
@ -94,9 +90,7 @@ describe('topic validators', () => {
const invalidRpc2 = {
subscriptions: [],
msgs: [{
from: peer.id.toB58String(),
data: uint8ArrayFromString('a different message'),
seqno: utils.randomSeqno(),
topicIDs: [filteredTopic]
}]
}