feat: address manager

src/address-manager/README.md

@@ -0,0 +1,49 @@
+# Address Manager
+
+The Address manager is responsible for keeping an updated register of the peer's addresses. It includes 3 different types of Addresses: `Listen Addresses`, `Announce Addresses` and `No Announce Addresses`.
+
+These Addresses should be specified in your libp2p [configuration](../../doc/CONFIGURATION.md) when you create your node.
+
+## Listen Addresses
+
+A libp2p node should have a set of listen addresses, which will be used by libp2p underlying transports to listen for dials from other nodes in the network.
+
+Before a libp2p node starts, a set of listen addresses should be provided to the AddressManager, so that when the node is started, the libp2p transports can use them to listen for connections. Accordingly, listen addresses should be specified through the libp2p configuration, in order to have the `AddressManager` created with them.
+
+It is important pointing out that libp2p accepts to listen on addresses that intend to rely on any available local port. In this context, the provided listen addresses might not be exactly the same as the ones used by the transports. For example tcp may replace `/ip4/0.0.0.0/tcp/0` with something like `/ip4/0.0.0.0/tcp/8989`. As a consequence, libp2p should take into account this when advertising its addresses.
+
+## Announce Addresses
+
+In some scenarios, a libp2p node will need to announce addresses that it is not listening on. In other words, Announce Addresses are an amendment to the Listen Addresses that aim to enable other nodes to achieve connectivity to this node.
+
+Scenarios for Announce Addresses include:
+- when you setup a libp2p node in your private network at home, but you need to announce your public IP Address to the outside world;
+- when you want to announce a DNS address, which maps to your public IP Address.
+
+## No Announce Addresses
+
+While we need to add Announce Addresses to enable peers' connectivity, we can also not announce addresses that will not be reachable. This way, No Announce Addresses should be specified so that they are not announced by the peer as addresses that other peers can use to dial it.
+
+As stated into the Listen Addresses section, Listen Addresses might get modified after libp2p transports get in action and use them to listen for new connections. This way, libp2p should also take into account these changes so that they can be matched when No Announce Addresses are being filtered out for advertising addresses.
+
+## Implementation
+
+When a libp2p node is created, the Address Manager will be populated from the provided addresses through the libp2p configuration. Once the node is started, the Transport Manager component will gather the listen addresses from the Address Manager, so that the libp2p transports use them to listen on.
+
+Libp2p will use the the Address Manager as the source of truth when advertising the peers addresses. After all transports are ready, other libp2p components/subsystems will kickoff, namely the Identify Service and the DHT. Both of them will announce the node addresses to the other peers in the network. The announce and noAnnounce addresses will have an important role here and will be gathered by libp2p to compute its current addresses to advertise everytime it is needed.
+
+## Future Considerations
+
+### Dynamic address modifications 
+
+In a future iteration, we can enable these addresses to be modified in runtime. For this, the Address Manager should be responsible for notifying interested subsystems of these changes, through an Event Emitter.
+
+#### Modify Listen Addresses
+
+While adding new addresses to listen on runtime is a feasible operation, removing one listen address might have bad implications for the node, since all the connections using that listen address will be closed. With this in mind and taking also into consideration the lack of good use cases for removing listen addresses, the Address Manager API only allows libp2p users to add new Listen Addresses on runtime.
+
+Every time a new listen address is added, the Address Manager should emit an event with the new multiaddrs to listen. The Transport Manager should listen to this events and act accordingly.
+
+#### Modify Announce Addresses
+
+When the announce addresses are modified, the Address Manager should emit an event so that other subsystems can act accordingly. For example, libp2p identify service should use the libp2p push protocol to inform other peers about these changes.

src/address-manager/index.js

@@ -0,0 +1,55 @@
+'use strict'
+
+const debug = require('debug')
+const log = debug('libp2p:addresses')
+log.error = debug('libp2p:addresses:error')
+
+const multiaddr = require('multiaddr')
+
+/**
+ * Responsible for managing the peer addresses.
+ * Peers can specify their listen, announce and noAnnounce addresses.
+ * The listen addresses will be used by the libp2p transports to listen for new connections,
+ * while the announce an noAnnounce addresses will be combined with the listen addresses for
+ * address adverstising to other peers in the network.
+ */
+class AddressManager {
+  /**
+   * @constructor
+   * @param {object} [options]
+   * @param {Array<string>} [options.listen = []] list of multiaddrs string representation to listen.
+   * @param {Array<string>} [options.announce = []] list of multiaddrs string representation to announce.
+   * @param {Array<string>} [options.noAnnounce = []] list of multiaddrs string representation to not announce.
+   */
+  constructor ({ listen = [], announce = [], noAnnounce = [] } = {}) {
+    this.listen = new Set(listen)
+    this.announce = new Set(announce)
+    this.noAnnounce = new Set(noAnnounce)
+  }
+
+  /**
+   * Get peer listen multiaddrs.
+   * @return {Array<Multiaddr>}
+   */
+  getListenMultiaddrs () {
+    return Array.from(this.listen).map((a) => multiaddr(a))
+  }
+
+  /**
+   * Get peer announcing multiaddrs.
+   * @return {Array<Multiaddr>}
+   */
+  getAnnounceMultiaddrs () {
+    return Array.from(this.announce).map((a) => multiaddr(a))
+  }
+
+  /**
+   * Get peer noAnnouncing multiaddrs.
+   * @return {Array<Multiaddr>}
+   */
+  getNoAnnounceMultiaddrs () {
+    return Array.from(this.noAnnounce).map((a) => multiaddr(a))
+  }
+}
+
+module.exports = AddressManager

src/circuit/index.js

@@ -32,7 +32,7 @@ class Circuit {
     this._connectionManager = libp2p.connectionManager
     this._upgrader = upgrader
     this._options = libp2p._config.relay
-    this.addresses = libp2p.addresses
+    this.addressManager = libp2p.addressManager
     this.peerId = libp2p.peerId
     this._registrar.handle(multicodec, this._onProtocol.bind(this))
   }
@@ -122,7 +122,7 @@ class Circuit {
           type: CircuitPB.Type.HOP,
           srcPeer: {
             id: this.peerId.toBytes(),
-            addrs: this.addresses.listen.map(addr => addr.buffer)
+            addrs: this.addressManager.getListenMultiaddrs().map(addr => addr.buffer)
           },
           dstPeer: {
             id: destinationPeer.toBytes(),

src/config.js

@@ -5,7 +5,9 @@ const Constants = require('./constants')
 
 const DefaultConfig = {
   addresses: {
-    listen: []
+    listen: [],
+    announce: [],
+    noAnnounce: []
   },
   connectionManager: {
     minPeers: 25

src/identify/index.js

@@ -46,22 +46,20 @@ class IdentifyService {
   /**
    * @constructor
    * @param {object} options
-   * @param {PeerStore} options.peerStore
-   * @param {ConnectionManager} options.connectionManager
+   * @param {Libp2p} options.libp2p
    * @param {Map<string, handler>} options.protocols A reference to the protocols we support
-   * @param {PeerId} options.peerId The peer running the identify service
-   * @param {{ listen: Array<Multiaddr>}} options.addresses The peer addresses
    */
   constructor (options) {
     /**
      * @property {PeerStore}
      */
-    this.peerStore = options.peerStore
+    this.peerStore = options.libp2p.peerStore
 
     /**
      * @property {ConnectionManager}
      */
-    this.connectionManager = options.connectionManager
+    this.connectionManager = options.libp2p.connectionManager
+
     this.connectionManager.on('peer:connect', (connection) => {
       const peerId = connection.remotePeer
 
@@ -71,9 +69,12 @@ class IdentifyService {
     /**
      * @property {PeerId}
      */
-    this.peerId = options.peerId
+    this.peerId = options.libp2p.peerId
 
-    this.addresses = options.addresses || {}
+    /**
+     * @property {AddressManager}
+     */
+    this._libp2p = options.libp2p
 
     this._protocols = options.protocols
 
@@ -92,7 +93,7 @@ class IdentifyService {
 
         await pipe(
           [{
-            listenAddrs: this.addresses.listen.map((ma) => ma.buffer),
+            listenAddrs: this._libp2p.getAdvertisingMultiaddrs().map((ma) => ma.buffer),
             protocols: Array.from(this._protocols.keys())
           }],
           pb.encode(Message),
@@ -217,7 +218,7 @@ class IdentifyService {
       protocolVersion: PROTOCOL_VERSION,
       agentVersion: AGENT_VERSION,
       publicKey,
-      listenAddrs: this.addresses.listen.map((ma) => ma.buffer),
+      listenAddrs: this._libp2p.getAdvertisingMultiaddrs().map((ma) => ma.buffer),
       observedAddr: connection.remoteAddr.buffer,
       protocols: Array.from(this._protocols.keys())
     })

src/index.js

@@ -15,6 +15,7 @@ const getPeer = require('./get-peer')
 const { validate: validateConfig } = require('./config')
 const { codes } = require('./errors')
 
+const AddressManager = require('./address-manager')
 const ConnectionManager = require('./connection-manager')
 const Circuit = require('./circuit')
 const Dialer = require('./dialer')
@@ -48,6 +49,7 @@ class Libp2p extends EventEmitter {
 
     // Addresses {listen, announce, noAnnounce}
     this.addresses = this._options.addresses
+    this.addressManager = new AddressManager(this._options.addresses)
 
     this._modules = this._options.modules
     this._config = this._options.config
@@ -123,10 +125,7 @@ class Libp2p extends EventEmitter {
 
       // Add the identify service since we can multiplex
       this.identifyService = new IdentifyService({
-        peerStore: this.peerStore,
-        connectionManager: this.connectionManager,
-        peerId: this.peerId,
-        addresses: this.addresses,
+        libp2p: this,
         protocols: this.upgrader.protocols
       })
       this.handle(Object.values(IDENTIFY_PROTOCOLS), this.identifyService.handleMessage)
@@ -190,6 +189,8 @@ class Libp2p extends EventEmitter {
    */
   async start () {
     log('libp2p is starting')
+    // TODO: consider validate listen addresses on start?
+    // depend on transports?
     try {
       await this._onStarting()
       await this._onDidStart()
@@ -295,6 +296,54 @@ class Libp2p extends EventEmitter {
     return connection
   }
 
+  /**
+   * Get peer advertising multiaddrs by concating the addresses used
+   * by transports to listen with the announce addresses.
+   * Duplicated addresses and noAnnounce addresses are filtered out.
+   * This takes into account random ports on matching noAnnounce addresses.
+   * @return {Array<Multiaddr>}
+   */
+  getAdvertisingMultiaddrs () {
+    // Filter noAnnounce multiaddrs
+    const filterMa = this.addressManager.getNoAnnounceMultiaddrs()
+
+    // Special filter for noAnnounce addresses using a random port
+    // eg /ip4/0.0.0.0/tcp/0 => /ip4/192.168.1.0/tcp/58751
+    const filterSpecial = filterMa
+      .map((ma) => ({
+        protos: ma.protos(),
+        ...ma.toOptions()
+      }))
+      .filter((op) => op.port === 0)
+
+    // Create advertising list
+    return this.transportManager.getAddrs()
+      .concat(this.addressManager.getAnnounceMultiaddrs())
+      .filter((ma, index, array) => {
+        // Filter out if repeated
+        if (array.findIndex((otherMa) => otherMa.equals(ma)) !== index) {
+          return false
+        }
+
+        // Filter out if in noAnnounceMultiaddrs
+        if (filterMa.find((fm) => fm.equals(ma))) {
+          return false
+        }
+
+        // Filter out if in the special filter
+        const options = ma.toOptions()
+        if (filterSpecial.find((op) =>
+          op.family === options.family &&
+          op.host === options.host &&
+          op.transport === options.transport &&
+          op.protos.length === ma.protos().length
+        )) {
+          return false
+        }
+        return true
+      })
+  }
+
   /**
    * Disconnects all connections to the given `peer`
    * @param {PeerId|multiaddr|string} peer the peer to close connections to
@@ -362,14 +411,8 @@ class Libp2p extends EventEmitter {
   }
 
   async _onStarting () {
-    // Listen on the addresses provided
-    const multiaddrs = this.addresses.listen
-
-    await this.transportManager.listen(multiaddrs)
-
-    // The addresses may change once the listener starts
-    // eg /ip4/0.0.0.0/tcp/0 => /ip4/192.168.1.0/tcp/58751
-    this.addresses.listen = this.transportManager.getAddrs()
+    // Listen on the provided transports
+    await this.transportManager.listen()
 
     if (this._config.pubsub.enabled) {
       this.pubsub && this.pubsub.start()
@@ -475,7 +518,6 @@ class Libp2p extends EventEmitter {
         if (typeof DiscoveryService === 'function') {
           discoveryService = new DiscoveryService(Object.assign({}, config, {
             peerId: this.peerId,
-            multiaddrs: this.addresses.listen,
             libp2p: this
           }))
         } else {

src/transport-manager.js

@@ -127,11 +127,13 @@ class TransportManager {
   }
 
   /**
-   * Starts listeners for each given Multiaddr.
+   * Starts listeners for each listen Multiaddr.
+   * Update listen multiaddrs of the Address Manager after the operation.
    * @async
-   * @param {Multiaddr[]} addrs
    */
-  async listen (addrs) {
+  async listen () {
+    const addrs = this.libp2p.addressManager.getListenMultiaddrs()
+
     if (addrs.length === 0) {
       log('no addresses were provided for listening, this node is dial only')
       return