78 Commits

Author SHA1 Message Date
Jeromy
00b46e0cf7 Fixes for dht findpeer queries
First, we use Alpha instead of K as the number of peers we grab from the
routing table (as per the kademlia paper).

Second, we don't use a size limited set for the 'GetClosestPeers' query.
We're going to process more than K peers before we find the K closest
peers.

Third, Change GetClosestPeers to actually return the K Closest peers,
not a hodge podge of peers that it found on the way to finding the
closest peers.
2017-03-05 21:36:46 -08:00
Jeromy
6894f33507 update dependencies after package extraction funtime 2016-10-05 12:34:28 -07:00
Jeromy
e00b3ca0a4 use stdlib context and clean up deps 2016-09-30 10:24:03 -07:00
Jeromy
c372d79e42 switch to strings and cids instead of keys 2016-09-30 10:13:57 -07:00
George Antoniadis
30d43d22e0 Fix dependencies 2016-09-02 20:21:23 +01:00
George Antoniadis
5f3e63aa46 Undo gx rewrites and add gx package.json 2016-08-21 17:18:58 +01:00
Jakub Sztandera
22af8d1151 Update go-log in whole dependency tree (#2898)
* Update golog in go-ipfs

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-secio for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-crypto for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-peer for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Import peersore, it wasn't imported

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peerstore

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peer

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update secio

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 09:38:07 -07:00
Jeromy
63b6db0589 pull in libp2p updates with utp fixes
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-06-11 10:33:44 -07:00
Jakub Sztandera
2b3334c4a4 Update go-log
https://github.com/ipfs/go-log/pull/3

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-09 23:37:09 +02:00
Jeromy
5f777f323b update libp2p to v3.3.1
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-06-01 16:45:46 -07:00
Jeromy
26beb1f3d0 update go-libp2p 3.2.2, nil maddr fixes
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-05-17 10:23:10 -07:00
Jeromy
b9f86c8c87 update libp2p with go-multiaddr and go-stream-muxer updates
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-05-10 16:06:28 -07:00
Hector Sanjuan
0a60fad1cd Update go-log to 1.1.0 and fix calls to go-log.Uuid
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2016-05-04 23:17:21 +02:00
Lars Gierth
38ff911ada Use extracted go-libp2p-crypto, -secio, -peer packages
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
2016-04-16 21:48:06 -07:00
Lars Gierth
aa46264d8a Update go-libp2p
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
2016-04-16 21:23:47 -07:00
Jeromy
75844fe336 update libp2p dep to fix hanging listeners problem
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-04-11 12:52:54 -07:00
Jeromy
3b906cf41c switch to new libp2p with mss crypto
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-04-07 15:55:41 -07:00
Jeromy
81971d0a2b update utp and cleanup more godeps along the way
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-03-29 19:18:14 -07:00
Jeromy
17edbc0725 update libp2p dep
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-03-09 09:53:19 -08:00
Jeromy
98ab3675cc Use gx vendored go-ipfs-utils where possible
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-12 17:21:40 -08:00
Jeromy
e87bf15e58 remove goprocess from godeps, use gx vendored one
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-12 17:21:40 -08:00
Jeromy
77f09ba180 do that last thing again
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-31 15:37:39 -08:00
Jeromy
32f70d517b update libp2p dep
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-31 10:19:50 -08:00
Jeromy
b9e978714f correct go-log dep
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:10 -08:00
Jeromy
7467be3711 go-keyspace dep from libp2p added
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:08 -08:00
Jeromy
a5e5c57f3a initial vendoring of libp2p outside of the repo with gx
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:06 -08:00
Jeromy
bc9ceab87e drop error log down to debug
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-11-09 23:18:38 -08:00
Juan Benet
440f66f1c8 Merge pull request #1943 from ipfs/dht/speed
fix swarm dial backoff
2015-11-10 06:41:54 +00:00
Jeromy
39d1e2146d Add in some more notifications to help profile queries
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-11-08 13:09:36 -08:00
Jeromy
f125d741ec vendor logging lib update
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-11-05 15:57:21 -08:00
Jeromy
675ed09ad8 update code to use new logging changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-10-27 11:04:56 -07:00
Jeromy
596864e1a3 replace imports with absolute path instead of using symlink
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-10-03 14:30:50 -07:00
Jeromy
badc73888c Fix dht queries
Queries previously would sometimes only query three (alpha value) peers
before halting the operation. This PR changes the number of peers
grabbed from the routing table to start a query to K.

Dht nodes would also not respond with enough peers, as per the kademlia
paper, this has been changed to from 4 to 'K'.

The query mechanism itself also was flawed in that it would pull all the
peers it had yet to query out of the queue and 'start' the query for
them. The concurrency rate limiting was done inside the 'queryPeer'
method after the goroutine was spawned. This did not allow for peers
receiver from query replies to be properly queried in order of distance.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-29 09:47:47 -07:00
Jeromy
892da9236f extract logging
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-14 17:35:50 -07:00
Jeromy
ea4a944533 use new methods from goprocess/context, remove thirdparty/waitable
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-08 21:37:40 -07:00
rht
4124663871 Make sure process context is set last
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2015-07-05 09:35:36 +07:00
Jeromy
a84b8b7bed move util.Key into its own package under blocks 2015-06-01 16:10:08 -07:00
Ho-Sheng Hsiao
0f9082aa6c Reorged imports from jbenet/go-ipfs to ipfs/go-ipfs
- Modified Godeps/Godeps.json by hand
- [TEST] Updated welcome docs hash to sharness
- [TEST] Updated contact doc
- [TEST] disabled breaking test (t0080-repo refs local)
2015-03-31 12:52:25 -07:00
Juan Batiz-Benet
be9c9b8d78 query: fix race condition: redux
this time just move to goprocess
2015-03-07 04:08:26 -08:00
Juan Batiz-Benet
c65a96f38f query: fixed race condition 2015-03-07 02:44:20 -08:00
Henry
9bc5b3d5d3 rewrote import paths of go.net/context to use golang.org/x/context
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go

- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo

- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)

Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.

Andrew Gerrand (1):
go.net: use golang.org/x/... import paths

Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.

Damien Neil (1):
context: fix removal of cancelled timer contexts from parent

David Symonds (2):
context: Fix WithValue example code.
net: add import comments.

Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
2015-02-25 11:58:19 +01:00
Juan Batiz-Benet
a293b33a2d dht/query: make sure to cancel all contexts.
We are leaking peer queues:

http://gateway.ipfs.io/ipfs/QmQxVA48CzVwwNYExUiFe56VrUBn8u368ZfchnCLoc7fSC/moriarty
2015-02-03 12:19:17 -08:00
Juan Batiz-Benet
dacbe3cf89 AddrManager: use addr manager with smarter TTLs
This addr manager should seriously help with the addrsplosion
problem.
2015-02-02 21:04:14 -08:00
Brian Tiger Chow
4e85858067 log(dht): remove lots of query debug logs
the debug log is flooded with pages upon pages of...

we've gotta be more judicious with our use of console logs. i'm sure
there's interesting actionable information in here. let's use the
console logging more like a sniper rifle and less like birdshot.

feel free to revert if there are specific critical statements in this
changeset

03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) QUERY worker for: <peer.ID QmSoLp> - not
found, and no closer peers. prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) completed prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) finished prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) 0 provider entries prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) 0 provider entries decoded prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) got closer peers: 0 [] prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) end prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) query finished prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) QUERY worker for: <peer.ID QmSoLn> - not
found, and no closer peers. prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) completed prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) finished prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
all peers ended prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
spawnWorkers end prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
failure: %s routing: not found prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
end prefixlog.go:107
2015-01-30 06:57:41 -08:00
Brian Tiger Chow
7429f19731 remove prefix logger 2015-01-24 00:30:56 -08:00
Jeromy
9c0fe1300b implement dht findprovs and add error output to dht query 2015-01-24 03:56:46 +00:00
Juan Batiz-Benet
356dcff831 dont rate limit query during dials 2015-01-24 03:56:46 +00:00
Juan Batiz-Benet
2322f6fe41 dht/query: err return NotFound case
When some queries finished, but we got no result, it should
be a simple NotFoundError. Only when every single query ended
in error do we externalize those to the client, in case
something major is going wrong
2015-01-23 02:08:29 -08:00
Brian Tiger Chow
9cc5c86e2a move generic packages to thirdparty (see thirdparty/README.md) 2015-01-18 14:04:45 -08:00
Juan Batiz-Benet
38c3e88e61 addr-explosion mitigated adding
mitigated adding our own addresses where received
from peers

see #573
2015-01-16 11:38:46 -08:00