251 Commits

Author SHA1 Message Date
Adin Schmahmann
fcbc5f9e7f
Asynchronous lookups (#498)
* feat(query): fully async implementation of Kademlia lookup. peers returned from the lookup are not guaranteed to be alive (i.e. we're only guaranteed to have dialed the closest beta peers to the target), but given stable and correct routing tables the expectation that most of the peers returned are alive is high.
* feat(query): add wrapper lookup followup function to followup after the lookup is completed and ensure that the closest k returned peers from a lookup have been queried even for beta < k
* refactor(query) modified the structure returned from lookups to be a useful subset of the full query state instead of the entire query state
* feat(options): beta parameter exposed as the Resiliency parameter
* feat(routing): do not mark the routing table as updated after a FindPeer query
* feat(routing): FindPeer can return addresses even if not Connected as long as it was either recently connected (CanConnect) or was discovered during the lookup
* feat(bootstrap): bootstrap logic now uses GetClosestPeers instead of FindPeer
* refactor(dht): stopFn no longer takes any state
* fix(test): changed GetClosestPeers test to only assume beta instead of k peers since that is now more appropriate given the query logic changes and that the routing tables in that test are bad, i.e. a ring network with arbitrary peerIDs

Co-authored-by: Petar Maymounkov <petarm@gmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
2020-03-24 11:17:48 -04:00
Adin Schmahmann
8eade909a7 feat: calling FindProvidersAsync with a count of zero now completes the query 2020-03-23 01:59:57 -04:00
Adin Schmahmann
147dc1244d cleanup: documentation and code cleanup for search/getvalue quorum defaulting to zero 2020-03-23 01:59:57 -04:00
Aarsh Shah
e42ee94214 Cypress Disjoint Query & Query Termination code cleanup (#489)
* kpeerset refactoring
* query code cleanup
2020-03-13 15:04:25 -07:00
Steven Allen
376300a6ff
fix: actually ignore deadline exceeded errors in Provide (#480) 2020-03-06 00:00:51 -05:00
Steven Allen
e50572360e fix: improve context deadline handling
1. Continue to best-effort provide, but still return an error when we fail to
send provider records to the _best_ peers.
2. Continue returning the best peer's we've found in GetClosestPeers, but also
return an error to indicate that we didn't find the closest ones.

And fix the hang test.
2020-03-05 20:01:39 -08:00
Adin Schmahmann
8ef9d774c9 cleanup unused code 2020-03-04 22:11:53 -08:00
Adin Schmahmann
6b25a6c008 try another peer sorting metric. fix bug in when findproviders returns 2020-03-04 22:11:53 -08:00
Adin Schmahmann
1984feb89e added todo comment about FindPeer and peer connectedness 2020-03-04 22:11:53 -08:00
Adin Schmahmann
c79bac3a7a SearchValue will only update k closest peers to have latest record 2020-03-04 22:11:53 -08:00
Adin Schmahmann
719a16ff88 error on queries to empty routing table 2020-03-04 22:11:53 -08:00
Adin Schmahmann
a9e21d2672 Search/GetValues stops if using quorum 2020-03-04 22:11:53 -08:00
Adin Schmahmann
5cabdf6d13 fix refreshing buckets during query. getting topK peers is more efficient. 2020-03-04 22:11:53 -08:00
Adin Schmahmann
68b116932f fix Search/GetValue to be Kad compliant. Default quorum is now 0 which means do not abort the query early 2020-03-04 22:11:53 -08:00
Adin Schmahmann
c4e94ce0fe first pass at proper kad 2020-03-04 22:11:53 -08:00
Adin Schmahmann
5d2e3df37d make all key types loggable 2019-12-19 12:04:25 -05:00
Adin Schmahmann
3e24f352aa provider record keys can be an arbitrary byte array less than 80 bytes instead of only a multihash 2019-12-19 12:00:51 -05:00
Adin Schmahmann
a4b38eebee feat(dht): provider records use multihashes instead of CIDs 2019-12-19 12:00:51 -05:00
Aarsh Shah
ef6ffec073 refresh cpl in dht 2019-12-17 01:25:57 +08:00
Steven Allen
ba86f51884 fix: return a closed channel from FindProvidersAsync when providers are disabled. 2019-12-06 09:27:09 -05:00
Steven Allen
52747fc1f0 feat: allow disabling value and provider storage/messages
fixes #274
2019-12-05 19:11:46 -05:00
Aarsh Shah
5da16344fe refresh bucket only when query is successful 2019-10-11 13:14:07 +09:00
aarshkshah1992
585d6725be refresh a bucket whenever we lookup a key in it 2019-10-11 13:13:53 +09:00
Raúl Kripalani
fed99afe6e
Make routing table bucket size configurable (#396) 2019-10-04 22:12:00 +09:00
Thomas Preindl
8a40bef78c fix deadline calculation in Provide function of routing.go (#377) 2019-07-23 15:19:50 -07:00
Steven Allen
49ccd212f2 feat: make sure to leave time to send provider records 2019-06-29 13:06:08 +02:00
Raúl Kripalani
31765355df
migrate to consolidated types. (#344) 2019-05-26 23:33:15 +01:00
Steven Allen
1cccee0f65 query: fix a goroutine leak when the routing table is empty
When the routing table is empty, `Run` would fail but _not_ close the
process (leaking some query goroutines). This patch fixes this in multiple
places by:

1. Not starting queries with no peers.
2. Failing queries with no peers earlier.
2019-04-26 12:55:45 -07:00
Raúl Kripalani
120a585ebf Revert "Test go mod in travis and use major versioning in import paths (#236)"
This reverts commit 7e68ac3c60c15243c7d2e622b91aed92245a2677.
2019-02-12 14:12:53 +00:00
Adrian Lanzafame
7e68ac3c60 Test go mod in travis and use major versioning in import paths (#236) 2019-02-12 17:38:06 +11:00
Matt Joiner
8b37385468 Rename log to logger 2019-02-03 13:14:35 +11:00
Steven Allen
201eea5e0b
Revert "Tidy up bootstrapping" 2019-01-24 14:06:38 -08:00
Matt Joiner
f423e38edd Simplify the bootstrap logic 2019-01-24 09:52:56 +11:00
Kevin Atkinson
a303c45371 gx update and fix code to use new Cid type 2018-09-11 21:54:47 -04:00
Steven Allen
f5423a6a72 don't double-validate values
Also, de-duplicate some logic.
2018-08-10 13:57:45 -07:00
Steven Allen
cd3cac0194 go fmt 2018-08-10 13:51:01 -07:00
Łukasz Magiera
246579e78b SearchValue: respect ctx on output 2018-08-10 12:13:17 +02:00
Łukasz Magiera
6ca5dd7bf4 SearchValues: more review addressing 2018-08-10 12:06:32 +02:00
Łukasz Magiera
d72432caf1 SearchValue: address review 2018-08-10 12:06:32 +02:00
Łukasz Magiera
51c475d117 SearchValue: make default quorum infinite for search 2018-08-10 12:06:32 +02:00
Łukasz Magiera
307660c78d SearchValue: Make -1 quorum mean infinite on get 2018-08-10 12:06:32 +02:00
Łukasz Magiera
e28b133f80 SearchValue: add backward-compatible GetValues 2018-08-10 12:06:32 +02:00
Łukasz Magiera
547c0a7170 SearchValue: simplify error handling further 2018-08-10 12:06:32 +02:00
Łukasz Magiera
64d1622c72 SearchValue: simplify error handling 2018-08-10 12:06:32 +02:00
Łukasz Magiera
e260fe510f Implement SearchValue 2018-08-10 12:06:32 +02:00
Steven Allen
4d76fd28ed upgrade protobuf and switch to bytes keys
fixes #177
2018-08-07 18:50:04 -07:00
Steven Allen
993399a183 don't treat a canceled context as a successful query
fixes #172
2018-07-10 17:29:02 +02:00
Steven Allen
97131ef4c2 remove random print from test 2018-06-15 14:13:07 -07:00
Steven Allen
ed35a2d66c return ErrNotFound when offline routing is requested and we have no local value
err is nil in this case
2018-06-13 19:20:00 -07:00
Steven Allen
6c4aa35c94 validate and compare record on local put 2018-06-13 17:01:53 -07:00