36 Commits

Author SHA1 Message Date
Juan Batiz-Benet
dacbe3cf89 AddrManager: use addr manager with smarter TTLs
This addr manager should seriously help with the addrsplosion
problem.
2015-02-02 21:04:14 -08:00
Brian Tiger Chow
4e85858067 log(dht): remove lots of query debug logs
the debug log is flooded with pages upon pages of...

we've gotta be more judicious with our use of console logs. i'm sure
there's interesting actionable information in here. let's use the
console logging more like a sniper rifle and less like birdshot.

feel free to revert if there are specific critical statements in this
changeset

03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) QUERY worker for: <peer.ID QmSoLp> - not
found, and no closer peers. prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) completed prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) finished prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) 0 provider entries prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) 0 provider entries decoded prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) got closer peers: 0 [] prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) end prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) query finished prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) QUERY worker for: <peer.ID QmSoLn> - not
found, and no closer peers. prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) completed prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) finished prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
all peers ended prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
spawnWorkers end prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
failure: %s routing: not found prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
end prefixlog.go:107
2015-01-30 06:57:41 -08:00
Brian Tiger Chow
7429f19731 remove prefix logger 2015-01-24 00:30:56 -08:00
Jeromy
9c0fe1300b implement dht findprovs and add error output to dht query 2015-01-24 03:56:46 +00:00
Juan Batiz-Benet
356dcff831 dont rate limit query during dials 2015-01-24 03:56:46 +00:00
Juan Batiz-Benet
2322f6fe41 dht/query: err return NotFound case
When some queries finished, but we got no result, it should
be a simple NotFoundError. Only when every single query ended
in error do we externalize those to the client, in case
something major is going wrong
2015-01-23 02:08:29 -08:00
Brian Tiger Chow
9cc5c86e2a move generic packages to thirdparty (see thirdparty/README.md) 2015-01-18 14:04:45 -08:00
Juan Batiz-Benet
38c3e88e61 addr-explosion mitigated adding
mitigated adding our own addresses where received
from peers

see #573
2015-01-16 11:38:46 -08:00
Juan Batiz-Benet
92c24c08ae dht: even more logging. 2015-01-05 04:55:02 -08:00
Juan Batiz-Benet
719cd7c51c swap net2 -> net 2015-01-02 08:46:46 -08:00
Juan Batiz-Benet
6bc26e01f3 net -> p2p/net
The net package is the next to move. It will be massaged
a bit still to fix the Network / "NetworkBackend" conflict.
2015-01-02 08:46:45 -08:00
Juan Batiz-Benet
2b086634b2 introducing p2p pkg
I think it's time to move a lot of the peer-to-peer networking
but-not-ipfs-specific things into its own package: p2p.
This could in the future be split off into its own library.
The first thing to go is the peer.
2015-01-02 08:46:45 -08:00
Jeromy
0547622e00 rewrite of provides to better select peers to send RPCs to
refactor test peer creation to be deterministic and reliable

a bit of cleanup trying to figure out TestGetFailure

add test to verify deterministic peer creation

switch put RPC over to use getClosestPeers

rm 0xDEADC0DE

fix queries not searching peer if its not actually closer
2015-01-02 07:42:07 +00:00
Juan Batiz-Benet
8fea5bcb58 dht/query: log when dialing a closerpeer 2014-12-24 03:45:20 -08:00
Juan Batiz-Benet
6d76c57c64 dht: removing extra newlines 2014-12-24 03:42:43 -08:00
Juan Batiz-Benet
1f317e756e dht/query: fix important panic
Withe queries (particularly providers), it was possible to
exit the query runner's Run BEFORE all its children were done,
because the runner itself only listened to the context. This
introduced the possibility of a panic (you can go check it
out by running the TestProvidersMany test on dht_test in commits
before this one).  Thankfully, ctxgroup saved the day with
almost _zero_ changes to the sync flow, and now we have the
guarantee that the query runner will only exit if all its
children are done. ❤️

Conflicts:
	routing/dht/query.go
2014-12-24 02:13:38 -08:00
Juan Batiz-Benet
ed7ef491e6 dht: helpful debugging for no closer peers 2014-12-23 09:03:26 -08:00
Juan Batiz-Benet
3b37c43171 peer change: peer.Peer -> peer.ID
this is a major refactor of the entire codebase
it changes the monolithic peer.Peer into using
a peer.ID and a peer.Peerstore.

Other changes:
- removed handshake3.
-	testutil vastly simplified peer
-	secio bugfix + debugging logs
-	testutil: RandKeyPair
-	backpressure bugfix: w.o.w.
-	peer: added hex enc/dec
-	peer: added a PeerInfo struct
  PeerInfo is a small struct used to pass around a peer with
 	a set of addresses and keys. This is not meant to be a
 	complete view of the system, but rather to model updates to
 	the peerstore. It is used by things like the routing system.
-	updated peer/queue + peerset
-	latency metrics
-	testutil: use crand for PeerID gen
 	RandPeerID generates random "valid" peer IDs. it does not
 	NEED to generate keys because it is as if we lost the key
 	right away. fine to read some randomness and hash it. to
 	generate proper keys and an ID, use:
 	  sk, pk, _ := testutil.RandKeyPair()
 	  id, _ := peer.IDFromPublicKey(pk)
 	Also added RandPeerIDFatal helper
- removed old spipe
- updated seccat
- core: cleanup initIdentity
- removed old getFromPeerList
2014-12-23 08:33:32 -08:00
Juan Batiz-Benet
3dd6ee1f86 dht: update to use net.LocalPeer 2014-12-08 20:52:45 -08:00
Brian Tiger Chow
077a34c247 fix(net) pass contexts to dial peer 2014-11-05 10:04:20 -08:00
Jeromy
c76d7467bb remove dagwriter in favor of new importer function 2014-10-31 05:03:42 +00:00
Brian Tiger Chow
a211f1ecdb fix(all) log.Debug -> log.Debugf 2014-10-30 06:35:29 -07:00
Brian Tiger Chow
e3d7d2b50c refactor(routing) use routing.ErrNotFound 2014-10-28 02:17:46 -07:00
Juan Batiz-Benet
5582e282d4 Dialer for dht
dht doesn't need the whole network interface, only needs a Dialer.
(much reduced surface of possible errors)
2014-10-22 03:24:05 -07:00
Juan Batiz-Benet
840a868c6f notes 2014-10-22 03:24:05 -07:00
Juan Batiz-Benet
5a2dc1a9f5 query wasnt ensuring conn
The query-- once it's actually attempting to connect to a peer--
should be the one connecting.
2014-10-22 03:24:05 -07:00
Juan Batiz-Benet
28e083b902 peer.Peer is now an interface
![](http://m.memegen.com/77n7dk.jpg)
2014-10-20 03:26:46 -07:00
Juan Batiz-Benet
740b381804 changed logging, in dht and elsewhere
- use log.* instead of u.*
- use automatic type conversions to .String()
  (Peer.String() prints nicely, and avoids calling b58 encoding
  until needed)
2014-10-07 21:33:59 -07:00
Jeromy
9bf956a8cf implement initial ipns filesystem interface as well as plumbing command for publishing 2014-10-01 00:39:57 -07:00
Jeromy
9dc86ca5cb add basic publish command, needs polish 2014-10-01 00:39:57 -07:00
Juan Batiz-Benet
34e171e2b3 fixed get/put 2014-09-22 04:06:12 -07:00
Juan Batiz-Benet
709f932e6e Fixed connections all over. 2014-09-22 04:06:12 -07:00
Juan Batiz-Benet
ffa167674c use Alpha as the concurrency.
cc @whyrusleeping
2014-09-22 04:06:12 -07:00
Juan Batiz-Benet
306285a1ac added some logging 2014-09-22 04:05:23 -07:00
Juan Batiz-Benet
b5f0a42bd6 better query processing (runner) 2014-09-22 04:05:22 -07:00
Juan Batiz-Benet
fdea4d9c5f got everything to build 2014-09-22 04:05:21 -07:00