23 Commits

Author SHA1 Message Date
Steven Allen
d4818f1a9d fix(dialqueue): fix a timer leak 2020-02-27 17:26:09 -08:00
Raúl Kripalani
31765355df
migrate to consolidated types. (#344) 2019-05-26 23:33:15 +01:00
Steven Allen
29d5b4af70 dialQueue: start the control loop later
Don't start the control loop till we start the queue. The _in_ queue will still
work, we just won't start processing peers from it until we start the control loop.

fixes #311
2019-03-27 09:49:47 +00:00
Matt Joiner
86d78dc072
golangci-lint run -D errcheck (#298) 2019-03-14 11:07:15 +11:00
Raúl Kripalani
bd60c95d07 replace atomics with sync.Once. 2019-03-13 21:39:16 +00:00
Raúl Kripalani
b649bcbec6 defer dialqueue action until initial peers have been added. 2019-03-13 21:25:36 +00:00
Steven Allen
c78d1e6786 fix(dialQueue): account for failed dials
fixes #276
2019-02-26 19:57:54 -07:00
Raúl Kripalani
4f0cf486fa dialqueue: rename local var for clarity. 2019-02-19 18:12:17 +00:00
Raúl Kripalani
8bacc6cdd9 dialqueue: fix possible worker goroutine leak. 2019-02-19 18:10:01 +00:00
Matt Joiner
8b37385468 Rename log to logger 2019-02-03 13:14:35 +11:00
Raúl Kripalani
94682b4df2 fix a race in dial queue. 2019-02-01 19:20:32 +00:00
Raúl Kripalani
ebcfcd46a6 make dial queue parameters configurable. 2019-02-01 13:43:27 +11:00
Matt Joiner
7246a3b0f4 Fix races with DialQueue variables 2019-01-31 13:08:45 +11:00
Raúl Kripalani
74d22f3f5e park waiters in slice; revise closure logic. 2019-01-30 23:25:10 +00:00
Raúl Kripalani
f2df3ec540 fix shutdown logic; fix timer logic. 2019-01-29 20:49:04 +00:00
Raúl Kripalani
72f9d4c2c0 add godoc on global vars. 2019-01-29 20:48:38 +00:00
Raúl Kripalani
83660414ce cleanup channels on context cancellation. 2019-01-29 17:11:40 +00:00
Raúl Kripalani
abacfe5fc9 refactor interface of Consume(). 2019-01-29 16:47:42 +00:00
Raúl Kripalani
7cd14f5d1e replace concurrency level error with panic. 2019-01-29 16:34:39 +00:00
Raúl Kripalani
24f0a2dec8 remove unneeded lk; group global vars. 2019-01-29 16:15:40 +00:00
Raúl Kripalani
95a0975dd7 enhance logging and import prefixes. 2019-01-29 00:56:12 +00:00
Raúl Kripalani
f044043c3d adjust godoc. 2019-01-29 00:56:12 +00:00
Raúl Kripalani
5e74c4a6f2 introduce adaptive queue for DHT dials.
This patch introduces an adaptive dial queue that spawns a dynamically sized
set of goroutines to preemptively stage dials for later handoff to the DHT
protocol for RPC. It identifies backpressure on both ends (dial consumers and
dial producers), and takes compensating action by adjusting the worker pool.

We start with `DialQueueMinParallelism` number of workers (6), and scale up
and down based on demand and supply of dialled peers.

The following events trigger scaling:
- we scale up when we can't immediately return a successful dial to a new
  consumer.
- we scale down when we've been idle for a while waiting for new dial
  attempts.
- we scale down when we complete a dial and realise nobody was waiting for it.

Dialler throttling (e.g. FD limit exceeded) is a concern, as we can easily
spin up more workers to compensate, and end up adding fuel to the fire. Since
we have no deterministic way to detect this for now, we hard-limit concurrency
to `DialQueueMaxParallelism` (20).
2019-01-29 00:56:12 +00:00