94 Commits

Author SHA1 Message Date
yutianwu
fcce9ed4db fix lint 2019-05-21 14:23:50 +08:00
yutianwu
c21b4fcc93 fix bug of proxy 2019-05-21 14:12:40 +08:00
Anton Kaliaev
a7358bc69f
libs/db: conditional compilation (#3628)
* libs/db: conditional compilation

For cleveldb: go build -tags cleveldb
For boltdb: go build -tags boltdb

Fixes #3611

* document db_backend param better

* remove deprecated LevelDBBackend

* update changelog

* add missing lines

* add new line

* fix TestRemoteDB

* add a line about boltdb tag

* Revert "remove deprecated LevelDBBackend"

This reverts commit 1aa85453f76605e0c4d967601bbe26240e668d51.

* make PR non breaking

* change DEPRECATED label format

https://stackoverflow.com/a/36360323/820520
2019-05-07 12:33:47 +04:00
Ismail Khoffi
1e3469789d Ensure WriteTimeout > TimeoutBroadcastTxCommit (#3443)
* Make sure config.TimeoutBroadcastTxCommit < rpcserver.WriteTimeout()

* remove redundant comment

* libs/rpc/http_server: move Read/WriteTimeout into Config

* increase defaults for read/write timeouts

Based on this article
https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration

* WriteTimeout should be larger than TimeoutBroadcastTxCommit

* set a deadline for subscribing to txs

* extract duration into const

* add two changelog entries

* Update CHANGELOG_PENDING.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* Update CHANGELOG_PENDING.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* 12 -> 10

* changelog

* changelog
2019-03-19 19:45:51 -04:00
Anton Kaliaev
d741c7b478
limit number of /subscribe clients and queries per client (#3269)
* limit number of /subscribe clients and queries per client

Add the following config variables (under [rpc] section):
  * max_subscription_clients
  * max_subscriptions_per_client
  * timeout_broadcast_tx_commit

Fixes #2826

new HTTPClient interface for subscriptions

finalize HTTPClient events interface

remove EventSubscriber

fix data race

```
WARNING: DATA RACE
Read at 0x00c000a36060 by goroutine 129:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0

Previous write at 0x00c000a36060 by goroutine 132:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 129 (running) created at:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 132 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
```

lite client works (tested manually)

godoc comments

httpclient: do not close the out channel

use TimeoutBroadcastTxCommit

no timeout for unsubscribe

but 1s Local (5s HTTP) timeout for resubscribe

format code

change Subscribe#out cap to 1

and replace config vars with RPCConfig

TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout

rpc: Context as first parameter to all functions

reformat code

fixes after my own review

fixes after Ethan's review

add test stubs

fix config.toml

* fixes after manual testing

- rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes
Tendermint resources (pubsub)
- rpc: better error in Subscribe and BroadcastTxCommit
- HTTPClient: do not resubscribe if err = ErrAlreadySubscribed

* fixes after Ismail's review

* Update rpc/grpc/grpc_test.go

Co-Authored-By: melekes <anton.kalyaev@gmail.com>
2019-03-11 22:45:58 +04:00
Yumin Xia
b021f1e505 libs/db: close batch (#3397)
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842
2019-03-10 12:46:32 +04:00
Anton Kaliaev
b6a510a3e7
make ineffassign linter pass (#3386)
Refs #3262

This fixes two small bugs:

1) lite/dbprovider: return `ok` instead of true in parse* functions. It's weird that we're ignoring `ok` value before.
2) consensus/state: previously because of the shadowing we almost never output "Error with msg". Now we declare both `added` and `err` in the beginning of the function, so there's no shadowing.
2019-03-08 09:46:09 +04:00
zjubfd
3421e4dcd7 make lightd availbe (#3364)
1."abci_query": rpcserver.NewRPCFunc(c.ABCIQuery, "path,data,prove")
"validators": rpcserver.NewRPCFunc(c.Validators, "height"),
the parameters and function do not match, cause index out of range error.
2. the prove of query is forced to be true, while default option is false.
3. fix the wrong key of merkle
2019-03-02 16:36:52 -05:00
Anton Kaliaev
8283ca7ddb
3291 follow-up (#3323)
* changelog: use issue number instead of PR number

* follow up to #3291

- rpc/test/helpers.go add StopTendermint(node) func
- remove ensureDir(filepath.Dir(walFile), 0700)
- mempool/mempool_test.go add type cleanupFunc func()

* cmd/show_validator: wrap err to make it more clear
2019-02-18 13:23:40 +04:00
Alessio Treglia
59cc6d36c9 improve ResetTestRootWithChainID() concurrency safety (#3291)
* improve ResetTestRootWithChainID() concurrency safety

Rely on ioutil.TempDir() to create test root directories and ensure
multiple same-chain id test cases can run in parallel.

* Update config/toml.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* clean up test directories after completion

Closes: #1034

* Remove redundant EnsureDir call

* s/PanicSafety()/panic()/s

* Put create dir functionality back in ResetTestRootWithChainID

* Place test directories in OS's tempdir

In modern UNIX and UNIX-like systems /tmp is very often
mounted as tmpfs. This might speed test execution a bit.

* Set 0700 to a const

* rootsDirs -> configRootDirs

* Don't double remove directories

* Avoid global variables

* Fix consensus tests

* Reduce defer stack

* Address review comments

* Try to fix tests

* Update CHANGELOG_PENDING.md

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* Update consensus/common_test.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* Update consensus/common_test.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>
2019-02-18 11:45:27 +04:00
Ethan Buchman
4f2ef36701
types.NewCommit (#3275)
* types.NewCommit

* use types.NewCommit everywhere

* fix log in unsafe_reset

* memoize height and round in constructor

* notes about deprecating toVote

* bring back memoizeHeightRound
2019-02-08 18:40:41 -05:00
Anton Kaliaev
6941d1bb35
use nolint label instead of commenting 2019-02-06 18:21:32 +04:00
Anton Kaliaev
ffd3bf8448
remove or comment out unused code 2019-02-06 15:16:38 +04:00
Ethan Buchman
1809efa350
Introduce CommitSig alias for Vote in Commit (#3245)
* types: memoize height/round in commit instead of first vote

* types: commit.ValidateBasic in VerifyCommit

* types: new CommitSig alias for Vote

In preparation for reducing the redundancy in Commits, we introduce the
CommitSig as an alias for Vote. This is non-breaking on the protocol,
and minor breaking on the Go API, as Commit now contains a list of
CommitSig instead of Vote.

* remove dependence on ToVote

* update some comments

* fix tests

* fix tests

* fixes from review
2019-02-04 13:01:59 -05:00
Infinytum
fbd1e79465 docs: fix lite client formatting (#3198)
Closes #3180
2019-01-24 12:10:34 +04:00
Ethan Buchman
de5a6010f0
fix DynamicVerifier for large validator set changes (#3171)
* base verifier: bc->bv and check chainid

* improve some comments

* comments in dynamic verifier

* fix comment in doc about BaseVerifier

It requires the validator set to perfectly match.

* failing test for #2862

* move errTooMuchChange to types. fixes #2862

* changelog, comments

* ic -> dv

* update comment, link to issue
2019-01-21 09:21:04 -05:00
Dev Ojha
ec53ce359b Simple merkle rfc compatibility (#2713)
* Begin simple merkle compatibility PR

* Fix query_test

* Use trillian test vectors

* Change the split point per RFC 6962

* update spec

* refactor innerhash to match spec

* Update changelog

* Address @liamsi's comments

* Write the comment requested by @liamsi
2019-01-13 18:02:38 -05:00
Ismail Khoffi
3f987adc92 Set accum of freshly added validator -(total voting power) (#2785)
* set the accum of a new validator to (-total voting power):

- disincentivize validators to unbond, then rebon to reset their
negative Accum to zero

additional unrelated changes:
- do not capitalize error msgs
- fix typo

* review comments: (re)capitalize errors & delete obsolete comments

* More changes suggested by @melekes

* WIP: do not batch clip (#2809)

* substract avgAccum on each iteration

- temporarily skip test

* remove unused method safeMulClip / safeMul

* always substract the avg accum

 - temp. skip another test

* remove overflow / underflow tests & add tests for avgAccum:

- add test for computeAvgAccum
- as we substract the avgAccum now we will not trivially over/underflow

* address @cwgoes' comments

* shift by avg at the end of IncrementAccum

* Add comment to MaxTotalVotingPower

* Guard inputs to not exceed MaxTotalVotingPower

* Address review comments:

 - do not fetch current validator from set again
 - update error message

* Address a few review comments:

 - fix typo
 - extract variable

* address more review comments:

 - clarify 1.125*totalVotingPower == totalVotingPower + (totalVotingPower >> 3)

* review comments: panic instead of "clipping":

 - total voting power is guarded to not exceed MaxTotalVotingPower ->
 panic if this invariant is violated

* fix failing test
2018-11-28 13:12:17 -05:00
Jae Kwon
416d143bf7 R4R: Swap start/end in ReverseIterator (#2913)
* Swap start/end in ReverseIterator

* update CHANGELOG_PENDING

* fixes from review
2018-11-28 08:49:24 -05:00
Alessio Treglia
b646437ec7 Decouple StartHTTP{,AndTLS}Server from Listen() (#2791)
* Decouple StartHTTP{,AndTLS}Server from Listen()

This should help solve cosmos/cosmos-sdk#2715

* Fix small mistake

* Update StartGRPCServer

* s/rpc/rpcserver/

* Start grpccore.StartGRPCServer in a goroutine

* Reinstate l.Close()

* Fix rpc/lib/test/main.go

* Update code comment

* update changelog and comments

* fix tm-monitor. more comments
2018-11-15 15:33:04 -05:00
Ismail Khoffi
6643c5dd11 Catch up with amino 0.13.0 (#2690)
* catch up with amino changes in
https://github.com/tendermint/go-amino/pull/222

* WIP: update to amino v0.13.0

* update to fixed amino release
2018-10-24 21:34:01 -04:00
Ismail Khoffi
8888595b94 [R4R] Fixed sized and reordered fields for Vote/Proposal/Heartbeat SignBytes (#2598)
* WIP: switching to fixed offsets for SignBytes

* add version field to sign bytes and update order

* more comments on test-cases and add a tc with a chainID

* remove amino:"write_empty" tag

- it doesn't affect if default fixed size fields ((u)int64) are
written or not
- add comment about int->int64 casting

* update CHANGELOG_PENDING

* update documentation

* add back link to issue #1622 in documentation

* remove JSON tags and add (failing test-case)

* fix failing test

* update test-vectors due to added `Type` field

* change Type field from string to byte and add new type alias

- SignedMsgType replaces VoteTypePrevote, VoteTypePrecommit and adds new
ProposalType to separate votes from proposal when signed

- update test-vectors

* fix remains from rebasing

* use SignMessageType instead of byte everywhere

* fixes from review
2018-10-12 19:21:46 -04:00
bradyjoestar
5b120d788a lite support maxOpenConnections (#2413) 2018-10-04 20:39:24 -04:00
Zarko Milosevic
12675ecd92 consensus: Wait timeout precommit before starting new round (#2493)
* Disable transitioning to new round upon 2/3+ of Precommit nils

Pull in ensureVote test function from https://github.com/tendermint/tendermint/pull/2132

* Add several ensureX test methods to wrap channel read with timeout

* Revert panic in tests
2018-10-04 09:37:13 -04:00
Ethan Buchman
f36ed7e7ff
General Merkle Follow Up (#2510)
* tmlibs -> libs

* update changelog

* address some comments from review of #2298
2018-09-28 23:32:13 -04:00
Joon
71a34adfe5 General Merkle Proof (#2298)
* first commit

finalize rebase

add protoc_merkle to Makefile

* in progress

* fix kvstore

* fix tests

* remove iavl dependency

* fix tx_test

* fix test_abci_cli

fix test_apps

* fix test_apps

* fix test_cover

* rm rebase residue

* address comment in progress

* finalize rebase
2018-09-28 20:03:19 -04:00
HaoyangLiu
8dda3c3b28 lite: Add synchronization in lite verify (#2396)
* Implement issues 2386: add synchronization in lite verify and change all Certify to Verify

* Replace make(chan struct{}, 0) with make(chan struct{})

* Parameterize memroy cache size and add concurrent test

* Refactor import order
2018-09-28 19:23:21 -04:00
Zarko Milosevic
7b88172f41 Implement BFT time (#2203)
* Implement BFT time

* set LastValidators when creating state in state helper

for heights >= 2
2018-08-31 19:33:51 -04:00
Dev Ojha
2756be5a59 libs: Remove usage of custom Fmt, in favor of fmt.Sprintf (#2199)
* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!
2018-08-10 09:25:57 +04:00
Jae Kwon
e719a93d1d Addressed review for #1815 except those marked as 'TODO make issue' 2018-08-02 03:10:50 -07:00
Ethan Buchman
f6705f02c7 fixes post merge 2018-07-23 23:39:22 -04:00
Ethan Buchman
ea31c4836a Merge branch 'develop' into jae/literefactor4 2018-07-23 23:28:14 -04:00
ValarDragon
c798702764 crypto: Remove Ed25519 and Secp256k1 suffix on GenPrivKey 2018-07-20 10:44:21 -07:00
ValarDragon
17c0029233 Merge remote-tracking branch 'origin/develop' into dev/refactor_crypto 2018-07-20 08:59:41 -07:00
Alexander Simmerl
ed8714e40c
Merge pull request #1965 from tendermint/693-part-2
make Block Header and Data non-pointers
2018-07-20 17:42:42 +02:00
ValarDragon
99e582d79a crypto: Refactor to move files out of the top level directory
Currently the top level directory contains basically all of the code
for the crypto package. This PR moves the crypto code into submodules
in a similar manner to what `golang/x/crypto` does. This improves code
organization.

Ref discussion: https://github.com/tendermint/tendermint/pull/1966

Closes #1956
2018-07-18 08:38:44 -07:00
Dev Ojha
dae7dc30e0 Switch usage of math/rand to cmn's rand (#1980)
This commit switches all usage of math/rand to cmn's rand. The only
exceptions are within the random file itself, the tools package, and the
crypto package. In tools you don't want it to lock between the go-routines.
The crypto package doesn't use it so the crypto package have no other
dependencies within tendermint/tendermint for easier portability.

Crypto/rand usage is unadjusted.

Closes #1343
2018-07-16 11:20:37 +04:00
Anton Kaliaev
270659f03f
make Block Header and Data non-pointers
make BlockMeta Header a non-pointer

Refs #693
2018-07-13 12:05:54 +04:00
Ethan Buchman
9018acde5f tmlibs -> tendermint/libs 2018-07-02 14:58:07 -04:00
Ethan Buchman
5453aa6169 Merge branch 'develop' into jae/literefactor4 2018-07-02 14:57:30 -04:00
Ethan Buchman
d55243f0e6 fix import paths 2018-07-01 22:36:49 -04:00
Jae Kwon
b51ed132f7 Fix test/p2p/pex circle tests; update consensus 2018-06-27 16:24:21 -07:00
Jae Kwon
37ef5485b4 Add logs to lite/*; Fix rpc status to return consensus height, not blockstore height 2018-06-26 16:53:06 -07:00
Jae Kwon
538c410bcd Fixes from review 2018-06-25 18:16:16 -07:00
Jae Kwon
c3296f2e01 Garbage collect DBProvider (unoptimized); Certifier creation takes a client 2018-06-25 17:13:00 -07:00
Jae Kwon
242a6037e8 Fixes from review 2018-06-25 17:12:52 -07:00
Jae Kwon
bf0ff212b9 Refactor "lite" to handle delayed validator set changes.
Also, fix consensus liveness issue.
2018-06-25 17:12:25 -07:00
Alexander Simmerl
f62d6651e3
Merge pull request #1781 from tendermint/1740-node-crashes-when-too-many-rpc-connections
limit number of open connections
2018-06-23 01:15:03 +02:00
Anton Kaliaev
936a655990
limit number of open connections
Refs #1740

also, expose limit option for number concurrent streams for gRPC
(unlimited by default)
2018-06-22 19:25:11 +04:00
Alexander Simmerl
a605b66c5a
Move abci imports 2018-06-22 07:28:07 +02:00