Compare commits

...

293 Commits

Author SHA1 Message Date
Ethan Buchman
9c236ffd6c Merge pull request #2975 from tendermint/release/v0.27.0
Release/v0.27.0
2018-12-05 16:50:36 -05:00
Ethan Buchman
9f8761d105 update changelog and upgrading (#2974) 2018-12-05 15:19:30 -05:00
Anton Kaliaev
5413c11150 kv indexer: add separator to start key when matching ranges (#2925)
* kv indexer: add separator to start key when matching ranges

to avoid including false positives

Refs #2908

* refactor code

* add a test case
2018-12-05 14:21:46 -05:00
Ethan Buchman
a14fd8eba0 p2p: fix peer count mismatch #2332 (#2969)
* p2p: test case for peer count mismatch #2332

* p2p: fix peer count mismatch #2332

* changelog

* use httptest.Server to scrape Prometheus metrics
2018-12-05 07:32:27 -05:00
Ethan Buchman
1bb7e31d63 p2p: panic on transport error (#2968)
* p2p: panic on transport error

Addresses #2823. Currently, the acceptRoutine exits if the transport returns
an error trying to accept a new connection. Once this happens, the node
can't accept any new connections. So here, we panic instead. While we
could potentially be more intelligent by rerunning the acceptRoutine, the
error may indicate something more fundamental (eg. file desriptor limit)
that requires a restart anyways. We can leave it to process managers to
handle that restart, and notify operators about the panic.

* changelog
2018-12-04 19:16:06 -05:00
Ethan Buchman
222b8978c8 Minor log changes (#2959)
* node: allow state and code to have diff block versions

* node: pex is a log module
2018-12-04 08:30:29 -05:00
Anton Kaliaev
d9a1aad5c5 docs: add client#Start/Stop to examples in RPC docs (#2939)
follow-up on https://github.com/tendermint/tendermint/pull/2936
2018-12-03 16:17:06 +04:00
Anton Kaliaev
8ef0c2681d check if deliverTxResCh is still open, return an err otherwise (#2947)
deliverTxResCh, like any other eventBus (pubsub) channel, is closed when
eventBus is stopped. We must check if the channel is still open. The
alternative approach is to not close any channels, which seems a bit
odd.

Fixes #2408
2018-12-03 16:15:36 +04:00
Ismail Khoffi
c4d93fd27b explicitly type MaxTotalVotingPower to int64 (#2953) 2018-11-30 14:43:16 -05:00
Ethan Buchman
dc2a338d96 Bucky/v0.27.0 (#2950)
* update changelog

* changelog, upgrading, version
2018-11-30 14:05:16 -05:00
Ismail Khoffi
725ed7969a Add some ProposerPriority tests (#2946)
* WIP: tests for #2785

* rebase onto develop

* add Bucky's test without changing ValidatorSet.Update

* make TestValidatorSetBasic fail

* add ProposerPriority preserving fix to ValidatorSet.Update to fix
TestValidatorSetBasic

* fix randValidator_ to stay in bounds of MaxTotalVotingPower

* check for expected proposer and remove some duplicate code

* actually limit the voting power of random validator ...

* fix test
2018-11-29 17:03:41 -05:00
Ethan Buchman
44b769b1ac types: ValidatorSet.Update preserves Accum (#2941)
* types: ValidatorSet.Update preserves ProposerPriority

This solves the other issue discovered as part of #2718,
where Accum (now called ProposerPriority) is reset to
0 every time a validator is updated.

* update changelog

* add test

* update comment

* Update types/validator_set_test.go

Co-Authored-By: ebuchman <ethan@coinculture.info>
2018-11-29 08:26:12 -05:00
Anton Kaliaev
380afaa678 docs: update ecosystem.json: add Rust ABCI (#2945) 2018-11-29 15:57:11 +04:00
Ismail Khoffi
b30c34e713 rename Accum -> ProposerPriority: (#2932)
- rename fields, methods, comments, tests
2018-11-28 15:35:09 -05:00
Dev Ojha
4039276085 remove unnecessary "crypto" import alias (#2940) 2018-11-28 23:53:04 +04:00
Ismail Khoffi
3f987adc92 Set accum of freshly added validator -(total voting power) (#2785)
* set the accum of a new validator to (-total voting power):

- disincentivize validators to unbond, then rebon to reset their
negative Accum to zero

additional unrelated changes:
- do not capitalize error msgs
- fix typo

* review comments: (re)capitalize errors & delete obsolete comments

* More changes suggested by @melekes

* WIP: do not batch clip (#2809)

* substract avgAccum on each iteration

- temporarily skip test

* remove unused method safeMulClip / safeMul

* always substract the avg accum

 - temp. skip another test

* remove overflow / underflow tests & add tests for avgAccum:

- add test for computeAvgAccum
- as we substract the avgAccum now we will not trivially over/underflow

* address @cwgoes' comments

* shift by avg at the end of IncrementAccum

* Add comment to MaxTotalVotingPower

* Guard inputs to not exceed MaxTotalVotingPower

* Address review comments:

 - do not fetch current validator from set again
 - update error message

* Address a few review comments:

 - fix typo
 - extract variable

* address more review comments:

 - clarify 1.125*totalVotingPower == totalVotingPower + (totalVotingPower >> 3)

* review comments: panic instead of "clipping":

 - total voting power is guarded to not exceed MaxTotalVotingPower ->
 panic if this invariant is violated

* fix failing test
2018-11-28 13:12:17 -05:00
Ethan Buchman
b11788d36d types: NewValidatorSet doesn't panic on empty valz list (#2938)
* types: NewValidatorSet doesn't panic on empty valz list

* changelog
2018-11-28 13:09:29 -05:00
Daniil Lashin
9adcfe2804 docs: add client.Start() to RPC WS examples (#2936) 2018-11-28 20:55:18 +04:00
Zach
3d15579e0c docs: fix js-abci example (#2935) 2018-11-28 20:29:26 +04:00
Dev Ojha
4571f0fbe8 Enforce validators can only use the correct pubkey type (#2739)
* Enforce validators can only use the correct pubkey type

* adapt to variable renames

* Address comments from #2636

* separate updating and validation logic

* update spec

* Add test case for TestStringSliceEqual, clarify slice copying code

* Address @ebuchman's comments

* Split up testing validator update execution, and its validation
2018-11-28 09:09:27 -05:00
Ethan Buchman
8a73feae14 Merge pull request #2934 from tendermint/master
Merge pull request #2922 from tendermint/release/v0.26.4
2018-11-28 08:56:39 -05:00
srmo
e291fbbebe 2871 remove proposalHeartbeat infrastructure (#2874)
* 2871 remove proposalHeartbeat infrastructure

* 2871 add preliminary changelog entry
2018-11-28 08:52:34 -05:00
Jae Kwon
416d143bf7 R4R: Swap start/end in ReverseIterator (#2913)
* Swap start/end in ReverseIterator

* update CHANGELOG_PENDING

* fixes from review
2018-11-28 08:49:24 -05:00
Daniil Lashin
7213869fc6 Refactor updateState #2865 (#2929)
* Refactor updateState #2865

* Apply suggestions from code review

Co-Authored-By: danil-lashin <danil-lashin@yandex.ru>

* Apply suggestions from code review
2018-11-28 08:32:16 -05:00
Anton Kaliaev
ef9902e602 docs: small improvements (#2933)
* update docs

- make install_c cmd (install)
- explain node IDs (quick-start)
- update UPGRADING section (using-tendermint)

* use git clone with JS example

JS devs may not have Go installed and we should not force them to.

* rewrite sentence
2018-11-28 08:25:23 -05:00
Ethan Buchman
b771798d48 Merge pull request #2922 from tendermint/release/v0.26.4
Release/v0.26.4
2018-11-27 08:51:45 -05:00
Ethan Buchman
1abf34aa91 Prepare v0.26.4 changelog (#2921)
* prepare changelog

* linkify changelog

* changelog and version

* update changelog
2018-11-27 08:43:21 -05:00
Anton Kaliaev
92dc5fc77a don't return false positives when searching for a prefix of a tag value (#2919)
Fixes #2908
2018-11-27 08:12:28 -05:00
nagarajmanjunath
bef39f3346 Updated Marshal and unmarshal methods to make compatible with protobuf (#2918)
* upadtes in grpc Marshal and unmarshal

* update comments
2018-11-27 08:07:20 -05:00
Anton Kaliaev
94e63be922 [indexer] order results by index if height is the same (#2900)
Fixes #2775
2018-11-27 07:53:06 -05:00
Anton Kaliaev
9570ac4d3e rpc: Fix tx.height range queries (#2899)
Modify lookForHeight to return a height only there's a equal operator.
Previously, it was returning a height even for range conditions: "height
< 10000".

Fixes #2759
2018-11-27 07:47:50 -05:00
Jernej Kos
99b9c9bf60 types: Emit tags from BeginBlock/EndBlock (#2747)
This commit makes both EventNewBlock and EventNewBlockHeader emit tags
on the event bus, so subscribers can use them in queries.
2018-11-26 22:21:42 -05:00
Ethan Buchman
47a0669d12 Fix fast sync stack with wrong block #2457 (#2731)
* fix fastsync may stuck by a wrong block

* fixes from updates

* fixes from review

* Align spec with the changes

* fmt
2018-11-26 15:31:11 -05:00
JamesRay
fe3b97fd66 It's better read from genDoc than from state.validators when appHeight==0 in replay (#2893)
* optimize addProposalBlockPart

* optimize addProposalBlockPart

* if ProposalBlockParts and LockedBlockParts both exist,let LockedBlockParts overwrite ProposalBlockParts.

* fix tryAddBlock

* broadcast lockedBlockParts in higher priority

* when appHeight==0, it's better fetch genDoc than state.validators.
2018-11-26 08:03:08 -05:00
Ismail Khoffi
56052c0a87 update encoding spec (#2903)
Quick fix for #2902
2018-11-26 12:24:32 +04:00
Anton Kaliaev
98e442a8de return back initially allowed level if we encounter allowed key (#2889)
Fixes #2868 where module=main setting overrides all others
2018-11-25 23:34:22 -05:00
Tomas Tauber
b12488b5f1 Handling integer IDs in JSON-RPC requests -- fixes #2366 (#2811)
* Fixed accepting integer IDs in requests for Tendermint RPC server (#2366)

* added a wrapper interface `jsonrpcid` that represents both string and int IDs in JSON-RPC requests/responses + custom JSON unmarshallers

* changed client-side code in RPC that uses it

* added extra tests for integer IDs

* updated CHANGELOG_PENDING, as suggested by PR instructions

* addressed PR comments

* added table driven tests for request type marshalling/unmarshalling
* expanded handler test to check IDs
* changed pending changelog note

* changed json rpc request/response unmarshalling to use empty interfaces and type switches on ID

* some cleanup
2018-11-25 23:33:40 -05:00
Anton Kaliaev
b487feba42 node: refactor privValidator ext client code & tests (#2895)
* update ConsensusState#OnStop comment

* consensus: set logger for WAL in tests

* refactor privValidator client code and tests

follow-up on https://github.com/tendermint/tendermint/pull/2866
2018-11-21 21:24:13 +04:00
Joe Bowman
72f86b5192 [pv] add ability to use ipc validator (#2866)
Ref #2827

(I have since seen #2847 which is a fix for the same issue; this PR has tests and docs too ;) )
2018-11-21 10:45:20 +04:00
Jae Kwon
42592d9ae0 IncrementAccum upon RPC /validators; Sanity checks and comments (#2808) 2018-11-21 10:43:02 +04:00
Dev Ojha
1610a05cbd Remove counter from every mempoolTx (#2891)
Within every tx in the mempool, we store a 64 bit counter,
as an index for when it was inserted into the mempool.
This counter doesn't really serve any purpose.
It was likely added for debugging at one point,
Removing the counter reclaims memory,
which enables greater mempool sizes / mitigates resources at the same size.

Closes #2835
2018-11-21 10:33:41 +04:00
Anton Kaliaev
2d525bf2b8 mempool: add txs from Update to cache
We should add txs that come in from mempool.Update to the mempool's
cache, so that they never hit a potentially expensive check tx.

Originally posted by @ValarDragon in #2846
https://github.com/tendermint/tendermint/issues/2846#issuecomment-439216656

Refs #2855
2018-11-20 10:47:30 +04:00
Anton Kaliaev
e9efbfe267 refactor mempool.Update
- rename filterTxs to removeTxs
- move txsMap into removeTxs func
- rename goodTxs to txsLeft
2018-11-20 10:47:30 +04:00
Anton Kaliaev
7b883a5457 docs/install: prepend cp to /usr/local with sudo (#2885)
Closes #2884
2018-11-20 10:27:58 +04:00
cong
fd8d1d6b69 add BlockTimeIota to the config.toml (#2878)
Refs #2877
2018-11-19 20:15:23 +04:00
Ethan Buchman
5abdd254e7 Merge pull request #2875 from tendermint/master
Merge pull request #2873 from tendermint/release/v0.26.3
2018-11-17 18:25:46 -05:00
Ethan Buchman
22dcc92232 Merge pull request #2873 from tendermint/release/v0.26.3
Release/v0.26.3
2018-11-17 18:25:12 -05:00
Ethan Buchman
ccf6b2b512 Bucky/v0.26.3 (#2872)
* update CONTRIBUTING with notes on CHANGELOG

* update changelog

* changelog and version
2018-11-17 16:04:05 -05:00
srmo
1466a2cc9f #2815 do not broadcast heartbeat proposal when we are non-validator (#2819)
* #2815 do not broadcast heartbeat proposal when we are non-validator

* #2815 adding preliminary changelog entry

* #2815 cosmetics and added test

* #2815 missed a little detail

- it's enough to call getAddress() once here

* #2815 remove debug logging from tests

* #2815 OK. I seem to be doing something fundamentally wrong here

* #2815 next iteration of proposalHeartbeat tests

- try and use "ensure" pattern in common_test

* 2815 incorporate review comments
2018-11-17 15:23:39 -05:00
Ethan Buchman
6168b404a7 p2p: NewMultiplexTransport takes an MConnConfig (#2869)
* p2p: NewMultiplexTransport takes an MConnConfig

* changelog

* move test func to test file
2018-11-17 03:16:49 -05:00
Zaki Manian
e6fc10faf6 R4R: Add timeouts to http servers (#2780)
* Replaces our current http servers where connections stay open forever with ones with timeouts to prevent file descriptor exhaustion

* Use the correct handler

* Put in go routines

* fix err

* changelog

* rpc: export Read/WriteTimeout

The `broadcast_tx_commit` endpoint has it's own timeout.
If this is longer than the http server's WriteTimeout, the
user will receive an error. Here, we export the WriteTimeout
and set the broadcast_tx_commit timeout to be less than it.

In the future, we should use a config struct for the timeouts
to avoid the need to export. The broadcast_tx_commit timeout
may also become configurable, but we must check that it's less
than the server's WriteTimeout.
2018-11-17 03:10:22 -05:00
Anton Kaliaev
60018d6148 comment out until someone decides to tackle #2285 (#2760)
current code results in panic and we certainly don't want that.
https://github.com/tendermint/tendermint/pull/2286#issuecomment-418281846
2018-11-16 18:17:07 -05:00
Ethan Buchman
0d5e0d2f13 p2p/conn: FlushStop. Use in pex. Closes #2092 (#2802)
* p2p/conn: FlushStop. Use in pex. Closes #2092

In seed mode, we call StopPeer immediately after Send.
Since flushing msgs to the peer happens in the background,
the peer connection is often closed before the messages are
actually sent out. The new FlushStop method allows all msgs
to first be written and flushed out on the conn before it is closed.

* fix dummy peer

* typo

* fixes from review

* more comments

* ensure pex doesn't call FlushStop more than once

FlushStop is not safe to call more than once,
but we call it from Receive in a go-routine so Receive
doesn't block.

To ensure we only call it once, we use the lastReceivedRequests
map - if an entry already exists, then FlushStop should already have
been called and we can return.
2018-11-16 17:44:19 -05:00
Daniil Lashin
2cfdef6111 Make "Update to validators" msg value pretty (#2848)
* Make "Update to validators" msg value pretty #2765

* New format for logging validator updates

* Refactor logging validator updates

* Fix changelog item

* fix merge conflict
2018-11-16 17:38:22 -05:00
krhubert
b90e06a37c More verbose error log (#2864) 2018-11-16 17:36:42 -05:00
Anton Kaliaev
e6a0d098e8 small fixes to spec & http_server & Vagrantfile (#2859)
* Vagrantfile: install dev_tools

Follow-up on https://github.com/tendermint/tendermint/pull/2824

* update consensus params spec

* fix test name

* rpc_test: panic if failed to start listener

also
- remove http_server#MustListen
- align StartHTTPServer and StartHTTPAndTLSServer functions

* dep: allow minor releases for grpc
2018-11-16 12:58:30 -05:00
Ethan Buchman
d8ab8509de p2p: log 'Send failed' on Debug (#2857) 2018-11-16 11:37:58 +04:00
Ethan Buchman
85bba82154 Merge pull request #2858 from tendermint/master
Merge pull request #2851 from tendermint/release/v0.26.2
2018-11-15 18:42:47 -05:00
Ethan Buchman
d5a05eccba Merge pull request #2851 from tendermint/release/v0.26.2
Release/v0.26.2
2018-11-15 18:41:54 -05:00
kevlubkcm
a676c71678 [R4R] Add proposer to NewRound event and proposal info to CompleteProposal event (#2767)
* add proposer info to EventCompleteProposal

* create separate EventData structure for CompleteProposal

* cant us rs.Proposal to get BlockID because it is not guaranteed to be set yet

* copying RoundState isnt helping us here

* add Step back to make compatible with original RoundState event. update changelog

* add NewRound event

* fix test

* remove unneeded RoundState

* put height round step into a struct

* pull out ValidatorInfo struct. add ensureProposal assert

* remove height-round-state sub-struct refactor

* minor fixes from review
2018-11-15 18:40:42 -05:00
Zach
c033975a53 docs ADR (#2828)
* wip

* use same ADR template as SDK

* finish docs adr

* lil fixes
2018-11-15 18:08:24 -05:00
Anton Kaliaev
06225e332e Config option for JSON output formatter (#2843)
* Introduce a structured logging option

* rename StructuredLog to LogFormat

* add changelog entry

* move log_format under log_level
2018-11-15 18:05:06 -05:00
Alessio Treglia
b646437ec7 Decouple StartHTTP{,AndTLS}Server from Listen() (#2791)
* Decouple StartHTTP{,AndTLS}Server from Listen()

This should help solve cosmos/cosmos-sdk#2715

* Fix small mistake

* Update StartGRPCServer

* s/rpc/rpcserver/

* Start grpccore.StartGRPCServer in a goroutine

* Reinstate l.Close()

* Fix rpc/lib/test/main.go

* Update code comment

* update changelog and comments

* fix tm-monitor. more comments
2018-11-15 15:33:04 -05:00
Zach
be8c2d5018 use a github link (#2849) 2018-11-15 14:41:40 -05:00
Anton Kaliaev
e93b492efe do not pin deps to exact versions (#2844)
because
- they are locked in .lock file already
- individual dependencies can be updated with `dep ensure -update XXX`
- review process (and ^^^) should help us prevent accidental updates

Closes #2798
2018-11-15 14:40:18 -05:00
Ethan Buchman
9973decff9 changelog, versionbump (#2850) 2018-11-15 12:16:24 -05:00
Ethan Buchman
a0412357c1 update some top-level markdown files (#2841)
* update some top-level markdown files

* Update README.md

Co-Authored-By: ebuchman <ethan@coinculture.info>
2018-11-15 12:04:47 -05:00
Zeyu Zhu
a70a53254d Optimize: using parameters in func (#2845)
Signed-off-by: Zeyu Zhu <zhuzeyu0409@gmail.com>
2018-11-15 19:57:13 +04:00
zramsay
1ce24a6282 docs: update config: ref #2800 & #2837 2018-11-15 10:23:00 +04:00
zramsay
814a88a69b more maintainable/useful install scripts 2018-11-15 10:23:00 +04:00
Hleb Albau
6353862ac0 2582 Enable CORS on RPC API (#2800) 2018-11-14 16:47:41 +04:00
Zach
3af11c43f2 cleanup ecosystem docs (#2829) 2018-11-14 14:52:01 +04:00
Zach
27fcf96556 update genesis docs, closes #2814 (#2831) 2018-11-14 14:34:10 +04:00
Zach
bb0e17dbf0 arm: add install script, fix Makefile (#2824)
* be like the SDK makefile

* arm: add install script, fix Makefile

* ...
2018-11-14 14:17:07 +04:00
Anton Kaliaev
5a6822c8ac abci: localClient improvements & bugfixes & pubsub Unsubscribe issues (#2748)
* use READ lock/unlock in ConsensusState#GetLastHeight

Refs #2721

* do not use defers when there's no need

* fix peer formatting (output its address instead of the pointer)

```
[54310]: E[11-02|11:59:39.851] Connection failed @ sendRoutine              module=p2p peer=0xb78f00 conn=MConn{74.207.236.148:26656} err="pong timeout"
```

https://github.com/tendermint/tendermint/issues/2721#issuecomment-435326581

* panic if peer has no state

https://github.com/tendermint/tendermint/issues/2721#issuecomment-435347165

It's confusing that sometimes we check if peer has a state, but most of
the times we expect it to be there

1. add79700b5/mempool/reactor.go (L138)
2. add79700b5/rpc/core/consensus.go (L196) (edited)

I will change everything to always assume peer has a state and panic
otherwise

that should help identify issues earlier

* abci/localclient: extend lock on app callback

App callback should be protected by lock as well (note this was already
done for InitChainAsync, why not for others???). Otherwise, when we
execute the block, tx might come in and call the callback in the same
time we're updating it in execBlockOnProxyApp => DATA RACE

Fixes #2721

Consensus state is locked

```
goroutine 113333 [semacquire, 309 minutes]:
sync.runtime_SemacquireMutex(0xc00180009c, 0xc0000c7e00)
        /usr/local/go/src/runtime/sema.go:71 +0x3d
sync.(*RWMutex).RLock(0xc001800090)
        /usr/local/go/src/sync/rwmutex.go:50 +0x4e
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).GetRoundState(0xc001800000, 0x0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:218 +0x46
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusReactor).queryMaj23Routine(0xc0017def80, 0x11104a0, 0xc0072488f0, 0xc007248
9c0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/reactor.go:735 +0x16d
created by github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusReactor).AddPeer
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/reactor.go:172 +0x236
```

because localClient is locked

```
goroutine 1899 [semacquire, 309 minutes]:
sync.runtime_SemacquireMutex(0xc00003363c, 0xc0000cb500)
        /usr/local/go/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc000033638)
        /usr/local/go/src/sync/mutex.go:134 +0xff
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/abci/client.(*localClient).SetResponseCallback(0xc0001fb560, 0xc007868540)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/abci/client/local_client.go:32 +0x33
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/proxy.(*appConnConsensus).SetResponseCallback(0xc00002f750, 0xc007868540)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/proxy/app_conn.go:57 +0x40
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/state.execBlockOnProxyApp(0x1104e20, 0xc002ca0ba0, 0x11092a0, 0xc00002f750, 0xc0001fe960, 0xc000bfc660, 0x110cfe0, 0xc000090330, 0xc9d12, 0xc000d9d5a0, ...)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/state/execution.go:230 +0x1fd
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(0xc002c2a230, 0x7, 0x0, 0xc000eae880, 0x6, 0xc002e52c60, 0x16, 0x1f927, 0xc9d12, 0xc000d9d5a0, ...)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/state/execution.go:96 +0x142
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit(0xc001800000, 0x1f928)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:1339 +0xa3e
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit(0xc001800000, 0x1f928)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:1270 +0x451
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1(0xc001800000, 0x0, 0x1f928)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:1218 +0x90
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit(0xc001800000, 0x1f928, 0x0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:1247 +0x6b8
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote(0xc001800000, 0xc003d8dea0, 0xc000cf4cc0, 0x28, 0xf1, 0xc003bc7ad0, 0xc003bc7b10)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:1659 +0xbad
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote(0xc001800000, 0xc003d8dea0, 0xc000cf4cc0, 0x28, 0xf1, 0xf1, 0xf1)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:1517 +0x59
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg(0xc001800000, 0xd98200, 0xc0070dbed0, 0xc000cf4cc0, 0x28)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:660 +0x64b
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine(0xc001800000, 0x0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:617 +0x670
created by github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus.(*ConsensusState).OnStart
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/consensus/state.go:311 +0x132
```

tx comes in and CheckTx is executed right when we execute the block

```
goroutine 111044 [semacquire, 309 minutes]:
sync.runtime_SemacquireMutex(0xc00003363c, 0x0)
        /usr/local/go/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc000033638)
        /usr/local/go/src/sync/mutex.go:134 +0xff
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc0001fb0e0, 0xc002d94500, 0x13f, 0x280, 0x0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/abci/client/local_client.go:85 +0x47
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc00002f720, 0xc002d94500, 0x13f, 0x280, 0x1)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/proxy/app_conn.go:114 +0x51
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/mempool.(*Mempool).CheckTx(0xc002d3a320, 0xc002d94500, 0x13f, 0x280, 0xc0072355f0, 0x0, 0x0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/mempool/mempool.go:316 +0x17b
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/core.BroadcastTxSync(0xc002d94500, 0x13f, 0x280, 0x0, 0x0, 0x0)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/core/mempool.go:93 +0xb8
reflect.Value.call(0xd85560, 0x10326c0, 0x13, 0xec7b8b, 0x4, 0xc00663f180, 0x1, 0x1, 0xc00663f180, 0xc00663f188, ...)
        /usr/local/go/src/reflect/value.go:447 +0x449
reflect.Value.Call(0xd85560, 0x10326c0, 0x13, 0xc00663f180, 0x1, 0x1, 0x0, 0x0, 0xc005cc9344)
        /usr/local/go/src/reflect/value.go:308 +0xa4
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/lib/server.makeHTTPHandler.func2(0x1102060, 0xc00663f100, 0xc0082d7900)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/lib/server/handlers.go:269 +0x188
net/http.HandlerFunc.ServeHTTP(0xc002c81f20, 0x1102060, 0xc00663f100, 0xc0082d7900)
        /usr/local/go/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0xc002c81b60, 0x1102060, 0xc00663f100, 0xc0082d7900)
        /usr/local/go/src/net/http/server.go:2361 +0x127
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/lib/server.maxBytesHandler.ServeHTTP(0x10f8a40, 0xc002c81b60, 0xf4240, 0x1102060, 0xc00663f100, 0xc0082d7900)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/lib/server/http_server.go:219 +0xcf
github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/lib/server.RecoverAndLogHandler.func1(0x1103220, 0xc00121e620, 0xc0082d7900)
        /root/go/src/github.com/MinterTeam/minter-go-node/vendor/github.com/tendermint/tendermint/rpc/lib/server/http_server.go:192 +0x394
net/http.HandlerFunc.ServeHTTP(0xc002c06ea0, 0x1103220, 0xc00121e620, 0xc0082d7900)
        /usr/local/go/src/net/http/server.go:1964 +0x44
net/http.serverHandler.ServeHTTP(0xc001a1aa90, 0x1103220, 0xc00121e620, 0xc0082d7900)
        /usr/local/go/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00785a3c0, 0x11041a0, 0xc000f844c0)
        /usr/local/go/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2851 +0x2f5
```

* consensus: use read lock in Receive#VoteMessage

* use defer to unlock mutex because application might panic

* use defer in every method of the localClient

* add a changelog entry

* drain channels before Unsubscribe(All)

Read 55362ed766/libs/pubsub/pubsub.go (L13)
for the detailed explanation of the issue.

We'll need to fix it someday. Make sure to keep an eye on
https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-033-pubsub.md

* retry instead of panic when peer has no state in reactors other than consensus

in /dump_consensus_state RPC endpoint, skip a peer with no state

* rpc/core/mempool: simplify error messages

* rpc/core/mempool: use time.After instead of timer

also, do not log DeliverTx result (to be consistent with other memthods)

* unlock before calling the callback in reqRes#SetCallback
2018-11-13 11:32:51 -05:00
Zach
fb10209a96 update to amino 0.14.1 (#2822) 2018-11-13 11:54:43 +04:00
Ethan Buchman
0f793a5a00 Merge pull request #2813 from tendermint/master
Merge pull request #2807 from tendermint/release/v0.26.1
2018-11-12 08:05:12 -05:00
Ethan Buchman
80d0a36250 Merge pull request #2807 from tendermint/release/v0.26.1
Release/v0.26.1
2018-11-12 08:04:27 -05:00
yutianwu
e11699038d [R4R] Add adr-034: PrivValidator file structure (#2751)
* add adr-034

* update changelog

* minor changes

* do some refactor
2018-11-11 14:47:34 -05:00
Ethan Buchman
533b3cfb73 Release/v0.26.1 (#2803)
* changelog and version

* fix changelog
2018-11-11 12:08:28 -05:00
Ismail Khoffi
3ff820bdf4 fix amino overhead computation for Tx (#2792)
* fix amino overhead computation for Tx:

- also count the fieldnum / typ3
- add method to compute overhead per Tx
- slightly clarify comment on MaxAminoOverheadForBlock
- add tests

* fix TestReapMaxBytesMaxGas according to amino overhead

* fix TestMempoolFilters according to amino overhead

* address review comments:

 - add a note about fieldNum = 1
 - add forgotten godoc comment

* fix and use sm.TxPreCheck

* fix test

* remove print statement
2018-11-11 10:09:33 -05:00
Mehmet Gurevin
905abf1388 p2p: re-check after sleeps (#2664)
* p2p: re-check after sleeps

* use NodeInfo as an interface

* Revert "use NodeInfo as an interface"

This reverts commit 5f7d055e6c745ac8c8e5a9a7f0bd5ea5bc3d448c.

* Revert "p2p: re-check after sleeps"

This reverts commit 7f41070da070eadd3312efce1cc821aaf3e23771.

* preserve dial to itself

* ignore ensured connections while re-connecting

* re-check after sleep

* keep protocol definition on net addresses

* decrease log level

* Revert "preserve dial to itself"

This reverts commit 0c6e0fc58da78c378c32bb9ded2dd04ad5e754a9.

* correct func comment according to modification

Co-Authored-By: mgurevin <mehmet@gurevin.net>
2018-11-11 08:14:52 -05:00
Ismail Khoffi
7a4b62d3be check the result of ps.peer.Send before calling ps.setHasVote (#2787)
- actually call `ps.SetHasVote` instead to avoid carrying around
`votes.Height()`, `votes.Round()`, `types.SignedMsgType(votes.Type())`
2018-11-11 07:57:08 -05:00
Jae Kwon
5b19fcf204 p2p: AddressBook requires addresses to have IDs; Do not close conn immediately after sending pex addrs in seed mode (#2797)
* Require addressbook to only store addresses with valid ID

* Do not shut down peer immediately after sending pex addrs in SeedMode

* p2p: fix #2773

* seed mode: use go-routine to sleep before stopping peer
2018-11-11 06:50:25 -05:00
Anton Kaliaev
1944d8534b test AutoFile#Size (happy path) 2018-11-09 16:29:43 +01:00
Anton Kaliaev
13badc1d29 [autofile/group] do not panic when checking size
It's OK if the head will grow a little bit bigger, but we'll avoid
panic.

Refs #2703
2018-11-09 16:29:43 +01:00
Anton Kaliaev
091d2c3e5e openFile creates a file if not exist => ErrNotExist is not possible 2018-11-09 16:29:43 +01:00
Anton Kaliaev
d178ea9eaf use our logger in autofile/group 2018-11-09 16:29:43 +01:00
Catalin Pirvu
46d32af055 Add tests for ValidateBasic methods (#2754)
Fixes #2740
2018-11-09 15:59:04 +01:00
Zach
8b77328313 [docs] improve organization of ABCI docs & fix links (#2749)
* dedup with spec/abci/client-server

* fixup abci/readme

* link to getting started in abci/README

* https

* spec/abci: some deduplication

* docs: remove extraneous comment
2018-11-09 15:11:06 +01:00
Ethan Buchman
6e9aee5460 p2p: peer-id -> peer_id (#2771)
* p2p: peer-id -> peer_id

* update changelog
2018-11-06 21:12:46 -08:00
Anton Kaliaev
d460df1335 mempool: print postCheck error (#2762)
This is a follow-up from https://github.com/tendermint/tendermint/pull/2724

Closes #2761
2018-11-06 20:23:44 -08:00
Jae Kwon
03e42d2e38 Fix crypto/merkle ProofOperators.Verify to check bounds on keypath pa… (#2756)
* Fix crypto/merkle ProofOperators.Verify to check bounds on keypath parts.

* Update PENDING
2018-11-05 22:53:44 -08:00
Anton Kaliaev
b8a9b0bf78 Mempool WAL is still created by default in home directory, leads to permission errors (#2758)
* only invoke InitWAL/CloseWAL if WalPath is not empty

Closes #2717

* panic if WAL is not initialized when calling CloseWAL

* add a changelog entry
2018-11-05 22:39:05 -08:00
Ethan Buchman
7246ffc48f mempool: ErrPreCheck and more log info (#2724)
* mempool: ErrPreCheck and more log info

* change Pre/PostCheckFunc to return errors

also, continue execution when checking txs in mempool_test if
err=PreCheckErr
2018-11-05 22:32:52 -08:00
Ethan Buchman
071ebdd514 Merge pull request #2704 from tendermint/2702-proposal-pol-round-validation
if some process locks a block in round 0, then 0 is valid proposal.PO…
2018-11-05 22:32:15 -08:00
Ethan Buchman
8760c5b4f9 Merge branch 'develop' into 2702-proposal-pol-round-validation 2018-11-05 20:53:05 -08:00
Ethan Buchman
59b75d3f28 Merge pull request #2753 from tendermint/master
Merge pull request #2750 from tendermint/release/v0.26.0
2018-11-03 12:46:55 -07:00
Ethan Buchman
c086d0a341 Merge pull request #2750 from tendermint/release/v0.26.0
Release/v0.26.0
2018-11-03 12:46:14 -07:00
Ethan Buchman
322cee9156 Release/v0.26.0 (#2726)
* changelog_pending -> changelog

* update changelog

* update changelog

* update changelog and upgrading
2018-11-02 13:55:09 -04:00
Anton Kaliaev
80e4fe6c0d [ADR] [DRAFT] pubsub 2.0 (#2532)
* pubsub adr

Refs #951, #1879, #1880

* highlight question

* fix typos after Ismail's review
2018-11-02 10:16:29 +01:00
Anton Kaliaev
fb91ef7462 validate reactor messages (#2711)
* validate reactor messages

Refs #2683

* validate blockchain messages

Refs #2683

* validate evidence messages

Refs #2683

* todo

* check ProposalPOL and signature sizes

* add a changelog entry

* check addr is valid when we add it to the addrbook

* validate incoming netAddr (not just nil check!)

* fixes after Bucky's review

* check timestamps

* beef up block#ValidateBasic

* move some checks into bcBlockResponseMessage

* update Gopkg.lock

Fix

```
grouped write of manifest, lock and vendor: failed to export github.com/tendermint/go-amino: fatal: failed to unpack tree object 6dcc6ddc143e116455c94b25c1004c99e0d0ca12
```

by running `dep ensure -update`

* bump year since now we check it

* generate test/p2p/data on the fly using tendermint testnet

* allow sync chains older than 1 year

* use full path when creating a testnet

* move testnet gen to test/docker/Dockerfile

* relax LastCommitRound check

Refs #2737

* fix conflicts after merge

* add small comment

* some ValidateBasic updates

* fixes

* AppHash length is not fixed
2018-11-01 02:07:18 -04:00
Ethan Buchman
a22c962e28 TMHASH is 32 bytes. Closes #1990 (#2732)
* tmhash is fully 32 bytes. closes #1990

* AddressSize

* fix tests

* fix max sizes
2018-10-31 12:42:05 -04:00
Jae Kwon
1660e30ffe Fix general merkle keypath to start w/ last op's key (#2733)
* Fix general merkle keypath to start w/ last op's key

* Update CHANGELOG_PENDING.md
2018-10-31 11:02:13 -04:00
Zarko Milosevic
a83c268d7f Fix spec (#2736) 2018-10-31 10:59:01 -04:00
Zarko Milosevic
c5905900eb Simplify proposal msg (#2735)
* Align Proposal message with spec

* Update spec
2018-10-31 10:27:11 -04:00
Zarko Milosevic
7a03344480 Introduce EventValidBlock for informing peers about wanted block (#2652)
* Introduce EventValidBlock for informing peer about wanted block

* Merge with develop

* Add isCommit flag to NewValidBlock message

- Add test for the case of +2/3 Precommit from the previous round
2018-10-31 09:20:36 -04:00
Ismail Khoffi
a530352f61 Align Vote/Proposal fields with canonical order and fields (#2730)
* reorder fields

* add TestVoteString & update tests

* remove redundant info from Proposal.String()

* update spec

* revert changes on vote.String() -> more human friendly
2018-10-30 12:16:55 -04:00
yutianwu
60437953ac [R4R] libs/log: add year to log format (#2707)
* add year to log format

* update documentation
2018-10-30 11:46:55 -04:00
Dev Ojha
56d7160606 Add ValidatorPubkeyTypes as a consensus param (#2636)
* Add ValidatorPubkeyTypes as a consensus param

Ref #2414

* update spec

* address anton's comment

* Switch to Validator and Validator Params

* Correct changelog entry

* Address bucky's comments!

* forgot to update changelog

* fix typo

* fix Params naming
2018-10-30 11:36:53 -04:00
Zach
cdc252b818 add fail-test file instead of dep, closes #2638 (#2728)
original author of this file is @ebuchman:

https://github.com/ebuchman/fail-test
2018-10-30 10:34:51 -04:00
Ethan Buchman
b24de1c01c update changelog pending and readme (#2725) 2018-10-29 10:13:11 -04:00
Ismail Khoffi
b6d5b8b745 Update to amino 0.14.0 (#2710)
* WIP: update to amino 0.14.0

* update Changelog

* Update to latest amino version (v0.14.0)
2018-10-29 09:16:50 -04:00
Anton Kaliaev
a67ae81469 if some process locks a block in round 0, then 0 is valid proposal.POLRound in rounds > 0
This condition is really hard to get. Initially, lockedRound and
validRound are set to -1 as we start with round 0.

Refs #2702
2018-10-25 16:40:20 +02:00
zhangzheng
bbf15b3d09 tm-monitor: update health after we added / removed node (#2694)
Refs #2693
2018-10-25 12:27:32 +02:00
Ismail Khoffi
6643c5dd11 Catch up with amino 0.13.0 (#2690)
* catch up with amino changes in
https://github.com/tendermint/go-amino/pull/222

* WIP: update to amino v0.13.0

* update to fixed amino release
2018-10-24 21:34:01 -04:00
Jun Kimura
9795e12ef2 fix RecoverAndLogHandler not to call multiple writeheader (#2688) 2018-10-24 10:07:33 +02:00
Ethan Buchman
be929acd6a Update to Amino v0.13.0-rc0 (#2687)
* types: test tm2pm on fully populated header

* upgrade for amino v0.13.0-rc0

* fix lint

* comment out final test
2018-10-23 13:21:47 -04:00
Ethan Buchman
fe1d59ab7b Set protocol versions in NodeInfo from state (#2686)
* use types.NewValidator

* node: set p2p.ProtocolVersion from state, not globals
2018-10-22 17:55:49 -04:00
Ethan Buchman
f94eb42ebe Version bump; Update Upgrading.md; linkify Changelog (#2679)
* version bump

* update UPGRADING.md

* add missing pr numbers to changelog pending

* linkify changelog
2018-10-19 20:27:00 -04:00
Ethan Buchman
9d62bd0ad3 crypto: use stdlib crypto/rand. ref #2099 (#2669)
* crypto: use stdlib crypto/rand. ref #2099

* comment
2018-10-19 14:29:45 -04:00
Ethan Buchman
30519e8361 types: first field in Canonical structs is Type (#2675)
* types: first field in Canonical structs is Type

* fix spec
2018-10-19 14:23:14 -04:00
Ethan Buchman
7c6519adbd Bucky/changelog (#2673)
* update changelog, add authors script

* update changelog

* update changelog
2018-10-19 13:49:04 -04:00
Ethan Buchman
f536089f0b types: dont use SimpleHashFromMap for header. closes #1841 (#2670)
* types: dont use SimpleHashFromMap for header. closes #1841

* changelog and spec

* comments
2018-10-19 11:39:27 -04:00
Ethan Buchman
746d137f86 p2p: Restore OriginalAddr (#2668)
* p2p: bring back OriginalAddr

* p2p: set OriginalAddr

* update changelog
2018-10-18 18:26:32 -04:00
Ethan Buchman
e798766a27 types: remove Version from CanonicalXxx (#2666) 2018-10-18 18:02:20 -04:00
Ethan Buchman
c3384e88e5 adr-016: update int64->uint64; add version to ConsensusParams (#2667) 2018-10-18 17:32:53 -04:00
Ethan Buchman
ed4ce5ff6c ADR-016: Update ABCI Info method for versions (#2662)
* abci: update RequestInfo for versions

* abci: update ResponseInfo for versions

* compile fix

* fix test

* software_version -> version

* comment fix

* update spec

* add test

* comments and fix test
2018-10-18 16:51:17 -04:00
Anton Kaliaev
055d7adffb add tm-abci python ABCI server (fork of py-abci) (#2658)
It utilises async IO -> greater performance.
2018-10-18 20:03:45 +04:00
Ethan Buchman
14c1baeb24 ADR-016: Add protocol Version to NodeInfo (#2654)
* p2p: add protocol Version to NodeInfo

* update node pkg. remove extraneous version files

* update changelog and docs

* fix test

* p2p: Version -> ProtocolVersion; more ValidateBasic and tests
2018-10-18 10:29:59 -04:00
Ethan Buchman
455d34134c ADR-016: Add versions to Block and State (#2644)
* types: add Version to Header

* abci: add Version to Header

* state: add Version to State

* node: check software and state protocol versions match

* update changelog

* docs/spec: update for versions

* state: more tests

* remove TODOs

* remove empty test
2018-10-17 15:30:53 -04:00
Jack Zampolin
6a07f415e9 Change error output format for better SDK and Voyager UX (#2648)
* Change error output format

* Update tests

* 🤦

* apply suggestion
2018-10-17 14:26:14 -04:00
Anton Kaliaev
d20693fb16 install scripts: update go version, remove upgrade cmd (#2650)
* [install scripts] update go version, remove upgrade

- upgrading OS is out of scope of installation scripts

* add missing log statement

Refs https://github.com/tendermint/tendermint/pull/2642#discussion_r225786794
2018-10-17 12:47:56 -04:00
Hendrik Hofstadt
f60713bca8 privval: Add IPCPV and fix SocketPV (#2568)
Ref: #2563

I added IPC as an unencrypted alternative to SocketPV.

Besides I fixed the following aspects of SocketPV:

Added locking since we are operating on a single socket
The connection deadline is extended every time a successful packet exchange happens; otherwise the connection would always die permanently x seconds after the connection was established.
Added a ping/heartbeat mechanism to keep the connection alive; native TCP keepalives do not work in this use-case


* Extend the SecureConn socket to extend its deadline
* Add locking & ping/heartbeat packets to SocketPV
* Implement IPC PV and abstract socket signing
* Refactored IPC and SocketPV
* Implement @melekes comments
* Fixes to rebase
2018-10-17 10:26:14 +02:00
Uzair1995
ed107d0e84 [scripts/install_tendermint_ubuntu] change /root to local user (#2647) 2018-10-17 10:12:31 +04:00
Peng Zhong
80562669bf add Google Analytics for documentation pages (#2645)
We need Google Analytics to start measuring how many developers are viewing our documentation. That way we can gauge how successful an/or useful various pages are. VuePress supports GA and all we have to provide is the tracking code.

This PR also renames the static docs site to "Tendermint Documentation", which is a better representation of the contents than only "Tendermint Core".
2018-10-16 10:10:52 +04:00
Anton Kaliaev
55362ed766 [pubsub] document design shortcomings (#2641)
Refs https://github.com/tendermint/tendermint/issues/1811#issuecomment-427825250
2018-10-16 10:09:24 +04:00
Dev Ojha
124d0db1e0 Make txs and evidencelist use merkle.SimpleHashFromBytes to create hash (#2635)
This is a performance regression, but will also spare the types directory
from knowing about RFC 6962, which is a more correct abstraction. For txs
this performance hit will be fixed soon with #2603. For evidence, the
performance impact is negligible due to it being capped at a small number.
2018-10-15 16:42:47 -04:00
Joon
4ab7dcf3ac [R4R] Unmerklize ConsensusParams.Hash() (#2609)
* Hash() uses tmhash instead of merkle.SimpleHashFromMap

* marshal whole struct

* update comments

* update docs
2018-10-15 16:31:27 -04:00
Joon
26462025bc standardize header.Hash() (#2610) 2018-10-15 16:28:49 -04:00
Zarko Milosevic
287b25a059 Align with spec (#2642) 2018-10-15 16:05:13 -04:00
Anton Kaliaev
37928cb990 set next validators along with validators while replay (#2637)
Closes #2634
2018-10-14 22:28:41 -04:00
Dev Ojha
0790223518 Comment about ed25519 private key format on Sign (#2632)
Closes #2001
2018-10-13 20:01:21 -04:00
Ethan Buchman
0baa7588c2 p2p: NodeInfo is an interface; General cleanup (#2556)
* p2p: NodeInfo is an interface

* (squash) fixes from review

* (squash) more fixes from review

* p2p: remove peerConn.HandshakeTimeout

* p2p: NodeInfo is two interfaces. Remove String()

* fixes from review

* remove test code from peer.RemoteIP()

* p2p: remove peer.OriginalAddr(). See #2618

* use a mockPeer in peer_set_test.go

* p2p: fix testNodeInfo naming

* p2p: remove unused var

* remove testRandNodeInfo

* fix linter

* fix retry dialing self

* fix rpc
2018-10-12 19:25:33 -04:00
Ismail Khoffi
8888595b94 [R4R] Fixed sized and reordered fields for Vote/Proposal/Heartbeat SignBytes (#2598)
* WIP: switching to fixed offsets for SignBytes

* add version field to sign bytes and update order

* more comments on test-cases and add a tc with a chainID

* remove amino:"write_empty" tag

- it doesn't affect if default fixed size fields ((u)int64) are
written or not
- add comment about int->int64 casting

* update CHANGELOG_PENDING

* update documentation

* add back link to issue #1622 in documentation

* remove JSON tags and add (failing test-case)

* fix failing test

* update test-vectors due to added `Type` field

* change Type field from string to byte and add new type alias

- SignedMsgType replaces VoteTypePrevote, VoteTypePrecommit and adds new
ProposalType to separate votes from proposal when signed

- update test-vectors

* fix remains from rebasing

* use SignMessageType instead of byte everywhere

* fixes from review
2018-10-12 19:21:46 -04:00
Dev Ojha
1b51cf3f46 Remove unnecessary layer of indirection / unnecessary allocation of hashes (#2620) 2018-10-12 17:48:00 -04:00
Zarko Milosevic
2363d88979 consensus: Wait for proposal or timeout before prevote (#2540)
* Fix termination issues and improve tests

* Improve formatting and tests based on reviewer feedback
2018-10-12 16:13:01 -04:00
Anton Kaliaev
e1538bf67e state: require block.Time of the fist block to be genesis time (#2594)
* require block.Time of the fist block to be genesis time

Refs #2587:

```
We only start validating block.Time when Height > 1, because there is no
commit to compute the median timestamp from for the first block. This
means a faulty proposer could make the first block with whatever time
they want.

Instead, we should require the timestamp of block 1 to match the genesis
time.

I discovered this while refactoring the ValidateBlock tests to be
table-driven while working on tests for #2560.
```

* do not accept blocks with negative height

* update changelog and spec

* nanos precision for test genesis time

* Fix failing test (#2607)
2018-10-12 01:03:58 -04:00
Alessio Treglia
3744e8271d [R4R] Pass nil to NewValidatorSet() when genesis file's Validators field is nil (#2617)
* Pass nil to NewValidatorSet() when genesis file's Validators field is nil

Closes: #2616

* Update CHANGELOG_PENDING.md
2018-10-11 23:37:21 -04:00
Ethan Buchman
7b48ea1788 privval: set deadline in readMsg (#2548)
* privval: set deadline in readMsg

* fixes from review
2018-10-11 13:55:36 -04:00
Dev Ojha
69ecda18f9 refactor nop_event_bus.go into event_bus.go (#2605)
This is just to match the style of the rest of the codebase, and to
reduce the number of files in types.
2018-10-11 10:16:25 -04:00
Connor Stein
cc0bea522c Minor doc fix regarding testnet on non-linux OS (#2601) 2018-10-11 10:03:34 -04:00
Ethan Buchman
feb08fa4f8 ed25519: use golang/x/crypto fork (#2558)
* ed25519: use golang/x/crypto fork

* changelog

* gix GenerateFromPassword

* fixes from review
2018-10-11 10:01:53 -04:00
Anton Kaliaev
9a6cdaddf2 fix contributor's name in CHANGELOG_PENDING (#2599)
Refs https://github.com/tendermint/tendermint/pull/2506#issuecomment-428458974
2018-10-10 21:29:13 +04:00
Dev Ojha
12fa9d1cab crypto/merkle: Remove byter in favor of plain byte slices (#2595)
* crypto/merkle: Remove byter in favor of plain byte slices

This PR is fully backwards compatible in terms of function output!
(The Go API differs though) The only test case changes was to refactor
it to be table driven.

* Update godocs per review comments
2018-10-10 12:46:09 -04:00
Matthew Slipper
92343ef484 Add additional metrics (#2500)
* Add additional metrics
Continues addressing https://github.com/cosmos/cosmos-sdk/issues/2169.
* Add nop metrics to fix NPE
* Tweak buckets, code review
* Update buckets
* Update docs with new metrics
* Code review updates
2018-10-10 18:27:43 +02:00
Dev Ojha
ee7b3d260e crypto/amino: Address anton's comment on PubkeyAminoRoute (#2592) 2018-10-10 12:13:42 +04:00
Ethan Buchman
6ec52a9233 types: cap evidence in block validation (#2560)
* cap evidence in block validation

* state: use table-driven test for ValidateBlockHeader

* state: test evidence cap

* fixes from review
2018-10-09 13:31:21 -04:00
Anton Kaliaev
05a119aab5 libs: Test deadlock from listener removal inside callback (#2588)
Closes #2575
2018-10-09 19:31:06 +02:00
Dev Ojha
d7341c4057 distribution: Lock binary dependencies to specific commits (#2550) 2018-10-09 19:28:15 +02:00
Anton Kaliaev
3fcb62b931 :tools: Update docs & fix build-docker Makefile target (#2584)
bump alpine version to 3.8
2018-10-09 16:04:15 +02:00
Dev Ojha
8761b27489 crypto: Add a way to go from pubkey to route (#2574)
This is intended for use in a future PR for #2414
2018-10-09 14:41:33 +02:00
Joon
e7708850c0 libs: Let prefixIterator implements Iterator correctly (#2581)
Fixes #2577
2018-10-09 14:21:36 +02:00
Overbool
561fc2d717 test(db): Test itr.Value in checkValuePanics (#2580)
Fixes #2573
2018-10-09 14:19:00 +02:00
Anton Kaliaev
724e264ff5 separate mock evidence from real evidence (#2571)
Closes #2525
2018-10-09 14:10:05 +02:00
Anton Kaliaev
989a2f32b1 libs: Refactor & document events code (#2576)
* [libs/events] add more godoc comments
* [libs/events] refactor code
- improve var naming
- improve code structure
- do not use defers for unlocking mutexes (defer takes time)
2018-10-09 13:09:40 +02:00
goolAdapter
4b2bf023dd libs: Fix event concurrency flaw (#2519)
* fix event concurrency flaw
* modify changelog
* fix a mistake
* fix a lint issue
* modify changelog
* modify for review issue
* modify for review issue
* modify for review issue
2018-10-08 15:36:31 +02:00
Dev Ojha
c17547ac2f Switch nodeID to use tmhash.Size, add test names for net addr tests (#2559)
* Switch nodeID to be tmhash.Size, add test names for net addr tests
Both of these came up when locally trying to change tmhash size.
* fix error introduced by merge
2018-10-08 15:20:41 +02:00
Dev Ojha
b1e7fac787 crypto/random: Use chacha20, add forward secrecy (#2562)
Ref #2099
2018-10-08 15:15:56 +02:00
Anton Kaliaev
35b671214c tools: Refactor tm-bench (#2570)
* specify time unit for FlushThrottleTimeout in TestP2PConfig
Refs #2555
* [tm-bench] refactor code
https://github.com/tendermint/tendermint/pull/2405#pullrequestreview-157166387
2018-10-08 15:03:38 +02:00
bradyjoestar
4c0c6e0116 [tm-bench] exit on CTRL-C (#2405) 2018-10-08 11:06:01 +04:00
Zach
b8556b97b8 circle: save p2p logs as artifacts (#2566) 2018-10-08 10:49:50 +04:00
goolAdapter
5f88fe0e9b fix p2p switch FlushThrottle value (#2569) 2018-10-08 10:05:12 +04:00
Pierrick Hymbert
1d8348d707 [p2p] Malformed external address causes SIGSEGV (if URL has empty host) (#2564)
fix #2071

Signed-off-by: phymbert <pierrick.hymbert@gmail.com>
2018-10-06 09:53:52 -04:00
Ethan Buchman
f471fc4963 abci: codespace (#2557)
* abci: codespace

* changelog
2018-10-06 09:20:15 -04:00
Joon
2d726a620b add adr (#2553) 2018-10-05 23:44:53 -04:00
Dev Ojha
dfda7b442f types: Remove pubkey from validator hash (#2512)
* types: Remove pubkey from validator hash

* undo lock file change

* Update Spec
2018-10-05 19:26:52 -04:00
Zach
6e5f58191e add spec/abci/readme to sidebar (#2551) 2018-10-05 09:51:23 -04:00
Dev Ojha
c648c93807 Fix random distribution in bitArray.PickRandom (#2534)
* Fix random distribution in bitArray.PickRandom

Previously it was very biased. 63 "_" followed by a single "x" had
much greater odds of being chosen. Additionally, the last element was
skewed. This fixes that by first preproccessing the set of all true
indices, and then randomly selecting a single element from there.

This commit also makes the code here significantly simpler, and
improves test cases.

* unlock mtx right after we select true indices
2018-10-05 11:00:50 +04:00
bradyjoestar
5b120d788a lite support maxOpenConnections (#2413) 2018-10-04 20:39:24 -04:00
Alexander Simmerl
e6a55b7be0 consensus: Add ADR for first stage consensus refactor (#2462) 2018-10-04 20:35:35 -04:00
Ismail Khoffi
d2be7482e1 [ADR][DRAFT] 024: SignBytes and validator types in privval (#2445)
* first draft for ADR summarizing discussion from:
https://github.com/tendermint/tendermint/issues/1622

* fix link and add comment about pub-key per message and fix link

* fix link and add comment about pub-key per message; also:
 - fix link
 - add little diagram
 - fix typo

* Add a slightly different approach

* typo and ADR number
2018-10-04 20:28:27 -04:00
Ethan Buchman
44a72fb642 Merge pull request #2544 from tendermint/bucky/adr-029
Bucky/adr 029
2018-10-04 20:09:34 -04:00
Ethan Buchman
c15fc9ff63 adr-029: update CheckBlock 2018-10-04 20:11:21 -04:00
JamesRay
be1760cc25 Create adr-021-check block txs before prevote.md 2018-10-04 18:43:17 -04:00
Anton Kaliaev
5b1b1ea58a [libs/autofile] fix DATA RACE by removing openFile() call (#2539)
There's a time window after we call RotateFile() where autofile#index+1
does not exist. It will be created during the next call to Write(). BUT
if somebody calls NewReader() before Write(), it will fail with "open
  /tmp/wal#index+1/wal: no such file or directory"

We must create file (either by calling gr.Head.openFile() or directly)
during NewReader() to ensure read calls succeed.

Closes #2538
2018-10-04 17:57:59 -04:00
Jeremiah Andrews
f11aef20a0 Add ADR for Commit changes (#2374) 2018-10-04 17:54:45 -04:00
Zach
303649818c update docs links & sidebar (#2541)
* docs: fix links

* docs: add readme from each section to the sidebar
2018-10-04 17:22:41 -04:00
Zarko Milosevic
12675ecd92 consensus: Wait timeout precommit before starting new round (#2493)
* Disable transitioning to new round upon 2/3+ of Precommit nils

Pull in ensureVote test function from https://github.com/tendermint/tendermint/pull/2132

* Add several ensureX test methods to wrap channel read with timeout

* Revert panic in tests
2018-10-04 09:37:13 -04:00
Anton Kaliaev
cb2e58411f add a missing changelog entry 2018-10-03 11:29:04 +04:00
ValarDragon
0755a5203d bit_array: Simplify subtraction
also, fix potential bug in Or function
2018-10-03 11:29:04 +04:00
JamesRay
c94133ed1b Fix a bug in bit_array's sub function (#2506) 2018-10-03 10:28:46 +04:00
Anton Kaliaev
f3d08f969d [rpc] fix /abci_query: trusted was renamed to prove (#2531) 2018-10-02 20:31:04 -04:00
goolAdapter
5c6999cf8f fix evidence db iter leak (#2516)
Also make reversing a slice more efficient
2018-10-02 11:52:56 +04:00
Dev Ojha
fd1b8598bc Make block_test.go more table driven (#2526) 2018-10-02 11:47:20 +04:00
Anton Kaliaev
32e274cff0 config: Refactor ValidateBasic (#2503)
* timeouts as time.Duration are also breaking for old configs
* split BaseConfig#ValidateBasic into smaller methods
2018-10-01 14:38:35 +02:00
Ethan Buchman
ccd04587ff docs/spec/abci: consensus params and general merkle (#2524)
* docs: links to dirs need a slash

* docs/spec/abci: consensus params and general merkle
2018-09-30 15:08:01 -04:00
Ethan Buchman
52e21cebcf remove some xxx comments and the config.mempool.recheck_empty (#2505)
* remove some XXX

* config: remove Mempool.RecheckEmpty

* docs: remove recheck_empty
2018-09-30 13:28:34 -04:00
Dev Ojha
69c7aa77bc clist: speedup Next by removing defers (#2511)
This change doubles the speed of the mempool's reaping.
Before:

BenchmarkReap-8   	    5000	    365390 ns/op	  122887 B/op

After:

BenchmarkReap-8   	   10000	    158274 ns/op	  122882 B/op
2018-09-30 13:26:14 -04:00
Ethan Buchman
ead9fc0179 Docs cleanup (#2522)
* minor doc cleanup

* docs/tools: link fixes and readme

* docs/networks: networks/local/README.md

* docs: update vuepress config

* docs: fixes from review
2018-09-30 12:35:52 -04:00
Ethan Buchman
f36ed7e7ff General Merkle Follow Up (#2510)
* tmlibs -> libs

* update changelog

* address some comments from review of #2298
2018-09-28 23:32:13 -04:00
Joon
71a34adfe5 General Merkle Proof (#2298)
* first commit

finalize rebase

add protoc_merkle to Makefile

* in progress

* fix kvstore

* fix tests

* remove iavl dependency

* fix tx_test

* fix test_abci_cli

fix test_apps

* fix test_apps

* fix test_cover

* rm rebase residue

* address comment in progress

* finalize rebase
2018-09-28 20:03:19 -04:00
Ismail Khoffi
fc073746a0 privval: Switch to amino encoding in SignBytes (#2459)
* switch to amino for SignBytes and add Reply with error message

- currently only Vote is done

* switch Reply type in socket for other messages

 - add error description on error

* add TODOs regarding error handling

* address comments from peer review session (thx @xla)

 - contains all changes besides the test-coverage / error'ing branches

* increase test coverage:

 - add tests for each newly introduced error'ing code path

* return error if received wrong response

* add test for wrong response branches (ErrUnexpectedResponse)

* update CHANGELOG_PENDING and related documentation (spec)

* fix typo: s/CanonicallockID/CanonicalBlockID

* fixes from review
2018-09-28 19:57:29 -04:00
Anton Kaliaev
47bc15c27a disable mempool WAL by default (#2490) 2018-09-28 19:28:42 -04:00
HaoyangLiu
8dda3c3b28 lite: Add synchronization in lite verify (#2396)
* Implement issues 2386: add synchronization in lite verify and change all Certify to Verify

* Replace make(chan struct{}, 0) with make(chan struct{})

* Parameterize memroy cache size and add concurrent test

* Refactor import order
2018-09-28 19:23:21 -04:00
Ethan Buchman
5173fe9414 Merge pull request #2497 from tendermint/zach/version
docs: Add version
2018-09-27 16:41:13 -04:00
zramsay
d007ade6c3 add version to docs 2018-09-26 17:49:20 -04:00
Anton Kaliaev
4c4a95ca53 config: Add ValidateBasic (#2485)
* add missing options to config.toml template and docs
Refs #2232
* config#ValidateBasic
Refs #2232
* [config] timeouts as time.Duration, not ints
Why:
- native type provides better guarantees than ", in ms" comment (harder
to shoot yourself in the leg)
- flexibility: you can change units
2018-09-26 12:04:44 +02:00
zramsay
df329e8f27 rpc/libs/doc: formatting for godoc, closes #2420 2018-09-26 11:37:26 +04:00
zramsay
cf8b42d813 rpc/core: ints are strings in responses, closes #1896 2018-09-26 11:37:26 +04:00
goolAdapter
110b07fb3f libs: Call Flush() before rename #2428 (#2439)
* fix Group.RotateFile need call Flush() before rename. #2428
* fix some review issue. #2428
 refactor Group's config: replace  setting member with initial option
* fix a handwriting mistake
* fix a time window error between rename and write.
* fix a syntax mistake.
* change option name Get_ to With_
* fix review issue
* fix review issue
2018-09-25 13:22:45 +02:00
Matthew Slipper
587116dae1 metrics: Add additional metrics to p2p and consensus (#2425)
* Add additional metrics to p2p and consensus
Partially addresses https://github.com/cosmos/cosmos-sdk/issues/2169.
* WIP
* Updates from code review
* Updates from code review
* Add instrumentation namespace to configuration
* Fix test failure
* Updates from code review
* Add quotes
* Add atomic load
* Use storeint64
* Use addInt64 in writePacketMsgTo
2018-09-25 13:14:38 +02:00
Anton Kaliaev
eb0da7f9cb libs: Handle SIGHUP explicitly inside autofile (#2480)
* handle SIGHUP explicitly inside autofile
Refs #2260
* libs: Use consistent channel suffix
2018-09-25 12:43:28 +02:00
Anton Kaliaev
d12e55c494 node: Respond always to OS interrupts (#2479)
* stop node upon receiving SIGTERM or CTRL-Ceven during genesis sleep by setting up interrupt before starting a node

Closes #2434

* call Start, not OnStart when starting a component to avoid:

```
E[09-24|10:13:15.805] Not stopping PubSub -- have not been started yet module=pubsub impl=PubSub
```

being printed on exit
2018-09-25 12:24:18 +02:00
Ethan Buchman
d297f02227 Merge pull request #2438 from tendermint/bucky/versions
version: types
2018-09-23 16:24:14 -04:00
Ethan Buchman
d0f6864c69 Merge pull request #2476 from tendermint/master
Master back to develop
2018-09-23 10:23:17 -04:00
Ethan Buchman
0c9c3292c9 Merge pull request #2473 from tendermint/release/v0.25.0
Release/v0.25.0
2018-09-23 10:22:46 -04:00
Ethan Buchman
e4ee34cfbc version: types 2018-09-23 09:19:12 -04:00
Anton Kaliaev
d16f52eab3 copy without sudo (#2474) 2018-09-23 08:31:19 -04:00
Ethan Buchman
27ba6e8a42 Minor docs cleanup (#2472)
* docs: link consensus to blockchain spec. closes #2422

* docs: deprecate research section. closes #2401

* docs: fix some links

* docs: fix some markdown lists

* docs: fix more links
2018-09-23 02:25:33 -04:00
Ethan Buchman
a8eee4ab28 Merge pull request #2471 from tendermint/master
Merge master to develop
2018-09-23 01:54:33 -04:00
Ethan Buchman
cd172acee8 Merge pull request #2467 from tendermint/release/v0.25.0
Release/v0.25.0
2018-09-23 01:33:35 -04:00
Ethan Buchman
97b43d875a update changelog 2018-09-23 01:25:10 -04:00
Ethan Buchman
b394bd5b5c Merge branch 'develop' into release/v0.25.0 2018-09-23 01:24:50 -04:00
Ethan Buchman
f5824bc837 Update abci and app docs (#2470)
* mempool: update some comments

* make build_c

* docs: notes about databases and WAL files

* docs: determinism. closes #1279

* docs: small note about query paths. closes #2090

* docs: gas

* docs: abci consensus params
2018-09-23 01:14:05 -04:00
Dev Ojha
111e627037 mempool: Filter new txs if they have insufficient gas (#2385)
This also refactors the prior mempool to filter to be known as
"precheck filter" and this new filter is called "postcheck filter"

This PR also fixes a bug where the precheck filter previously didn't
account for the amino overhead, which could a maliciously sized tx to
halt blocks from getting any txs in them.

* Move maxGas outside of function definition to avoid race condition
* Type filter funcs and make public
* Use helper method for post check
* Remove superfluous Filter suffix
* Move default pre/post checks into package
* Fix broken references
* Fix typos
* Expand on examples for checks
2018-09-22 02:50:06 +02:00
Ethan Buchman
ee8b8bbefb flush changelog pending, bump version 2018-09-21 17:41:02 -04:00
Ethan Buchman
dde0936fb8 linkify changelog 2018-09-21 17:37:40 -04:00
Ethan Buchman
2dfde37f44 update changelog and upgrading 2018-09-21 17:34:36 -04:00
Zarko Milosevic
f99e4010f2 Add stats related channel between consensus state and reactor (#2388) 2018-09-21 14:36:48 -04:00
Alessio Treglia
f11db8c1b0 Pass http.ServeTLS() errors back to the caller (#2461)
Closes: #2460

* Pass http.ServeTLS() errors back to the caller
* Update CHANGELOG
* Amend StartHTTPServer() too for consistency's sake
* Revert "Amend StartHTTPServer() too for consistency's sake"
This reverts commit 23bfb4c2e917f581702291fe3ea69fce23f8c89d.
2018-09-21 18:12:29 +02:00
Zach
886a83dfb8 docs: Add assets/instructions for local docs build (#2453)
* ungitignore
* add docs/.vuepress to enable local builds
* config.js needs to be here, one less step
* docs: make spec in sidebar nicer
* docs: local build instructions
2018-09-21 13:39:55 +02:00
Anton Kaliaev
8d50bb9dad conesnsu: follow up to removing some consensus params (#2427)
* follow up to removing some consensus params Refs #2382
* change args type to int64 in state#makeParams
* make valsCount and evidenceCount ints again
* MaxEvidenceBytesPerBlock: include magic number in godoc
* [spec] creating a proposal
* test state#TxFilter
* panic if MaxDataBytes is less than 0
* fixes after review
* use amino#UvarintSize to calculate overhead
0c74291f3b/encoder.go (L85-L90)
* avoid cyclic imports
* you can do better Go, come on
* remove testdouble package
2018-09-21 11:00:36 +02:00
Ethan Buchman
7b727bf3d0 Minor changelog fixes (#2449)
* readme: add some libs to semver

* changelog: some updates
2018-09-20 11:55:36 -04:00
Aravind
84b518b8d3 rpc: Add /consensus_params endpoint (#2415)
* Add /consensus_params endpoint
* Incorporated change https://github.com/tendermint/tendermint/pull/2415#discussion_r219078049
* Fixed an error in pervious commit
2018-09-20 14:31:20 +02:00
Anton Kaliaev
bd951171db docs: Add missing changelog entry and comment (#2451)
Follow-up on https://github.com/tendermint/tendermint/pull/2411
2018-09-20 11:14:02 +02:00
Dev Ojha
0d6b75bd53 common: Delete unused functions (#2452)
These functions were not used anywhere within tendermint, or the
cosmos-sdk. (The functionality is already duplicated in the cosmos-sdk
types package)

* common: Delete unused functions within byteslice
* remove more unused code from strings.go and int.go
* Remove more unused code from int.go
* Fix testcase
2018-09-20 11:12:42 +02:00
ValarDragon
f76312ffe6 docs: Update secure-p2p doc to match the spec + current implementation
Closes #2421.
I am of the opinion that the spec is easier to read than this though,
and we shouldn't really explain this here other than that we use a variant
of station to station protocol, with X25519 for the diffie hellman, and we
describe the related security properties.
2018-09-20 09:56:15 +02:00
bradyjoestar
8aad09d9d4 Output error instead of panic when the given db_backend is not initialised (#2411)
Closes #2371
2018-09-20 09:53:25 +04:00
Ethan Buchman
faa3509646 adr-021: note about tag spacers (#2362) 2018-09-19 18:56:23 -04:00
Ethan Buchman
a045c562a2 update adr-016 (#2435) 2018-09-19 18:11:11 -04:00
bradyjoestar
26aa978456 Make mempool cache a proper LRU (#2407)
Closes #2399
2018-09-19 21:40:22 +04:00
Anton Kaliaev
aa5495f24e p2p: add RPCAddress to NodeInfoOther.String() (#2442)
- remove second AminoVersion from
https://github.com/tendermint/tendermint/pull/2426/files#diff-f2fefc7a06ea0d1e0a910196901e50aaR86

Refs #2433
2018-09-19 15:54:14 +02:00
Dev Ojha
c0cdb9d441 libs : Remove libs/common/word.go (#2431)
We didn't use this code anywhere in the codebase. As such, we probably
should reduce the surface area we support. In the event that we do
in fact require 256 bit words inside of tendermint, we should adapt
the stdlibs' internal word representations, which also handles SIMD.

Inside of the SDK, a separate solution for big ints / larger words
is employed, which uses big ints. This in turn does utilize the stdlibs
SIMD support.
2018-09-19 15:38:30 +02:00
Ethan Buchman
91a8767083 proxy: remove Handshaker from proxy pkg (#2437)
Handshaker was removed from proxy package so it can be called
independently of starting the abci app connections and can return a
result to the caller.
2018-09-19 15:35:09 +02:00
zhangzheng
3e099f75c7 minor note in indexing-transactions.md (#2443) (#2444)
Fix inconsistent documents and code
2018-09-19 13:46:37 +04:00
Alexander Simmerl
bdd01310a0 p2p: Integrate new Transport
We are swapping the exisiting listener implementation with the newly
introduced Transport and its default implementation MultiplexTransport,
removing a large chunk of old connection setup and handling scattered
over the Peer and Switch code. The Switch requires a Transport now and
handles externally passed Peer filters.
2018-09-18 22:26:43 +02:00
Alexander Simmerl
be5d68ea4f p2p: Implement PeerTransport
This is the implementation for the design described in ADR 12[0]. It's
the first step of a larger refactor of the p2p package as tracked in
interface bundling all concerns of low-level connection handling and
isolating the rest of peer lifecycle management from the specifics of
the low-level internet protocols. Even if the swappable implementation
will never be utilised, already the isolation of conn related code in
one place will help with the reasoning about execution path and
addressation of security sensitive issues surfaced through bounty
programs and audits.

We deliberately decided to not have Peer filtering and other management
in the Transport, its sole responsibility is the translation of
connections to Peers, handing those to the caller fully setup. It's the
responsibility of the caller to reject those and or keep track. Peer
filtering will take place in the Switch and can be inspected in a the
following commit.

This changeset additionally is an exercise in clean separation of logic
and other infrastructural concerns like logging and instrumentation. By
leveraging a clean and minimal interface. How this looks can be seen in
a follow-up change.

Design #2069[2]
Refs #2067[3]
Fixes #2047[4]
Fixes #2046[5]

changes:
* describe Transport interface
* implement new default Transport: MultiplexTransport
* test MultiplexTransport with new constraints
* implement ConnSet for concurrent management of net.Conn, synchronous
to PeerSet
* implement and expose duplicate IP filter
* implemnt TransportOption for optional parametirisation

[0] https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-012-peer-transport.md
[1] https://github.com/tendermint/tendermint/issues/2067
[2] https://github.com/tendermint/tendermint/pull/2069
[3] https://github.com/tendermint/tendermint/issues/2067
[4] https://github.com/tendermint/tendermint/issues/2047
[5] https://github.com/tendermint/tendermint/issues/2046
2018-09-18 22:26:43 +02:00
Anton Kaliaev
89462c52d9 spec: add missing field to NodeInfoOther (#2426)
and fix formatting

Refs https://github.com/tendermint/tendermint/pull/2417#discussion_r218080500
2018-09-18 11:28:32 +02:00
Zarko Milosevic
2fbf810cd8 Delay starting node until Genesis time (#2389) 2018-09-18 13:16:50 +04:00
Anton Kaliaev
64fc8f8157 Merge pull request #2337 from tendermint/anton/cleveldb
BlockIntervalSeconds metric & DB tests cleanup & benchmark improv.
2018-09-18 12:48:45 +04:00
Anton Kaliaev
e1bda36c6c switch back to original fork 2018-09-18 12:16:03 +04:00
Anton Kaliaev
ff9d0cdfb6 generate random txs
otherwise we're benchmarking overriding single key (because hash stays
the same!)
2018-09-18 12:16:03 +04:00
Anton Kaliaev
788474d08d change consensus_block_interval_seconds metric type to gauge
Otherwise, it's impossible to see outliers
https://github.com/tendermint/tendermint/issues/1835#issuecomment-402054099
2018-09-18 12:16:01 +04:00
Anton Kaliaev
484194789f update Vagrantfile to install go1.11 2018-09-18 12:15:44 +04:00
Anton Kaliaev
747797bf3b cleanup after tests! 2018-09-18 12:15:44 +04:00
Anton Kaliaev
76302c651f remove LICENSE from libs/db in favor of root license 2018-09-18 12:15:44 +04:00
Anton Kaliaev
5bfb9001eb switch from jmhodges/levigo to DataDog/leveldb
Why:
original fork is abandoned and not supported anymore.

Changes:
- LevelDB 1.19 (LevelDB and Snappy are both compiled and linked statically, so while you will not need them installed on your target machine, you should have a roughly compatible version of libstdc++.)
- snappy and lz4 libs included by default
2018-09-18 12:15:43 +04:00
Anton Kaliaev
38bced2440 [types] add Address to GenesisValidator (#2418)
Refs #1714
2018-09-18 11:59:52 +04:00
Zach
4fe9906361 docs: Update README (#2393)
* update DOCS_README
* add spec to docs & other lil fixes (#2402)
2018-09-17 18:43:10 +02:00
Anton Kaliaev
fc7f9bcaf6 rpc: Transform /status result.node_info.other into map (#2417)
* [rpc] transform /status result.node_info.other into map
* amino does not support maps, duh

Refs #2391
2018-09-17 18:39:52 +02:00
Ismail Khoffi
8ae3334423 [libs/autofile & db/fsdb] Throw error if file permissions change (#2286)
* Enforce file permissions in case they've changed

* test behaviour for autofile

* use testify in tests and rename `fInf` to `fileInfo`

* return an error if file permissions have changed

- if we can't read the file, we'll still panic

* get rid of "github.com/pkg/errors" dependency

* address review comments:

- prefix instead of suffix
- add state to err and construct formatting in Error() method

* address review comments:

- move error to libs/errors
2018-09-17 14:38:29 +04:00
zhangzheng
c6c0b52d0c tools/tm-bench: bounds check for txSize and improving test cases (#2410)
Fixes #2409
2018-09-17 13:08:47 +04:00
Anton Kaliaev
e3e3c13741 [common] revert started flag when service already stopped (#2326)
also, return ErrNotStarted when trying to stop a not-running service
2018-09-12 19:07:29 -04:00
Dev Ojha
1ea64fc27f Make mempool aware of MaxGas requirement (#2360)
* Make mempool aware of MaxGas requirement

* update spec

* Add tests

* Switch GasWanted from kv store to persistent kv store

* Fix typo in test name

* switch back to using kvstore, not persistent kv store
2018-09-12 16:41:19 -04:00
Anton Kaliaev
0e1cd88863 Remove ConsensusParams.TxSize and ConsensusParams.BlockGossip (#2364)
* remove ConsensusParams.TxSize and ConsensusParams.BlockGossip

Refs #2347

* block part size is now fixed

Refs #2347

* use max data size, not max bytes for tx limit

Refs #2347
2018-09-12 15:44:43 -04:00
Zach
33b4617e9a docs: update link to rpc (#2361)
* md links dont work in slate

* docs: link to rpc

* docs: use unsafe_reset_all

* do not advertise unsafe_reset_priv_validator
2018-09-12 21:03:17 +04:00
Ethan Buchman
503de8c9b8 docs/spec/abci: improve docs on AppHash (#2363) 2018-09-10 09:10:53 -04:00
xiaoping
dea4e96f66 fix docs links (#2352) 2018-09-10 10:42:48 +04:00
Ethan Buchman
a57aae7072 [ADR] ABCI errors and events (#2314)
* Start of ADR

* flesh out abci events and errors adrs

* adr: move 012 to 023

* adr-022: add note from cwgoes
2018-09-09 14:04:01 -04:00
Zach
0bec20a1e3 update readme, clsoes #2357 (#2359) 2018-09-08 16:45:12 +04:00
Ethan Buchman
83a7c04bce Merge pull request #2356 from tendermint/master
Merge master back to develop
2018-09-07 07:47:36 -04:00
Ethan Buchman
d419fffe18 Merge pull request #2350 from tendermint/release/v0.24.0
Release/v0.24.0
2018-09-07 07:46:44 -04:00
Ethan Buchman
c8895dab98 minor note in UPGRADING.md 2018-09-07 07:43:41 -04:00
Ethan Buchman
8b94deca73 Merge pull request #2354 from tendermint/anton/fix-typos-in-spec
Fix typos in spec
2018-09-07 07:28:50 -04:00
Ethan Buchman
4cd2e40fb1 TMBIN -> Amino 2018-09-07 07:28:58 -04:00
Anton Kaliaev
47dc4e6428 fix a few typos in spec 2018-09-07 11:40:16 +04:00
Ethan Buchman
94288006ba minor fixes 2018-09-06 22:47:05 -04:00
Ethan Buchman
22445a5029 Merge pull request #2349 from tendermint/release/v0.24.0
update UPGRADING.md and minor docs fixes
2018-09-06 22:25:20 -04:00
Ethan Buchman
299d46304d update UPGRADING.md and minor docs fixes 2018-09-06 22:35:31 -04:00
Ethan Buchman
5106af484f docs: add abci spec to config.js 2018-09-06 22:18:15 -04:00
Ethan Buchman
114c405120 docs/spec/abci: fixes and more from #2249 2018-09-06 22:17:00 -04:00
Ethan Buchman
246a56283a Merge pull request #2343 from tendermint/release/v0.24.0
Major spec update to prepare v0.24.0 for release
2018-09-06 20:44:54 -04:00
Ethan Buchman
1144e72c61 docs: refactor ABCI docs
* move spec/software/abci.md to spec/abci/apps.md and improve it
* move some of app-dev/app-development.md to spec/abci/client-server.md
2018-09-06 20:51:36 -04:00
Ethan Buchman
3fd54c5df5 docs/spec/abci: update spec
* better overview section
* section on tags
* remove notes about state/concurrency from CheckTx
* incorporate feedback from #2249
* explain how validator set updates effect future blocks
2018-09-06 19:36:30 -04:00
Ethan Buchman
20c55cffc4 docs: move app-dev/abci-spec.md to spec/abci/abci.md 2018-09-06 18:36:11 -04:00
Ethan Buchman
dea34506fb types/time: add note about stripping monotonic part 2018-09-06 17:58:52 -04:00
Ethan Buchman
54fe6ef73c version: add and bump abci version 2018-09-06 17:58:12 -04:00
Ethan Buchman
6fd79d1545 docs/spec/blockchain: remove tags from result for now 2018-09-06 12:47:26 -04:00
Ethan Buchman
8fcabe8b05 docs: fix note about ChainID size 2018-09-06 12:42:23 -04:00
Ethan Buchman
e0fa827a53 docs/spec/blockchain: specify consensus params in state.md 2018-09-06 12:41:57 -04:00
Ethan Buchman
bdf3238710 docs/spec/blockchain: bring blockchain.md up-to-date 2018-09-06 12:41:33 -04:00
Ethan Buchman
ed9e00a8a7 docs/spec/blockchain: fix encoding JSON 2018-09-06 12:41:02 -04:00
422 changed files with 25605 additions and 9154 deletions

View File

@@ -92,6 +92,7 @@ jobs:
command: |
export PATH="$GOBIN:$PATH"
make get_tools
make get_dev_tools
- run:
name: dependencies
command: |
@@ -301,6 +302,8 @@ jobs:
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- run: bash test/p2p/circleci.sh
- store_artifacts:
path: /home/circleci/project/test/p2p/logs
upload_coverage:
<<: *defaults

5
.gitignore vendored
View File

@@ -14,6 +14,7 @@ test/p2p/data/
test/logs
coverage.txt
docs/_build
docs/dist
*.log
abci-cli
docs/node_modules/
@@ -25,6 +26,8 @@ scripts/cutWALUntil/cutWALUntil
.idea/
*.iml
.vscode/
libs/pubsub/query/fuzz_test/output
shunit2
@@ -38,4 +41,4 @@ terraform.tfstate
terraform.tfstate.backup
terraform.tfstate.d
.vscode
.vscode

View File

@@ -1,5 +1,404 @@
# Changelog
## v0.27.0
*December 5th, 2018*
Special thanks to external contributors on this release:
@danil-lashin, @srmo
Special thanks to @dlguddus for discovering a [major
issue](https://github.com/tendermint/tendermint/issues/2718#issuecomment-440888677)
in the proposer selection algorithm.
Friendly reminder, we have a [bug bounty
program](https://hackerone.com/tendermint).
This release is primarily about fixes to the proposer selection algorithm
in preparation for the [Cosmos Game of
Stakes](https://blog.cosmos.network/the-game-of-stakes-is-open-for-registration-83a404746ee6).
It also makes use of the `ConsensusParams.Validator.PubKeyTypes` to restrict the
key types that can be used by validators, and removes the `Heartbeat` consensus
message.
### BREAKING CHANGES:
* CLI/RPC/Config
- [rpc] [\#2932](https://github.com/tendermint/tendermint/issues/2932) Rename `accum` to `proposer_priority`
* Go API
- [db] [\#2913](https://github.com/tendermint/tendermint/pull/2913)
ReverseIterator API change: start < end, and end is exclusive.
- [types] [\#2932](https://github.com/tendermint/tendermint/issues/2932) Rename `Validator.Accum` to `Validator.ProposerPriority`
* Blockchain Protocol
- [state] [\#2714](https://github.com/tendermint/tendermint/issues/2714) Validators can now only use pubkeys allowed within
ConsensusParams.Validator.PubKeyTypes
* P2P Protocol
- [consensus] [\#2871](https://github.com/tendermint/tendermint/issues/2871)
Remove *ProposalHeartbeat* message as it serves no real purpose (@srmo)
- [state] Fixes for proposer selection:
- [\#2785](https://github.com/tendermint/tendermint/issues/2785) Accum for new validators is `-1.125*totalVotingPower` instead of 0
- [\#2941](https://github.com/tendermint/tendermint/issues/2941) val.Accum is preserved during ValidatorSet.Update to avoid being
reset to 0
### IMPROVEMENTS:
- [state] [\#2929](https://github.com/tendermint/tendermint/issues/2929) Minor refactor of updateState logic (@danil-lashin)
- [node] \#2959 Allow node to start even if software's BlockProtocol is
different from state's BlockProtocol
- [pex] \#2959 Pex reactor logger uses `module=pex`
### BUG FIXES:
- [p2p] \#2968 Panic on transport error rather than continuing to run but not
accept new connections
- [p2p] \#2969 Fix mismatch in peer count between `/net_info` and the prometheus
metrics
- [rpc] \#2408 `/broadcast_tx_commit`: Fix "interface conversion: interface {} in nil, not EventDataTx" panic (could happen if somebody sent a tx using `/broadcast_tx_commit` while Tendermint was being stopped)
- [state] [\#2785](https://github.com/tendermint/tendermint/issues/2785) Fix accum for new validators to be `-1.125*totalVotingPower`
instead of 0, forcing them to wait before becoming the proposer. Also:
- do not batch clip
- keep accums averaged near 0
- [txindex/kv] [\#2925](https://github.com/tendermint/tendermint/issues/2925) Don't return false positives when range searching for a prefix of a tag value
- [types] [\#2938](https://github.com/tendermint/tendermint/issues/2938) Fix regression in v0.26.4 where we panic on empty
genDoc.Validators
- [types] [\#2941](https://github.com/tendermint/tendermint/issues/2941) Preserve val.Accum during ValidatorSet.Update to avoid it being
reset to 0 every time a validator is updated
## v0.26.4
*November 27th, 2018*
Special thanks to external contributors on this release:
@ackratos, @goolAdapter, @james-ray, @joe-bowman, @kostko,
@nagarajmanjunath, @tomtau
Friendly reminder, we have a [bug bounty
program](https://hackerone.com/tendermint).
### FEATURES:
- [rpc] [\#2747](https://github.com/tendermint/tendermint/issues/2747) Enable subscription to tags emitted from `BeginBlock`/`EndBlock` (@kostko)
- [types] [\#2747](https://github.com/tendermint/tendermint/issues/2747) Add `ResultBeginBlock` and `ResultEndBlock` fields to `EventDataNewBlock`
and `EventDataNewBlockHeader` to support subscriptions (@kostko)
- [types] [\#2918](https://github.com/tendermint/tendermint/issues/2918) Add Marshal, MarshalTo, Unmarshal methods to various structs
to support Protobuf compatibility (@nagarajmanjunath)
### IMPROVEMENTS:
- [config] [\#2877](https://github.com/tendermint/tendermint/issues/2877) Add `blocktime_iota` to the config.toml (@ackratos)
- NOTE: this should be a ConsensusParam, not part of the config, and will be
removed from the config at a later date
([\#2920](https://github.com/tendermint/tendermint/issues/2920).
- [mempool] [\#2882](https://github.com/tendermint/tendermint/issues/2882) Add txs from Update to cache
- [mempool] [\#2891](https://github.com/tendermint/tendermint/issues/2891) Remove local int64 counter from being stored in every tx
- [node] [\#2866](https://github.com/tendermint/tendermint/issues/2866) Add ability to instantiate IPCVal (@joe-bowman)
### BUG FIXES:
- [blockchain] [\#2731](https://github.com/tendermint/tendermint/issues/2731) Retry both blocks if either is bad to avoid getting stuck during fast sync (@goolAdapter)
- [consensus] [\#2893](https://github.com/tendermint/tendermint/issues/2893) Use genDoc.Validators instead of state.NextValidators on replay when appHeight==0 (@james-ray)
- [log] [\#2868](https://github.com/tendermint/tendermint/issues/2868) Fix `module=main` setting overriding all others
- NOTE: this changes the default logging behaviour to be much less verbose.
Set `log_level="info"` to restore the previous behaviour.
- [rpc] [\#2808](https://github.com/tendermint/tendermint/issues/2808) Fix `accum` field in `/validators` by calling `IncrementAccum` if necessary
- [rpc] [\#2811](https://github.com/tendermint/tendermint/issues/2811) Allow integer IDs in JSON-RPC requests (@tomtau)
- [txindex/kv] [\#2759](https://github.com/tendermint/tendermint/issues/2759) Fix tx.height range queries
- [txindex/kv] [\#2775](https://github.com/tendermint/tendermint/issues/2775) Order tx results by index if height is the same
- [txindex/kv] [\#2908](https://github.com/tendermint/tendermint/issues/2908) Don't return false positives when searching for a prefix of a tag value
## v0.26.3
*November 17th, 2018*
Special thanks to external contributors on this release:
@danil-lashin, @kevlubkcm, @krhubert, @srmo
Friendly reminder, we have a [bug bounty
program](https://hackerone.com/tendermint).
### BREAKING CHANGES:
* Go API
- [rpc] [\#2791](https://github.com/tendermint/tendermint/issues/2791) Functions that start HTTP servers are now blocking:
- Impacts `StartHTTPServer`, `StartHTTPAndTLSServer`, and `StartGRPCServer`
- These functions now take a `net.Listener` instead of an address
- [rpc] [\#2767](https://github.com/tendermint/tendermint/issues/2767) Subscribing to events
`NewRound` and `CompleteProposal` return new types `EventDataNewRound` and
`EventDataCompleteProposal`, respectively, instead of the generic `EventDataRoundState`. (@kevlubkcm)
### FEATURES:
- [log] [\#2843](https://github.com/tendermint/tendermint/issues/2843) New `log_format` config option, which can be set to 'plain' for colored
text or 'json' for JSON output
- [types] [\#2767](https://github.com/tendermint/tendermint/issues/2767) New event types EventDataNewRound (with ProposerInfo) and EventDataCompleteProposal (with BlockID). (@kevlubkcm)
### IMPROVEMENTS:
- [dep] [\#2844](https://github.com/tendermint/tendermint/issues/2844) Dependencies are no longer pinned to an exact version in the
Gopkg.toml:
- Serialization libs are allowed to vary by patch release
- Other libs are allowed to vary by minor release
- [p2p] [\#2857](https://github.com/tendermint/tendermint/issues/2857) "Send failed" is logged at debug level instead of error.
- [rpc] [\#2780](https://github.com/tendermint/tendermint/issues/2780) Add read and write timeouts to HTTP servers
- [state] [\#2848](https://github.com/tendermint/tendermint/issues/2848) Make "Update to validators" msg value pretty (@danil-lashin)
### BUG FIXES:
- [consensus] [\#2819](https://github.com/tendermint/tendermint/issues/2819) Don't send proposalHearbeat if not a validator
- [docs] [\#2859](https://github.com/tendermint/tendermint/issues/2859) Fix ConsensusParams details in spec
- [libs/autofile] [\#2760](https://github.com/tendermint/tendermint/issues/2760) Comment out autofile permissions check - should fix
running Tendermint on Windows
- [p2p] [\#2869](https://github.com/tendermint/tendermint/issues/2869) Set connection config properly instead of always using default
- [p2p/pex] [\#2802](https://github.com/tendermint/tendermint/issues/2802) Seed mode fixes:
- Only disconnect from inbound peers
- Use FlushStop instead of Sleep to ensure all messages are sent before
disconnecting
## v0.26.2
*November 15th, 2018*
Special thanks to external contributors on this release: @hleb-albau, @zhuzeyu
Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint).
### FEATURES:
- [rpc] [\#2582](https://github.com/tendermint/tendermint/issues/2582) Enable CORS on RPC API (@hleb-albau)
### BUG FIXES:
- [abci] [\#2748](https://github.com/tendermint/tendermint/issues/2748) Unlock mutex in localClient so even when app panics (e.g. during CheckTx), consensus continue working
- [abci] [\#2748](https://github.com/tendermint/tendermint/issues/2748) Fix DATA RACE in localClient
- [amino] [\#2822](https://github.com/tendermint/tendermint/issues/2822) Update to v0.14.1 to support compiling on 32-bit platforms
- [rpc] [\#2748](https://github.com/tendermint/tendermint/issues/2748) Drain channel before calling Unsubscribe(All) in `/broadcast_tx_commit`
## v0.26.1
*November 11, 2018*
Special thanks to external contributors on this release: @katakonst
Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint).
### IMPROVEMENTS:
- [consensus] [\#2704](https://github.com/tendermint/tendermint/issues/2704) Simplify valid POL round logic
- [docs] [\#2749](https://github.com/tendermint/tendermint/issues/2749) Deduplicate some ABCI docs
- [mempool] More detailed log messages
- [\#2724](https://github.com/tendermint/tendermint/issues/2724)
- [\#2762](https://github.com/tendermint/tendermint/issues/2762)
### BUG FIXES:
- [autofile] [\#2703](https://github.com/tendermint/tendermint/issues/2703) Do not panic when checking Head size
- [crypto/merkle] [\#2756](https://github.com/tendermint/tendermint/issues/2756) Fix crypto/merkle ProofOperators.Verify to check bounds on keypath parts.
- [mempool] fix a bug where we create a WAL despite `wal_dir` being empty
- [p2p] [\#2771](https://github.com/tendermint/tendermint/issues/2771) Fix `peer-id` label name to `peer_id` in prometheus metrics
- [p2p] [\#2797](https://github.com/tendermint/tendermint/pull/2797) Fix IDs in peer NodeInfo and require them for addresses
in AddressBook
- [p2p] [\#2797](https://github.com/tendermint/tendermint/pull/2797) Do not close conn immediately after sending pex addrs in seed mode. Partial fix for [\#2092](https://github.com/tendermint/tendermint/issues/2092).
## v0.26.0
*November 2, 2018*
Special thanks to external contributors on this release:
@bradyjoestar, @connorwstein, @goolAdapter, @HaoyangLiu,
@james-ray, @overbool, @phymbert, @Slamper, @Uzair1995, @yutianwu.
Special thanks to @Slamper for a series of bug reports in our [bug bounty
program](https://hackerone.com/tendermint) which are fixed in this release.
This release is primarily about adding Version fields to various data structures,
optimizing consensus messages for signing and verification in
restricted environments (like HSMs and the Ethereum Virtual Machine), and
aligning the consensus code with the [specification](https://arxiv.org/abs/1807.04938).
It also includes our first take at a generalized merkle proof system, and
changes the length of hashes used for hashing data structures from 20 to 32
bytes.
See the [UPGRADING.md](UPGRADING.md#v0.26.0) for details on upgrading to the new
version.
Please note that we are still making breaking changes to the protocols.
While the new Version fields should help us to keep the software backwards compatible
even while upgrading the protocols, we cannot guarantee that new releases will
be compatible with old chains just yet. We expect there will be another breaking
release or two before the Cosmos Hub launch, but we will otherwise be paying
increasing attention to backwards compatibility. Thanks for bearing with us!
### BREAKING CHANGES:
* CLI/RPC/Config
* [config] [\#2232](https://github.com/tendermint/tendermint/issues/2232) Timeouts are now strings like "3s" and "100ms", not ints
* [config] [\#2505](https://github.com/tendermint/tendermint/issues/2505) Remove Mempool.RecheckEmpty (it was effectively useless anyways)
* [config] [\#2490](https://github.com/tendermint/tendermint/issues/2490) `mempool.wal` is disabled by default
* [privval] [\#2459](https://github.com/tendermint/tendermint/issues/2459) Split `SocketPVMsg`s implementations into Request and Response, where the Response may contain a error message (returned by the remote signer)
* [state] [\#2644](https://github.com/tendermint/tendermint/issues/2644) Add Version field to State, breaking the format of State as
encoded on disk.
* [rpc] [\#2298](https://github.com/tendermint/tendermint/issues/2298) `/abci_query` takes `prove` argument instead of `trusted` and switches the default
behaviour to `prove=false`
* [rpc] [\#2654](https://github.com/tendermint/tendermint/issues/2654) Remove all `node_info.other.*_version` fields in `/status` and
`/net_info`
* [rpc] [\#2636](https://github.com/tendermint/tendermint/issues/2636) Remove
`_params` suffix from fields in `consensus_params`.
* Apps
* [abci] [\#2298](https://github.com/tendermint/tendermint/issues/2298) ResponseQuery.Proof is now a structured merkle.Proof, not just
arbitrary bytes
* [abci] [\#2644](https://github.com/tendermint/tendermint/issues/2644) Add Version to Header and shift all fields by one
* [abci] [\#2662](https://github.com/tendermint/tendermint/issues/2662) Bump the field numbers for some `ResponseInfo` fields to make room for
`AppVersion`
* [abci] [\#2636](https://github.com/tendermint/tendermint/issues/2636) Updates to ConsensusParams
* Remove `Params` suffix from field names
* Add `Params` suffix to message types
* Add new field and type, `Validator ValidatorParams`, to control what types of validator keys are allowed.
* Go API
* [config] [\#2232](https://github.com/tendermint/tendermint/issues/2232) Timeouts are time.Duration, not ints
* [crypto/merkle & lite] [\#2298](https://github.com/tendermint/tendermint/issues/2298) Various changes to accomodate General Merkle trees
* [crypto/merkle] [\#2595](https://github.com/tendermint/tendermint/issues/2595) Remove all Hasher objects in favor of byte slices
* [crypto/merkle] [\#2635](https://github.com/tendermint/tendermint/issues/2635) merkle.SimpleHashFromTwoHashes is no longer exported
* [node] [\#2479](https://github.com/tendermint/tendermint/issues/2479) Remove node.RunForever
* [rpc/client] [\#2298](https://github.com/tendermint/tendermint/issues/2298) `ABCIQueryOptions.Trusted` -> `ABCIQueryOptions.Prove`
* [types] [\#2298](https://github.com/tendermint/tendermint/issues/2298) Remove `Index` and `Total` fields from `TxProof`.
* [types] [\#2598](https://github.com/tendermint/tendermint/issues/2598)
`VoteTypeXxx` are now of type `SignedMsgType byte` and named `XxxType`, eg.
`PrevoteType`, `PrecommitType`.
* [types] [\#2636](https://github.com/tendermint/tendermint/issues/2636) Rename fields in ConsensusParams to remove `Params` suffixes
* [types] [\#2735](https://github.com/tendermint/tendermint/issues/2735) Simplify Proposal message to align with spec
* Blockchain Protocol
* [crypto/tmhash] [\#2732](https://github.com/tendermint/tendermint/issues/2732) TMHASH is now full 32-byte SHA256
* All hashes in the block header and Merkle trees are now 32-bytes
* PubKey Addresses are still only 20-bytes
* [state] [\#2587](https://github.com/tendermint/tendermint/issues/2587) Require block.Time of the fist block to be genesis time
* [state] [\#2644](https://github.com/tendermint/tendermint/issues/2644) Require block.Version to match state.Version
* [types] Update SignBytes for `Vote`/`Proposal`/`Heartbeat`:
* [\#2459](https://github.com/tendermint/tendermint/issues/2459) Use amino encoding instead of JSON in `SignBytes`.
* [\#2598](https://github.com/tendermint/tendermint/issues/2598) Reorder fields and use fixed sized encoding.
* [\#2598](https://github.com/tendermint/tendermint/issues/2598) Change `Type` field from `string` to `byte` and use new
`SignedMsgType` to enumerate.
* [types] [\#2730](https://github.com/tendermint/tendermint/issues/2730) Use
same order for fields in `Vote` as in the SignBytes
* [types] [\#2732](https://github.com/tendermint/tendermint/issues/2732) Remove the address field from the validator hash
* [types] [\#2644](https://github.com/tendermint/tendermint/issues/2644) Add Version struct to Header
* [types] [\#2609](https://github.com/tendermint/tendermint/issues/2609) ConsensusParams.Hash() is the hash of the amino encoded
struct instead of the Merkle tree of the fields
* [types] [\#2670](https://github.com/tendermint/tendermint/issues/2670) Header.Hash() builds Merkle tree out of fields in the same
order they appear in the header, instead of sorting by field name
* [types] [\#2682](https://github.com/tendermint/tendermint/issues/2682) Use proto3 `varint` encoding for ints that are usually unsigned (instead of zigzag encoding).
* [types] [\#2636](https://github.com/tendermint/tendermint/issues/2636) Add Validator field to ConsensusParams
(Used to control which pubkey types validators can use, by abci type).
* P2P Protocol
* [consensus] [\#2652](https://github.com/tendermint/tendermint/issues/2652)
Replace `CommitStepMessage` with `NewValidBlockMessage`
* [consensus] [\#2735](https://github.com/tendermint/tendermint/issues/2735) Simplify `Proposal` message to align with spec
* [consensus] [\#2730](https://github.com/tendermint/tendermint/issues/2730)
Add `Type` field to `Proposal` and use same order of fields as in the
SignBytes for both `Proposal` and `Vote`
* [p2p] [\#2654](https://github.com/tendermint/tendermint/issues/2654) Add `ProtocolVersion` struct with protocol versions to top of
DefaultNodeInfo and require `ProtocolVersion.Block` to match during peer handshake
### FEATURES:
- [abci] [\#2557](https://github.com/tendermint/tendermint/issues/2557) Add `Codespace` field to `Response{CheckTx, DeliverTx, Query}`
- [abci] [\#2662](https://github.com/tendermint/tendermint/issues/2662) Add `BlockVersion` and `P2PVersion` to `RequestInfo`
- [crypto/merkle] [\#2298](https://github.com/tendermint/tendermint/issues/2298) General Merkle Proof scheme for chaining various types of Merkle trees together
- [docs/architecture] [\#1181](https://github.com/tendermint/tendermint/issues/1181) S
plit immutable and mutable parts of priv_validator.json
### IMPROVEMENTS:
- Additional Metrics
- [consensus] [\#2169](https://github.com/cosmos/cosmos-sdk/issues/2169)
- [p2p] [\#2169](https://github.com/cosmos/cosmos-sdk/issues/2169)
- [config] [\#2232](https://github.com/tendermint/tendermint/issues/2232) Added ValidateBasic method, which performs basic checks
- [crypto/ed25519] [\#2558](https://github.com/tendermint/tendermint/issues/2558) Switch to use latest `golang.org/x/crypto` through our fork at
github.com/tendermint/crypto
- [libs/log] [\#2707](https://github.com/tendermint/tendermint/issues/2707) Add year to log format (@yutianwu)
- [tools] [\#2238](https://github.com/tendermint/tendermint/issues/2238) Binary dependencies are now locked to a specific git commit
### BUG FIXES:
- [\#2711](https://github.com/tendermint/tendermint/issues/2711) Validate all incoming reactor messages. Fixes various bugs due to negative ints.
- [autofile] [\#2428](https://github.com/tendermint/tendermint/issues/2428) Group.RotateFile need call Flush() before rename (@goolAdapter)
- [common] [\#2533](https://github.com/tendermint/tendermint/issues/2533) Fixed a bug in the `BitArray.Or` method
- [common] [\#2506](https://github.com/tendermint/tendermint/issues/2506) Fixed a bug in the `BitArray.Sub` method (@james-ray)
- [common] [\#2534](https://github.com/tendermint/tendermint/issues/2534) Fix `BitArray.PickRandom` to choose uniformly from true bits
- [consensus] [\#1690](https://github.com/tendermint/tendermint/issues/1690) Wait for
timeoutPrecommit before starting next round
- [consensus] [\#1745](https://github.com/tendermint/tendermint/issues/1745) Wait for
Proposal or timeoutProposal before entering prevote
- [consensus] [\#2642](https://github.com/tendermint/tendermint/issues/2642) Only propose ValidBlock, not LockedBlock
- [consensus] [\#2642](https://github.com/tendermint/tendermint/issues/2642) Initialized ValidRound and LockedRound to -1
- [consensus] [\#1637](https://github.com/tendermint/tendermint/issues/1637) Limit the amount of evidence that can be included in a
block
- [consensus] [\#2652](https://github.com/tendermint/tendermint/issues/2652) Ensure valid block property with faulty proposer
- [evidence] [\#2515](https://github.com/tendermint/tendermint/issues/2515) Fix db iter leak (@goolAdapter)
- [libs/event] [\#2518](https://github.com/tendermint/tendermint/issues/2518) Fix event concurrency flaw (@goolAdapter)
- [node] [\#2434](https://github.com/tendermint/tendermint/issues/2434) Make node respond to signal interrupts while sleeping for genesis time
- [state] [\#2616](https://github.com/tendermint/tendermint/issues/2616) Pass nil to NewValidatorSet() when genesis file's Validators field is nil
- [p2p] [\#2555](https://github.com/tendermint/tendermint/issues/2555) Fix p2p switch FlushThrottle value (@goolAdapter)
- [p2p] [\#2668](https://github.com/tendermint/tendermint/issues/2668) Reconnect to originally dialed address (not self-reported address) for persistent peers
## v0.25.0
*September 22, 2018*
Special thanks to external contributors on this release:
@scriptionist, @bradyjoestar, @WALL-E
This release is mostly about the ConsensusParams - removing fields and enforcing MaxGas.
It also addresses some issues found via security audit, removes various unused
functions from `libs/common`, and implements
[ADR-012](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-012-peer-transport.md).
Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint).
BREAKING CHANGES:
* CLI/RPC/Config
* [rpc] [\#2391](https://github.com/tendermint/tendermint/issues/2391) /status `result.node_info.other` became a map
* [types] [\#2364](https://github.com/tendermint/tendermint/issues/2364) Remove `TxSize` and `BlockGossip` from `ConsensusParams`
* Maximum tx size is now set implicitly via the `BlockSize.MaxBytes`
* The size of block parts in the consensus is now fixed to 64kB
* Apps
* [mempool] [\#2360](https://github.com/tendermint/tendermint/issues/2360) Mempool tracks the `ResponseCheckTx.GasWanted` and
`ConsensusParams.BlockSize.MaxGas` and enforces:
- `GasWanted <= MaxGas` for every tx
- `(sum of GasWanted in block) <= MaxGas` for block proposal
* Go API
* [libs/common] [\#2431](https://github.com/tendermint/tendermint/issues/2431) Remove Word256 due to lack of use
* [libs/common] [\#2452](https://github.com/tendermint/tendermint/issues/2452) Remove the following functions due to lack of use:
* byteslice.go: cmn.IsZeros, cmn.RightPadBytes, cmn.LeftPadBytes, cmn.PrefixEndBytes
* strings.go: cmn.IsHex, cmn.StripHex
* int.go: Uint64Slice, all put/get int64 methods
FEATURES:
- [rpc] [\#2415](https://github.com/tendermint/tendermint/issues/2415) New `/consensus_params?height=X` endpoint to query the consensus
params at any height (@scriptonist)
- [types] [\#1714](https://github.com/tendermint/tendermint/issues/1714) Add Address to GenesisValidator
- [metrics] [\#2337](https://github.com/tendermint/tendermint/issues/2337) `consensus.block_interval_metrics` is now gauge, not histogram (you will be able to see spikes, if any)
- [libs] [\#2286](https://github.com/tendermint/tendermint/issues/2286) Panic if `autofile` or `db/fsdb` permissions change from 0600.
IMPROVEMENTS:
- [libs/db] [\#2371](https://github.com/tendermint/tendermint/issues/2371) Output error instead of panic when the given `db_backend` is not initialised (@bradyjoestar)
- [mempool] [\#2399](https://github.com/tendermint/tendermint/issues/2399) Make mempool cache a proper LRU (@bradyjoestar)
- [p2p] [\#2126](https://github.com/tendermint/tendermint/issues/2126) Introduce PeerTransport interface to improve isolation of concerns
- [libs/common] [\#2326](https://github.com/tendermint/tendermint/issues/2326) Service returns ErrNotStarted
BUG FIXES:
- [node] [\#2294](https://github.com/tendermint/tendermint/issues/2294) Delay starting node until Genesis time
- [consensus] [\#2048](https://github.com/tendermint/tendermint/issues/2048) Correct peer statistics for marking peer as good
- [rpc] [\#2460](https://github.com/tendermint/tendermint/issues/2460) StartHTTPAndTLSServer() now passes StartTLS() errors back to the caller rather than hanging forever.
- [p2p] [\#2047](https://github.com/tendermint/tendermint/issues/2047) Accept new connections asynchronously
- [tm-bench] [\#2410](https://github.com/tendermint/tendermint/issues/2410) Enforce minimum transaction size (@WALL-E)
## 0.24.0
*September 6th, 2018*
@@ -9,9 +408,13 @@ peerlink, Ahmah2009, bluele, b00f.
This release includes breaking upgrades in the block header,
including the long awaited changes for delaying validator set updates by one
block. It also fixes enforcement on the max size of blocks, and includes a BFT
timestamp in each block that can be safely used by applications. There are also some
minor breaking changes to the rpc, config, and ABCI.
block to better support light clients.
It also fixes enforcement on the maximum size of blocks, and includes a BFT
timestamp in each block that can be safely used by applications.
There are also some minor breaking changes to the rpc, config, and ABCI.
See the [UPGRADING.md](UPGRADING.md#v0.24.0) for details on upgrading to the new
version.
From here on, breaking changes will be broken down to better reflect how users
are affected by a change.
@@ -20,6 +423,8 @@ A few more breaking changes are in the works - each will come with a clear
Architecture Decision Record (ADR) explaining the change. You can review ADRs
[here](https://github.com/tendermint/tendermint/tree/develop/docs/architecture)
or in the [open Pull Requests](https://github.com/tendermint/tendermint/pulls).
You can also check in on the [issues marked as
breaking](https://github.com/tendermint/tendermint/issues?q=is%3Aopen+is%3Aissue+label%3Abreaking).
BREAKING CHANGES:
@@ -84,6 +489,7 @@ FEATURES:
IMPROVEMENTS:
- [docs] Lint documentation with `write-good` and `stop-words`.
- [docs] [\#2249](https://github.com/tendermint/tendermint/issues/2249) Refactor, deduplicate, and improve the ABCI docs and spec (with thanks to @ttmc).
- [scripts] [\#2196](https://github.com/tendermint/tendermint/issues/2196) Added json2wal tool, which is supposed to help our users restore (@bradyjoestar)
corrupted WAL files and compose test WAL files (@bradyjoestar)
- [mempool] [\#2234](https://github.com/tendermint/tendermint/issues/2234) Now stores txs by hash inside of the cache, to mitigate memory leakage
@@ -102,8 +508,8 @@ BUG FIXES:
*August 22nd, 2018*
BUG FIXES:
- [libs/autofile] \#2261 Fix log rotation so it actually happens.
- Fixes issues with consensus WAL growing unbounded ala \#2259
- [libs/autofile] [\#2261](https://github.com/tendermint/tendermint/issues/2261) Fix log rotation so it actually happens.
- Fixes issues with consensus WAL growing unbounded ala [\#2259](https://github.com/tendermint/tendermint/issues/2259)
## 0.23.0

View File

@@ -1,8 +1,10 @@
# Pending
## v0.27.1
Special thanks to external contributors with PRs included in this release:
*TBD*
BREAKING CHANGES:
Special thanks to external contributors on this release:
### BREAKING CHANGES:
* CLI/RPC/Config
@@ -14,9 +16,8 @@ BREAKING CHANGES:
* P2P Protocol
### FEATURES:
FEATURES:
### IMPROVEMENTS:
IMPROVEMENTS:
BUG FIXES:
### BUG FIXES:

View File

@@ -6,7 +6,7 @@ This code of conduct applies to all projects run by the Tendermint/COSMOS team a
# Conduct
## Contact: adrian@tendermint.com
## Contact: conduct@tendermint.com
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.

View File

@@ -27,8 +27,8 @@ Of course, replace `ebuchman` with your git handle.
To pull in updates from the origin repo, run
* `git fetch upstream`
* `git rebase upstream/master` (or whatever branch you want)
* `git fetch upstream`
* `git rebase upstream/master` (or whatever branch you want)
Please don't make Pull Requests to `master`.
@@ -50,6 +50,11 @@ as apps, tools, and the core, should use dep.
Run `dep status` to get a list of vendor dependencies that may not be
up-to-date.
When updating dependencies, please only update the particular dependencies you
need. Instead of running `dep ensure -update`, which will update anything,
specify exactly the dependency you want to update, eg.
`dep ensure -update github.com/tendermint/go-amino`.
## Vagrant
If you are a [Vagrant](https://www.vagrantup.com/) user, you can get started
@@ -64,43 +69,74 @@ vagrant ssh
make test
```
## Testing
## Changelog
All repos should be hooked up to [CircleCI](https://circleci.com/).
Every fix, improvement, feature, or breaking change should be made in a
pull-request that includes an update to the `CHANGELOG_PENDING.md` file.
If they have `.go` files in the root directory, they will be automatically
tested by circle using `go test -v -race ./...`. If not, they will need a
`circle.yml`. Ideally, every repo has a `Makefile` that defines `make test` and
includes its continuous integration status using a badge in the `README.md`.
Changelog entries should be formatted as follows:
```
- [module] \#xxx Some description about the change (@contributor)
```
Here, `module` is the part of the code that changed (typically a
top-level Go package), `xxx` is the pull-request number, and `contributor`
is the author/s of the change.
It's also acceptable for `xxx` to refer to the relevent issue number, but pull-request
numbers are preferred.
Note this means pull-requests should be opened first so the changelog can then
be updated with the pull-request's number.
There is no need to include the full link, as this will be added
automatically during release. But please include the backslash and pound, eg. `\#2313`.
Changelog entries should be ordered alphabetically according to the
`module`, and numerically according to the pull-request number.
Changes with multiple classifications should be doubly included (eg. a bug fix
that is also a breaking change should be recorded under both).
Breaking changes are further subdivided according to the APIs/users they impact.
Any change that effects multiple APIs/users should be recorded multiply - for
instance, a change to the `Blockchain Protocol` that removes a field from the
header should also be recorded under `CLI/RPC/Config` since the field will be
removed from the header in rpc responses as well.
## Branching Model and Release
User-facing repos should adhere to the branching model: http://nvie.com/posts/a-successful-git-branching-model/.
That is, these repos should be well versioned, and any merge to master requires a version bump and tagged release.
Libraries need not follow the model strictly, but would be wise to,
especially `go-p2p` and `go-rpc`, as their versions are referenced in tendermint core.
All repos should adhere to the branching model: http://nvie.com/posts/a-successful-git-branching-model/.
This means that all pull-requests should be made against develop. Any merge to
master constitutes a tagged release.
### Development Procedure:
- the latest state of development is on `develop`
- `develop` must never fail `make test`
- no --force onto `develop` (except when reverting a broken commit, which should seldom happen)
- never --force onto `develop` (except when reverting a broken commit, which should seldom happen)
- create a development branch either on github.com/tendermint/tendermint, or your fork (using `git remote add origin`)
- before submitting a pull request, begin `git rebase` on top of `develop`
- make changes and update the `CHANGELOG_PENDING.md` to record your change
- before submitting a pull request, run `git rebase` on top of the latest `develop`
### Pull Merge Procedure:
- ensure pull branch is rebased on develop
- ensure pull branch is based on a recent develop
- run `make test` to ensure that all tests pass
- merge pull request
- the `unstable` branch may be used to aggregate pull merges before testing once
- push master may request that pull requests be rebased on top of `unstable`
- the `unstable` branch may be used to aggregate pull merges before fixing tests
### Release Procedure:
- start on `develop`
- run integration tests (see `test_integrations` in Makefile)
- prepare changelog/release issue
- prepare changelog:
- copy `CHANGELOG_PENDING.md` to top of `CHANGELOG.md`
- run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for
all issues
- run `bash ./scripts/authors.sh` to get a list of authors since the latest
release, and add the github aliases of external contributors to the top of
the changelog. To lookup an alias from an email, try `bash
./scripts/authors.sh <email>`
- reset the `CHANGELOG_PENDING.md`
- bump versions
- push to release-vX.X.X to run the extended integration tests on the CI
- push to release/vX.X.X to run the extended integration tests on the CI
- merge to master
- merge master back to develop
@@ -115,3 +151,13 @@ especially `go-p2p` and `go-rpc`, as their versions are referenced in tendermint
- merge hotfix-vX.X.X to master
- merge hotfix-vX.X.X to develop
- delete the hotfix-vX.X.X branch
## Testing
All repos should be hooked up to [CircleCI](https://circleci.com/).
If they have `.go` files in the root directory, they will be automatically
tested by circle using `go test -v -race ./...`. If not, they will need a
`circle.yml`. Ideally, every repo has a `Makefile` that defines `make test` and
includes its continuous integration status using a badge in the `README.md`.

124
Gopkg.lock generated
View File

@@ -11,11 +11,11 @@
[[projects]]
branch = "master"
digest = "1:2c00f064ba355903866cbfbf3f7f4c0fe64af6638cc7d1b8bdcf3181bc67f1d8"
digest = "1:c0decf632843204d2b8781de7b26e7038584e2dcccc7e2f401e88ae85b1df2b7"
name = "github.com/btcsuite/btcd"
packages = ["btcec"]
pruneopts = "UT"
revision = "f5e261fc9ec3437697fb31d8b38453c293204b29"
revision = "67e573d211ace594f1366b4ce9d39726c4b19bd0"
[[projects]]
digest = "1:1d8e1cb71c33a9470bbbae09bfec09db43c6bf358dfcae13cd8807c4e2a9a2bf"
@@ -28,19 +28,12 @@
revision = "d4cc87b860166d00d6b5b9e0d3b3d71d6088d4d4"
[[projects]]
digest = "1:a2c1d0e43bd3baaa071d1b9ed72c27d78169b2b269f71c105ac4ba34b1be4a39"
digest = "1:ffe9824d294da03b391f44e1ae8281281b4afc1bdaa9588c9097785e3af10cec"
name = "github.com/davecgh/go-spew"
packages = ["spew"]
pruneopts = "UT"
revision = "346938d642f2ec3594ed81d874461961cd0faa76"
version = "v1.1.0"
[[projects]]
digest = "1:c7644c73a3d23741fdba8a99b1464e021a224b7e205be497271a8003a15ca41b"
name = "github.com/ebuchman/fail-test"
packages = ["."]
pruneopts = "UT"
revision = "95f809107225be108efcf10a3509e4ea6ceef3c4"
revision = "8991bc29aa16c548c550c7ff78260e27b9ab7c73"
version = "v1.1.1"
[[projects]]
digest = "1:544229a3ca0fb2dd5ebc2896d3d2ff7ce096d9751635301e44e37e761349ee70"
@@ -83,12 +76,12 @@
version = "v0.3.0"
[[projects]]
digest = "1:c4a2528ccbcabf90f9f3c464a5fc9e302d592861bbfd0b7135a7de8a943d0406"
digest = "1:586ea76dbd0374d6fb649a91d70d652b7fe0ccffb8910a77468e7702e7901f3d"
name = "github.com/go-stack/stack"
packages = ["."]
pruneopts = "UT"
revision = "259ab82a6cad3992b4e21ff5cac294ccb06474bc"
version = "v1.7.0"
revision = "2fee6af1a9795aafbe0253a0cfbdf668e1fb8a9a"
version = "v1.8.0"
[[projects]]
digest = "1:35621fe20f140f05a0c4ef662c26c0ab4ee50bca78aa30fe87d33120bd28165e"
@@ -136,8 +129,7 @@
version = "v1.2.0"
[[projects]]
branch = "master"
digest = "1:12247a2e99a060cc692f6680e5272c8adf0b8f572e6bce0d7095e624c958a240"
digest = "1:ea40c24cdbacd054a6ae9de03e62c5f252479b96c716375aace5c120d68647c8"
name = "github.com/hashicorp/hcl"
packages = [
".",
@@ -151,7 +143,8 @@
"json/token",
]
pruneopts = "UT"
revision = "ef8a98b0bbce4a65b5aa4c368430a80ddc533168"
revision = "8cb6e5b959231cc1119e43259c4a608f9c51a241"
version = "v1.0.0"
[[projects]]
digest = "1:870d441fe217b8e689d7949fef6e43efbc787e50f200cb1e70dbca9204a1d6be"
@@ -193,12 +186,12 @@
version = "v1.0.1"
[[projects]]
branch = "master"
digest = "1:5ab79470a1d0fb19b041a624415612f8236b3c06070161a910562f2b2d064355"
digest = "1:53bc4cd4914cd7cd52139990d5170d6dc99067ae31c56530621b18b35fc30318"
name = "github.com/mitchellh/mapstructure"
packages = ["."]
pruneopts = "UT"
revision = "f15292f7a699fcc1a38a80977f80a046874ba8ac"
revision = "3536a929edddb9a5b34bd6861dc4a9647cb459fe"
version = "v1.1.2"
[[projects]]
digest = "1:95741de3af260a92cc5c7f3f3061e85273f5a81b5db20d4bd68da74bd521675e"
@@ -225,14 +218,16 @@
version = "v1.0.0"
[[projects]]
digest = "1:c1a04665f9613e082e1209cf288bf64f4068dcd6c87a64bf1c4ff006ad422ba0"
digest = "1:26663fafdea73a38075b07e8e9d82fc0056379d2be8bb4e13899e8fda7c7dd23"
name = "github.com/prometheus/client_golang"
packages = [
"prometheus",
"prometheus/internal",
"prometheus/promhttp",
]
pruneopts = "UT"
revision = "ae27198cdd90bf12cd134ad79d1366a6cf49f632"
revision = "abad2d1bd44235a26707c172eab6bca5bf2dbad3"
version = "v0.9.1"
[[projects]]
branch = "master"
@@ -244,7 +239,7 @@
[[projects]]
branch = "master"
digest = "1:63b68062b8968092eb86bedc4e68894bd096ea6b24920faca8b9dcf451f54bb5"
digest = "1:db712fde5d12d6cdbdf14b777f0c230f4ff5ab0be8e35b239fc319953ed577a4"
name = "github.com/prometheus/common"
packages = [
"expfmt",
@@ -252,11 +247,11 @@
"model",
]
pruneopts = "UT"
revision = "c7de2306084e37d54b8be01f3541a8464345e9a5"
revision = "7e9e6cabbd393fc208072eedef99188d0ce788b6"
[[projects]]
branch = "master"
digest = "1:8c49953a1414305f2ff5465147ee576dd705487c35b15918fcd4efdc0cb7a290"
digest = "1:ef74914912f99c79434d9c09658274678bc85080ebe3ab32bec3940ebce5e1fc"
name = "github.com/prometheus/procfs"
packages = [
".",
@@ -265,7 +260,7 @@
"xfs",
]
pruneopts = "UT"
revision = "05ee40e3a273f7245e8777337fc7b46e533a9a92"
revision = "185b4288413d2a0dd0806f78c90dde719829e5ae"
[[projects]]
digest = "1:c4556a44e350b50a490544d9b06e9fba9c286c21d6c0e47f54f3a9214597298c"
@@ -275,23 +270,31 @@
revision = "e2704e165165ec55d062f5919b4b29494e9fa790"
[[projects]]
digest = "1:bd1ae00087d17c5a748660b8e89e1043e1e5479d0fea743352cda2f8dd8c4f84"
digest = "1:b0c25f00bad20d783d259af2af8666969e2fc343fa0dc9efe52936bbd67fb758"
name = "github.com/rs/cors"
packages = ["."]
pruneopts = "UT"
revision = "9a47f48565a795472d43519dd49aac781f3034fb"
version = "v1.6.0"
[[projects]]
digest = "1:6a4a11ba764a56d2758899ec6f3848d24698d48442ebce85ee7a3f63284526cd"
name = "github.com/spf13/afero"
packages = [
".",
"mem",
]
pruneopts = "UT"
revision = "787d034dfe70e44075ccc060d346146ef53270ad"
version = "v1.1.1"
revision = "d40851caa0d747393da1ffb28f7f9d8b4eeffebd"
version = "v1.1.2"
[[projects]]
digest = "1:516e71bed754268937f57d4ecb190e01958452336fa73dbac880894164e91c1f"
digest = "1:08d65904057412fc0270fc4812a1c90c594186819243160dc779a402d4b6d0bc"
name = "github.com/spf13/cast"
packages = ["."]
pruneopts = "UT"
revision = "8965335b8c7107321228e3e3702cab9832751bac"
version = "v1.2.0"
revision = "8c9545af88b134710ab1cd196795e7f2388358d7"
version = "v1.3.0"
[[projects]]
digest = "1:7ffc0983035bc7e297da3688d9fe19d60a420e9c38bef23f845c53788ed6a05e"
@@ -302,20 +305,20 @@
version = "v0.0.1"
[[projects]]
branch = "master"
digest = "1:080e5f630945ad754f4b920e60b4d3095ba0237ebf88dc462eb28002932e3805"
digest = "1:68ea4e23713989dc20b1bded5d9da2c5f9be14ff9885beef481848edd18c26cb"
name = "github.com/spf13/jwalterweatherman"
packages = ["."]
pruneopts = "UT"
revision = "7c0cea34c8ece3fbeb2b27ab9b59511d360fb394"
revision = "4a4406e478ca629068e7768fc33f3f044173c0a6"
version = "v1.0.0"
[[projects]]
digest = "1:9424f440bba8f7508b69414634aef3b2b3a877e522d8a4624692412805407bb7"
digest = "1:c1b1102241e7f645bc8e0c22ae352e8f0dc6484b6cb4d132fa9f24174e0119e2"
name = "github.com/spf13/pflag"
packages = ["."]
pruneopts = "UT"
revision = "583c0c0531f06d5278b7d917446061adc344b5cd"
version = "v1.0.1"
revision = "298182f68c66c05229eb03ac171abe6e309ee79a"
version = "v1.0.3"
[[projects]]
digest = "1:f8e1a678a2571e265f4bf91a3e5e32aa6b1474a55cb0ea849750cc177b664d96"
@@ -338,7 +341,7 @@
[[projects]]
branch = "master"
digest = "1:b3cfb8d82b1601a846417c3f31c03a7961862cb2c98dcf0959c473843e6d9a2b"
digest = "1:59483b8e8183f10ab21a85ba1f4cbb4a2335d48891801f79ed7b9499f44d383c"
name = "github.com/syndtr/goleveldb"
packages = [
"leveldb",
@@ -355,7 +358,7 @@
"leveldb/util",
]
pruneopts = "UT"
revision = "c4c61651e9e37fa117f53c5a906d3b63090d8445"
revision = "6b91fda63f2e36186f1c9d0e48578defb69c5d43"
[[projects]]
digest = "1:605b6546f3f43745695298ec2d342d3e952b6d91cdf9f349bea9315f677d759f"
@@ -365,34 +368,23 @@
revision = "e5840949ff4fff0c56f9b6a541e22b63581ea9df"
[[projects]]
branch = "master"
digest = "1:087aaa7920e5d0bf79586feb57ce01c35c830396ab4392798112e8aae8c47722"
name = "github.com/tendermint/ed25519"
packages = [
".",
"edwards25519",
"extra25519",
]
pruneopts = "UT"
revision = "d8387025d2b9d158cf4efb07e7ebf814bcce2057"
[[projects]]
digest = "1:e0a2a4be1e20c305badc2b0a7a9ab7fef6da500763bec23ab81df3b5f9eec9ee"
digest = "1:ad9c4c1a4e7875330b1f62906f2830f043a23edb5db997e3a5ac5d3e6eadf80a"
name = "github.com/tendermint/go-amino"
packages = ["."]
pruneopts = "UT"
revision = "a8328986c1608950fa5d3d1c0472cccc4f8fc02c"
version = "v0.12.0-rc0"
revision = "dc14acf9ef15f85828bfbc561ed9dd9d2a284885"
version = "v0.14.1"
[[projects]]
branch = "master"
digest = "1:c31a37cafc12315b8bd745c8ad6a006ac25350472488162a821e557b3e739d67"
digest = "1:72b71e3a29775e5752ed7a8012052a3dee165e27ec18cedddae5288058f09acf"
name = "golang.org/x/crypto"
packages = [
"bcrypt",
"blowfish",
"chacha20poly1305",
"curve25519",
"ed25519",
"ed25519/internal/edwards25519",
"hkdf",
"internal/chacha20",
"internal/subtle",
@@ -405,7 +397,8 @@
"salsa20/salsa",
]
pruneopts = "UT"
revision = "56440b844dfe139a8ac053f4ecac0b20b79058f4"
revision = "3764759f34a542a3aef74d6b02e35be7ab893bba"
source = "github.com/tendermint/crypto"
[[projects]]
digest = "1:d36f55a999540d29b6ea3c2ea29d71c76b1d9853fdcd3e5c5cb4836f2ba118f1"
@@ -425,14 +418,14 @@
[[projects]]
branch = "master"
digest = "1:bb0fe59917bdd5b89f49b9a8b26e5f465e325d9223b3a8e32254314bdf51e0f1"
digest = "1:6f86e2f2e2217cd4d74dec6786163cf80e4d2b99adb341ecc60a45113b844dca"
name = "golang.org/x/sys"
packages = [
"cpu",
"unix",
]
pruneopts = "UT"
revision = "3dc4335d56c789b04b0ba99b7a37249d9b614314"
revision = "7e31e0c00fa05cb5fbf4347b585621d6709e19a4"
[[projects]]
digest = "1:a2ab62866c75542dd18d2b069fec854577a20211d7c0ea6ae746072a1dccdd18"
@@ -459,11 +452,11 @@
[[projects]]
branch = "master"
digest = "1:077c1c599507b3b3e9156d17d36e1e61928ee9b53a5b420f10f28ebd4a0b275c"
digest = "1:56b0bca90b7e5d1facf5fbdacba23e4e0ce069d25381b8e2f70ef1e7ebfb9c1a"
name = "google.golang.org/genproto"
packages = ["googleapis/rpc/status"]
pruneopts = "UT"
revision = "daca94659cb50e9f37c1b834680f2e46358f10b0"
revision = "b69ba1387ce2108ac9bc8e8e5e5a46e7d5c72313"
[[projects]]
digest = "1:2dab32a43451e320e49608ff4542fdfc653c95dcc35d0065ec9c6c3dd540ed74"
@@ -513,7 +506,6 @@
input-imports = [
"github.com/btcsuite/btcutil/base58",
"github.com/btcsuite/btcutil/bech32",
"github.com/ebuchman/fail-test",
"github.com/fortytw2/leaktest",
"github.com/go-kit/kit/log",
"github.com/go-kit/kit/log/level",
@@ -534,6 +526,7 @@
"github.com/prometheus/client_golang/prometheus",
"github.com/prometheus/client_golang/prometheus/promhttp",
"github.com/rcrowley/go-metrics",
"github.com/rs/cors",
"github.com/spf13/cobra",
"github.com/spf13/viper",
"github.com/stretchr/testify/assert",
@@ -543,12 +536,11 @@
"github.com/syndtr/goleveldb/leveldb/iterator",
"github.com/syndtr/goleveldb/leveldb/opt",
"github.com/tendermint/btcd/btcec",
"github.com/tendermint/ed25519",
"github.com/tendermint/ed25519/extra25519",
"github.com/tendermint/go-amino",
"golang.org/x/crypto/bcrypt",
"golang.org/x/crypto/chacha20poly1305",
"golang.org/x/crypto/curve25519",
"golang.org/x/crypto/ed25519",
"golang.org/x/crypto/hkdf",
"golang.org/x/crypto/nacl/box",
"golang.org/x/crypto/nacl/secretbox",

View File

@@ -20,66 +20,74 @@
# unused-packages = true
#
###########################################################
# NOTE: All packages should be pinned to specific versions.
# Packages without releases must pin to a commit.
# Allow only patch releases for serialization libraries
[[constraint]]
name = "github.com/go-kit/kit"
version = "=0.6.0"
name = "github.com/tendermint/go-amino"
version = "~0.14.1"
[[constraint]]
name = "github.com/gogo/protobuf"
version = "=1.1.1"
version = "~1.1.1"
[[constraint]]
name = "github.com/golang/protobuf"
version = "=1.1.0"
version = "~1.1.0"
# Allow only minor releases for other libraries
[[constraint]]
name = "github.com/go-kit/kit"
version = "^0.6.0"
[[constraint]]
name = "github.com/gorilla/websocket"
version = "=1.2.0"
version = "^1.2.0"
[[constraint]]
name = "github.com/rs/cors"
version = "^1.6.0"
[[constraint]]
name = "github.com/pkg/errors"
version = "=0.8.0"
version = "^0.8.0"
[[constraint]]
name = "github.com/spf13/cobra"
version = "=0.0.1"
version = "^0.0.1"
[[constraint]]
name = "github.com/spf13/viper"
version = "=1.0.0"
version = "^1.0.0"
[[constraint]]
name = "github.com/stretchr/testify"
version = "=1.2.1"
[[constraint]]
name = "github.com/tendermint/go-amino"
version = "v0.12.0-rc0"
version = "^1.2.1"
[[constraint]]
name = "google.golang.org/grpc"
version = "=1.13.0"
version = "^1.13.0"
[[constraint]]
name = "github.com/fortytw2/leaktest"
version = "=1.2.0"
version = "^1.2.0"
[[constraint]]
name = "github.com/prometheus/client_golang"
version = "^0.9.1"
###################################
## Some repos dont have releases.
## Pin to revision
[[constraint]]
name = "golang.org/x/crypto"
source = "github.com/tendermint/crypto"
revision = "3764759f34a542a3aef74d6b02e35be7ab893bba"
[[override]]
name = "github.com/jmhodges/levigo"
revision = "c42d9e0ca023e2198120196f842701bb4c55d7b9"
[[constraint]]
name = "github.com/ebuchman/fail-test"
revision = "95f809107225be108efcf10a3509e4ea6ceef3c4"
# last revision used by go-crypto
[[constraint]]
name = "github.com/btcsuite/btcutil"
@@ -89,11 +97,6 @@
name = "github.com/tendermint/btcd"
revision = "e5840949ff4fff0c56f9b6a541e22b63581ea9df"
# Haven't made a release since 2016.
[[constraint]]
name = "github.com/prometheus/client_golang"
revision = "ae27198cdd90bf12cd134ad79d1366a6cf49f632"
[[constraint]]
name = "github.com/rcrowley/go-metrics"
revision = "e2704e165165ec55d062f5919b4b29494e9fa790"

View File

@@ -1,38 +1,44 @@
GOTOOLS = \
github.com/mitchellh/gox \
github.com/golang/dep/cmd/dep \
gopkg.in/alecthomas/gometalinter.v2 \
github.com/alecthomas/gometalinter \
github.com/gogo/protobuf/protoc-gen-gogo \
github.com/square/certstrap
GOBIN?=${GOPATH}/bin
PACKAGES=$(shell go list ./...)
INCLUDE = -I=. -I=${GOPATH}/src -I=${GOPATH}/src/github.com/gogo/protobuf/protobuf
BUILD_TAGS?='tendermint'
BUILD_FLAGS = -ldflags "-X github.com/tendermint/tendermint/version.GitCommit=`git rev-parse --short=8 HEAD`"
LINT_FLAGS = --exclude '.*\.pb\.go' --vendor --deadline=600s
LINT_FLAGS = --exclude '.*\.pb\.go' --exclude 'vendor/*' --vendor --deadline=600s
all: check build test install
check: check_tools get_vendor_deps
########################################
### Build Tendermint
build:
CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint/
build_c:
CGO_ENABLED=1 go build $(BUILD_FLAGS) -tags "$(BUILD_TAGS) gcc" -o build/tendermint ./cmd/tendermint/
build_race:
CGO_ENABLED=0 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint
install:
CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
install_c:
CGO_ENABLED=1 go install $(BUILD_FLAGS) -tags "$(BUILD_TAGS) gcc" ./cmd/tendermint
########################################
### Protobuf
protoc_all: protoc_libs protoc_abci protoc_grpc
protoc_all: protoc_libs protoc_merkle protoc_abci protoc_grpc protoc_proto3types
%.pb.go: %.proto
## If you get the following error,
@@ -48,6 +54,8 @@ protoc_all: protoc_libs protoc_abci protoc_grpc
# see protobuf section above
protoc_abci: abci/types/types.pb.go
protoc_proto3types: types/proto3/block.pb.go
build_abci:
@go build -i ./abci/cmd/...
@@ -72,12 +80,15 @@ check_tools:
get_tools:
@echo "--> Installing tools"
go get -u -v $(GOTOOLS)
@gometalinter.v2 --install
./scripts/get_tools.sh
get_dev_tools:
@echo "--> Downloading linters (this may take awhile)"
$(GOPATH)/src/github.com/alecthomas/gometalinter/scripts/install.sh -b $(GOBIN)
update_tools:
@echo "--> Updating tools"
go get -u -v $(GOTOOLS)
./scripts/get_tools.sh
#Update dependencies
get_vendor_deps:
@@ -134,6 +145,8 @@ grpc_dbserver:
protoc_grpc: rpc/grpc/types.pb.go
protoc_merkle: crypto/merkle/merkle.pb.go
########################################
### Testing
@@ -177,6 +190,9 @@ test_p2p:
cd ..
# requires 'tester' the image from above
bash test/p2p/test.sh tester
# the `docker cp` takes a really long time; uncomment for debugging
#
# mkdir -p test/p2p/logs && docker cp rsyslog:/var/log test/p2p/logs
test_integrations:
make build_docker_test_image
@@ -219,7 +235,7 @@ fmt:
metalinter:
@echo "--> Running linter"
@gometalinter.v2 $(LINT_FLAGS) --disable-all \
@gometalinter $(LINT_FLAGS) --disable-all \
--enable=deadcode \
--enable=gosimple \
--enable=misspell \
@@ -248,7 +264,7 @@ metalinter:
metalinter_all:
@echo "--> Running linter (all)"
gometalinter.v2 $(LINT_FLAGS) --enable-all --disable=lll ./...
gometalinter $(LINT_FLAGS) --enable-all --disable=lll ./...
DESTINATION = ./index.html.md
@@ -315,4 +331,4 @@ build-slate:
# To avoid unintended conflicts with file names, always add to .PHONY
# unless there is a reason not to.
# https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html
.PHONY: check build build_race build_abci dist install install_abci check_dep check_tools get_tools update_tools get_vendor_deps draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all
.PHONY: check build build_race build_abci dist install install_abci check_dep check_tools get_tools get_dev_tools update_tools get_vendor_deps draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all build_c install_c

View File

@@ -8,7 +8,7 @@ Or [Blockchain](https://en.wikipedia.org/wiki/Blockchain_(database)) for short.
[![API Reference](
https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667
)](https://godoc.org/github.com/tendermint/tendermint)
[![Go version](https://img.shields.io/badge/go-1.9.2-blue.svg)](https://github.com/moovweb/gvm)
[![Go version](https://img.shields.io/badge/go-1.10.4-blue.svg)](https://github.com/moovweb/gvm)
[![riot.im](https://img.shields.io/badge/riot.im-JOIN%20CHAT-green.svg)](https://riot.im/app/#/room/#tendermint:matrix.org)
[![license](https://img.shields.io/github/license/tendermint/tendermint.svg)](https://github.com/tendermint/tendermint/blob/master/LICENSE)
[![](https://tokei.rs/b1/github/tendermint/tendermint?category=lines)](https://github.com/tendermint/tendermint)
@@ -22,7 +22,10 @@ develop | [![CircleCI](https://circleci.com/gh/tendermint/tendermint/tree/deve
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language -
and securely replicates it on many machines.
For protocol details, see [the specification](/docs/spec). For a consensus proof and detailed protocol analysis checkout our recent paper, "[The latest gossip on BFT consensus](https://arxiv.org/abs/1807.04938)".
For protocol details, see [the specification](/docs/spec).
For detailed analysis of the consensus protocol, including safety and liveness proofs,
see our recent paper, "[The latest gossip on BFT consensus](https://arxiv.org/abs/1807.04938)".
## A Note on Production Readiness
@@ -30,15 +33,15 @@ While Tendermint is being used in production in private, permissioned
environments, we are still working actively to harden and audit it in preparation
for use in public blockchains, such as the [Cosmos Network](https://cosmos.network/).
We are also still making breaking changes to the protocol and the APIs.
Thus we tag the releases as *alpha software*.
Thus, we tag the releases as *alpha software*.
In any case, if you intend to run Tendermint in production,
please [contact us](https://riot.im/app/#/room/#tendermint:matrix.org) :)
please [contact us](mailto:partners@tendermint.com) and [join the chat](https://riot.im/app/#/room/#tendermint:matrix.org).
## Security
To report a security vulnerability, see our [bug bounty
program](https://tendermint.com/security).
program](https://hackerone.com/tendermint)
For examples of the kinds of bugs we're looking for, see [SECURITY.md](SECURITY.md)
@@ -46,18 +49,22 @@ For examples of the kinds of bugs we're looking for, see [SECURITY.md](SECURITY.
Requirement|Notes
---|---
Go version | Go1.9 or higher
Go version | Go1.10 or higher
## Install
## Documentation
Complete documentation can be found on the [website](https://tendermint.com/docs/).
### Install
See the [install instructions](/docs/introduction/install.md)
## Quick Start
### Quick Start
- [Single node](/docs/using-tendermint.md)
- [Local cluster using docker-compose](/networks/local)
- [Single node](/docs/introduction/quick-start.md)
- [Local cluster using docker-compose](/docs/networks/docker-compose.md)
- [Remote cluster using terraform and ansible](/docs/networks/terraform-and-ansible.md)
- [Join the public testnet](https://cosmos.network/testnet)
- [Join the Cosmos testnet](https://cosmos.network/testnet)
## Resources
@@ -66,30 +73,32 @@ See the [install instructions](/docs/introduction/install.md)
For details about the blockchain data structures and the p2p protocols, see the
the [Tendermint specification](/docs/spec).
For details on using the software, [Read The Docs](https://tendermint.readthedocs.io/en/master/).
Additional information about some - and eventually all - of the sub-projects below, can be found at Read The Docs.
For details on using the software, see the [documentation](/docs/) which is also
hosted at: https://tendermint.com/docs/
### Tools
Benchmarking and monitoring is provided by `tm-bench` and `tm-monitor`, respectively.
Their code is found [here](/tools) and these binaries need to be built seperately.
Additional documentation is found [here](/docs/tools).
### Sub-projects
* [Amino](http://github.com/tendermint/go-amino), a reflection-based improvement on proto3
* [IAVL](http://github.com/tendermint/iavl), Merkleized IAVL+ Tree implementation
### Tools
* [Deployment, Benchmarking, and Monitoring](http://tendermint.readthedocs.io/projects/tools/en/develop/index.html#tendermint-tools)
### Applications
* [Cosmos SDK](http://github.com/cosmos/cosmos-sdk); a cryptocurrency application framework
* [Ethermint](http://github.com/tendermint/ethermint); Ethereum on Tendermint
* [Many more](https://tendermint.readthedocs.io/en/master/ecosystem.html#abci-applications)
* [Ethermint](http://github.com/cosmos/ethermint); Ethereum on Tendermint
* [Many more](https://tendermint.com/ecosystem)
### More
### Research
* [The latest gossip on BFT consensus](https://arxiv.org/abs/1807.04938)
* [Master's Thesis on Tendermint](https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769)
* [Original Whitepaper](https://tendermint.com/static/docs/tendermint.pdf)
* [Tendermint Blog](https://blog.cosmos.network/tendermint/home)
* [Cosmos Blog](https://blog.cosmos.network)
* [Blog](https://blog.cosmos.network/tendermint/home)
## Contributing
@@ -114,6 +123,12 @@ CHANGELOG even if they don't lead to MINOR version bumps:
- rpc/client
- config
- node
- libs
- bech32
- common
- db
- errors
- log
Exported objects in these packages that are not covered by the versioning scheme
are explicitly marked by `// UNSTABLE` in their go doc comment and may change at any
@@ -130,6 +145,8 @@ data into the new chain.
However, any bump in the PATCH version should be compatible with existing histories
(if not please open an [issue](https://github.com/tendermint/tendermint/issues)).
For more information on upgrading, see [here](./UPGRADING.md)
## Code of Conduct
Please read, understand and adhere to our [code of conduct](CODE_OF_CONDUCT.md).

View File

@@ -1,7 +1,8 @@
# Security
As part of our [Coordinated Vulnerability Disclosure
Policy](https://tendermint.com/security), we operate a bug bounty.
Policy](https://tendermint.com/security), we operate a [bug
bounty](https://hackerone.com/tendermint).
See the policy for more details on submissions and rewards.
Here is a list of examples of the kinds of bugs we're most interested in:

View File

@@ -3,10 +3,52 @@
This guide provides steps to be followed when you upgrade your applications to
a newer version of Tendermint Core.
## Upgrading from 0.23.0 to 0.24.0
## v0.27.0
New 0.24.0 release contains a lot of changes to the state and types. It's not
compatible to the old versions.
This release contains some breaking changes to the block and p2p protocols,
but does not change any core data structures, so it should be compatible with
existing blockchains from the v0.26 series that only used Ed25519 validator keys.
Blockchains using Secp256k1 for validators will not be compatible. This is due
to the fact that we now enforce which key types validators can use as a
consensus param. The default is Ed25519, and Secp256k1 must be activated
explicitly.
It is recommended to upgrade all nodes at once to avoid incompatibilities at the
peer layer - namely, the heartbeat consensus message has been removed (only
relevant if `create_empty_blocks=false` or `create_empty_blocks_interval > 0`),
and the proposer selection algorithm has changed. Since proposer information is
never included in the blockchain, this change only affects the peer layer.
### Go API Changes
#### libs/db
The ReverseIterator API has changed the meaning of `start` and `end`.
Before, iteration was from `start` to `end`, where
`start > end`. Now, iteration is from `end` to `start`, where `start < end`.
The iterator also excludes `end`. This change allows a simplified and more
intuitive logic, aligning the semantic meaning of `start` and `end` in the
`Iterator` and `ReverseIterator`.
### Applications
This release enforces a new consensus parameter, the
ValidatorParams.PubKeyTypes. Applications must ensure that they only return
validator updates with the allowed PubKeyTypes. If a validator update includes a
pubkey type that is not included in the ConsensusParams.Validator.PubKeyTypes,
block execution will fail and the consensus will halt.
By default, only Ed25519 pubkeys may be used for validators. Enabling
Secp256k1 requires explicit modification of the ConsensusParams.
Please update your application accordingly (ie. restrict validators to only be
able to use Ed25519 keys, or explicitly add additional key types to the genesis
file).
## v0.26.0
This release contains a lot of changes to core data types and protocols. It is not
compatible to the old versions and there is no straight forward way to update
old data to be compatible with the new version.
To reset the state do:
@@ -14,10 +56,98 @@ To reset the state do:
$ tendermint unsafe_reset_all
```
Here we summarize some other notable changes to be mindful of.
### Config Changes
All timeouts must be changed from integers to strings with their duration, for
instance `flush_throttle_timeout = 100` would be changed to
`flush_throttle_timeout = "100ms"` and `timeout_propose = 3000` would be changed
to `timeout_propose = "3s"`.
### RPC Changes
The default behaviour of `/abci_query` has been changed to not return a proof,
and the name of the parameter that controls this has been changed from `trusted`
to `prove`. To get proofs with your queries, ensure you set `prove=true`.
Various version fields like `amino_version`, `p2p_version`, `consensus_version`,
and `rpc_version` have been removed from the `node_info.other` and are
consolidated under the tendermint semantic version (ie. `node_info.version`) and
the new `block` and `p2p` protocol versions under `node_info.protocol_version`.
### ABCI Changes
Field numbers were bumped in the `Header` and `ResponseInfo` messages to make
room for new `version` fields. It should be straight forward to recompile the
protobuf file for these changes.
#### Proofs
The `ResponseQuery.Proof` field is now structured as a `[]ProofOp` to support
generalized Merkle tree constructions where the leaves of one Merkle tree are
the root of another. If you don't need this functionality, and you used to
return `<proof bytes>` here, you should instead return a single `ProofOp` with
just the `Data` field set:
```
[]ProofOp{
ProofOp{
Data: <proof bytes>,
}
}
```
For more information, see:
- [ADR-026](https://github.com/tendermint/tendermint/blob/30519e8361c19f4bf320ef4d26288ebc621ad725/docs/architecture/adr-026-general-merkle-proof.md)
- [Relevant ABCI
documentation](https://github.com/tendermint/tendermint/blob/30519e8361c19f4bf320ef4d26288ebc621ad725/docs/spec/abci/apps.md#query-proofs)
- [Description of
keys](https://github.com/tendermint/tendermint/blob/30519e8361c19f4bf320ef4d26288ebc621ad725/crypto/merkle/proof_key_path.go#L14)
### Go API Changes
#### crypto/merkle
The `merkle.Hasher` interface was removed. Functions which used to take `Hasher`
now simply take `[]byte`. This means that any objects being Merklized should be
serialized before they are passed in.
#### node
The `node.RunForever` function was removed. Signal handling and running forever
should instead be explicitly configured by the caller. See how we do it
[here](https://github.com/tendermint/tendermint/blob/30519e8361c19f4bf320ef4d26288ebc621ad725/cmd/tendermint/commands/run_node.go#L60).
### Other
All hashes, except for public key addresses, are now 32-bytes.
## v0.25.0
This release has minimal impact.
If you use GasWanted in ABCI and want to enforce it, set the MaxGas in the genesis file (default is no max).
## v0.24.0
New 0.24.0 release contains a lot of changes to the state and types. It's not
compatible to the old versions and there is no straight forward way to update
old data to be compatible with the new version.
To reset the state do:
```
$ tendermint unsafe_reset_all
```
Here we summarize some other notable changes to be mindful of.
### Config changes
`p2p.max_num_peers` was removed in favor of `p2p.max_num_inbound_peers` and
`p2p.max_num_outbound_peers`.
`p2p.max_num_outbound_peers`.
```
# Maximum number of inbound peers
@@ -28,5 +158,36 @@ max_num_outbound_peers = 10
```
As you can see, the default ratio of inbound/outbound peers is 4/1. The reason
as we want it to be easier for new nodes to connect to the network. You can
is we want it to be easier for new nodes to connect to the network. You can
tweak these parameters to alter the network topology.
### RPC Changes
The result of `/commit` used to contain `header` and `commit` fields at the top level. These are now contained under the `signed_header` field.
### ABCI Changes
The header has been upgraded and contains new fields, but none of the existing
fields were changed, except their order.
The `Validator` type was split into two, one containing an `Address` and one
containing a `PubKey`. When processing `RequestBeginBlock`, use the `Validator`
type, which contains just the `Address`. When returning `ResponseEndBlock`, use
the `ValidatorUpdate` type, which contains just the `PubKey`.
### Validator Set Updates
Validator set updates returned in ResponseEndBlock for height `H` used to take
effect immediately at height `H+1`. Now they will be delayed one block, to take
effect at height `H+2`. Note this means that the change will be seen by the ABCI
app in the `RequestBeginBlock.LastCommitInfo` at block `H+3`. Apps were already
required to maintain a map from validator addresses to pubkeys since v0.23 (when
pubkeys were removed from RequestBeginBlock), but now they may need to track
multiple validator sets at once to accomodate this delay.
### Block Size
The `ConsensusParams.BlockSize.MaxTxs` was removed in favour of
`ConsensusParams.BlockSize.MaxBytes`, which is now enforced. This means blocks
are limitted only by byte-size, not by number of transactions.

8
Vagrantfile vendored
View File

@@ -29,10 +29,10 @@ Vagrant.configure("2") do |config|
usermod -a -G docker vagrant
# install go
wget -q https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz
tar -xvf go1.10.1.linux-amd64.tar.gz
wget -q https://dl.google.com/go/go1.11.linux-amd64.tar.gz
tar -xvf go1.11.linux-amd64.tar.gz
mv go /usr/local
rm -f go1.10.1.linux-amd64.tar.gz
rm -f go1.11.linux-amd64.tar.gz
# cleanup
apt-get autoremove -y
@@ -53,6 +53,6 @@ Vagrant.configure("2") do |config|
# get all deps and tools, ready to install/test
su - vagrant -c 'source /home/vagrant/.bash_profile'
su - vagrant -c 'cd /home/vagrant/go/src/github.com/tendermint/tendermint && make get_tools && make get_vendor_deps'
su - vagrant -c 'cd /home/vagrant/go/src/github.com/tendermint/tendermint && make get_tools && make get_dev_tools && make get_vendor_deps'
SHELL
end

View File

@@ -1,7 +1,5 @@
# Application BlockChain Interface (ABCI)
[![CircleCI](https://circleci.com/gh/tendermint/abci.svg?style=svg)](https://circleci.com/gh/tendermint/abci)
Blockchains are systems for multi-master state machine replication.
**ABCI** is an interface that defines the boundary between the replication engine (the blockchain),
and the state machine (the application).
@@ -12,160 +10,28 @@ Previously, the ABCI was referred to as TMSP.
The community has provided a number of addtional implementations, see the [Tendermint Ecosystem](https://tendermint.com/ecosystem)
## Installation & Usage
To get up and running quickly, see the [getting started guide](../docs/app-dev/getting-started.md) along with the [abci-cli documentation](../docs/app-dev/abci-cli.md) which will go through the examples found in the [examples](./example/) directory.
## Specification
A detailed description of the ABCI methods and message types is contained in:
- [A prose specification](specification.md)
- [A protobuf file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto)
- [A Go interface](https://github.com/tendermint/tendermint/blob/master/abci/types/application.go).
For more background information on ABCI, motivations, and tendermint, please visit [the documentation](https://tendermint.com/docs/).
The two guides to focus on are the `Application Development Guide` and `Using ABCI-CLI`.
- [The main spec](../docs/spec/abci/abci.md)
- [A protobuf file](./types/types.proto)
- [A Go interface](./types/application.go)
## Protocol Buffers
To compile the protobuf file, run:
To compile the protobuf file, run (from the root of the repo):
```
cd $GOPATH/src/github.com/tendermint/tendermint/; make protoc_abci
make protoc_abci
```
See `protoc --help` and [the Protocol Buffers site](https://developers.google.com/protocol-buffers)
for details on compiling for other languages. Note we also include a [GRPC](http://www.grpc.io/docs)
for details on compiling for other languages. Note we also include a [GRPC](https://www.grpc.io/docs)
service definition.
## Install ABCI-CLI
The `abci-cli` is a simple tool for debugging ABCI servers and running some
example apps. To install it:
```
mkdir -p $GOPATH/src/github.com/tendermint
cd $GOPATH/src/github.com/tendermint
git clone https://github.com/tendermint/tendermint.git
cd tendermint
make get_tools
make get_vendor_deps
make install_abci
```
## Implementation
We provide three implementations of the ABCI in Go:
- Golang in-process
- ABCI-socket
- GRPC
Note the GRPC version is maintained primarily to simplify onboarding and prototyping and is not receiving the same
attention to security and performance as the others
### In Process
The simplest implementation just uses function calls within Go.
This means ABCI applications written in Golang can be compiled with TendermintCore and run as a single binary.
See the [examples](#examples) below for more information.
### Socket (TSP)
ABCI is best implemented as a streaming protocol.
The socket implementation provides for asynchronous, ordered message passing over unix or tcp.
Messages are serialized using Protobuf3 and length-prefixed with a [signed Varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-integers)
For example, if the Protobuf3 encoded ABCI message is `0xDEADBEEF` (4 bytes), the length-prefixed message is `0x08DEADBEEF`, since `0x08` is the signed varint
encoding of `4`. If the Protobuf3 encoded ABCI message is 65535 bytes long, the length-prefixed message would be like `0xFEFF07...`.
Note the benefit of using this `varint` encoding over the old version (where integers were encoded as `<len of len><big endian len>` is that
it is the standard way to encode integers in Protobuf. It is also generally shorter.
### GRPC
GRPC is an rpc framework native to Protocol Buffers with support in many languages.
Implementing the ABCI using GRPC can allow for faster prototyping, but is expected to be much slower than
the ordered, asynchronous socket protocol. The implementation has also not received as much testing or review.
Note the length-prefixing used in the socket implementation does not apply for GRPC.
## Usage
The `abci-cli` tool wraps an ABCI client and can be used for probing/testing an ABCI server.
For instance, `abci-cli test` will run a test sequence against a listening server running the Counter application (see below).
It can also be used to run some example applications.
See [the documentation](https://tendermint.com/docs/) for more details.
### Examples
Check out the variety of example applications in the [example directory](example/).
It also contains the code refered to by the `counter` and `kvstore` apps; these apps come
built into the `abci-cli` binary.
#### Counter
The `abci-cli counter` application illustrates nonce checking in transactions. It's code looks like:
```golang
func cmdCounter(cmd *cobra.Command, args []string) error {
app := counter.NewCounterApplication(flagSerial)
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
// Start the listener
srv, err := server.NewServer(flagAddrC, flagAbci, app)
if err != nil {
return err
}
srv.SetLogger(logger.With("module", "abci-server"))
if err := srv.Start(); err != nil {
return err
}
// Wait forever
cmn.TrapSignal(func() {
// Cleanup
srv.Stop()
})
return nil
}
```
and can be found in [this file](cmd/abci-cli/abci-cli.go).
#### kvstore
The `abci-cli kvstore` application, which illustrates a simple key-value Merkle tree
```golang
func cmdKVStore(cmd *cobra.Command, args []string) error {
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
// Create the application - in memory or persisted to disk
var app types.Application
if flagPersist == "" {
app = kvstore.NewKVStoreApplication()
} else {
app = kvstore.NewPersistentKVStoreApplication(flagPersist)
app.(*kvstore.PersistentKVStoreApplication).SetLogger(logger.With("module", "kvstore"))
}
// Start the listener
srv, err := server.NewServer(flagAddrD, flagAbci, app)
if err != nil {
return err
}
srv.SetLogger(logger.With("module", "abci-server"))
if err := srv.Start(); err != nil {
return err
}
// Wait forever
cmn.TrapSignal(func() {
// Cleanup
srv.Stop()
})
return nil
}
```

View File

@@ -105,8 +105,8 @@ func (reqRes *ReqRes) SetCallback(cb func(res *types.Response)) {
return
}
defer reqRes.mtx.Unlock()
reqRes.cb = cb
reqRes.mtx.Unlock()
}
func (reqRes *ReqRes) GetCallback() func(*types.Response) {

View File

@@ -54,7 +54,7 @@ RETRY_LOOP:
if cli.mustConnect {
return err
}
cli.Logger.Error(fmt.Sprintf("abci.grpcClient failed to connect to %v. Retrying...\n", cli.addr))
cli.Logger.Error(fmt.Sprintf("abci.grpcClient failed to connect to %v. Retrying...\n", cli.addr), "err", err)
time.Sleep(time.Second * dialRetryIntervalSeconds)
continue RETRY_LOOP
}
@@ -111,8 +111,8 @@ func (cli *grpcClient) Error() error {
// NOTE: callback may get internally generated flush responses.
func (cli *grpcClient) SetResponseCallback(resCb Callback) {
cli.mtx.Lock()
defer cli.mtx.Unlock()
cli.resCb = resCb
cli.mtx.Unlock()
}
//----------------------------------------

View File

@@ -9,8 +9,13 @@ import (
var _ Client = (*localClient)(nil)
// NOTE: use defer to unlock mutex because Application might panic (e.g., in
// case of malicious tx or query). It only makes sense for publicly exposed
// methods like CheckTx (/broadcast_tx_* RPC endpoint) or Query (/abci_query
// RPC endpoint), but defers are used everywhere for the sake of consistency.
type localClient struct {
cmn.BaseService
mtx *sync.Mutex
types.Application
Callback
@@ -30,8 +35,8 @@ func NewLocalClient(mtx *sync.Mutex, app types.Application) *localClient {
func (app *localClient) SetResponseCallback(cb Callback) {
app.mtx.Lock()
defer app.mtx.Unlock()
app.Callback = cb
app.mtx.Unlock()
}
// TODO: change types.Application to include Error()?
@@ -45,6 +50,9 @@ func (app *localClient) FlushAsync() *ReqRes {
}
func (app *localClient) EchoAsync(msg string) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
return app.callback(
types.ToRequestEcho(msg),
types.ToResponseEcho(msg),
@@ -53,8 +61,9 @@ func (app *localClient) EchoAsync(msg string) *ReqRes {
func (app *localClient) InfoAsync(req types.RequestInfo) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.Info(req)
app.mtx.Unlock()
return app.callback(
types.ToRequestInfo(req),
types.ToResponseInfo(res),
@@ -63,8 +72,9 @@ func (app *localClient) InfoAsync(req types.RequestInfo) *ReqRes {
func (app *localClient) SetOptionAsync(req types.RequestSetOption) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.SetOption(req)
app.mtx.Unlock()
return app.callback(
types.ToRequestSetOption(req),
types.ToResponseSetOption(res),
@@ -73,8 +83,9 @@ func (app *localClient) SetOptionAsync(req types.RequestSetOption) *ReqRes {
func (app *localClient) DeliverTxAsync(tx []byte) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.DeliverTx(tx)
app.mtx.Unlock()
return app.callback(
types.ToRequestDeliverTx(tx),
types.ToResponseDeliverTx(res),
@@ -83,8 +94,9 @@ func (app *localClient) DeliverTxAsync(tx []byte) *ReqRes {
func (app *localClient) CheckTxAsync(tx []byte) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.CheckTx(tx)
app.mtx.Unlock()
return app.callback(
types.ToRequestCheckTx(tx),
types.ToResponseCheckTx(res),
@@ -93,8 +105,9 @@ func (app *localClient) CheckTxAsync(tx []byte) *ReqRes {
func (app *localClient) QueryAsync(req types.RequestQuery) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.Query(req)
app.mtx.Unlock()
return app.callback(
types.ToRequestQuery(req),
types.ToResponseQuery(res),
@@ -103,8 +116,9 @@ func (app *localClient) QueryAsync(req types.RequestQuery) *ReqRes {
func (app *localClient) CommitAsync() *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.Commit()
app.mtx.Unlock()
return app.callback(
types.ToRequestCommit(),
types.ToResponseCommit(res),
@@ -113,19 +127,20 @@ func (app *localClient) CommitAsync() *ReqRes {
func (app *localClient) InitChainAsync(req types.RequestInitChain) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.InitChain(req)
reqRes := app.callback(
return app.callback(
types.ToRequestInitChain(req),
types.ToResponseInitChain(res),
)
app.mtx.Unlock()
return reqRes
}
func (app *localClient) BeginBlockAsync(req types.RequestBeginBlock) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.BeginBlock(req)
app.mtx.Unlock()
return app.callback(
types.ToRequestBeginBlock(req),
types.ToResponseBeginBlock(res),
@@ -134,8 +149,9 @@ func (app *localClient) BeginBlockAsync(req types.RequestBeginBlock) *ReqRes {
func (app *localClient) EndBlockAsync(req types.RequestEndBlock) *ReqRes {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.EndBlock(req)
app.mtx.Unlock()
return app.callback(
types.ToRequestEndBlock(req),
types.ToResponseEndBlock(res),
@@ -154,64 +170,73 @@ func (app *localClient) EchoSync(msg string) (*types.ResponseEcho, error) {
func (app *localClient) InfoSync(req types.RequestInfo) (*types.ResponseInfo, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.Info(req)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) SetOptionSync(req types.RequestSetOption) (*types.ResponseSetOption, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.SetOption(req)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) DeliverTxSync(tx []byte) (*types.ResponseDeliverTx, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.DeliverTx(tx)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) CheckTxSync(tx []byte) (*types.ResponseCheckTx, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.CheckTx(tx)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) QuerySync(req types.RequestQuery) (*types.ResponseQuery, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.Query(req)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) CommitSync() (*types.ResponseCommit, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.Commit()
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) InitChainSync(req types.RequestInitChain) (*types.ResponseInitChain, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.InitChain(req)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) BeginBlockSync(req types.RequestBeginBlock) (*types.ResponseBeginBlock, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.BeginBlock(req)
app.mtx.Unlock()
return &res, nil
}
func (app *localClient) EndBlockSync(req types.RequestEndBlock) (*types.ResponseEndBlock, error) {
app.mtx.Lock()
defer app.mtx.Unlock()
res := app.Application.EndBlock(req)
app.mtx.Unlock()
return &res, nil
}

View File

@@ -67,7 +67,7 @@ RETRY_LOOP:
if cli.mustConnect {
return err
}
cli.Logger.Error(fmt.Sprintf("abci.socketClient failed to connect to %v. Retrying...", cli.addr))
cli.Logger.Error(fmt.Sprintf("abci.socketClient failed to connect to %v. Retrying...", cli.addr), "err", err)
time.Sleep(time.Second * dialRetryIntervalSeconds)
continue RETRY_LOOP
}
@@ -118,8 +118,8 @@ func (cli *socketClient) Error() error {
// NOTE: callback may get internally generated flush responses.
func (cli *socketClient) SetResponseCallback(resCb Callback) {
cli.mtx.Lock()
defer cli.mtx.Unlock()
cli.resCb = resCb
cli.mtx.Unlock()
}
//----------------------------------------

View File

@@ -22,6 +22,7 @@ import (
servertest "github.com/tendermint/tendermint/abci/tests/server"
"github.com/tendermint/tendermint/abci/types"
"github.com/tendermint/tendermint/abci/version"
"github.com/tendermint/tendermint/crypto/merkle"
)
// client is a global variable so it can be reused by the console
@@ -100,7 +101,7 @@ type queryResponse struct {
Key []byte
Value []byte
Height int64
Proof []byte
Proof *merkle.Proof
}
func Execute() error {
@@ -748,7 +749,7 @@ func printResponse(cmd *cobra.Command, args []string, rsp response) {
fmt.Printf("-> value.hex: %X\n", rsp.Query.Value)
}
if rsp.Query.Proof != nil {
fmt.Printf("-> proof: %X\n", rsp.Query.Proof)
fmt.Printf("-> proof: %#v\n", rsp.Query.Proof)
}
}
}

View File

@@ -6,4 +6,5 @@ const (
CodeTypeEncodingError uint32 = 1
CodeTypeBadNonce uint32 = 2
CodeTypeUnauthorized uint32 = 3
CodeTypeUnknownError uint32 = 4
)

View File

@@ -10,11 +10,14 @@ import (
"github.com/tendermint/tendermint/abci/types"
cmn "github.com/tendermint/tendermint/libs/common"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/version"
)
var (
stateKey = []byte("stateKey")
kvPairPrefixKey = []byte("kvPairKey:")
ProtocolVersion version.Protocol = 0x1
)
type State struct {
@@ -65,7 +68,11 @@ func NewKVStoreApplication() *KVStoreApplication {
}
func (app *KVStoreApplication) Info(req types.RequestInfo) (resInfo types.ResponseInfo) {
return types.ResponseInfo{Data: fmt.Sprintf("{\"size\":%v}", app.state.Size)}
return types.ResponseInfo{
Data: fmt.Sprintf("{\"size\":%v}", app.state.Size),
Version: version.ABCIVersion,
AppVersion: ProtocolVersion.Uint64(),
}
}
// tx is either "key=value" or just arbitrary bytes
@@ -81,14 +88,14 @@ func (app *KVStoreApplication) DeliverTx(tx []byte) types.ResponseDeliverTx {
app.state.Size += 1
tags := []cmn.KVPair{
{Key: []byte("app.creator"), Value: []byte("jae")},
{Key: []byte("app.creator"), Value: []byte("Cosmoshi Netowoko")},
{Key: []byte("app.key"), Value: key},
}
return types.ResponseDeliverTx{Code: code.CodeTypeOK, Tags: tags}
}
func (app *KVStoreApplication) CheckTx(tx []byte) types.ResponseCheckTx {
return types.ResponseCheckTx{Code: code.CodeTypeOK}
return types.ResponseCheckTx{Code: code.CodeTypeOK, GasWanted: 1}
}
func (app *KVStoreApplication) Commit() types.ResponseCommit {
@@ -114,6 +121,7 @@ func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery type
}
return
} else {
resQuery.Key = reqQuery.Data
value := app.state.db.Get(prefixKey(reqQuery.Data))
resQuery.Value = value
if value != nil {

View File

@@ -28,6 +28,8 @@
-> code: OK
-> log: exists
-> height: 0
-> key: abc
-> key.hex: 616263
-> value: abc
-> value.hex: 616263
@@ -42,6 +44,8 @@
-> code: OK
-> log: exists
-> height: 0
-> key: def
-> key.hex: 646566
-> value: xyz
-> value.hex: 78797A

File diff suppressed because it is too large Load Diff

View File

@@ -6,6 +6,7 @@ package types;
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
import "google/protobuf/timestamp.proto";
import "github.com/tendermint/tendermint/libs/common/types.proto";
import "github.com/tendermint/tendermint/crypto/merkle/merkle.proto";
// This file is copied from http://github.com/tendermint/abci
// NOTE: When using custom types, mind the warnings.
@@ -48,6 +49,8 @@ message RequestFlush {
message RequestInfo {
string version = 1;
uint64 block_version = 2;
uint64 p2p_version = 3;
}
// nondeterministic
@@ -71,7 +74,6 @@ message RequestQuery {
bool prove = 4;
}
// NOTE: validators here have empty pubkeys.
message RequestBeginBlock {
bytes hash = 1;
Header header = 2 [(gogoproto.nullable)=false];
@@ -128,9 +130,12 @@ message ResponseFlush {
message ResponseInfo {
string data = 1;
string version = 2;
int64 last_block_height = 3;
bytes last_block_app_hash = 4;
uint64 app_version = 3;
int64 last_block_height = 4;
bytes last_block_app_hash = 5;
}
// nondeterministic
@@ -154,8 +159,9 @@ message ResponseQuery {
int64 index = 5;
bytes key = 6;
bytes value = 7;
bytes proof = 8;
merkle.Proof proof = 8;
int64 height = 9;
string codespace = 10;
}
message ResponseBeginBlock {
@@ -170,6 +176,7 @@ message ResponseCheckTx {
int64 gas_wanted = 5;
int64 gas_used = 6;
repeated common.KVPair tags = 7 [(gogoproto.nullable)=false, (gogoproto.jsontag)="tags,omitempty"];
string codespace = 8;
}
message ResponseDeliverTx {
@@ -180,6 +187,7 @@ message ResponseDeliverTx {
int64 gas_wanted = 5;
int64 gas_used = 6;
repeated common.KVPair tags = 7 [(gogoproto.nullable)=false, (gogoproto.jsontag)="tags,omitempty"];
string codespace = 8;
}
message ResponseEndBlock {
@@ -199,28 +207,28 @@ message ResponseCommit {
// ConsensusParams contains all consensus-relevant parameters
// that can be adjusted by the abci app
message ConsensusParams {
BlockSize block_size = 1;
TxSize tx_size = 2;
BlockGossip block_gossip = 3;
BlockSizeParams block_size = 1;
EvidenceParams evidence = 2;
ValidatorParams validator = 3;
}
// BlockSize contains limits on the block size.
message BlockSize {
int32 max_bytes = 1;
message BlockSizeParams {
// Note: must be greater than 0
int64 max_bytes = 1;
// Note: must be greater or equal to -1
int64 max_gas = 2;
}
// TxSize contains limits on the tx size.
message TxSize {
int32 max_bytes = 1;
int64 max_gas = 2;
// EvidenceParams contains limits on the evidence.
message EvidenceParams {
// Note: must be greater than 0
int64 max_age = 1;
}
// BlockGossip determine consensus critical
// elements of how blocks are gossiped
message BlockGossip {
// Note: must not be 0
int32 block_part_size_bytes = 1;
// ValidatorParams contains limits on validators.
message ValidatorParams {
repeated string pub_key_types = 1;
}
message LastCommitInfo {
@@ -233,31 +241,38 @@ message LastCommitInfo {
message Header {
// basic block info
string chain_id = 1 [(gogoproto.customname)="ChainID"];
int64 height = 2;
google.protobuf.Timestamp time = 3 [(gogoproto.nullable)=false, (gogoproto.stdtime)=true];
int64 num_txs = 4;
int64 total_txs = 5;
Version version = 1 [(gogoproto.nullable)=false];
string chain_id = 2 [(gogoproto.customname)="ChainID"];
int64 height = 3;
google.protobuf.Timestamp time = 4 [(gogoproto.nullable)=false, (gogoproto.stdtime)=true];
int64 num_txs = 5;
int64 total_txs = 6;
// prev block info
BlockID last_block_id = 6 [(gogoproto.nullable)=false];
BlockID last_block_id = 7 [(gogoproto.nullable)=false];
// hashes of block data
bytes last_commit_hash = 7; // commit from validators from the last block
bytes data_hash = 8; // transactions
bytes last_commit_hash = 8; // commit from validators from the last block
bytes data_hash = 9; // transactions
// hashes from the app output from the prev block
bytes validators_hash = 9; // validators for the current block
bytes next_validators_hash = 10; // validators for the next block
bytes consensus_hash = 11; // consensus params for current block
bytes app_hash = 12; // state after txs from the previous block
bytes last_results_hash = 13;// root hash of all results from the txs from the previous block
bytes validators_hash = 10; // validators for the current block
bytes next_validators_hash = 11; // validators for the next block
bytes consensus_hash = 12; // consensus params for current block
bytes app_hash = 13; // state after txs from the previous block
bytes last_results_hash = 14;// root hash of all results from the txs from the previous block
// consensus info
bytes evidence_hash = 14; // evidence included in the block
bytes proposer_address = 15; // original proposer of the block
bytes evidence_hash = 15; // evidence included in the block
bytes proposer_address = 16; // original proposer of the block
}
message Version {
uint64 Block = 1;
uint64 App = 2;
}
message BlockID {
bytes hash = 1;
PartSetHeader parts_header = 2 [(gogoproto.nullable)=false];

View File

@@ -14,6 +14,7 @@ import fmt "fmt"
import math "math"
import _ "github.com/gogo/protobuf/gogoproto"
import _ "github.com/golang/protobuf/ptypes/timestamp"
import _ "github.com/tendermint/tendermint/crypto/merkle"
import _ "github.com/tendermint/tendermint/libs/common"
// Reference imports to suppress errors if they are not otherwise used.
@@ -1478,15 +1479,15 @@ func TestConsensusParamsMarshalTo(t *testing.T) {
}
}
func TestBlockSizeProto(t *testing.T) {
func TestBlockSizeParamsProto(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSize(popr, false)
p := NewPopulatedBlockSizeParams(popr, false)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockSize{}
msg := &BlockSizeParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1509,10 +1510,10 @@ func TestBlockSizeProto(t *testing.T) {
}
}
func TestBlockSizeMarshalTo(t *testing.T) {
func TestBlockSizeParamsMarshalTo(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSize(popr, false)
p := NewPopulatedBlockSizeParams(popr, false)
size := p.Size()
dAtA := make([]byte, size)
for i := range dAtA {
@@ -1522,7 +1523,7 @@ func TestBlockSizeMarshalTo(t *testing.T) {
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockSize{}
msg := &BlockSizeParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1534,15 +1535,15 @@ func TestBlockSizeMarshalTo(t *testing.T) {
}
}
func TestTxSizeProto(t *testing.T) {
func TestEvidenceParamsProto(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedTxSize(popr, false)
p := NewPopulatedEvidenceParams(popr, false)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &TxSize{}
msg := &EvidenceParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1565,10 +1566,10 @@ func TestTxSizeProto(t *testing.T) {
}
}
func TestTxSizeMarshalTo(t *testing.T) {
func TestEvidenceParamsMarshalTo(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedTxSize(popr, false)
p := NewPopulatedEvidenceParams(popr, false)
size := p.Size()
dAtA := make([]byte, size)
for i := range dAtA {
@@ -1578,7 +1579,7 @@ func TestTxSizeMarshalTo(t *testing.T) {
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &TxSize{}
msg := &EvidenceParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1590,15 +1591,15 @@ func TestTxSizeMarshalTo(t *testing.T) {
}
}
func TestBlockGossipProto(t *testing.T) {
func TestValidatorParamsProto(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockGossip(popr, false)
p := NewPopulatedValidatorParams(popr, false)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockGossip{}
msg := &ValidatorParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1621,10 +1622,10 @@ func TestBlockGossipProto(t *testing.T) {
}
}
func TestBlockGossipMarshalTo(t *testing.T) {
func TestValidatorParamsMarshalTo(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockGossip(popr, false)
p := NewPopulatedValidatorParams(popr, false)
size := p.Size()
dAtA := make([]byte, size)
for i := range dAtA {
@@ -1634,7 +1635,7 @@ func TestBlockGossipMarshalTo(t *testing.T) {
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockGossip{}
msg := &ValidatorParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1758,6 +1759,62 @@ func TestHeaderMarshalTo(t *testing.T) {
}
}
func TestVersionProto(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedVersion(popr, false)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &Version{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
littlefuzz := make([]byte, len(dAtA))
copy(littlefuzz, dAtA)
for i := range dAtA {
dAtA[i] = byte(popr.Intn(256))
}
if !p.Equal(msg) {
t.Fatalf("seed = %d, %#v !Proto %#v", seed, msg, p)
}
if len(littlefuzz) > 0 {
fuzzamount := 100
for i := 0; i < fuzzamount; i++ {
littlefuzz[popr.Intn(len(littlefuzz))] = byte(popr.Intn(256))
littlefuzz = append(littlefuzz, byte(popr.Intn(256)))
}
// shouldn't panic
_ = github_com_gogo_protobuf_proto.Unmarshal(littlefuzz, msg)
}
}
func TestVersionMarshalTo(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedVersion(popr, false)
size := p.Size()
dAtA := make([]byte, size)
for i := range dAtA {
dAtA[i] = byte(popr.Intn(256))
}
_, err := p.MarshalTo(dAtA)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &Version{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
for i := range dAtA {
dAtA[i] = byte(popr.Intn(256))
}
if !p.Equal(msg) {
t.Fatalf("seed = %d, %#v !Proto %#v", seed, msg, p)
}
}
func TestBlockIDProto(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
@@ -2618,16 +2675,16 @@ func TestConsensusParamsJSON(t *testing.T) {
t.Fatalf("seed = %d, %#v !Json Equal %#v", seed, msg, p)
}
}
func TestBlockSizeJSON(t *testing.T) {
func TestBlockSizeParamsJSON(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSize(popr, true)
p := NewPopulatedBlockSizeParams(popr, true)
marshaler := github_com_gogo_protobuf_jsonpb.Marshaler{}
jsondata, err := marshaler.MarshalToString(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockSize{}
msg := &BlockSizeParams{}
err = github_com_gogo_protobuf_jsonpb.UnmarshalString(jsondata, msg)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
@@ -2636,16 +2693,16 @@ func TestBlockSizeJSON(t *testing.T) {
t.Fatalf("seed = %d, %#v !Json Equal %#v", seed, msg, p)
}
}
func TestTxSizeJSON(t *testing.T) {
func TestEvidenceParamsJSON(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedTxSize(popr, true)
p := NewPopulatedEvidenceParams(popr, true)
marshaler := github_com_gogo_protobuf_jsonpb.Marshaler{}
jsondata, err := marshaler.MarshalToString(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &TxSize{}
msg := &EvidenceParams{}
err = github_com_gogo_protobuf_jsonpb.UnmarshalString(jsondata, msg)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
@@ -2654,16 +2711,16 @@ func TestTxSizeJSON(t *testing.T) {
t.Fatalf("seed = %d, %#v !Json Equal %#v", seed, msg, p)
}
}
func TestBlockGossipJSON(t *testing.T) {
func TestValidatorParamsJSON(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockGossip(popr, true)
p := NewPopulatedValidatorParams(popr, true)
marshaler := github_com_gogo_protobuf_jsonpb.Marshaler{}
jsondata, err := marshaler.MarshalToString(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockGossip{}
msg := &ValidatorParams{}
err = github_com_gogo_protobuf_jsonpb.UnmarshalString(jsondata, msg)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
@@ -2708,6 +2765,24 @@ func TestHeaderJSON(t *testing.T) {
t.Fatalf("seed = %d, %#v !Json Equal %#v", seed, msg, p)
}
}
func TestVersionJSON(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedVersion(popr, true)
marshaler := github_com_gogo_protobuf_jsonpb.Marshaler{}
jsondata, err := marshaler.MarshalToString(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &Version{}
err = github_com_gogo_protobuf_jsonpb.UnmarshalString(jsondata, msg)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
if !p.Equal(msg) {
t.Fatalf("seed = %d, %#v !Json Equal %#v", seed, msg, p)
}
}
func TestBlockIDJSON(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
@@ -3562,12 +3637,12 @@ func TestConsensusParamsProtoCompactText(t *testing.T) {
}
}
func TestBlockSizeProtoText(t *testing.T) {
func TestBlockSizeParamsProtoText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSize(popr, true)
p := NewPopulatedBlockSizeParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.MarshalTextString(p)
msg := &BlockSize{}
msg := &BlockSizeParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3576,12 +3651,12 @@ func TestBlockSizeProtoText(t *testing.T) {
}
}
func TestBlockSizeProtoCompactText(t *testing.T) {
func TestBlockSizeParamsProtoCompactText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSize(popr, true)
p := NewPopulatedBlockSizeParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.CompactTextString(p)
msg := &BlockSize{}
msg := &BlockSizeParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3590,12 +3665,12 @@ func TestBlockSizeProtoCompactText(t *testing.T) {
}
}
func TestTxSizeProtoText(t *testing.T) {
func TestEvidenceParamsProtoText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedTxSize(popr, true)
p := NewPopulatedEvidenceParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.MarshalTextString(p)
msg := &TxSize{}
msg := &EvidenceParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3604,12 +3679,12 @@ func TestTxSizeProtoText(t *testing.T) {
}
}
func TestTxSizeProtoCompactText(t *testing.T) {
func TestEvidenceParamsProtoCompactText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedTxSize(popr, true)
p := NewPopulatedEvidenceParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.CompactTextString(p)
msg := &TxSize{}
msg := &EvidenceParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3618,12 +3693,12 @@ func TestTxSizeProtoCompactText(t *testing.T) {
}
}
func TestBlockGossipProtoText(t *testing.T) {
func TestValidatorParamsProtoText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockGossip(popr, true)
p := NewPopulatedValidatorParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.MarshalTextString(p)
msg := &BlockGossip{}
msg := &ValidatorParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3632,12 +3707,12 @@ func TestBlockGossipProtoText(t *testing.T) {
}
}
func TestBlockGossipProtoCompactText(t *testing.T) {
func TestValidatorParamsProtoCompactText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockGossip(popr, true)
p := NewPopulatedValidatorParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.CompactTextString(p)
msg := &BlockGossip{}
msg := &ValidatorParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3702,6 +3777,34 @@ func TestHeaderProtoCompactText(t *testing.T) {
}
}
func TestVersionProtoText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedVersion(popr, true)
dAtA := github_com_gogo_protobuf_proto.MarshalTextString(p)
msg := &Version{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
if !p.Equal(msg) {
t.Fatalf("seed = %d, %#v !Proto %#v", seed, msg, p)
}
}
func TestVersionProtoCompactText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedVersion(popr, true)
dAtA := github_com_gogo_protobuf_proto.CompactTextString(p)
msg := &Version{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
if !p.Equal(msg) {
t.Fatalf("seed = %d, %#v !Proto %#v", seed, msg, p)
}
}
func TestBlockIDProtoText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
@@ -4470,10 +4573,10 @@ func TestConsensusParamsSize(t *testing.T) {
}
}
func TestBlockSizeSize(t *testing.T) {
func TestBlockSizeParamsSize(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSize(popr, true)
p := NewPopulatedBlockSizeParams(popr, true)
size2 := github_com_gogo_protobuf_proto.Size(p)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
@@ -4492,10 +4595,10 @@ func TestBlockSizeSize(t *testing.T) {
}
}
func TestTxSizeSize(t *testing.T) {
func TestEvidenceParamsSize(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedTxSize(popr, true)
p := NewPopulatedEvidenceParams(popr, true)
size2 := github_com_gogo_protobuf_proto.Size(p)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
@@ -4514,10 +4617,10 @@ func TestTxSizeSize(t *testing.T) {
}
}
func TestBlockGossipSize(t *testing.T) {
func TestValidatorParamsSize(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockGossip(popr, true)
p := NewPopulatedValidatorParams(popr, true)
size2 := github_com_gogo_protobuf_proto.Size(p)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
@@ -4580,6 +4683,28 @@ func TestHeaderSize(t *testing.T) {
}
}
func TestVersionSize(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedVersion(popr, true)
size2 := github_com_gogo_protobuf_proto.Size(p)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
size := p.Size()
if len(dAtA) != size {
t.Errorf("seed = %d, size %v != marshalled size %v", seed, size, len(dAtA))
}
if size2 != size {
t.Errorf("seed = %d, size %v != before marshal proto.Size %v", seed, size, size2)
}
size3 := github_com_gogo_protobuf_proto.Size(p)
if size3 != size {
t.Errorf("seed = %d, size %v != after marshal proto.Size %v", seed, size, size3)
}
}
func TestBlockIDSize(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))

View File

@@ -1,9 +1,9 @@
package version
// NOTE: we should probably be versioning the ABCI and the abci-cli separately
import (
"github.com/tendermint/tendermint/version"
)
const Maj = "0"
const Min = "12"
const Fix = "0"
// TODO: eliminate this after some version refactor
const Version = "0.12.0"
const Version = version.ABCIVersion

View File

@@ -12,20 +12,28 @@ import (
ctypes "github.com/tendermint/tendermint/rpc/core/types"
)
func testNodeInfo(id p2p.ID) p2p.DefaultNodeInfo {
return p2p.DefaultNodeInfo{
ProtocolVersion: p2p.ProtocolVersion{1, 2, 3},
ID_: id,
Moniker: "SOMENAME",
Network: "SOMENAME",
ListenAddr: "SOMEADDR",
Version: "SOMEVER",
Other: p2p.DefaultNodeInfoOther{
TxIndex: "on",
RPCAddress: "0.0.0.0:26657",
},
}
}
func BenchmarkEncodeStatusWire(b *testing.B) {
b.StopTimer()
cdc := amino.NewCodec()
ctypes.RegisterAmino(cdc)
nodeKey := p2p.NodeKey{PrivKey: ed25519.GenPrivKey()}
status := &ctypes.ResultStatus{
NodeInfo: p2p.NodeInfo{
ID: nodeKey.ID(),
Moniker: "SOMENAME",
Network: "SOMENAME",
ListenAddr: "SOMEADDR",
Version: "SOMEVER",
Other: []string{"SOMESTRING", "OTHERSTRING"},
},
NodeInfo: testNodeInfo(nodeKey.ID()),
SyncInfo: ctypes.SyncInfo{
LatestBlockHash: []byte("SOMEBYTES"),
LatestBlockHeight: 123,
@@ -53,14 +61,7 @@ func BenchmarkEncodeNodeInfoWire(b *testing.B) {
cdc := amino.NewCodec()
ctypes.RegisterAmino(cdc)
nodeKey := p2p.NodeKey{PrivKey: ed25519.GenPrivKey()}
nodeInfo := p2p.NodeInfo{
ID: nodeKey.ID(),
Moniker: "SOMENAME",
Network: "SOMENAME",
ListenAddr: "SOMEADDR",
Version: "SOMEVER",
Other: []string{"SOMESTRING", "OTHERSTRING"},
}
nodeInfo := testNodeInfo(nodeKey.ID())
b.StartTimer()
counter := 0
@@ -78,14 +79,7 @@ func BenchmarkEncodeNodeInfoBinary(b *testing.B) {
cdc := amino.NewCodec()
ctypes.RegisterAmino(cdc)
nodeKey := p2p.NodeKey{PrivKey: ed25519.GenPrivKey()}
nodeInfo := p2p.NodeInfo{
ID: nodeKey.ID(),
Moniker: "SOMENAME",
Network: "SOMENAME",
ListenAddr: "SOMEADDR",
Version: "SOMEVER",
Other: []string{"SOMESTRING", "OTHERSTRING"},
}
nodeInfo := testNodeInfo(nodeKey.ID())
b.StartTimer()
counter := 0

View File

@@ -168,9 +168,12 @@ func (pool *BlockPool) IsCaughtUp() bool {
return false
}
// some conditions to determine if we're caught up
receivedBlockOrTimedOut := (pool.height > 0 || time.Since(pool.startTime) > 5*time.Second)
ourChainIsLongestAmongPeers := pool.maxPeerHeight == 0 || pool.height >= pool.maxPeerHeight
// Some conditions to determine if we're caught up.
// Ensures we've either received a block or waited some amount of time,
// and that we're synced to the highest known height. Note we use maxPeerHeight - 1
// because to sync block H requires block H+1 to verify the LastCommit.
receivedBlockOrTimedOut := pool.height > 0 || time.Since(pool.startTime) > 5*time.Second
ourChainIsLongestAmongPeers := pool.maxPeerHeight == 0 || pool.height >= (pool.maxPeerHeight-1)
isCaughtUp := receivedBlockOrTimedOut && ourChainIsLongestAmongPeers
return isCaughtUp
}
@@ -252,7 +255,8 @@ func (pool *BlockPool) AddBlock(peerID p2p.ID, block *types.Block, blockSize int
peer.decrPending(blockSize)
}
} else {
// Bad peer?
pool.Logger.Info("invalid peer", "peer", peerID, "blockHeight", block.Height)
pool.sendError(errors.New("invalid peer"), peerID)
}
}
@@ -292,7 +296,7 @@ func (pool *BlockPool) RemovePeer(peerID p2p.ID) {
func (pool *BlockPool) removePeer(peerID p2p.ID) {
for _, requester := range pool.requesters {
if requester.getPeerID() == peerID {
requester.redo()
requester.redo(peerID)
}
}
delete(pool.peers, peerID)
@@ -326,8 +330,11 @@ func (pool *BlockPool) makeNextRequester() {
defer pool.mtx.Unlock()
nextHeight := pool.height + pool.requestersLen()
if nextHeight > pool.maxPeerHeight {
return
}
request := newBPRequester(pool, nextHeight)
// request.SetLogger(pool.Logger.With("height", nextHeight))
pool.requesters[nextHeight] = request
atomic.AddInt32(&pool.numPending, 1)
@@ -453,7 +460,7 @@ type bpRequester struct {
pool *BlockPool
height int64
gotBlockCh chan struct{}
redoCh chan struct{}
redoCh chan p2p.ID //redo may send multitime, add peerId to identify repeat
mtx sync.Mutex
peerID p2p.ID
@@ -465,7 +472,7 @@ func newBPRequester(pool *BlockPool, height int64) *bpRequester {
pool: pool,
height: height,
gotBlockCh: make(chan struct{}, 1),
redoCh: make(chan struct{}, 1),
redoCh: make(chan p2p.ID, 1),
peerID: "",
block: nil,
@@ -524,9 +531,9 @@ func (bpr *bpRequester) reset() {
// Tells bpRequester to pick another peer and try again.
// NOTE: Nonblocking, and does nothing if another redo
// was already requested.
func (bpr *bpRequester) redo() {
func (bpr *bpRequester) redo(peerId p2p.ID) {
select {
case bpr.redoCh <- struct{}{}:
case bpr.redoCh <- peerId:
default:
}
}
@@ -565,9 +572,13 @@ OUTER_LOOP:
return
case <-bpr.Quit():
return
case <-bpr.redoCh:
bpr.reset()
continue OUTER_LOOP
case peerID := <-bpr.redoCh:
if peerID == bpr.peerID {
bpr.reset()
continue OUTER_LOOP
} else {
continue WAIT_LOOP
}
case <-bpr.gotBlockCh:
// We got a block!
// Continue the for-loop and wait til Quit.

View File

@@ -16,16 +16,52 @@ func init() {
}
type testPeer struct {
id p2p.ID
height int64
id p2p.ID
height int64
inputChan chan inputData //make sure each peer's data is sequential
}
func makePeers(numPeers int, minHeight, maxHeight int64) map[p2p.ID]testPeer {
peers := make(map[p2p.ID]testPeer, numPeers)
type inputData struct {
t *testing.T
pool *BlockPool
request BlockRequest
}
func (p testPeer) runInputRoutine() {
go func() {
for input := range p.inputChan {
p.simulateInput(input)
}
}()
}
// Request desired, pretend like we got the block immediately.
func (p testPeer) simulateInput(input inputData) {
block := &types.Block{Header: types.Header{Height: input.request.Height}}
input.pool.AddBlock(input.request.PeerID, block, 123)
input.t.Logf("Added block from peer %v (height: %v)", input.request.PeerID, input.request.Height)
}
type testPeers map[p2p.ID]testPeer
func (ps testPeers) start() {
for _, v := range ps {
v.runInputRoutine()
}
}
func (ps testPeers) stop() {
for _, v := range ps {
close(v.inputChan)
}
}
func makePeers(numPeers int, minHeight, maxHeight int64) testPeers {
peers := make(testPeers, numPeers)
for i := 0; i < numPeers; i++ {
peerID := p2p.ID(cmn.RandStr(12))
height := minHeight + cmn.RandInt63n(maxHeight-minHeight)
peers[peerID] = testPeer{peerID, height}
peers[peerID] = testPeer{peerID, height, make(chan inputData, 10)}
}
return peers
}
@@ -45,6 +81,9 @@ func TestBasic(t *testing.T) {
defer pool.Stop()
peers.start()
defer peers.stop()
// Introduce each peer.
go func() {
for _, peer := range peers {
@@ -77,12 +116,8 @@ func TestBasic(t *testing.T) {
if request.Height == 300 {
return // Done!
}
// Request desired, pretend like we got the block immediately.
go func() {
block := &types.Block{Header: types.Header{Height: request.Height}}
pool.AddBlock(request.PeerID, block, 123)
t.Logf("Added block from peer %v (height: %v)", request.PeerID, request.Height)
}()
peers[request.PeerID].inputChan <- inputData{t, pool, request}
}
}
}

View File

@@ -1,6 +1,7 @@
package blockchain
import (
"errors"
"fmt"
"reflect"
"time"
@@ -180,6 +181,12 @@ func (bcR *BlockchainReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
return
}
if err = msg.ValidateBasic(); err != nil {
bcR.Logger.Error("Peer sent us invalid msg", "peer", src, "msg", msg, "err", err)
bcR.Switch.StopPeerForError(src, err)
return
}
bcR.Logger.Debug("Receive", "src", src, "chID", chID, "msg", msg)
switch msg := msg.(type) {
@@ -188,7 +195,6 @@ func (bcR *BlockchainReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
// Unfortunately not queued since the queue is full.
}
case *bcBlockResponseMessage:
// Got a block.
bcR.pool.AddBlock(src.ID(), msg.Block, len(msgBytes))
case *bcStatusRequestMessage:
// Send peer our state.
@@ -258,8 +264,12 @@ FOR_LOOP:
bcR.Logger.Info("Time to switch to consensus reactor!", "height", height)
bcR.pool.Stop()
conR := bcR.Switch.Reactor("CONSENSUS").(consensusReactor)
conR.SwitchToConsensus(state, blocksSynced)
conR, ok := bcR.Switch.Reactor("CONSENSUS").(consensusReactor)
if ok {
conR.SwitchToConsensus(state, blocksSynced)
} else {
// should only happen during testing
}
break FOR_LOOP
}
@@ -290,7 +300,7 @@ FOR_LOOP:
didProcessCh <- struct{}{}
}
firstParts := first.MakePartSet(state.ConsensusParams.BlockPartSizeBytes)
firstParts := first.MakePartSet(types.BlockPartSizeBytes)
firstPartsHeader := firstParts.Header()
firstID := types.BlockID{first.Hash(), firstPartsHeader}
// Finally, verify the first block using the second's commit
@@ -308,6 +318,13 @@ FOR_LOOP:
// still need to clean up the rest.
bcR.Switch.StopPeerForError(peer, fmt.Errorf("BlockchainReactor validation error: %v", err))
}
peerID2 := bcR.pool.RedoRequest(second.Height)
peer2 := bcR.Switch.Peers().Get(peerID2)
if peer2 != nil && peer2 != peer {
// NOTE: we've already removed the peer's request, but we
// still need to clean up the rest.
bcR.Switch.StopPeerForError(peer2, fmt.Errorf("BlockchainReactor validation error: %v", err))
}
continue FOR_LOOP
} else {
bcR.pool.PopRequest()
@@ -352,7 +369,9 @@ func (bcR *BlockchainReactor) BroadcastStatusRequest() error {
// Messages
// BlockchainMessage is a generic message for this reactor.
type BlockchainMessage interface{}
type BlockchainMessage interface {
ValidateBasic() error
}
func RegisterBlockchainMessages(cdc *amino.Codec) {
cdc.RegisterInterface((*BlockchainMessage)(nil), nil)
@@ -377,6 +396,14 @@ type bcBlockRequestMessage struct {
Height int64
}
// ValidateBasic performs basic validation.
func (m *bcBlockRequestMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
return nil
}
func (m *bcBlockRequestMessage) String() string {
return fmt.Sprintf("[bcBlockRequestMessage %v]", m.Height)
}
@@ -385,6 +412,14 @@ type bcNoBlockResponseMessage struct {
Height int64
}
// ValidateBasic performs basic validation.
func (m *bcNoBlockResponseMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
return nil
}
func (brm *bcNoBlockResponseMessage) String() string {
return fmt.Sprintf("[bcNoBlockResponseMessage %d]", brm.Height)
}
@@ -395,6 +430,15 @@ type bcBlockResponseMessage struct {
Block *types.Block
}
// ValidateBasic performs basic validation.
func (m *bcBlockResponseMessage) ValidateBasic() error {
if err := m.Block.ValidateBasic(); err != nil {
return err
}
return nil
}
func (m *bcBlockResponseMessage) String() string {
return fmt.Sprintf("[bcBlockResponseMessage %v]", m.Block.Height)
}
@@ -405,6 +449,14 @@ type bcStatusRequestMessage struct {
Height int64
}
// ValidateBasic performs basic validation.
func (m *bcStatusRequestMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
return nil
}
func (m *bcStatusRequestMessage) String() string {
return fmt.Sprintf("[bcStatusRequestMessage %v]", m.Height)
}
@@ -415,6 +467,14 @@ type bcStatusResponseMessage struct {
Height int64
}
// ValidateBasic performs basic validation.
func (m *bcStatusResponseMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
return nil
}
func (m *bcStatusResponseMessage) String() string {
return fmt.Sprintf("[bcStatusResponseMessage %v]", m.Height)
}

View File

@@ -1,72 +1,151 @@
package blockchain
import (
"net"
"sort"
"testing"
"time"
"github.com/stretchr/testify/assert"
abci "github.com/tendermint/tendermint/abci/types"
cfg "github.com/tendermint/tendermint/config"
cmn "github.com/tendermint/tendermint/libs/common"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/libs/log"
cfg "github.com/tendermint/tendermint/config"
"github.com/tendermint/tendermint/p2p"
"github.com/tendermint/tendermint/proxy"
sm "github.com/tendermint/tendermint/state"
"github.com/tendermint/tendermint/types"
tmtime "github.com/tendermint/tendermint/types/time"
)
func makeStateAndBlockStore(logger log.Logger) (sm.State, *BlockStore) {
config := cfg.ResetTestRoot("blockchain_reactor_test")
// blockDB := dbm.NewDebugDB("blockDB", dbm.NewMemDB())
// stateDB := dbm.NewDebugDB("stateDB", dbm.NewMemDB())
var config *cfg.Config
func randGenesisDoc(numValidators int, randPower bool, minPower int64) (*types.GenesisDoc, []types.PrivValidator) {
validators := make([]types.GenesisValidator, numValidators)
privValidators := make([]types.PrivValidator, numValidators)
for i := 0; i < numValidators; i++ {
val, privVal := types.RandValidator(randPower, minPower)
validators[i] = types.GenesisValidator{
PubKey: val.PubKey,
Power: val.VotingPower,
}
privValidators[i] = privVal
}
sort.Sort(types.PrivValidatorsByAddress(privValidators))
return &types.GenesisDoc{
GenesisTime: tmtime.Now(),
ChainID: config.ChainID(),
Validators: validators,
}, privValidators
}
func makeVote(header *types.Header, blockID types.BlockID, valset *types.ValidatorSet, privVal types.PrivValidator) *types.Vote {
addr := privVal.GetAddress()
idx, _ := valset.GetByAddress(addr)
vote := &types.Vote{
ValidatorAddress: addr,
ValidatorIndex: idx,
Height: header.Height,
Round: 1,
Timestamp: tmtime.Now(),
Type: types.PrecommitType,
BlockID: blockID,
}
privVal.SignVote(header.ChainID, vote)
return vote
}
type BlockchainReactorPair struct {
reactor *BlockchainReactor
app proxy.AppConns
}
func newBlockchainReactor(logger log.Logger, genDoc *types.GenesisDoc, privVals []types.PrivValidator, maxBlockHeight int64) BlockchainReactorPair {
if len(privVals) != 1 {
panic("only support one validator")
}
app := &testApp{}
cc := proxy.NewLocalClientCreator(app)
proxyApp := proxy.NewAppConns(cc)
err := proxyApp.Start()
if err != nil {
panic(cmn.ErrorWrap(err, "error start app"))
}
blockDB := dbm.NewMemDB()
stateDB := dbm.NewMemDB()
blockStore := NewBlockStore(blockDB)
state, err := sm.LoadStateFromDBOrGenesisFile(stateDB, config.GenesisFile())
state, err := sm.LoadStateFromDBOrGenesisDoc(stateDB, genDoc)
if err != nil {
panic(cmn.ErrorWrap(err, "error constructing state from genesis file"))
}
return state, blockStore
}
func newBlockchainReactor(logger log.Logger, maxBlockHeight int64) *BlockchainReactor {
state, blockStore := makeStateAndBlockStore(logger)
// Make the blockchainReactor itself
// Make the BlockchainReactor itself.
// NOTE we have to create and commit the blocks first because
// pool.height is determined from the store.
fastSync := true
var nilApp proxy.AppConnConsensus
blockExec := sm.NewBlockExecutor(dbm.NewMemDB(), log.TestingLogger(), nilApp,
blockExec := sm.NewBlockExecutor(dbm.NewMemDB(), log.TestingLogger(), proxyApp.Consensus(),
sm.MockMempool{}, sm.MockEvidencePool{})
// let's add some blocks in
for blockHeight := int64(1); blockHeight <= maxBlockHeight; blockHeight++ {
lastCommit := &types.Commit{}
if blockHeight > 1 {
lastBlockMeta := blockStore.LoadBlockMeta(blockHeight - 1)
lastBlock := blockStore.LoadBlock(blockHeight - 1)
vote := makeVote(&lastBlock.Header, lastBlockMeta.BlockID, state.Validators, privVals[0])
lastCommit = &types.Commit{Precommits: []*types.Vote{vote}, BlockID: lastBlockMeta.BlockID}
}
thisBlock := makeBlock(blockHeight, state, lastCommit)
thisParts := thisBlock.MakePartSet(types.BlockPartSizeBytes)
blockID := types.BlockID{thisBlock.Hash(), thisParts.Header()}
state, err = blockExec.ApplyBlock(state, blockID, thisBlock)
if err != nil {
panic(cmn.ErrorWrap(err, "error apply block"))
}
blockStore.SaveBlock(thisBlock, thisParts, lastCommit)
}
bcReactor := NewBlockchainReactor(state.Copy(), blockExec, blockStore, fastSync)
bcReactor.SetLogger(logger.With("module", "blockchain"))
// Next: we need to set a switch in order for peers to be added in
bcReactor.Switch = p2p.NewSwitch(cfg.DefaultP2PConfig())
// Lastly: let's add some blocks in
for blockHeight := int64(1); blockHeight <= maxBlockHeight; blockHeight++ {
firstBlock := makeBlock(blockHeight, state)
secondBlock := makeBlock(blockHeight+1, state)
firstParts := firstBlock.MakePartSet(state.ConsensusParams.BlockGossip.BlockPartSizeBytes)
blockStore.SaveBlock(firstBlock, firstParts, secondBlock.LastCommit)
}
return bcReactor
return BlockchainReactorPair{bcReactor, proxyApp}
}
func TestNoBlockResponse(t *testing.T) {
maxBlockHeight := int64(20)
config = cfg.ResetTestRoot("blockchain_reactor_test")
genDoc, privVals := randGenesisDoc(1, false, 30)
bcr := newBlockchainReactor(log.TestingLogger(), maxBlockHeight)
bcr.Start()
defer bcr.Stop()
maxBlockHeight := int64(65)
// Add some peers in
peer := newbcrTestPeer(p2p.ID(cmn.RandStr(12)))
bcr.AddPeer(peer)
reactorPairs := make([]BlockchainReactorPair, 2)
chID := byte(0x01)
reactorPairs[0] = newBlockchainReactor(log.TestingLogger(), genDoc, privVals, maxBlockHeight)
reactorPairs[1] = newBlockchainReactor(log.TestingLogger(), genDoc, privVals, 0)
p2p.MakeConnectedSwitches(config.P2P, 2, func(i int, s *p2p.Switch) *p2p.Switch {
s.AddReactor("BLOCKCHAIN", reactorPairs[i].reactor)
return s
}, p2p.Connect2Switches)
defer func() {
for _, r := range reactorPairs {
r.reactor.Stop()
r.app.Stop()
}
}()
tests := []struct {
height int64
@@ -78,72 +157,100 @@ func TestNoBlockResponse(t *testing.T) {
{100, false},
}
// receive a request message from peer,
// wait for our response to be received on the peer
for _, tt := range tests {
reqBlockMsg := &bcBlockRequestMessage{tt.height}
reqBlockBytes := cdc.MustMarshalBinaryBare(reqBlockMsg)
bcr.Receive(chID, peer, reqBlockBytes)
msg := peer.lastBlockchainMessage()
for {
if reactorPairs[1].reactor.pool.IsCaughtUp() {
break
}
time.Sleep(10 * time.Millisecond)
}
assert.Equal(t, maxBlockHeight, reactorPairs[0].reactor.store.Height())
for _, tt := range tests {
block := reactorPairs[1].reactor.store.LoadBlock(tt.height)
if tt.existent {
if blockMsg, ok := msg.(*bcBlockResponseMessage); !ok {
t.Fatalf("Expected to receive a block response for height %d", tt.height)
} else if blockMsg.Block.Height != tt.height {
t.Fatalf("Expected response to be for height %d, got %d", tt.height, blockMsg.Block.Height)
}
assert.True(t, block != nil)
} else {
if noBlockMsg, ok := msg.(*bcNoBlockResponseMessage); !ok {
t.Fatalf("Expected to receive a no block response for height %d", tt.height)
} else if noBlockMsg.Height != tt.height {
t.Fatalf("Expected response to be for height %d, got %d", tt.height, noBlockMsg.Height)
}
assert.True(t, block == nil)
}
}
}
/*
// NOTE: This is too hard to test without
// an easy way to add test peer to switch
// or without significant refactoring of the module.
// Alternatively we could actually dial a TCP conn but
// that seems extreme.
func TestBadBlockStopsPeer(t *testing.T) {
maxBlockHeight := int64(20)
config = cfg.ResetTestRoot("blockchain_reactor_test")
genDoc, privVals := randGenesisDoc(1, false, 30)
bcr := newBlockchainReactor(log.TestingLogger(), maxBlockHeight)
bcr.Start()
defer bcr.Stop()
maxBlockHeight := int64(148)
// Add some peers in
peer := newbcrTestPeer(p2p.ID(cmn.RandStr(12)))
otherChain := newBlockchainReactor(log.TestingLogger(), genDoc, privVals, maxBlockHeight)
defer func() {
otherChain.reactor.Stop()
otherChain.app.Stop()
}()
// XXX: This doesn't add the peer to anything,
// so it's hard to check that it's later removed
bcr.AddPeer(peer)
assert.True(t, bcr.Switch.Peers().Size() > 0)
reactorPairs := make([]BlockchainReactorPair, 4)
// send a bad block from the peer
// default blocks already dont have commits, so should fail
block := bcr.store.LoadBlock(3)
msg := &bcBlockResponseMessage{Block: block}
peer.Send(BlockchainChannel, struct{ BlockchainMessage }{msg})
reactorPairs[0] = newBlockchainReactor(log.TestingLogger(), genDoc, privVals, maxBlockHeight)
reactorPairs[1] = newBlockchainReactor(log.TestingLogger(), genDoc, privVals, 0)
reactorPairs[2] = newBlockchainReactor(log.TestingLogger(), genDoc, privVals, 0)
reactorPairs[3] = newBlockchainReactor(log.TestingLogger(), genDoc, privVals, 0)
ticker := time.NewTicker(time.Millisecond * 10)
timer := time.NewTimer(time.Second * 2)
LOOP:
for {
select {
case <-ticker.C:
if bcr.Switch.Peers().Size() == 0 {
break LOOP
}
case <-timer.C:
t.Fatal("Timed out waiting to disconnect peer")
switches := p2p.MakeConnectedSwitches(config.P2P, 4, func(i int, s *p2p.Switch) *p2p.Switch {
s.AddReactor("BLOCKCHAIN", reactorPairs[i].reactor)
return s
}, p2p.Connect2Switches)
defer func() {
for _, r := range reactorPairs {
r.reactor.Stop()
r.app.Stop()
}
}()
for {
if reactorPairs[3].reactor.pool.IsCaughtUp() {
break
}
time.Sleep(1 * time.Second)
}
//at this time, reactors[0-3] is the newest
assert.Equal(t, 3, reactorPairs[1].reactor.Switch.Peers().Size())
//mark reactorPairs[3] is an invalid peer
reactorPairs[3].reactor.store = otherChain.reactor.store
lastReactorPair := newBlockchainReactor(log.TestingLogger(), genDoc, privVals, 0)
reactorPairs = append(reactorPairs, lastReactorPair)
switches = append(switches, p2p.MakeConnectedSwitches(config.P2P, 1, func(i int, s *p2p.Switch) *p2p.Switch {
s.AddReactor("BLOCKCHAIN", reactorPairs[len(reactorPairs)-1].reactor)
return s
}, p2p.Connect2Switches)...)
for i := 0; i < len(reactorPairs)-1; i++ {
p2p.Connect2Switches(switches, i, len(reactorPairs)-1)
}
for {
if lastReactorPair.reactor.pool.IsCaughtUp() || lastReactorPair.reactor.Switch.Peers().Size() == 0 {
break
}
time.Sleep(1 * time.Second)
}
assert.True(t, lastReactorPair.reactor.Switch.Peers().Size() < len(reactorPairs)-1)
}
*/
//----------------------------------------------
// utility funcs
@@ -155,55 +262,41 @@ func makeTxs(height int64) (txs []types.Tx) {
return txs
}
func makeBlock(height int64, state sm.State) *types.Block {
block, _ := state.MakeBlock(height, makeTxs(height), new(types.Commit), nil, state.Validators.GetProposer().Address)
func makeBlock(height int64, state sm.State, lastCommit *types.Commit) *types.Block {
block, _ := state.MakeBlock(height, makeTxs(height), lastCommit, nil, state.Validators.GetProposer().Address)
return block
}
// The Test peer
type bcrTestPeer struct {
cmn.BaseService
id p2p.ID
ch chan interface{}
type testApp struct {
abci.BaseApplication
}
var _ p2p.Peer = (*bcrTestPeer)(nil)
var _ abci.Application = (*testApp)(nil)
func newbcrTestPeer(id p2p.ID) *bcrTestPeer {
bcr := &bcrTestPeer{
id: id,
ch: make(chan interface{}, 2),
}
bcr.BaseService = *cmn.NewBaseService(nil, "bcrTestPeer", bcr)
return bcr
func (app *testApp) Info(req abci.RequestInfo) (resInfo abci.ResponseInfo) {
return abci.ResponseInfo{}
}
func (tp *bcrTestPeer) lastBlockchainMessage() interface{} { return <-tp.ch }
func (tp *bcrTestPeer) TrySend(chID byte, msgBytes []byte) bool {
var msg BlockchainMessage
err := cdc.UnmarshalBinaryBare(msgBytes, &msg)
if err != nil {
panic(cmn.ErrorWrap(err, "Error while trying to parse a BlockchainMessage"))
}
if _, ok := msg.(*bcStatusResponseMessage); ok {
// Discard status response messages since they skew our results
// We only want to deal with:
// + bcBlockResponseMessage
// + bcNoBlockResponseMessage
} else {
tp.ch <- msg
}
return true
func (app *testApp) BeginBlock(req abci.RequestBeginBlock) abci.ResponseBeginBlock {
return abci.ResponseBeginBlock{}
}
func (tp *bcrTestPeer) Send(chID byte, msgBytes []byte) bool { return tp.TrySend(chID, msgBytes) }
func (tp *bcrTestPeer) NodeInfo() p2p.NodeInfo { return p2p.NodeInfo{} }
func (tp *bcrTestPeer) Status() p2p.ConnectionStatus { return p2p.ConnectionStatus{} }
func (tp *bcrTestPeer) ID() p2p.ID { return tp.id }
func (tp *bcrTestPeer) IsOutbound() bool { return false }
func (tp *bcrTestPeer) IsPersistent() bool { return true }
func (tp *bcrTestPeer) Get(s string) interface{} { return s }
func (tp *bcrTestPeer) Set(string, interface{}) {}
func (tp *bcrTestPeer) RemoteIP() net.IP { return []byte{127, 0, 0, 1} }
func (tp *bcrTestPeer) OriginalAddr() *p2p.NetAddress { return nil }
func (app *testApp) EndBlock(req abci.RequestEndBlock) abci.ResponseEndBlock {
return abci.ResponseEndBlock{}
}
func (app *testApp) DeliverTx(tx []byte) abci.ResponseDeliverTx {
return abci.ResponseDeliverTx{Tags: []cmn.KVPair{}}
}
func (app *testApp) CheckTx(tx []byte) abci.ResponseCheckTx {
return abci.ResponseCheckTx{}
}
func (app *testApp) Commit() abci.ResponseCommit {
return abci.ResponseCommit{}
}
func (app *testApp) Query(reqQuery abci.RequestQuery) (resQuery abci.ResponseQuery) {
return
}

View File

@@ -63,7 +63,7 @@ func (bs *BlockStore) LoadBlock(height int64) *types.Block {
part := bs.LoadBlockPart(height, i)
buf = append(buf, part.Bytes...)
}
err := cdc.UnmarshalBinary(buf, block)
err := cdc.UnmarshalBinaryLengthPrefixed(buf, block)
if err != nil {
// NOTE: The existence of meta should imply the existence of the
// block. So, make sure meta is only saved after blocks are saved.

View File

@@ -9,13 +9,30 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
cfg "github.com/tendermint/tendermint/config"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/libs/db"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/libs/log"
sm "github.com/tendermint/tendermint/state"
"github.com/tendermint/tendermint/types"
tmtime "github.com/tendermint/tendermint/types/time"
)
func makeStateAndBlockStore(logger log.Logger) (sm.State, *BlockStore) {
config := cfg.ResetTestRoot("blockchain_reactor_test")
// blockDB := dbm.NewDebugDB("blockDB", dbm.NewMemDB())
// stateDB := dbm.NewDebugDB("stateDB", dbm.NewMemDB())
blockDB := dbm.NewMemDB()
stateDB := dbm.NewMemDB()
state, err := sm.LoadStateFromDBOrGenesisFile(stateDB, config.GenesisFile())
if err != nil {
panic(cmn.ErrorWrap(err, "error constructing state from genesis file"))
}
return state, NewBlockStore(blockDB)
}
func TestLoadBlockStoreStateJSON(t *testing.T) {
db := db.NewMemDB()
@@ -65,7 +82,7 @@ func freshBlockStore() (*BlockStore, db.DB) {
var (
state, _ = makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
block = makeBlock(1, state)
block = makeBlock(1, state, new(types.Commit))
partSet = block.MakePartSet(2)
part1 = partSet.GetPart(0)
part2 = partSet.GetPart(1)
@@ -88,7 +105,7 @@ func TestBlockStoreSaveLoadBlock(t *testing.T) {
}
// save a block
block := makeBlock(bs.Height()+1, state)
block := makeBlock(bs.Height()+1, state, new(types.Commit))
validPartSet := block.MakePartSet(2)
seenCommit := &types.Commit{Precommits: []*types.Vote{{Height: 10,
Timestamp: tmtime.Now()}}}
@@ -331,7 +348,7 @@ func TestLoadBlockMeta(t *testing.T) {
func TestBlockFetchAtHeight(t *testing.T) {
state, bs := makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
require.Equal(t, bs.Height(), int64(0), "initially the height should be zero")
block := makeBlock(bs.Height()+1, state)
block := makeBlock(bs.Height()+1, state, new(types.Commit))
partSet := block.MakePartSet(2)
seenCommit := &types.Commit{Precommits: []*types.Vote{{Height: 10,

View File

@@ -58,8 +58,9 @@ func initFilesWithConfig(config *cfg.Config) error {
ConsensusParams: types.DefaultConsensusParams(),
}
genDoc.Validators = []types.GenesisValidator{{
PubKey: pv.GetPubKey(),
Power: 10,
Address: pv.GetPubKey().Address(),
PubKey: pv.GetPubKey(),
Power: 10,
}}
if err := genDoc.SaveAs(genFile); err != nil {

View File

@@ -26,10 +26,12 @@ just with added trust and running locally.`,
}
var (
listenAddr string
nodeAddr string
chainID string
home string
listenAddr string
nodeAddr string
chainID string
home string
maxOpenConnections int
cacheSize int
)
func init() {
@@ -37,6 +39,8 @@ func init() {
LiteCmd.Flags().StringVar(&nodeAddr, "node", "tcp://localhost:26657", "Connect to a Tendermint node at this address")
LiteCmd.Flags().StringVar(&chainID, "chain-id", "tendermint", "Specify the Tendermint chain ID")
LiteCmd.Flags().StringVar(&home, "home-dir", ".tendermint-lite", "Specify the home directory")
LiteCmd.Flags().IntVar(&maxOpenConnections, "max-open-connections", 900, "Maximum number of simultaneous connections (including WebSocket).")
LiteCmd.Flags().IntVar(&cacheSize, "cache-size", 10, "Specify the memory trust store cache size")
}
func ensureAddrHasSchemeOrDefaultToTCP(addr string) (string, error) {
@@ -69,7 +73,7 @@ func runProxy(cmd *cobra.Command, args []string) error {
node := rpcclient.NewHTTP(nodeAddr, "/websocket")
logger.Info("Constructing Verifier...")
cert, err := proxy.NewVerifier(chainID, home, node, logger)
cert, err := proxy.NewVerifier(chainID, home, node, logger, cacheSize)
if err != nil {
return cmn.ErrorWrap(err, "constructing Verifier")
}
@@ -77,7 +81,7 @@ func runProxy(cmd *cobra.Command, args []string) error {
sc := proxy.SecureClient(node, cert)
logger.Info("Starting proxy...")
err = proxy.StartProxy(sc, listenAddr, logger)
err = proxy.StartProxy(sc, listenAddr, logger, maxOpenConnections)
if err != nil {
return cmn.ErrorWrap(err, "starting proxy")
}

View File

@@ -1,6 +1,7 @@
package commands
import (
"fmt"
"os"
"github.com/spf13/cobra"
@@ -35,6 +36,9 @@ func ParseConfig() (*cfg.Config, error) {
}
conf.SetRoot(conf.RootDir)
cfg.EnsureRoot(conf.RootDir)
if err = conf.ValidateBasic(); err != nil {
return nil, fmt.Errorf("Error in config file: %v", err)
}
return conf, err
}
@@ -50,6 +54,9 @@ var RootCmd = &cobra.Command{
if err != nil {
return err
}
if config.LogFormat == cfg.LogFormatJSON {
logger = log.NewTMJSONLogger(log.NewSyncWriter(os.Stdout))
}
logger, err = tmflags.ParseLogLevel(config.LogLevel, logger, cfg.DefaultLogLevel())
if err != nil {
return err

View File

@@ -2,6 +2,9 @@ package commands
import (
"fmt"
"os"
"os/signal"
"syscall"
"github.com/spf13/cobra"
@@ -49,19 +52,31 @@ func NewRunNodeCmd(nodeProvider nm.NodeProvider) *cobra.Command {
Use: "node",
Short: "Run the tendermint node",
RunE: func(cmd *cobra.Command, args []string) error {
// Create & start node
n, err := nodeProvider(config, logger)
if err != nil {
return fmt.Errorf("Failed to create node: %v", err)
}
// Stop upon receiving SIGTERM or CTRL-C
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
go func() {
for sig := range c {
logger.Error(fmt.Sprintf("captured %v, exiting...", sig))
if n.IsRunning() {
n.Stop()
}
os.Exit(1)
}
}()
if err := n.Start(); err != nil {
return fmt.Errorf("Failed to start node: %v", err)
}
logger.Info("Started node", "nodeInfo", n.Switch().NodeInfo())
// Trap signal, run forever.
n.RunForever()
// Run forever
select {}
return nil
},

View File

@@ -91,9 +91,10 @@ func testnetFiles(cmd *cobra.Command, args []string) error {
pvFile := filepath.Join(nodeDir, config.BaseConfig.PrivValidator)
pv := privval.LoadFilePV(pvFile)
genVals[i] = types.GenesisValidator{
PubKey: pv.GetPubKey(),
Power: 1,
Name: nodeDirName,
Address: pv.GetPubKey().Address(),
PubKey: pv.GetPubKey(),
Power: 1,
Name: nodeDirName,
}
}

View File

@@ -5,6 +5,8 @@ import (
"os"
"path/filepath"
"time"
"github.com/pkg/errors"
)
const (
@@ -12,6 +14,11 @@ const (
FuzzModeDrop = iota
// FuzzModeDelay is a mode in which we randomly sleep
FuzzModeDelay
// LogFormatPlain is a format for colored text
LogFormatPlain = "plain"
// LogFormatJSON is a format for json output
LogFormatJSON = "json"
)
// NOTE: Most of the structs & relevant comments + the
@@ -19,7 +26,7 @@ const (
// generate the config.toml. Please reflect any changes
// made here in the defaultConfigTemplate constant in
// config/toml.go
// NOTE: tmlibs/cli must know to look in the config dir!
// NOTE: libs/cli must know to look in the config dir!
var (
DefaultTendermintDir = ".tendermint"
defaultConfigDir = "config"
@@ -89,6 +96,30 @@ func (cfg *Config) SetRoot(root string) *Config {
return cfg
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg *Config) ValidateBasic() error {
if err := cfg.BaseConfig.ValidateBasic(); err != nil {
return err
}
if err := cfg.RPC.ValidateBasic(); err != nil {
return errors.Wrap(err, "Error in [rpc] section")
}
if err := cfg.P2P.ValidateBasic(); err != nil {
return errors.Wrap(err, "Error in [p2p] section")
}
if err := cfg.Mempool.ValidateBasic(); err != nil {
return errors.Wrap(err, "Error in [mempool] section")
}
if err := cfg.Consensus.ValidateBasic(); err != nil {
return errors.Wrap(err, "Error in [consensus] section")
}
return errors.Wrap(
cfg.Instrumentation.ValidateBasic(),
"Error in [instrumentation] section",
)
}
//-----------------------------------------------------------------------------
// BaseConfig
@@ -113,7 +144,7 @@ type BaseConfig struct {
// and verifying their commits
FastSync bool `mapstructure:"fast_sync"`
// Database backend: leveldb | memdb
// Database backend: leveldb | memdb | cleveldb
DBBackend string `mapstructure:"db_backend"`
// Database directory
@@ -122,6 +153,9 @@ type BaseConfig struct {
// Output level for logging
LogLevel string `mapstructure:"log_level"`
// Output format: 'plain' (colored text) or 'json'
LogFormat string `mapstructure:"log_format"`
// Path to the JSON file containing the initial validator set and other meta data
Genesis string `mapstructure:"genesis_file"`
@@ -156,6 +190,7 @@ func DefaultBaseConfig() BaseConfig {
ProxyApp: "tcp://127.0.0.1:26658",
ABCI: "socket",
LogLevel: DefaultPackageLogLevels(),
LogFormat: LogFormatPlain,
ProfListenAddress: "",
FastSync: true,
FilterPeers: false,
@@ -198,6 +233,17 @@ func (cfg BaseConfig) DBDir() string {
return rootify(cfg.DBPath, cfg.RootDir)
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg BaseConfig) ValidateBasic() error {
switch cfg.LogFormat {
case LogFormatPlain, LogFormatJSON:
default:
return errors.New("unknown log_format (must be 'plain' or 'json')")
}
return nil
}
// DefaultLogLevel returns a default log level of "error"
func DefaultLogLevel() string {
return "error"
@@ -219,6 +265,18 @@ type RPCConfig struct {
// TCP or UNIX socket address for the RPC server to listen on
ListenAddress string `mapstructure:"laddr"`
// A list of origins a cross-domain request can be executed from.
// If the special '*' value is present in the list, all origins will be allowed.
// An origin may contain a wildcard (*) to replace 0 or more characters (i.e.: http://*.domain.com).
// Only one wildcard can be used per origin.
CORSAllowedOrigins []string `mapstructure:"cors_allowed_origins"`
// A list of methods the client is allowed to use with cross-domain requests.
CORSAllowedMethods []string `mapstructure:"cors_allowed_methods"`
// A list of non simple headers the client is allowed to use with cross-domain requests.
CORSAllowedHeaders []string `mapstructure:"cors_allowed_headers"`
// TCP or UNIX socket address for the gRPC server to listen on
// NOTE: This server only supports /broadcast_tx_commit
GRPCListenAddress string `mapstructure:"grpc_laddr"`
@@ -246,8 +304,10 @@ type RPCConfig struct {
// DefaultRPCConfig returns a default configuration for the RPC server
func DefaultRPCConfig() *RPCConfig {
return &RPCConfig{
ListenAddress: "tcp://0.0.0.0:26657",
ListenAddress: "tcp://0.0.0.0:26657",
CORSAllowedOrigins: []string{},
CORSAllowedMethods: []string{"HEAD", "GET", "POST"},
CORSAllowedHeaders: []string{"Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time"},
GRPCListenAddress: "",
GRPCMaxOpenConnections: 900,
@@ -265,6 +325,23 @@ func TestRPCConfig() *RPCConfig {
return cfg
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg *RPCConfig) ValidateBasic() error {
if cfg.GRPCMaxOpenConnections < 0 {
return errors.New("grpc_max_open_connections can't be negative")
}
if cfg.MaxOpenConnections < 0 {
return errors.New("max_open_connections can't be negative")
}
return nil
}
// IsCorsEnabled returns true if cross-origin resource sharing is enabled.
func (cfg *RPCConfig) IsCorsEnabled() bool {
return len(cfg.CORSAllowedOrigins) != 0
}
//-----------------------------------------------------------------------------
// P2PConfig
@@ -301,8 +378,8 @@ type P2PConfig struct {
// Maximum number of outbound peers to connect to, excluding persistent peers
MaxNumOutboundPeers int `mapstructure:"max_num_outbound_peers"`
// Time to wait before flushing messages out on the connection, in ms
FlushThrottleTimeout int `mapstructure:"flush_throttle_timeout"`
// Time to wait before flushing messages out on the connection
FlushThrottleTimeout time.Duration `mapstructure:"flush_throttle_timeout"`
// Maximum size of a message packet payload, in bytes
MaxPacketMsgPayloadSize int `mapstructure:"max_packet_msg_payload_size"`
@@ -351,7 +428,7 @@ func DefaultP2PConfig() *P2PConfig {
AddrBookStrict: true,
MaxNumInboundPeers: 40,
MaxNumOutboundPeers: 10,
FlushThrottleTimeout: 100,
FlushThrottleTimeout: 100 * time.Millisecond,
MaxPacketMsgPayloadSize: 1024, // 1 kB
SendRate: 5120000, // 5 mB/s
RecvRate: 5120000, // 5 mB/s
@@ -370,7 +447,7 @@ func DefaultP2PConfig() *P2PConfig {
func TestP2PConfig() *P2PConfig {
cfg := DefaultP2PConfig()
cfg.ListenAddress = "tcp://0.0.0.0:36656"
cfg.FlushThrottleTimeout = 10
cfg.FlushThrottleTimeout = 10 * time.Millisecond
cfg.AllowDuplicateIP = true
return cfg
}
@@ -380,6 +457,30 @@ func (cfg *P2PConfig) AddrBookFile() string {
return rootify(cfg.AddrBook, cfg.RootDir)
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg *P2PConfig) ValidateBasic() error {
if cfg.MaxNumInboundPeers < 0 {
return errors.New("max_num_inbound_peers can't be negative")
}
if cfg.MaxNumOutboundPeers < 0 {
return errors.New("max_num_outbound_peers can't be negative")
}
if cfg.FlushThrottleTimeout < 0 {
return errors.New("flush_throttle_timeout can't be negative")
}
if cfg.MaxPacketMsgPayloadSize < 0 {
return errors.New("max_packet_msg_payload_size can't be negative")
}
if cfg.SendRate < 0 {
return errors.New("send_rate can't be negative")
}
if cfg.RecvRate < 0 {
return errors.New("recv_rate can't be negative")
}
return nil
}
// FuzzConnConfig is a FuzzedConnection configuration.
type FuzzConnConfig struct {
Mode int
@@ -405,22 +506,20 @@ func DefaultFuzzConnConfig() *FuzzConnConfig {
// MempoolConfig defines the configuration options for the Tendermint mempool
type MempoolConfig struct {
RootDir string `mapstructure:"home"`
Recheck bool `mapstructure:"recheck"`
RecheckEmpty bool `mapstructure:"recheck_empty"`
Broadcast bool `mapstructure:"broadcast"`
WalPath string `mapstructure:"wal_dir"`
Size int `mapstructure:"size"`
CacheSize int `mapstructure:"cache_size"`
RootDir string `mapstructure:"home"`
Recheck bool `mapstructure:"recheck"`
Broadcast bool `mapstructure:"broadcast"`
WalPath string `mapstructure:"wal_dir"`
Size int `mapstructure:"size"`
CacheSize int `mapstructure:"cache_size"`
}
// DefaultMempoolConfig returns a default configuration for the Tendermint mempool
func DefaultMempoolConfig() *MempoolConfig {
return &MempoolConfig{
Recheck: true,
RecheckEmpty: true,
Broadcast: true,
WalPath: filepath.Join(defaultDataDir, "mempool.wal"),
Recheck: true,
Broadcast: true,
WalPath: "",
// Each signature verification takes .5ms, size reduced until we implement
// ABCI Recheck
Size: 5000,
@@ -440,6 +539,23 @@ func (cfg *MempoolConfig) WalDir() string {
return rootify(cfg.WalPath, cfg.RootDir)
}
// WalEnabled returns true if the WAL is enabled.
func (cfg *MempoolConfig) WalEnabled() bool {
return cfg.WalPath != ""
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg *MempoolConfig) ValidateBasic() error {
if cfg.Size < 0 {
return errors.New("size can't be negative")
}
if cfg.CacheSize < 0 {
return errors.New("cache_size can't be negative")
}
return nil
}
//-----------------------------------------------------------------------------
// ConsensusConfig
@@ -450,72 +566,70 @@ type ConsensusConfig struct {
WalPath string `mapstructure:"wal_file"`
walFile string // overrides WalPath if set
// All timeouts are in milliseconds
TimeoutPropose int `mapstructure:"timeout_propose"`
TimeoutProposeDelta int `mapstructure:"timeout_propose_delta"`
TimeoutPrevote int `mapstructure:"timeout_prevote"`
TimeoutPrevoteDelta int `mapstructure:"timeout_prevote_delta"`
TimeoutPrecommit int `mapstructure:"timeout_precommit"`
TimeoutPrecommitDelta int `mapstructure:"timeout_precommit_delta"`
TimeoutCommit int `mapstructure:"timeout_commit"`
TimeoutPropose time.Duration `mapstructure:"timeout_propose"`
TimeoutProposeDelta time.Duration `mapstructure:"timeout_propose_delta"`
TimeoutPrevote time.Duration `mapstructure:"timeout_prevote"`
TimeoutPrevoteDelta time.Duration `mapstructure:"timeout_prevote_delta"`
TimeoutPrecommit time.Duration `mapstructure:"timeout_precommit"`
TimeoutPrecommitDelta time.Duration `mapstructure:"timeout_precommit_delta"`
TimeoutCommit time.Duration `mapstructure:"timeout_commit"`
// Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
SkipTimeoutCommit bool `mapstructure:"skip_timeout_commit"`
// EmptyBlocks mode and possible interval between empty blocks in seconds
CreateEmptyBlocks bool `mapstructure:"create_empty_blocks"`
CreateEmptyBlocksInterval int `mapstructure:"create_empty_blocks_interval"`
// EmptyBlocks mode and possible interval between empty blocks
CreateEmptyBlocks bool `mapstructure:"create_empty_blocks"`
CreateEmptyBlocksInterval time.Duration `mapstructure:"create_empty_blocks_interval"`
// Reactor sleep duration parameters are in milliseconds
PeerGossipSleepDuration int `mapstructure:"peer_gossip_sleep_duration"`
PeerQueryMaj23SleepDuration int `mapstructure:"peer_query_maj23_sleep_duration"`
// Reactor sleep duration parameters
PeerGossipSleepDuration time.Duration `mapstructure:"peer_gossip_sleep_duration"`
PeerQueryMaj23SleepDuration time.Duration `mapstructure:"peer_query_maj23_sleep_duration"`
// Block time parameters in milliseconds. Corresponds to the minimum time increment between consecutive blocks.
BlockTimeIota int `mapstructure:"blocktime_iota"`
// Block time parameters. Corresponds to the minimum time increment between consecutive blocks.
BlockTimeIota time.Duration `mapstructure:"blocktime_iota"`
}
// DefaultConsensusConfig returns a default configuration for the consensus service
func DefaultConsensusConfig() *ConsensusConfig {
return &ConsensusConfig{
WalPath: filepath.Join(defaultDataDir, "cs.wal", "wal"),
TimeoutPropose: 3000,
TimeoutProposeDelta: 500,
TimeoutPrevote: 1000,
TimeoutPrevoteDelta: 500,
TimeoutPrecommit: 1000,
TimeoutPrecommitDelta: 500,
TimeoutCommit: 1000,
TimeoutPropose: 3000 * time.Millisecond,
TimeoutProposeDelta: 500 * time.Millisecond,
TimeoutPrevote: 1000 * time.Millisecond,
TimeoutPrevoteDelta: 500 * time.Millisecond,
TimeoutPrecommit: 1000 * time.Millisecond,
TimeoutPrecommitDelta: 500 * time.Millisecond,
TimeoutCommit: 1000 * time.Millisecond,
SkipTimeoutCommit: false,
CreateEmptyBlocks: true,
CreateEmptyBlocksInterval: 0,
PeerGossipSleepDuration: 100,
PeerQueryMaj23SleepDuration: 2000,
BlockTimeIota: 1000,
CreateEmptyBlocksInterval: 0 * time.Second,
PeerGossipSleepDuration: 100 * time.Millisecond,
PeerQueryMaj23SleepDuration: 2000 * time.Millisecond,
BlockTimeIota: 1000 * time.Millisecond,
}
}
// TestConsensusConfig returns a configuration for testing the consensus service
func TestConsensusConfig() *ConsensusConfig {
cfg := DefaultConsensusConfig()
cfg.TimeoutPropose = 100
cfg.TimeoutProposeDelta = 1
cfg.TimeoutPrevote = 10
cfg.TimeoutPrevoteDelta = 1
cfg.TimeoutPrecommit = 10
cfg.TimeoutPrecommitDelta = 1
cfg.TimeoutCommit = 10
cfg.TimeoutPropose = 40 * time.Millisecond
cfg.TimeoutProposeDelta = 1 * time.Millisecond
cfg.TimeoutPrevote = 10 * time.Millisecond
cfg.TimeoutPrevoteDelta = 1 * time.Millisecond
cfg.TimeoutPrecommit = 10 * time.Millisecond
cfg.TimeoutPrecommitDelta = 1 * time.Millisecond
cfg.TimeoutCommit = 10 * time.Millisecond
cfg.SkipTimeoutCommit = true
cfg.PeerGossipSleepDuration = 5
cfg.PeerQueryMaj23SleepDuration = 250
cfg.BlockTimeIota = 10
cfg.PeerGossipSleepDuration = 5 * time.Millisecond
cfg.PeerQueryMaj23SleepDuration = 250 * time.Millisecond
cfg.BlockTimeIota = 10 * time.Millisecond
return cfg
}
// MinValidVoteTime returns the minimum acceptable block time.
// See the [BFT time spec](https://godoc.org/github.com/tendermint/tendermint/docs/spec/consensus/bft-time.md).
func (cfg *ConsensusConfig) MinValidVoteTime(lastBlockTime time.Time) time.Time {
return lastBlockTime.
Add(time.Duration(cfg.BlockTimeIota) * time.Millisecond)
return lastBlockTime.Add(cfg.BlockTimeIota)
}
// WaitForTxs returns true if the consensus should wait for transactions before entering the propose step
@@ -523,39 +637,30 @@ func (cfg *ConsensusConfig) WaitForTxs() bool {
return !cfg.CreateEmptyBlocks || cfg.CreateEmptyBlocksInterval > 0
}
// EmptyBlocks returns the amount of time to wait before proposing an empty block or starting the propose timer if there are no txs available
func (cfg *ConsensusConfig) EmptyBlocksInterval() time.Duration {
return time.Duration(cfg.CreateEmptyBlocksInterval) * time.Second
}
// Propose returns the amount of time to wait for a proposal
func (cfg *ConsensusConfig) Propose(round int) time.Duration {
return time.Duration(cfg.TimeoutPropose+cfg.TimeoutProposeDelta*round) * time.Millisecond
return time.Duration(
cfg.TimeoutPropose.Nanoseconds()+cfg.TimeoutProposeDelta.Nanoseconds()*int64(round),
) * time.Nanosecond
}
// Prevote returns the amount of time to wait for straggler votes after receiving any +2/3 prevotes
func (cfg *ConsensusConfig) Prevote(round int) time.Duration {
return time.Duration(cfg.TimeoutPrevote+cfg.TimeoutPrevoteDelta*round) * time.Millisecond
return time.Duration(
cfg.TimeoutPrevote.Nanoseconds()+cfg.TimeoutPrevoteDelta.Nanoseconds()*int64(round),
) * time.Nanosecond
}
// Precommit returns the amount of time to wait for straggler votes after receiving any +2/3 precommits
func (cfg *ConsensusConfig) Precommit(round int) time.Duration {
return time.Duration(cfg.TimeoutPrecommit+cfg.TimeoutPrecommitDelta*round) * time.Millisecond
return time.Duration(
cfg.TimeoutPrecommit.Nanoseconds()+cfg.TimeoutPrecommitDelta.Nanoseconds()*int64(round),
) * time.Nanosecond
}
// Commit returns the amount of time to wait for straggler votes after receiving +2/3 precommits for a single block (ie. a commit).
func (cfg *ConsensusConfig) Commit(t time.Time) time.Time {
return t.Add(time.Duration(cfg.TimeoutCommit) * time.Millisecond)
}
// PeerGossipSleep returns the amount of time to sleep if there is nothing to send from the ConsensusReactor
func (cfg *ConsensusConfig) PeerGossipSleep() time.Duration {
return time.Duration(cfg.PeerGossipSleepDuration) * time.Millisecond
}
// PeerQueryMaj23Sleep returns the amount of time to sleep after each VoteSetMaj23Message is sent in the ConsensusReactor
func (cfg *ConsensusConfig) PeerQueryMaj23Sleep() time.Duration {
return time.Duration(cfg.PeerQueryMaj23SleepDuration) * time.Millisecond
return t.Add(cfg.TimeoutCommit)
}
// WalFile returns the full path to the write-ahead log file
@@ -571,6 +676,45 @@ func (cfg *ConsensusConfig) SetWalFile(walFile string) {
cfg.walFile = walFile
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg *ConsensusConfig) ValidateBasic() error {
if cfg.TimeoutPropose < 0 {
return errors.New("timeout_propose can't be negative")
}
if cfg.TimeoutProposeDelta < 0 {
return errors.New("timeout_propose_delta can't be negative")
}
if cfg.TimeoutPrevote < 0 {
return errors.New("timeout_prevote can't be negative")
}
if cfg.TimeoutPrevoteDelta < 0 {
return errors.New("timeout_prevote_delta can't be negative")
}
if cfg.TimeoutPrecommit < 0 {
return errors.New("timeout_precommit can't be negative")
}
if cfg.TimeoutPrecommitDelta < 0 {
return errors.New("timeout_precommit_delta can't be negative")
}
if cfg.TimeoutCommit < 0 {
return errors.New("timeout_commit can't be negative")
}
if cfg.CreateEmptyBlocksInterval < 0 {
return errors.New("create_empty_blocks_interval can't be negative")
}
if cfg.PeerGossipSleepDuration < 0 {
return errors.New("peer_gossip_sleep_duration can't be negative")
}
if cfg.PeerQueryMaj23SleepDuration < 0 {
return errors.New("peer_query_maj23_sleep_duration can't be negative")
}
if cfg.BlockTimeIota < 0 {
return errors.New("blocktime_iota can't be negative")
}
return nil
}
//-----------------------------------------------------------------------------
// TxIndexConfig
@@ -587,15 +731,15 @@ type TxIndexConfig struct {
// Comma-separated list of tags to index (by default the only tag is "tx.hash")
//
// You can also index transactions by height by adding "tx.height" tag here.
//
//
// It's recommended to index only a subset of tags due to possible memory
// bloat. This is, of course, depends on the indexer's DB and the volume of
// transactions.
IndexTags string `mapstructure:"index_tags"`
// When set to true, tells indexer to index all tags (predefined tags:
// "tx.hash", "tx.height" and all tags from DeliverTx responses).
//
// "tx.hash", "tx.height" and all tags from DeliverTx responses).
//
// Note this may be not desirable (see the comment above). IndexTags has a
// precedence over IndexAllTags (i.e. when given both, IndexTags will be
// indexed).
@@ -634,6 +778,9 @@ type InstrumentationConfig struct {
// you increase your OS limits.
// 0 - unlimited.
MaxOpenConnections int `mapstructure:"max_open_connections"`
// Tendermint instrumentation namespace.
Namespace string `mapstructure:"namespace"`
}
// DefaultInstrumentationConfig returns a default configuration for metrics
@@ -643,6 +790,7 @@ func DefaultInstrumentationConfig() *InstrumentationConfig {
Prometheus: false,
PrometheusListenAddr: ":26660",
MaxOpenConnections: 3,
Namespace: "tendermint",
}
}
@@ -652,6 +800,15 @@ func TestInstrumentationConfig() *InstrumentationConfig {
return DefaultInstrumentationConfig()
}
// ValidateBasic performs basic validation (checking param bounds, etc.) and
// returns an error if any check fails.
func (cfg *InstrumentationConfig) ValidateBasic() error {
if cfg.MaxOpenConnections < 0 {
return errors.New("max_open_connections can't be negative")
}
return nil
}
//-----------------------------------------------------------------------------
// Utils

View File

@@ -2,6 +2,7 @@ package config
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
)
@@ -26,3 +27,12 @@ func TestDefaultConfig(t *testing.T) {
assert.Equal("/foo/wal/mem", cfg.Mempool.WalDir())
}
func TestConfigValidateBasic(t *testing.T) {
cfg := DefaultConfig()
assert.NoError(t, cfg.ValidateBasic())
// tamper with timeout_propose
cfg.Consensus.TimeoutPropose = -10 * time.Second
assert.Error(t, cfg.ValidateBasic())
}

View File

@@ -77,7 +77,7 @@ moniker = "{{ .BaseConfig.Moniker }}"
# and verifying their commits
fast_sync = {{ .BaseConfig.FastSync }}
# Database backend: leveldb | memdb
# Database backend: leveldb | memdb | cleveldb
db_backend = "{{ .BaseConfig.DBBackend }}"
# Database directory
@@ -86,6 +86,9 @@ db_dir = "{{ js .BaseConfig.DBPath }}"
# Output level for logging, including package level options
log_level = "{{ .BaseConfig.LogLevel }}"
# Output format: 'plain' (colored text) or 'json'
log_format = "{{ .BaseConfig.LogFormat }}"
##### additional base config options #####
# Path to the JSON file containing the initial validator set and other meta data
@@ -99,7 +102,7 @@ priv_validator_file = "{{ js .BaseConfig.PrivValidator }}"
priv_validator_laddr = "{{ .BaseConfig.PrivValidatorListenAddr }}"
# Path to the JSON file containing the private key to use for node authentication in the p2p protocol
node_key_file = "{{ js .BaseConfig.NodeKey}}"
node_key_file = "{{ js .BaseConfig.NodeKey }}"
# Mechanism to connect to the ABCI application: socket | grpc
abci = "{{ .BaseConfig.ABCI }}"
@@ -119,6 +122,17 @@ filter_peers = {{ .BaseConfig.FilterPeers }}
# TCP or UNIX socket address for the RPC server to listen on
laddr = "{{ .RPC.ListenAddress }}"
# A list of origins a cross-domain request can be executed from
# Default value '[]' disables cors support
# Use '["*"]' to allow any origin
cors_allowed_origins = "{{ .RPC.CORSAllowedOrigins }}"
# A list of methods the client is allowed to use with cross-domain requests
cors_allowed_methods = "{{ .RPC.CORSAllowedMethods }}"
# A list of non simple headers the client is allowed to use with cross-domain requests
cors_allowed_headers = "{{ .RPC.CORSAllowedHeaders }}"
# TCP or UNIX socket address for the gRPC server to listen on
# NOTE: This server only supports /broadcast_tx_commit
grpc_laddr = "{{ .RPC.GRPCListenAddress }}"
@@ -172,15 +186,15 @@ addr_book_file = "{{ js .P2P.AddrBook }}"
# Set false for private or local networks
addr_book_strict = {{ .P2P.AddrBookStrict }}
# Time to wait before flushing messages out on the connection, in ms
flush_throttle_timeout = {{ .P2P.FlushThrottleTimeout }}
# Maximum number of inbound peers
max_num_inbound_peers = {{ .P2P.MaxNumInboundPeers }}
# Maximum number of outbound peers to connect to, excluding persistent peers
max_num_outbound_peers = {{ .P2P.MaxNumOutboundPeers }}
# Time to wait before flushing messages out on the connection
flush_throttle_timeout = "{{ .P2P.FlushThrottleTimeout }}"
# Maximum size of a message packet payload, in bytes
max_packet_msg_payload_size = {{ .P2P.MaxPacketMsgPayloadSize }}
@@ -202,11 +216,17 @@ seed_mode = {{ .P2P.SeedMode }}
# Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
private_peer_ids = "{{ .P2P.PrivatePeerIDs }}"
# Toggle to disable guard against peers connecting from the same ip.
allow_duplicate_ip = {{ .P2P.AllowDuplicateIP }}
# Peer connection configuration.
handshake_timeout = "{{ .P2P.HandshakeTimeout }}"
dial_timeout = "{{ .P2P.DialTimeout }}"
##### mempool configuration options #####
[mempool]
recheck = {{ .Mempool.Recheck }}
recheck_empty = {{ .Mempool.RecheckEmpty }}
broadcast = {{ .Mempool.Broadcast }}
wal_dir = "{{ js .Mempool.WalPath }}"
@@ -221,25 +241,27 @@ cache_size = {{ .Mempool.CacheSize }}
wal_file = "{{ js .Consensus.WalPath }}"
# All timeouts are in milliseconds
timeout_propose = {{ .Consensus.TimeoutPropose }}
timeout_propose_delta = {{ .Consensus.TimeoutProposeDelta }}
timeout_prevote = {{ .Consensus.TimeoutPrevote }}
timeout_prevote_delta = {{ .Consensus.TimeoutPrevoteDelta }}
timeout_precommit = {{ .Consensus.TimeoutPrecommit }}
timeout_precommit_delta = {{ .Consensus.TimeoutPrecommitDelta }}
timeout_commit = {{ .Consensus.TimeoutCommit }}
timeout_propose = "{{ .Consensus.TimeoutPropose }}"
timeout_propose_delta = "{{ .Consensus.TimeoutProposeDelta }}"
timeout_prevote = "{{ .Consensus.TimeoutPrevote }}"
timeout_prevote_delta = "{{ .Consensus.TimeoutPrevoteDelta }}"
timeout_precommit = "{{ .Consensus.TimeoutPrecommit }}"
timeout_precommit_delta = "{{ .Consensus.TimeoutPrecommitDelta }}"
timeout_commit = "{{ .Consensus.TimeoutCommit }}"
# Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
skip_timeout_commit = {{ .Consensus.SkipTimeoutCommit }}
# EmptyBlocks mode and possible interval between empty blocks in seconds
# EmptyBlocks mode and possible interval between empty blocks
create_empty_blocks = {{ .Consensus.CreateEmptyBlocks }}
create_empty_blocks_interval = {{ .Consensus.CreateEmptyBlocksInterval }}
create_empty_blocks_interval = "{{ .Consensus.CreateEmptyBlocksInterval }}"
# Reactor sleep duration parameters are in milliseconds
peer_gossip_sleep_duration = {{ .Consensus.PeerGossipSleepDuration }}
peer_query_maj23_sleep_duration = {{ .Consensus.PeerQueryMaj23SleepDuration }}
# Reactor sleep duration parameters
peer_gossip_sleep_duration = "{{ .Consensus.PeerGossipSleepDuration }}"
peer_query_maj23_sleep_duration = "{{ .Consensus.PeerQueryMaj23SleepDuration }}"
# Block time parameters. Corresponds to the minimum time increment between consecutive blocks.
blocktime_iota = "{{ .Consensus.BlockTimeIota }}"
##### transactions indexer configuration options #####
[tx_index]
@@ -284,6 +306,9 @@ prometheus_listen_addr = "{{ .Instrumentation.PrometheusListenAddr }}"
# you increase your OS limits.
# 0 - unlimited.
max_open_connections = {{ .Instrumentation.MaxOpenConnections }}
# Instrumentation namespace
namespace = "{{ .Instrumentation.Namespace }}"
`
/****** these are for test settings ***********/
@@ -334,7 +359,7 @@ func ResetTestRoot(testName string) *Config {
}
var testGenesis = `{
"genesis_time": "0001-01-01T00:00:00.000Z",
"genesis_time": "2018-10-10T08:20:13.695936996Z",
"chain_id": "tendermint_test",
"validators": [
{

View File

@@ -39,7 +39,13 @@ func TestByzantine(t *testing.T) {
switches := make([]*p2p.Switch, N)
p2pLogger := logger.With("module", "p2p")
for i := 0; i < N; i++ {
switches[i] = p2p.NewSwitch(config.P2P)
switches[i] = p2p.MakeSwitch(
config.P2P,
i,
"foo", "1.0.0",
func(i int, sw *p2p.Switch) *p2p.Switch {
return sw
})
switches[i].SetLogger(p2pLogger.With("validator", i))
}
@@ -173,16 +179,16 @@ func byzantineDecideProposalFunc(t *testing.T, height int64, round int, cs *Cons
// Create a new proposal block from state/txs from the mempool.
block1, blockParts1 := cs.createProposalBlock()
polRound, polBlockID := cs.Votes.POLInfo()
proposal1 := types.NewProposal(height, round, blockParts1.Header(), polRound, polBlockID)
polRound, propBlockID := cs.ValidRound, types.BlockID{block1.Hash(), blockParts1.Header()}
proposal1 := types.NewProposal(height, round, polRound, propBlockID)
if err := cs.privValidator.SignProposal(cs.state.ChainID, proposal1); err != nil {
t.Error(err)
}
// Create a new proposal block from state/txs from the mempool.
block2, blockParts2 := cs.createProposalBlock()
polRound, polBlockID = cs.Votes.POLInfo()
proposal2 := types.NewProposal(height, round, blockParts2.Header(), polRound, polBlockID)
polRound, propBlockID = cs.ValidRound, types.BlockID{block2.Hash(), blockParts2.Header()}
proposal2 := types.NewProposal(height, round, polRound, propBlockID)
if err := cs.privValidator.SignProposal(cs.state.ChainID, proposal2); err != nil {
t.Error(err)
}
@@ -220,8 +226,8 @@ func sendProposalAndParts(height int64, round int, cs *ConsensusState, peer p2p.
// votes
cs.mtx.Lock()
prevote, _ := cs.signVote(types.VoteTypePrevote, blockHash, parts.Header())
precommit, _ := cs.signVote(types.VoteTypePrecommit, blockHash, parts.Header())
prevote, _ := cs.signVote(types.PrevoteType, blockHash, parts.Header())
precommit, _ := cs.signVote(types.PrecommitType, blockHash, parts.Header())
cs.mtx.Unlock()
peer.Send(VoteChannel, cdc.MustMarshalBinaryBare(&VoteMessage{prevote}))
@@ -257,7 +263,7 @@ func (br *ByzantineReactor) AddPeer(peer p2p.Peer) {
// Send our state to peer.
// If we're fast_syncing, broadcast a RoundStepMessage later upon SwitchToConsensus().
if !br.reactor.fastSync {
br.reactor.sendNewRoundStepMessages(peer)
br.reactor.sendNewRoundStepMessage(peer)
}
}
func (br *ByzantineReactor) RemovePeer(peer p2p.Peer, reason interface{}) {

View File

@@ -7,6 +7,7 @@ import (
"io/ioutil"
"os"
"path"
"reflect"
"sort"
"sync"
"testing"
@@ -38,8 +39,8 @@ const (
)
// genesis, chain_id, priv_val
var config *cfg.Config // NOTE: must be reset for each _test.go file
var ensureTimeout = time.Second * 1 // must be in seconds because CreateEmptyBlocksInterval is
var config *cfg.Config // NOTE: must be reset for each _test.go file
var ensureTimeout = time.Millisecond * 100
func ensureDir(dir string, mode os.FileMode) {
if err := cmn.EnsureDir(dir, mode); err != nil {
@@ -70,7 +71,7 @@ func NewValidatorStub(privValidator types.PrivValidator, valIndex int) *validato
}
}
func (vs *validatorStub) signVote(voteType byte, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
func (vs *validatorStub) signVote(voteType types.SignedMsgType, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
vote := &types.Vote{
ValidatorIndex: vs.Index,
ValidatorAddress: vs.PrivValidator.GetAddress(),
@@ -85,7 +86,7 @@ func (vs *validatorStub) signVote(voteType byte, hash []byte, header types.PartS
}
// Sign vote for type/hash/header
func signVote(vs *validatorStub, voteType byte, hash []byte, header types.PartSetHeader) *types.Vote {
func signVote(vs *validatorStub, voteType types.SignedMsgType, hash []byte, header types.PartSetHeader) *types.Vote {
v, err := vs.signVote(voteType, hash, header)
if err != nil {
panic(fmt.Errorf("failed to sign vote: %v", err))
@@ -93,7 +94,7 @@ func signVote(vs *validatorStub, voteType byte, hash []byte, header types.PartSe
return v
}
func signVotes(voteType byte, hash []byte, header types.PartSetHeader, vss ...*validatorStub) []*types.Vote {
func signVotes(voteType types.SignedMsgType, hash []byte, header types.PartSetHeader, vss ...*validatorStub) []*types.Vote {
votes := make([]*types.Vote, len(vss))
for i, vs := range vss {
votes[i] = signVote(vs, voteType, hash, header)
@@ -129,8 +130,8 @@ func decideProposal(cs1 *ConsensusState, vs *validatorStub, height int64, round
}
// Make proposal
polRound, polBlockID := cs1.Votes.POLInfo()
proposal = types.NewProposal(height, round, blockParts.Header(), polRound, polBlockID)
polRound, propBlockID := cs1.ValidRound, types.BlockID{block.Hash(), blockParts.Header()}
proposal = types.NewProposal(height, round, polRound, propBlockID)
if err := vs.SignProposal(cs1.state.ChainID, proposal); err != nil {
panic(err)
}
@@ -143,7 +144,7 @@ func addVotes(to *ConsensusState, votes ...*types.Vote) {
}
}
func signAddVotes(to *ConsensusState, voteType byte, hash []byte, header types.PartSetHeader, vss ...*validatorStub) {
func signAddVotes(to *ConsensusState, voteType types.SignedMsgType, hash []byte, header types.PartSetHeader, vss ...*validatorStub) {
votes := signVotes(voteType, hash, header, vss...)
addVotes(to, votes...)
}
@@ -306,23 +307,260 @@ func randConsensusState(nValidators int) (*ConsensusState, []*validatorStub) {
//-------------------------------------------------------------------------------
func ensureNoNewStep(stepCh <-chan interface{}) {
timer := time.NewTimer(ensureTimeout)
func ensureNoNewEvent(ch <-chan interface{}, timeout time.Duration,
errorMessage string) {
select {
case <-timer.C:
case <-time.After(timeout):
break
case <-stepCh:
panic("We should be stuck waiting, not moving to the next step")
case <-ch:
panic(errorMessage)
}
}
func ensureNewStep(stepCh <-chan interface{}) {
timer := time.NewTimer(ensureTimeout)
func ensureNoNewEventOnChannel(ch <-chan interface{}) {
ensureNoNewEvent(
ch,
ensureTimeout,
"We should be stuck waiting, not receiving new event on the channel")
}
func ensureNoNewRoundStep(stepCh <-chan interface{}) {
ensureNoNewEvent(
stepCh,
ensureTimeout,
"We should be stuck waiting, not receiving NewRoundStep event")
}
func ensureNoNewUnlock(unlockCh <-chan interface{}) {
ensureNoNewEvent(
unlockCh,
ensureTimeout,
"We should be stuck waiting, not receiving Unlock event")
}
func ensureNoNewTimeout(stepCh <-chan interface{}, timeout int64) {
timeoutDuration := time.Duration(timeout*5) * time.Nanosecond
ensureNoNewEvent(
stepCh,
timeoutDuration,
"We should be stuck waiting, not receiving NewTimeout event")
}
func ensureNewEvent(
ch <-chan interface{},
height int64,
round int,
timeout time.Duration,
errorMessage string) {
select {
case <-timer.C:
panic("We shouldnt be stuck waiting")
case <-stepCh:
case <-time.After(timeout):
panic(errorMessage)
case ev := <-ch:
rs, ok := ev.(types.EventDataRoundState)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataRoundState, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
}
// TODO: We could check also for a step at this point!
}
}
func ensureNewRoundStep(stepCh <-chan interface{}, height int64, round int) {
ensureNewEvent(
stepCh,
height,
round,
ensureTimeout,
"Timeout expired while waiting for NewStep event")
}
func ensureNewVote(voteCh <-chan interface{}, height int64, round int) {
select {
case <-time.After(ensureTimeout):
break
case v := <-voteCh:
edv, ok := v.(types.EventDataVote)
if !ok {
panic(fmt.Sprintf("expected a *types.Vote, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(v)))
}
vote := edv.Vote
if vote.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, vote.Height))
}
if vote.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, vote.Round))
}
}
}
func ensureNewRound(roundCh <-chan interface{}, height int64, round int) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewRound event")
case ev := <-roundCh:
rs, ok := ev.(types.EventDataNewRound)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataNewRound, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
}
}
}
func ensureNewTimeout(timeoutCh <-chan interface{}, height int64, round int, timeout int64) {
timeoutDuration := time.Duration(timeout*3) * time.Nanosecond
ensureNewEvent(timeoutCh, height, round, timeoutDuration,
"Timeout expired while waiting for NewTimeout event")
}
func ensureNewProposal(proposalCh <-chan interface{}, height int64, round int) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewProposal event")
case ev := <-proposalCh:
rs, ok := ev.(types.EventDataCompleteProposal)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataCompleteProposal, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
}
}
}
func ensureNewValidBlock(validBlockCh <-chan interface{}, height int64, round int) {
ensureNewEvent(validBlockCh, height, round, ensureTimeout,
"Timeout expired while waiting for NewValidBlock event")
}
func ensureNewBlock(blockCh <-chan interface{}, height int64) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewBlock event")
case ev := <-blockCh:
block, ok := ev.(types.EventDataNewBlock)
if !ok {
panic(fmt.Sprintf("expected a *types.EventDataNewBlock, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(block)))
}
if block.Block.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, block.Block.Height))
}
}
}
func ensureNewBlockHeader(blockCh <-chan interface{}, height int64, blockHash cmn.HexBytes) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewBlockHeader event")
case ev := <-blockCh:
blockHeader, ok := ev.(types.EventDataNewBlockHeader)
if !ok {
panic(fmt.Sprintf("expected a *types.EventDataNewBlockHeader, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(blockHeader)))
}
if blockHeader.Header.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, blockHeader.Header.Height))
}
if !bytes.Equal(blockHeader.Header.Hash(), blockHash) {
panic(fmt.Sprintf("expected header %X, got %X", blockHash, blockHeader.Header.Hash()))
}
}
}
func ensureNewUnlock(unlockCh <-chan interface{}, height int64, round int) {
ensureNewEvent(unlockCh, height, round, ensureTimeout,
"Timeout expired while waiting for NewUnlock event")
}
func ensureVote(voteCh <-chan interface{}, height int64, round int,
voteType types.SignedMsgType) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewVote event")
case v := <-voteCh:
edv, ok := v.(types.EventDataVote)
if !ok {
panic(fmt.Sprintf("expected a *types.Vote, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(v)))
}
vote := edv.Vote
if vote.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, vote.Height))
}
if vote.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, vote.Round))
}
if vote.Type != voteType {
panic(fmt.Sprintf("expected type %v, got %v", voteType, vote.Type))
}
}
}
func ensureProposal(proposalCh <-chan interface{}, height int64, round int, propId types.BlockID) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewProposal event")
case ev := <-proposalCh:
rs, ok := ev.(types.EventDataCompleteProposal)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataCompleteProposal, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
}
if !rs.BlockID.Equals(propId) {
panic("Proposed block does not match expected block")
}
}
}
func ensurePrecommit(voteCh <-chan interface{}, height int64, round int) {
ensureVote(voteCh, height, round, types.PrecommitType)
}
func ensurePrevote(voteCh <-chan interface{}, height int64, round int) {
ensureVote(voteCh, height, round, types.PrevoteType)
}
func ensureNewEventOnChannel(ch <-chan interface{}) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for new activity on the channel")
case <-ch:
}
}
@@ -399,7 +637,7 @@ func randConsensusNetWithPeers(nValidators, nPeers int, testName string, tickerF
func getSwitchIndex(switches []*p2p.Switch, peer p2p.Peer) int {
for i, s := range switches {
if peer.NodeInfo().ID == s.NodeInfo().ID {
if peer.NodeInfo().ID() == s.NodeInfo().ID() {
return i
}
}
@@ -433,8 +671,6 @@ func randGenesisDoc(numValidators int, randPower bool, minPower int64) (*types.G
func randGenesisState(numValidators int, randPower bool, minPower int64) (sm.State, []types.PrivValidator) {
genDoc, privValidators := randGenesisDoc(numValidators, randPower, minPower)
s0, _ := sm.MakeGenesisState(genDoc)
db := dbm.NewMemDB() // remove this ?
sm.SaveState(db, s0)
return s0, privValidators
}

View File

@@ -10,7 +10,6 @@ import (
"github.com/tendermint/tendermint/abci/example/code"
abci "github.com/tendermint/tendermint/abci/types"
"github.com/tendermint/tendermint/types"
)
@@ -28,17 +27,17 @@ func TestMempoolNoProgressUntilTxsAvailable(t *testing.T) {
newBlockCh := subscribe(cs.eventBus, types.EventQueryNewBlock)
startTestRound(cs, height, round)
ensureNewStep(newBlockCh) // first block gets committed
ensureNoNewStep(newBlockCh)
ensureNewEventOnChannel(newBlockCh) // first block gets committed
ensureNoNewEventOnChannel(newBlockCh)
deliverTxsRange(cs, 0, 1)
ensureNewStep(newBlockCh) // commit txs
ensureNewStep(newBlockCh) // commit updated app hash
ensureNoNewStep(newBlockCh)
ensureNewEventOnChannel(newBlockCh) // commit txs
ensureNewEventOnChannel(newBlockCh) // commit updated app hash
ensureNoNewEventOnChannel(newBlockCh)
}
func TestMempoolProgressAfterCreateEmptyBlocksInterval(t *testing.T) {
config := ResetConfig("consensus_mempool_txs_available_test")
config.Consensus.CreateEmptyBlocksInterval = int(ensureTimeout.Seconds())
config.Consensus.CreateEmptyBlocksInterval = ensureTimeout
state, privVals := randGenesisState(1, false, 10)
cs := newConsensusStateWithConfig(config, state, privVals[0], NewCounterApplication())
cs.mempool.EnableTxsAvailable()
@@ -46,9 +45,9 @@ func TestMempoolProgressAfterCreateEmptyBlocksInterval(t *testing.T) {
newBlockCh := subscribe(cs.eventBus, types.EventQueryNewBlock)
startTestRound(cs, height, round)
ensureNewStep(newBlockCh) // first block gets committed
ensureNoNewStep(newBlockCh) // then we dont make a block ...
ensureNewStep(newBlockCh) // until the CreateEmptyBlocksInterval has passed
ensureNewEventOnChannel(newBlockCh) // first block gets committed
ensureNoNewEventOnChannel(newBlockCh) // then we dont make a block ...
ensureNewEventOnChannel(newBlockCh) // until the CreateEmptyBlocksInterval has passed
}
func TestMempoolProgressInHigherRound(t *testing.T) {
@@ -72,13 +71,19 @@ func TestMempoolProgressInHigherRound(t *testing.T) {
}
startTestRound(cs, height, round)
ensureNewStep(newRoundCh) // first round at first height
ensureNewStep(newBlockCh) // first block gets committed
ensureNewStep(newRoundCh) // first round at next height
deliverTxsRange(cs, 0, 1) // we deliver txs, but dont set a proposal so we get the next round
<-timeoutCh
ensureNewStep(newRoundCh) // wait for the next round
ensureNewStep(newBlockCh) // now we can commit the block
ensureNewRound(newRoundCh, height, round) // first round at first height
ensureNewEventOnChannel(newBlockCh) // first block gets committed
height = height + 1 // moving to the next height
round = 0
ensureNewRound(newRoundCh, height, round) // first round at next height
deliverTxsRange(cs, 0, 1) // we deliver txs, but dont set a proposal so we get the next round
ensureNewTimeout(timeoutCh, height, round, cs.config.TimeoutPropose.Nanoseconds())
round = round + 1 // moving to the next round
ensureNewRound(newRoundCh, height, round) // wait for the next round
ensureNewEventOnChannel(newBlockCh) // now we can commit the block
}
func deliverTxsRange(cs *ConsensusState, start, end int) {
@@ -148,7 +153,7 @@ func TestMempoolRmBadTx(t *testing.T) {
// check for the tx
for {
txs := cs.mempool.ReapMaxBytes(len(txBytes))
txs := cs.mempool.ReapMaxBytesMaxGas(int64(len(txBytes)), -1)
if len(txs) == 0 {
emptyMempoolCh <- struct{}{}
return

View File

@@ -8,6 +8,8 @@ import (
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const MetricsSubsystem = "consensus"
// Metrics contains metrics exposed by this package.
type Metrics struct {
// Height of the chain.
@@ -30,7 +32,7 @@ type Metrics struct {
ByzantineValidatorsPower metrics.Gauge
// Time between this and the last block.
BlockIntervalSeconds metrics.Histogram
BlockIntervalSeconds metrics.Gauge
// Number of transactions.
NumTxs metrics.Gauge
@@ -38,75 +40,111 @@ type Metrics struct {
BlockSizeBytes metrics.Gauge
// Total number of transactions.
TotalTxs metrics.Gauge
// The latest block height.
CommittedHeight metrics.Gauge
// Whether or not a node is fast syncing. 1 if yes, 0 if no.
FastSyncing metrics.Gauge
// Number of blockparts transmitted by peer.
BlockParts metrics.Counter
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
func PrometheusMetrics() *Metrics {
func PrometheusMetrics(namespace string) *Metrics {
return &Metrics{
Height: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "height",
Help: "Height of the chain.",
}, []string{}),
Rounds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "rounds",
Help: "Number of rounds.",
}, []string{}),
Validators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators",
Help: "Number of validators.",
}, []string{}),
ValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators_power",
Help: "Total power of all validators.",
}, []string{}),
MissingValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators",
Help: "Number of validators who did not sign.",
}, []string{}),
MissingValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators_power",
Help: "Total power of the missing validators.",
}, []string{}),
ByzantineValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators",
Help: "Number of validators who tried to double sign.",
}, []string{}),
ByzantineValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators_power",
Help: "Total power of the byzantine validators.",
}, []string{}),
BlockIntervalSeconds: prometheus.NewHistogramFrom(stdprometheus.HistogramOpts{
Subsystem: "consensus",
BlockIntervalSeconds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_interval_seconds",
Help: "Time between this and the last block.",
Buckets: []float64{1, 2.5, 5, 10, 60},
}, []string{}),
NumTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_txs",
Help: "Number of transactions.",
}, []string{}),
BlockSizeBytes: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_size_bytes",
Help: "Size of the block.",
}, []string{}),
TotalTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Subsystem: "consensus",
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "total_txs",
Help: "Total number of transactions.",
}, []string{}),
CommittedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "latest_block_height",
Help: "The latest block height.",
}, []string{}),
FastSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "fast_syncing",
Help: "Whether or not a node is fast syncing. 1 if yes, 0 if no.",
}, []string{}),
BlockParts: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_parts",
Help: "Number of blockparts transmitted by peer.",
}, []string{"peer_id"}),
}
}
@@ -124,10 +162,13 @@ func NopMetrics() *Metrics {
ByzantineValidators: discard.NewGauge(),
ByzantineValidatorsPower: discard.NewGauge(),
BlockIntervalSeconds: discard.NewHistogram(),
BlockIntervalSeconds: discard.NewGauge(),
NumTxs: discard.NewGauge(),
BlockSizeBytes: discard.NewGauge(),
TotalTxs: discard.NewGauge(),
NumTxs: discard.NewGauge(),
BlockSizeBytes: discard.NewGauge(),
TotalTxs: discard.NewGauge(),
CommittedHeight: discard.NewGauge(),
FastSyncing: discard.NewGauge(),
BlockParts: discard.NewCounter(),
}
}

View File

@@ -8,8 +8,7 @@ import (
"github.com/pkg/errors"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/go-amino"
cstypes "github.com/tendermint/tendermint/consensus/types"
cmn "github.com/tendermint/tendermint/libs/common"
tmevents "github.com/tendermint/tendermint/libs/events"
@@ -29,6 +28,7 @@ const (
maxMsgSize = 1048576 // 1MB; NOTE/TODO: keep in sync with types.PartSet sizes.
blocksToContributeToBecomeGoodPeer = 10000
votesToContributeToBecomeGoodPeer = 10000
)
//-----------------------------------------------------------------------------
@@ -42,16 +42,27 @@ type ConsensusReactor struct {
mtx sync.RWMutex
fastSync bool
eventBus *types.EventBus
metrics *Metrics
}
type ReactorOption func(*ConsensusReactor)
// NewConsensusReactor returns a new ConsensusReactor with the given
// consensusState.
func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *ConsensusReactor {
func NewConsensusReactor(consensusState *ConsensusState, fastSync bool, options ...ReactorOption) *ConsensusReactor {
conR := &ConsensusReactor{
conS: consensusState,
fastSync: fastSync,
metrics: NopMetrics(),
}
conR.updateFastSyncingMetric()
conR.BaseReactor = *p2p.NewBaseReactor("ConsensusReactor", conR)
for _, option := range options {
option(conR)
}
return conR
}
@@ -60,6 +71,9 @@ func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *Consens
func (conR *ConsensusReactor) OnStart() error {
conR.Logger.Info("ConsensusReactor ", "fastSync", conR.FastSync())
// start routine that computes peer statistics for evaluating peer quality
go conR.peerStatsRoutine()
conR.subscribeToBroadcastEvents()
if !conR.FastSync() {
@@ -94,6 +108,7 @@ func (conR *ConsensusReactor) SwitchToConsensus(state sm.State, blocksSynced int
conR.mtx.Lock()
conR.fastSync = false
conR.mtx.Unlock()
conR.metrics.FastSyncing.Set(0)
if blocksSynced > 0 {
// dont bother with the WAL if we fast synced
@@ -158,7 +173,7 @@ func (conR *ConsensusReactor) AddPeer(peer p2p.Peer) {
// Send our state to peer.
// If we're fast_syncing, broadcast a RoundStepMessage later upon SwitchToConsensus().
if !conR.FastSync() {
conR.sendNewRoundStepMessages(peer)
conR.sendNewRoundStepMessage(peer)
}
}
@@ -168,7 +183,11 @@ func (conR *ConsensusReactor) RemovePeer(peer p2p.Peer, reason interface{}) {
return
}
// TODO
//peer.Get(PeerStateKey).(*PeerState).Disconnect()
// ps, ok := peer.Get(PeerStateKey).(*PeerState)
// if !ok {
// panic(fmt.Sprintf("Peer %v has no state", peer))
// }
// ps.Disconnect()
}
// Receive implements Reactor
@@ -189,18 +208,28 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
conR.Switch.StopPeerForError(src, err)
return
}
if err = msg.ValidateBasic(); err != nil {
conR.Logger.Error("Peer sent us invalid msg", "peer", src, "msg", msg, "err", err)
conR.Switch.StopPeerForError(src, err)
return
}
conR.Logger.Debug("Receive", "src", src, "chId", chID, "msg", msg)
// Get peer states
ps := src.Get(types.PeerStateKey).(*PeerState)
ps, ok := src.Get(types.PeerStateKey).(*PeerState)
if !ok {
panic(fmt.Sprintf("Peer %v has no state", src))
}
switch chID {
case StateChannel:
switch msg := msg.(type) {
case *NewRoundStepMessage:
ps.ApplyNewRoundStepMessage(msg)
case *CommitStepMessage:
ps.ApplyCommitStepMessage(msg)
case *NewValidBlockMessage:
ps.ApplyNewValidBlockMessage(msg)
case *HasVoteMessage:
ps.ApplyHasVoteMessage(msg)
case *VoteSetMaj23Message:
@@ -221,13 +250,12 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
// (and consequently shows which we don't have)
var ourVotes *cmn.BitArray
switch msg.Type {
case types.VoteTypePrevote:
case types.PrevoteType:
ourVotes = votes.Prevotes(msg.Round).BitArrayByBlockID(msg.BlockID)
case types.VoteTypePrecommit:
case types.PrecommitType:
ourVotes = votes.Precommits(msg.Round).BitArrayByBlockID(msg.BlockID)
default:
conR.Logger.Error("Bad VoteSetBitsMessage field Type")
return
panic("Bad VoteSetBitsMessage field Type. Forgot to add a check in ValidateBasic?")
}
src.TrySend(VoteSetBitsChannel, cdc.MustMarshalBinaryBare(&VoteSetBitsMessage{
Height: msg.Height,
@@ -236,11 +264,6 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
BlockID: msg.BlockID,
Votes: ourVotes,
}))
case *ProposalHeartbeatMessage:
hb := msg.Heartbeat
conR.Logger.Debug("Received proposal heartbeat message",
"height", hb.Height, "round", hb.Round, "sequence", hb.Sequence,
"valIdx", hb.ValidatorIndex, "valAddr", hb.ValidatorAddress)
default:
conR.Logger.Error(fmt.Sprintf("Unknown message type %v", reflect.TypeOf(msg)))
}
@@ -258,9 +281,7 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
ps.ApplyProposalPOLMessage(msg)
case *BlockPartMessage:
ps.SetHasProposalBlockPart(msg.Height, msg.Round, msg.Part.Index)
if numBlocks := ps.RecordBlockPart(msg); numBlocks%blocksToContributeToBecomeGoodPeer == 0 {
conR.Switch.MarkPeerAsGood(src)
}
conR.metrics.BlockParts.With("peer_id", string(src.ID())).Add(1)
conR.conS.peerMsgQueue <- msgInfo{msg, src.ID()}
default:
conR.Logger.Error(fmt.Sprintf("Unknown message type %v", reflect.TypeOf(msg)))
@@ -274,15 +295,12 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
switch msg := msg.(type) {
case *VoteMessage:
cs := conR.conS
cs.mtx.Lock()
cs.mtx.RLock()
height, valSize, lastCommitSize := cs.Height, cs.Validators.Size(), cs.LastCommit.Size()
cs.mtx.Unlock()
cs.mtx.RUnlock()
ps.EnsureVoteBitArrays(height, valSize)
ps.EnsureVoteBitArrays(height-1, lastCommitSize)
ps.SetHasVote(msg.Vote)
if blocks := ps.RecordVote(msg.Vote); blocks%blocksToContributeToBecomeGoodPeer == 0 {
conR.Switch.MarkPeerAsGood(src)
}
cs.peerMsgQueue <- msgInfo{msg, src.ID()}
@@ -306,13 +324,12 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
if height == msg.Height {
var ourVotes *cmn.BitArray
switch msg.Type {
case types.VoteTypePrevote:
case types.PrevoteType:
ourVotes = votes.Prevotes(msg.Round).BitArrayByBlockID(msg.BlockID)
case types.VoteTypePrecommit:
case types.PrecommitType:
ourVotes = votes.Precommits(msg.Round).BitArrayByBlockID(msg.BlockID)
default:
conR.Logger.Error("Bad VoteSetBitsMessage field Type")
return
panic("Bad VoteSetBitsMessage field Type. Forgot to add a check in ValidateBasic?")
}
ps.ApplyVoteSetBitsMessage(msg, ourVotes)
} else {
@@ -347,14 +364,19 @@ func (conR *ConsensusReactor) FastSync() bool {
//--------------------------------------
// subscribeToBroadcastEvents subscribes for new round steps, votes and
// proposal heartbeats using internal pubsub defined on state to broadcast
// subscribeToBroadcastEvents subscribes for new round steps and votes
// using internal pubsub defined on state to broadcast
// them to peers upon receiving.
func (conR *ConsensusReactor) subscribeToBroadcastEvents() {
const subscriber = "consensus-reactor"
conR.conS.evsw.AddListenerForEvent(subscriber, types.EventNewRoundStep,
func(data tmevents.EventData) {
conR.broadcastNewRoundStepMessages(data.(*cstypes.RoundState))
conR.broadcastNewRoundStepMessage(data.(*cstypes.RoundState))
})
conR.conS.evsw.AddListenerForEvent(subscriber, types.EventValidBlock,
func(data tmevents.EventData) {
conR.broadcastNewValidBlockMessage(data.(*cstypes.RoundState))
})
conR.conS.evsw.AddListenerForEvent(subscriber, types.EventVote,
@@ -362,10 +384,6 @@ func (conR *ConsensusReactor) subscribeToBroadcastEvents() {
conR.broadcastHasVoteMessage(data.(*types.Vote))
})
conR.conS.evsw.AddListenerForEvent(subscriber, types.EventProposalHeartbeat,
func(data tmevents.EventData) {
conR.broadcastProposalHeartbeatMessage(data.(*types.Heartbeat))
})
}
func (conR *ConsensusReactor) unsubscribeFromBroadcastEvents() {
@@ -373,21 +391,20 @@ func (conR *ConsensusReactor) unsubscribeFromBroadcastEvents() {
conR.conS.evsw.RemoveListener(subscriber)
}
func (conR *ConsensusReactor) broadcastProposalHeartbeatMessage(hb *types.Heartbeat) {
conR.Logger.Debug("Broadcasting proposal heartbeat message",
"height", hb.Height, "round", hb.Round, "sequence", hb.Sequence)
msg := &ProposalHeartbeatMessage{hb}
conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(msg))
func (conR *ConsensusReactor) broadcastNewRoundStepMessage(rs *cstypes.RoundState) {
nrsMsg := makeRoundStepMessage(rs)
conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(nrsMsg))
}
func (conR *ConsensusReactor) broadcastNewRoundStepMessages(rs *cstypes.RoundState) {
nrsMsg, csMsg := makeRoundStepMessages(rs)
if nrsMsg != nil {
conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(nrsMsg))
}
if csMsg != nil {
conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(csMsg))
func (conR *ConsensusReactor) broadcastNewValidBlockMessage(rs *cstypes.RoundState) {
csMsg := &NewValidBlockMessage{
Height: rs.Height,
Round: rs.Round,
BlockPartsHeader: rs.ProposalBlockParts.Header(),
BlockParts: rs.ProposalBlockParts.BitArray(),
IsCommit: rs.Step == cstypes.RoundStepCommit,
}
conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(csMsg))
}
// Broadcasts HasVoteMessage to peers that care.
@@ -402,7 +419,10 @@ func (conR *ConsensusReactor) broadcastHasVoteMessage(vote *types.Vote) {
/*
// TODO: Make this broadcast more selective.
for _, peer := range conR.Switch.Peers().List() {
ps := peer.Get(PeerStateKey).(*PeerState)
ps, ok := peer.Get(PeerStateKey).(*PeerState)
if !ok {
panic(fmt.Sprintf("Peer %v has no state", peer))
}
prs := ps.GetRoundState()
if prs.Height == vote.Height {
// TODO: Also filter on round?
@@ -416,7 +436,7 @@ func (conR *ConsensusReactor) broadcastHasVoteMessage(vote *types.Vote) {
*/
}
func makeRoundStepMessages(rs *cstypes.RoundState) (nrsMsg *NewRoundStepMessage, csMsg *CommitStepMessage) {
func makeRoundStepMessage(rs *cstypes.RoundState) (nrsMsg *NewRoundStepMessage) {
nrsMsg = &NewRoundStepMessage{
Height: rs.Height,
Round: rs.Round,
@@ -424,25 +444,13 @@ func makeRoundStepMessages(rs *cstypes.RoundState) (nrsMsg *NewRoundStepMessage,
SecondsSinceStartTime: int(time.Since(rs.StartTime).Seconds()),
LastCommitRound: rs.LastCommit.Round(),
}
if rs.Step == cstypes.RoundStepCommit {
csMsg = &CommitStepMessage{
Height: rs.Height,
BlockPartsHeader: rs.ProposalBlockParts.Header(),
BlockParts: rs.ProposalBlockParts.BitArray(),
}
}
return
}
func (conR *ConsensusReactor) sendNewRoundStepMessages(peer p2p.Peer) {
func (conR *ConsensusReactor) sendNewRoundStepMessage(peer p2p.Peer) {
rs := conR.conS.GetRoundState()
nrsMsg, csMsg := makeRoundStepMessages(rs)
if nrsMsg != nil {
peer.Send(StateChannel, cdc.MustMarshalBinaryBare(nrsMsg))
}
if csMsg != nil {
peer.Send(StateChannel, cdc.MustMarshalBinaryBare(csMsg))
}
nrsMsg := makeRoundStepMessage(rs)
peer.Send(StateChannel, cdc.MustMarshalBinaryBare(nrsMsg))
}
func (conR *ConsensusReactor) gossipDataRoutine(peer p2p.Peer, ps *PeerState) {
@@ -497,7 +505,7 @@ OUTER_LOOP:
// If height and round don't match, sleep.
if (rs.Height != prs.Height) || (rs.Round != prs.Round) {
//logger.Info("Peer Height|Round mismatch, sleeping", "peerHeight", prs.Height, "peerRound", prs.Round, "peer", peer)
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
continue OUTER_LOOP
}
@@ -513,6 +521,7 @@ OUTER_LOOP:
msg := &ProposalMessage{Proposal: rs.Proposal}
logger.Debug("Sending proposal", "height", prs.Height, "round", prs.Round)
if peer.Send(DataChannel, cdc.MustMarshalBinaryBare(msg)) {
// NOTE[ZM]: A peer might have received different proposal msg so this Proposal msg will be rejected!
ps.SetHasProposal(rs.Proposal)
}
}
@@ -533,7 +542,7 @@ OUTER_LOOP:
}
// Nothing to do. Sleep.
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
continue OUTER_LOOP
}
}
@@ -547,12 +556,12 @@ func (conR *ConsensusReactor) gossipDataForCatchup(logger log.Logger, rs *cstype
if blockMeta == nil {
logger.Error("Failed to load block meta",
"ourHeight", rs.Height, "blockstoreHeight", conR.conS.blockStore.Height())
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
return
} else if !blockMeta.BlockID.PartsHeader.Equals(prs.ProposalBlockPartsHeader) {
logger.Info("Peer ProposalBlockPartsHeader mismatch, sleeping",
"blockPartsHeader", blockMeta.BlockID.PartsHeader, "peerBlockPartsHeader", prs.ProposalBlockPartsHeader)
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
return
}
// Load the part
@@ -560,7 +569,7 @@ func (conR *ConsensusReactor) gossipDataForCatchup(logger log.Logger, rs *cstype
if part == nil {
logger.Error("Could not load part", "index", index,
"blockPartsHeader", blockMeta.BlockID.PartsHeader, "peerBlockPartsHeader", prs.ProposalBlockPartsHeader)
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
return
}
// Send the part
@@ -578,7 +587,7 @@ func (conR *ConsensusReactor) gossipDataForCatchup(logger log.Logger, rs *cstype
return
}
//logger.Info("No parts to send in catch-up, sleeping")
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
}
func (conR *ConsensusReactor) gossipVotesRoutine(peer p2p.Peer, ps *PeerState) {
@@ -647,7 +656,7 @@ OUTER_LOOP:
sleeping = 1
}
time.Sleep(conR.conS.config.PeerGossipSleep())
time.Sleep(conR.conS.config.PeerGossipSleepDuration)
continue OUTER_LOOP
}
}
@@ -728,10 +737,10 @@ OUTER_LOOP:
peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
Height: prs.Height,
Round: prs.Round,
Type: types.VoteTypePrevote,
Type: types.PrevoteType,
BlockID: maj23,
}))
time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
}
}
}
@@ -745,10 +754,10 @@ OUTER_LOOP:
peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
Height: prs.Height,
Round: prs.Round,
Type: types.VoteTypePrecommit,
Type: types.PrecommitType,
BlockID: maj23,
}))
time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
}
}
}
@@ -762,10 +771,10 @@ OUTER_LOOP:
peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
Height: prs.Height,
Round: prs.ProposalPOLRound,
Type: types.VoteTypePrevote,
Type: types.PrevoteType,
BlockID: maj23,
}))
time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
}
}
}
@@ -781,19 +790,59 @@ OUTER_LOOP:
peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
Height: prs.Height,
Round: commit.Round(),
Type: types.VoteTypePrecommit,
Type: types.PrecommitType,
BlockID: commit.BlockID,
}))
time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
}
}
time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
continue OUTER_LOOP
}
}
func (conR *ConsensusReactor) peerStatsRoutine() {
for {
if !conR.IsRunning() {
conR.Logger.Info("Stopping peerStatsRoutine")
return
}
select {
case msg := <-conR.conS.statsMsgQueue:
// Get peer
peer := conR.Switch.Peers().Get(msg.PeerID)
if peer == nil {
conR.Logger.Debug("Attempt to update stats for non-existent peer",
"peer", msg.PeerID)
continue
}
// Get peer state
ps, ok := peer.Get(types.PeerStateKey).(*PeerState)
if !ok {
panic(fmt.Sprintf("Peer %v has no state", peer))
}
switch msg.Msg.(type) {
case *VoteMessage:
if numVotes := ps.RecordVote(); numVotes%votesToContributeToBecomeGoodPeer == 0 {
conR.Switch.MarkPeerAsGood(peer)
}
case *BlockPartMessage:
if numParts := ps.RecordBlockPart(); numParts%blocksToContributeToBecomeGoodPeer == 0 {
conR.Switch.MarkPeerAsGood(peer)
}
}
case <-conR.conS.Quit():
return
case <-conR.Quit():
return
}
}
}
// String returns a string representation of the ConsensusReactor.
// NOTE: For now, it is just a hard-coded string to avoid accessing unprotected shared variables.
// TODO: improve!
@@ -807,13 +856,31 @@ func (conR *ConsensusReactor) StringIndented(indent string) string {
s := "ConsensusReactor{\n"
s += indent + " " + conR.conS.StringIndented(indent+" ") + "\n"
for _, peer := range conR.Switch.Peers().List() {
ps := peer.Get(types.PeerStateKey).(*PeerState)
ps, ok := peer.Get(types.PeerStateKey).(*PeerState)
if !ok {
panic(fmt.Sprintf("Peer %v has no state", peer))
}
s += indent + " " + ps.StringIndented(indent+" ") + "\n"
}
s += indent + "}"
return s
}
func (conR *ConsensusReactor) updateFastSyncingMetric() {
var fastSyncing float64
if conR.fastSync {
fastSyncing = 1
} else {
fastSyncing = 0
}
conR.metrics.FastSyncing.Set(fastSyncing)
}
// ReactorMetrics sets the metrics
func ReactorMetrics(metrics *Metrics) ReactorOption {
return func(conR *ConsensusReactor) { conR.metrics = metrics }
}
//-----------------------------------------------------------------------------
var (
@@ -836,15 +903,13 @@ type PeerState struct {
// peerStateStats holds internal statistics for a peer.
type peerStateStats struct {
LastVoteHeight int64 `json:"last_vote_height"`
Votes int `json:"votes"`
LastBlockPartHeight int64 `json:"last_block_part_height"`
BlockParts int `json:"block_parts"`
Votes int `json:"votes"`
BlockParts int `json:"block_parts"`
}
func (pss peerStateStats) String() string {
return fmt.Sprintf("peerStateStats{lvh: %d, votes: %d, lbph: %d, blockParts: %d}",
pss.LastVoteHeight, pss.Votes, pss.LastBlockPartHeight, pss.BlockParts)
return fmt.Sprintf("peerStateStats{votes: %d, blockParts: %d}",
pss.Votes, pss.BlockParts)
}
// NewPeerState returns a new PeerState for the given Peer
@@ -903,13 +968,20 @@ func (ps *PeerState) SetHasProposal(proposal *types.Proposal) {
if ps.PRS.Height != proposal.Height || ps.PRS.Round != proposal.Round {
return
}
if ps.PRS.Proposal {
return
}
ps.PRS.Proposal = true
ps.PRS.ProposalBlockPartsHeader = proposal.BlockPartsHeader
ps.PRS.ProposalBlockParts = cmn.NewBitArray(proposal.BlockPartsHeader.Total)
// ps.PRS.ProposalBlockParts is set due to NewValidBlockMessage
if ps.PRS.ProposalBlockParts != nil {
return
}
ps.PRS.ProposalBlockPartsHeader = proposal.BlockID.PartsHeader
ps.PRS.ProposalBlockParts = cmn.NewBitArray(proposal.BlockID.PartsHeader.Total)
ps.PRS.ProposalPOLRound = proposal.POLRound
ps.PRS.ProposalPOL = nil // Nil until ProposalPOLMessage received.
}
@@ -945,7 +1017,11 @@ func (ps *PeerState) PickSendVote(votes types.VoteSetReader) bool {
if vote, ok := ps.PickVoteToSend(votes); ok {
msg := &VoteMessage{vote}
ps.logger.Debug("Sending vote message", "ps", ps, "vote", vote)
return ps.peer.Send(VoteChannel, cdc.MustMarshalBinaryBare(msg))
if ps.peer.Send(VoteChannel, cdc.MustMarshalBinaryBare(msg)) {
ps.SetHasVote(vote)
return true
}
return false
}
return false
}
@@ -961,7 +1037,7 @@ func (ps *PeerState) PickVoteToSend(votes types.VoteSetReader) (vote *types.Vote
return nil, false
}
height, round, type_, size := votes.Height(), votes.Round(), votes.Type(), votes.Size()
height, round, type_, size := votes.Height(), votes.Round(), types.SignedMsgType(votes.Type()), votes.Size()
// Lazily set data using 'votes'.
if votes.IsCommit() {
@@ -974,13 +1050,12 @@ func (ps *PeerState) PickVoteToSend(votes types.VoteSetReader) (vote *types.Vote
return nil, false // Not something worth sending
}
if index, ok := votes.BitArray().Sub(psVotes).PickRandom(); ok {
ps.setHasVote(height, round, type_, index)
return votes.GetByIndex(index), true
}
return nil, false
}
func (ps *PeerState) getVoteBitArray(height int64, round int, type_ byte) *cmn.BitArray {
func (ps *PeerState) getVoteBitArray(height int64, round int, type_ types.SignedMsgType) *cmn.BitArray {
if !types.IsVoteTypeValid(type_) {
return nil
}
@@ -988,25 +1063,25 @@ func (ps *PeerState) getVoteBitArray(height int64, round int, type_ byte) *cmn.B
if ps.PRS.Height == height {
if ps.PRS.Round == round {
switch type_ {
case types.VoteTypePrevote:
case types.PrevoteType:
return ps.PRS.Prevotes
case types.VoteTypePrecommit:
case types.PrecommitType:
return ps.PRS.Precommits
}
}
if ps.PRS.CatchupCommitRound == round {
switch type_ {
case types.VoteTypePrevote:
case types.PrevoteType:
return nil
case types.VoteTypePrecommit:
case types.PrecommitType:
return ps.PRS.CatchupCommit
}
}
if ps.PRS.ProposalPOLRound == round {
switch type_ {
case types.VoteTypePrevote:
case types.PrevoteType:
return ps.PRS.ProposalPOL
case types.VoteTypePrecommit:
case types.PrecommitType:
return nil
}
}
@@ -1015,9 +1090,9 @@ func (ps *PeerState) getVoteBitArray(height int64, round int, type_ byte) *cmn.B
if ps.PRS.Height == height+1 {
if ps.PRS.LastCommitRound == round {
switch type_ {
case types.VoteTypePrevote:
case types.PrevoteType:
return nil
case types.VoteTypePrecommit:
case types.PrecommitType:
return ps.PRS.LastCommit
}
}
@@ -1080,18 +1155,14 @@ func (ps *PeerState) ensureVoteBitArrays(height int64, numValidators int) {
}
}
// RecordVote updates internal statistics for this peer by recording the vote.
// It returns the total number of votes (1 per block). This essentially means
// the number of blocks for which peer has been sending us votes.
func (ps *PeerState) RecordVote(vote *types.Vote) int {
// RecordVote increments internal votes related statistics for this peer.
// It returns the total number of added votes.
func (ps *PeerState) RecordVote() int {
ps.mtx.Lock()
defer ps.mtx.Unlock()
if ps.Stats.LastVoteHeight >= vote.Height {
return ps.Stats.Votes
}
ps.Stats.LastVoteHeight = vote.Height
ps.Stats.Votes++
return ps.Stats.Votes
}
@@ -1104,25 +1175,17 @@ func (ps *PeerState) VotesSent() int {
return ps.Stats.Votes
}
// RecordBlockPart updates internal statistics for this peer by recording the
// block part. It returns the total number of block parts (1 per block). This
// essentially means the number of blocks for which peer has been sending us
// block parts.
func (ps *PeerState) RecordBlockPart(bp *BlockPartMessage) int {
// RecordBlockPart increments internal block part related statistics for this peer.
// It returns the total number of added block parts.
func (ps *PeerState) RecordBlockPart() int {
ps.mtx.Lock()
defer ps.mtx.Unlock()
if ps.Stats.LastBlockPartHeight >= bp.Height {
return ps.Stats.BlockParts
}
ps.Stats.LastBlockPartHeight = bp.Height
ps.Stats.BlockParts++
return ps.Stats.BlockParts
}
// BlockPartsSent returns the number of blocks for which peer has been sending
// us block parts.
// BlockPartsSent returns the number of useful block parts the peer has sent us.
func (ps *PeerState) BlockPartsSent() int {
ps.mtx.Lock()
defer ps.mtx.Unlock()
@@ -1138,7 +1201,7 @@ func (ps *PeerState) SetHasVote(vote *types.Vote) {
ps.setHasVote(vote.Height, vote.Round, vote.Type, vote.ValidatorIndex)
}
func (ps *PeerState) setHasVote(height int64, round int, type_ byte, index int) {
func (ps *PeerState) setHasVote(height int64, round int, type_ types.SignedMsgType, index int) {
logger := ps.logger.With("peerH/R", fmt.Sprintf("%d/%d", ps.PRS.Height, ps.PRS.Round), "H/R", fmt.Sprintf("%d/%d", height, round))
logger.Debug("setHasVote", "type", type_, "index", index)
@@ -1162,7 +1225,6 @@ func (ps *PeerState) ApplyNewRoundStepMessage(msg *NewRoundStepMessage) {
// Just remember these values.
psHeight := ps.PRS.Height
psRound := ps.PRS.Round
//psStep := ps.PRS.Step
psCatchupCommitRound := ps.PRS.CatchupCommitRound
psCatchupCommit := ps.PRS.CatchupCommit
@@ -1203,8 +1265,8 @@ func (ps *PeerState) ApplyNewRoundStepMessage(msg *NewRoundStepMessage) {
}
}
// ApplyCommitStepMessage updates the peer state for the new commit.
func (ps *PeerState) ApplyCommitStepMessage(msg *CommitStepMessage) {
// ApplyNewValidBlockMessage updates the peer state for the new valid block.
func (ps *PeerState) ApplyNewValidBlockMessage(msg *NewValidBlockMessage) {
ps.mtx.Lock()
defer ps.mtx.Unlock()
@@ -1212,6 +1274,10 @@ func (ps *PeerState) ApplyCommitStepMessage(msg *CommitStepMessage) {
return
}
if ps.PRS.Round != msg.Round && !msg.IsCommit {
return
}
ps.PRS.ProposalBlockPartsHeader = msg.BlockPartsHeader
ps.PRS.ProposalBlockParts = msg.BlockParts
}
@@ -1290,12 +1356,14 @@ func (ps *PeerState) StringIndented(indent string) string {
// Messages
// ConsensusMessage is a message that can be sent and received on the ConsensusReactor
type ConsensusMessage interface{}
type ConsensusMessage interface {
ValidateBasic() error
}
func RegisterConsensusMessages(cdc *amino.Codec) {
cdc.RegisterInterface((*ConsensusMessage)(nil), nil)
cdc.RegisterConcrete(&NewRoundStepMessage{}, "tendermint/NewRoundStepMessage", nil)
cdc.RegisterConcrete(&CommitStepMessage{}, "tendermint/CommitStep", nil)
cdc.RegisterConcrete(&NewValidBlockMessage{}, "tendermint/NewValidBlockMessage", nil)
cdc.RegisterConcrete(&ProposalMessage{}, "tendermint/Proposal", nil)
cdc.RegisterConcrete(&ProposalPOLMessage{}, "tendermint/ProposalPOL", nil)
cdc.RegisterConcrete(&BlockPartMessage{}, "tendermint/BlockPart", nil)
@@ -1303,7 +1371,6 @@ func RegisterConsensusMessages(cdc *amino.Codec) {
cdc.RegisterConcrete(&HasVoteMessage{}, "tendermint/HasVote", nil)
cdc.RegisterConcrete(&VoteSetMaj23Message{}, "tendermint/VoteSetMaj23", nil)
cdc.RegisterConcrete(&VoteSetBitsMessage{}, "tendermint/VoteSetBits", nil)
cdc.RegisterConcrete(&ProposalHeartbeatMessage{}, "tendermint/ProposalHeartbeat", nil)
}
func decodeMsg(bz []byte) (msg ConsensusMessage, err error) {
@@ -1326,6 +1393,27 @@ type NewRoundStepMessage struct {
LastCommitRound int
}
// ValidateBasic performs basic validation.
func (m *NewRoundStepMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.Round < 0 {
return errors.New("Negative Round")
}
if !m.Step.IsValid() {
return errors.New("Invalid Step")
}
// NOTE: SecondsSinceStartTime may be negative
if (m.Height == 1 && m.LastCommitRound != -1) ||
(m.Height > 1 && m.LastCommitRound < -1) { // TODO: #2737 LastCommitRound should always be >= 0 for heights > 1
return errors.New("Invalid LastCommitRound (for 1st block: -1, for others: >= 0)")
}
return nil
}
// String returns a string representation.
func (m *NewRoundStepMessage) String() string {
return fmt.Sprintf("[NewRoundStep H:%v R:%v S:%v LCR:%v]",
@@ -1334,16 +1422,40 @@ func (m *NewRoundStepMessage) String() string {
//-------------------------------------
// CommitStepMessage is sent when a block is committed.
type CommitStepMessage struct {
// NewValidBlockMessage is sent when a validator observes a valid block B in some round r,
//i.e., there is a Proposal for block B and 2/3+ prevotes for the block B in the round r.
// In case the block is also committed, then IsCommit flag is set to true.
type NewValidBlockMessage struct {
Height int64
Round int
BlockPartsHeader types.PartSetHeader
BlockParts *cmn.BitArray
IsCommit bool
}
// ValidateBasic performs basic validation.
func (m *NewValidBlockMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.Round < 0 {
return errors.New("Negative Round")
}
if err := m.BlockPartsHeader.ValidateBasic(); err != nil {
return fmt.Errorf("Wrong BlockPartsHeader: %v", err)
}
if m.BlockParts.Size() != m.BlockPartsHeader.Total {
return fmt.Errorf("BlockParts bit array size %d not equal to BlockPartsHeader.Total %d",
m.BlockParts.Size(),
m.BlockPartsHeader.Total)
}
return nil
}
// String returns a string representation.
func (m *CommitStepMessage) String() string {
return fmt.Sprintf("[CommitStep H:%v BP:%v BA:%v]", m.Height, m.BlockPartsHeader, m.BlockParts)
func (m *NewValidBlockMessage) String() string {
return fmt.Sprintf("[ValidBlockMessage H:%v R:%v BP:%v BA:%v IsCommit:%v]",
m.Height, m.Round, m.BlockPartsHeader, m.BlockParts, m.IsCommit)
}
//-------------------------------------
@@ -1353,6 +1465,11 @@ type ProposalMessage struct {
Proposal *types.Proposal
}
// ValidateBasic performs basic validation.
func (m *ProposalMessage) ValidateBasic() error {
return m.Proposal.ValidateBasic()
}
// String returns a string representation.
func (m *ProposalMessage) String() string {
return fmt.Sprintf("[Proposal %v]", m.Proposal)
@@ -1367,6 +1484,20 @@ type ProposalPOLMessage struct {
ProposalPOL *cmn.BitArray
}
// ValidateBasic performs basic validation.
func (m *ProposalPOLMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.ProposalPOLRound < 0 {
return errors.New("Negative ProposalPOLRound")
}
if m.ProposalPOL.Size() == 0 {
return errors.New("Empty ProposalPOL bit array")
}
return nil
}
// String returns a string representation.
func (m *ProposalPOLMessage) String() string {
return fmt.Sprintf("[ProposalPOL H:%v POLR:%v POL:%v]", m.Height, m.ProposalPOLRound, m.ProposalPOL)
@@ -1381,6 +1512,20 @@ type BlockPartMessage struct {
Part *types.Part
}
// ValidateBasic performs basic validation.
func (m *BlockPartMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.Round < 0 {
return errors.New("Negative Round")
}
if err := m.Part.ValidateBasic(); err != nil {
return fmt.Errorf("Wrong Part: %v", err)
}
return nil
}
// String returns a string representation.
func (m *BlockPartMessage) String() string {
return fmt.Sprintf("[BlockPart H:%v R:%v P:%v]", m.Height, m.Round, m.Part)
@@ -1393,6 +1538,11 @@ type VoteMessage struct {
Vote *types.Vote
}
// ValidateBasic performs basic validation.
func (m *VoteMessage) ValidateBasic() error {
return m.Vote.ValidateBasic()
}
// String returns a string representation.
func (m *VoteMessage) String() string {
return fmt.Sprintf("[Vote %v]", m.Vote)
@@ -1404,10 +1554,27 @@ func (m *VoteMessage) String() string {
type HasVoteMessage struct {
Height int64
Round int
Type byte
Type types.SignedMsgType
Index int
}
// ValidateBasic performs basic validation.
func (m *HasVoteMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.Round < 0 {
return errors.New("Negative Round")
}
if !types.IsVoteTypeValid(m.Type) {
return errors.New("Invalid Type")
}
if m.Index < 0 {
return errors.New("Negative Index")
}
return nil
}
// String returns a string representation.
func (m *HasVoteMessage) String() string {
return fmt.Sprintf("[HasVote VI:%v V:{%v/%02d/%v}]", m.Index, m.Height, m.Round, m.Type)
@@ -1419,10 +1586,27 @@ func (m *HasVoteMessage) String() string {
type VoteSetMaj23Message struct {
Height int64
Round int
Type byte
Type types.SignedMsgType
BlockID types.BlockID
}
// ValidateBasic performs basic validation.
func (m *VoteSetMaj23Message) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.Round < 0 {
return errors.New("Negative Round")
}
if !types.IsVoteTypeValid(m.Type) {
return errors.New("Invalid Type")
}
if err := m.BlockID.ValidateBasic(); err != nil {
return fmt.Errorf("Wrong BlockID: %v", err)
}
return nil
}
// String returns a string representation.
func (m *VoteSetMaj23Message) String() string {
return fmt.Sprintf("[VSM23 %v/%02d/%v %v]", m.Height, m.Round, m.Type, m.BlockID)
@@ -1434,24 +1618,32 @@ func (m *VoteSetMaj23Message) String() string {
type VoteSetBitsMessage struct {
Height int64
Round int
Type byte
Type types.SignedMsgType
BlockID types.BlockID
Votes *cmn.BitArray
}
// ValidateBasic performs basic validation.
func (m *VoteSetBitsMessage) ValidateBasic() error {
if m.Height < 0 {
return errors.New("Negative Height")
}
if m.Round < 0 {
return errors.New("Negative Round")
}
if !types.IsVoteTypeValid(m.Type) {
return errors.New("Invalid Type")
}
if err := m.BlockID.ValidateBasic(); err != nil {
return fmt.Errorf("Wrong BlockID: %v", err)
}
// NOTE: Votes.Size() can be zero if the node does not have any
return nil
}
// String returns a string representation.
func (m *VoteSetBitsMessage) String() string {
return fmt.Sprintf("[VSB %v/%02d/%v %v %v]", m.Height, m.Round, m.Type, m.BlockID, m.Votes)
}
//-------------------------------------
// ProposalHeartbeatMessage is sent to signal that a node is alive and waiting for transactions for a proposal.
type ProposalHeartbeatMessage struct {
Heartbeat *types.Heartbeat
}
// String returns a string representation.
func (m *ProposalHeartbeatMessage) String() string {
return fmt.Sprintf("[HEARTBEAT %v]", m.Heartbeat)
}

View File

@@ -11,24 +11,20 @@ import (
"testing"
"time"
abcicli "github.com/tendermint/tendermint/abci/client"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/tendermint/tendermint/abci/client"
"github.com/tendermint/tendermint/abci/example/kvstore"
abci "github.com/tendermint/tendermint/abci/types"
bc "github.com/tendermint/tendermint/blockchain"
cmn "github.com/tendermint/tendermint/libs/common"
cfg "github.com/tendermint/tendermint/config"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/libs/log"
mempl "github.com/tendermint/tendermint/mempool"
sm "github.com/tendermint/tendermint/state"
tmtime "github.com/tendermint/tendermint/types/time"
cfg "github.com/tendermint/tendermint/config"
"github.com/tendermint/tendermint/p2p"
p2pdummy "github.com/tendermint/tendermint/p2p/dummy"
sm "github.com/tendermint/tendermint/state"
"github.com/tendermint/tendermint/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func init() {
@@ -101,6 +97,9 @@ func TestReactorBasic(t *testing.T) {
// Ensure we can process blocks with evidence
func TestReactorWithEvidence(t *testing.T) {
types.RegisterMockEvidences(cdc)
types.RegisterMockEvidences(types.GetCodec())
nValidators := 4
testName := "consensus_reactor_test"
tickerFunc := newMockTickerFunc(true)
@@ -196,7 +195,7 @@ func newMockEvidencePool(val []byte) *mockEvidencePool {
}
// NOTE: maxBytes is ignored
func (m *mockEvidencePool) PendingEvidence(maxBytes int) []types.Evidence {
func (m *mockEvidencePool) PendingEvidence(maxBytes int64) []types.Evidence {
if m.height > 0 {
return m.ev
}
@@ -214,8 +213,8 @@ func (m *mockEvidencePool) Update(block *types.Block, state sm.State) {
//------------------------------------
// Ensure a testnet sends proposal heartbeats and makes blocks when there are txs
func TestReactorProposalHeartbeats(t *testing.T) {
// Ensure a testnet makes blocks when there are txs
func TestReactorCreatesBlockWhenEmptyBlocksFalse(t *testing.T) {
N := 4
css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter,
func(c *cfg.Config) {
@@ -223,17 +222,6 @@ func TestReactorProposalHeartbeats(t *testing.T) {
})
reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
heartbeatChans := make([]chan interface{}, N)
var err error
for i := 0; i < N; i++ {
heartbeatChans[i] = make(chan interface{}, 1)
err = eventBuses[i].Subscribe(context.Background(), testSubscriber, types.EventQueryProposalHeartbeat, heartbeatChans[i])
require.NoError(t, err)
}
// wait till everyone sends a proposal heartbeat
timeoutWaitGroup(t, N, func(j int) {
<-heartbeatChans[j]
}, css)
// send a tx
if err := css[3].mempool.CheckTx([]byte{1, 2, 3}, nil); err != nil {
@@ -246,110 +234,25 @@ func TestReactorProposalHeartbeats(t *testing.T) {
}, css)
}
// Test we record block parts from other peers
func TestReactorRecordsBlockParts(t *testing.T) {
// create dummy peer
peer := p2pdummy.NewPeer()
ps := NewPeerState(peer).SetLogger(log.TestingLogger())
peer.Set(types.PeerStateKey, ps)
// Test we record stats about votes and block parts from other peers.
func TestReactorRecordsVotesAndBlockParts(t *testing.T) {
N := 4
css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
// create reactor
css := randConsensusNet(1, "consensus_reactor_records_block_parts_test", newMockTickerFunc(true), newPersistentKVStore)
reactor := NewConsensusReactor(css[0], false) // so we dont start the consensus states
reactor.SetEventBus(css[0].eventBus)
reactor.SetLogger(log.TestingLogger())
sw := p2p.MakeSwitch(cfg.DefaultP2PConfig(), 1, "testing", "123.123.123", func(i int, sw *p2p.Switch) *p2p.Switch { return sw })
reactor.SetSwitch(sw)
err := reactor.Start()
require.NoError(t, err)
defer reactor.Stop()
// wait till everyone makes the first new block
timeoutWaitGroup(t, N, func(j int) {
<-eventChans[j]
}, css)
// 1) new block part
parts := types.NewPartSetFromData(cmn.RandBytes(100), 10)
msg := &BlockPartMessage{
Height: 2,
Round: 0,
Part: parts.GetPart(0),
}
bz, err := cdc.MarshalBinaryBare(msg)
require.NoError(t, err)
// Get peer
peer := reactors[1].Switch.Peers().List()[0]
// Get peer state
ps := peer.Get(types.PeerStateKey).(*PeerState)
reactor.Receive(DataChannel, peer, bz)
require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should have increased by 1")
// 2) block part with the same height, but different round
msg.Round = 1
bz, err = cdc.MarshalBinaryBare(msg)
require.NoError(t, err)
reactor.Receive(DataChannel, peer, bz)
require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should stay the same")
// 3) block part from earlier height
msg.Height = 1
msg.Round = 0
bz, err = cdc.MarshalBinaryBare(msg)
require.NoError(t, err)
reactor.Receive(DataChannel, peer, bz)
require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should stay the same")
}
// Test we record votes from other peers.
func TestReactorRecordsVotes(t *testing.T) {
// Create dummy peer.
peer := p2pdummy.NewPeer()
ps := NewPeerState(peer).SetLogger(log.TestingLogger())
peer.Set(types.PeerStateKey, ps)
// Create reactor.
css := randConsensusNet(1, "consensus_reactor_records_votes_test", newMockTickerFunc(true), newPersistentKVStore)
reactor := NewConsensusReactor(css[0], false) // so we dont start the consensus states
reactor.SetEventBus(css[0].eventBus)
reactor.SetLogger(log.TestingLogger())
sw := p2p.MakeSwitch(cfg.DefaultP2PConfig(), 1, "testing", "123.123.123", func(i int, sw *p2p.Switch) *p2p.Switch { return sw })
reactor.SetSwitch(sw)
err := reactor.Start()
require.NoError(t, err)
defer reactor.Stop()
_, val := css[0].state.Validators.GetByIndex(0)
// 1) new vote
vote := &types.Vote{
ValidatorIndex: 0,
ValidatorAddress: val.Address,
Height: 2,
Round: 0,
Timestamp: tmtime.Now(),
Type: types.VoteTypePrevote,
BlockID: types.BlockID{},
}
bz, err := cdc.MarshalBinaryBare(&VoteMessage{vote})
require.NoError(t, err)
reactor.Receive(VoteChannel, peer, bz)
assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should have increased by 1")
// 2) vote with the same height, but different round
vote.Round = 1
bz, err = cdc.MarshalBinaryBare(&VoteMessage{vote})
require.NoError(t, err)
reactor.Receive(VoteChannel, peer, bz)
assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should stay the same")
// 3) vote from earlier height
vote.Height = 1
vote.Round = 0
bz, err = cdc.MarshalBinaryBare(&VoteMessage{vote})
require.NoError(t, err)
reactor.Receive(VoteChannel, peer, bz)
assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should stay the same")
assert.Equal(t, true, ps.VotesSent() > 0, "number of votes sent should have increased")
assert.Equal(t, true, ps.BlockPartsSent() > 0, "number of votes sent should have increased")
}
//-------------------------------------------------------------

View File

@@ -11,6 +11,7 @@ import (
"time"
abci "github.com/tendermint/tendermint/abci/types"
"github.com/tendermint/tendermint/version"
//auto "github.com/tendermint/tendermint/libs/autofile"
cmn "github.com/tendermint/tendermint/libs/common"
dbm "github.com/tendermint/tendermint/libs/db"
@@ -19,7 +20,6 @@ import (
"github.com/tendermint/tendermint/proxy"
sm "github.com/tendermint/tendermint/state"
"github.com/tendermint/tendermint/types"
"github.com/tendermint/tendermint/version"
)
var crc32c = crc32.MakeTable(crc32.Castagnoli)
@@ -73,7 +73,7 @@ func (cs *ConsensusState) readReplayMessage(msg *TimedWALMessage, newStepCh chan
case *ProposalMessage:
p := msg.Proposal
cs.Logger.Info("Replay: Proposal", "height", p.Height, "round", p.Round, "header",
p.BlockPartsHeader, "pol", p.POLRound, "peer", peerID)
p.BlockID.PartsHeader, "pol", p.POLRound, "peer", peerID)
case *BlockPartMessage:
cs.Logger.Info("Replay: BlockPart", "height", msg.Height, "round", msg.Round, "peer", peerID)
case *VoteMessage:
@@ -227,7 +227,7 @@ func (h *Handshaker) NBlocks() int {
func (h *Handshaker) Handshake(proxyApp proxy.AppConns) error {
// Handshake is done via ABCI Info on the query conn.
res, err := proxyApp.Query().InfoSync(abci.RequestInfo{Version: version.Version})
res, err := proxyApp.Query().InfoSync(proxy.RequestInfo)
if err != nil {
return fmt.Errorf("Error calling Info: %v", err)
}
@@ -238,9 +238,15 @@ func (h *Handshaker) Handshake(proxyApp proxy.AppConns) error {
}
appHash := res.LastBlockAppHash
h.logger.Info("ABCI Handshake", "appHeight", blockHeight, "appHash", fmt.Sprintf("%X", appHash))
h.logger.Info("ABCI Handshake App Info",
"height", blockHeight,
"hash", fmt.Sprintf("%X", appHash),
"software-version", res.Version,
"protocol-version", res.AppVersion,
)
// TODO: check app version.
// Set AppVersion on the state.
h.initialState.Version.Consensus.App = version.Protocol(res.AppVersion)
// Replay blocks up to the latest in the blockstore.
_, err = h.ReplayBlocks(h.initialState, appHash, blockHeight, proxyApp)
@@ -258,15 +264,24 @@ func (h *Handshaker) Handshake(proxyApp proxy.AppConns) error {
// Replay all blocks since appBlockHeight and ensure the result matches the current state.
// Returns the final AppHash or an error.
func (h *Handshaker) ReplayBlocks(state sm.State, appHash []byte, appBlockHeight int64, proxyApp proxy.AppConns) ([]byte, error) {
func (h *Handshaker) ReplayBlocks(
state sm.State,
appHash []byte,
appBlockHeight int64,
proxyApp proxy.AppConns,
) ([]byte, error) {
storeBlockHeight := h.store.Height()
stateBlockHeight := state.LastBlockHeight
h.logger.Info("ABCI Replay Blocks", "appHeight", appBlockHeight, "storeHeight", storeBlockHeight, "stateHeight", stateBlockHeight)
// If appBlockHeight == 0 it means that we are at genesis and hence should send InitChain.
if appBlockHeight == 0 {
nextVals := types.TM2PB.ValidatorUpdates(state.NextValidators) // state.Validators would work too.
validators := make([]*types.Validator, len(h.genDoc.Validators))
for i, val := range h.genDoc.Validators {
validators[i] = types.NewValidator(val.PubKey, val.Power)
}
validatorSet := types.NewValidatorSet(validators)
nextVals := types.TM2PB.ValidatorUpdates(validatorSet)
csParams := types.TM2PB.ConsensusParams(h.genDoc.ConsensusParams)
req := abci.RequestInitChain{
Time: h.genDoc.GenesisTime,
@@ -287,6 +302,7 @@ func (h *Handshaker) ReplayBlocks(state sm.State, appHash []byte, appBlockHeight
return nil, err
}
state.Validators = types.NewValidatorSet(vals)
state.NextValidators = types.NewValidatorSet(vals)
}
if res.ConsensusParams != nil {
state.ConsensusParams = types.PB2TM.ConsensusParams(res.ConsensusParams)

View File

@@ -58,7 +58,18 @@ func (cs *ConsensusState) ReplayFile(file string, console bool) error {
if err != nil {
return errors.Errorf("failed to subscribe %s to %v", subscriber, types.EventQueryNewRoundStep)
}
defer cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
defer func() {
// drain newStepCh to make sure we don't block
LOOP:
for {
select {
case <-newStepCh:
default:
break LOOP
}
}
cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
}()
// just open the file for reading, no need to use wal
fp, err := os.OpenFile(file, os.O_RDONLY, 0600)
@@ -221,7 +232,18 @@ func (pb *playback) replayConsoleLoop() int {
if err != nil {
cmn.Exit(fmt.Sprintf("failed to subscribe %s to %v", subscriber, types.EventQueryNewRoundStep))
}
defer pb.cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
defer func() {
// drain newStepCh to make sure we don't block
LOOP:
for {
select {
case <-newStepCh:
default:
break LOOP
}
}
pb.cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
}()
if len(tokens) == 1 {
if err := pb.replayReset(1, newStepCh); err != nil {
@@ -298,13 +320,18 @@ func newConsensusStateForReplay(config cfg.BaseConfig, csConfig *cfg.ConsensusCo
// Create proxyAppConn connection (consensus, mempool, query)
clientCreator := proxy.DefaultClientCreator(config.ProxyApp, config.ABCI, config.DBDir())
proxyApp := proxy.NewAppConns(clientCreator,
NewHandshaker(stateDB, state, blockStore, gdoc))
proxyApp := proxy.NewAppConns(clientCreator)
err = proxyApp.Start()
if err != nil {
cmn.Exit(fmt.Sprintf("Error starting proxy app conns: %v", err))
}
handshaker := NewHandshaker(stateDB, state, blockStore, gdoc)
err = handshaker.Handshake(proxyApp)
if err != nil {
cmn.Exit(fmt.Sprintf("Error on handshake: %v", err))
}
eventBus := types.NewEventBus()
if err := eventBus.Start(); err != nil {
cmn.Exit(fmt.Sprintf("Failed to start event bus: %v", err))

View File

@@ -17,9 +17,10 @@ import (
"github.com/tendermint/tendermint/abci/example/kvstore"
abci "github.com/tendermint/tendermint/abci/types"
crypto "github.com/tendermint/tendermint/crypto"
"github.com/tendermint/tendermint/crypto"
auto "github.com/tendermint/tendermint/libs/autofile"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/version"
cfg "github.com/tendermint/tendermint/config"
"github.com/tendermint/tendermint/libs/log"
@@ -102,14 +103,6 @@ func TestWALCrash(t *testing.T) {
{"empty block",
func(stateDB dbm.DB, cs *ConsensusState, ctx context.Context) {},
1},
{"block with a smaller part size",
func(stateDB dbm.DB, cs *ConsensusState, ctx context.Context) {
// XXX: is there a better way to change BlockPartSizeBytes?
cs.state.ConsensusParams.BlockPartSizeBytes = 512
sm.SaveState(stateDB, cs.state)
go sendTxs(cs, ctx)
},
1},
{"many non-empty blocks",
func(stateDB dbm.DB, cs *ConsensusState, ctx context.Context) {
go sendTxs(cs, ctx)
@@ -322,30 +315,23 @@ func testHandshakeReplay(t *testing.T, nBlocks int, mode uint) {
config := ResetConfig("proxy_test_")
walBody, err := WALWithNBlocks(NUM_BLOCKS)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
walFile := tempWALWithData(walBody)
config.Consensus.SetWalFile(walFile)
privVal := privval.LoadFilePV(config.PrivValidatorFile())
wal, err := NewWAL(walFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
wal.SetLogger(log.TestingLogger())
if err := wal.Start(); err != nil {
t.Fatal(err)
}
err = wal.Start()
require.NoError(t, err)
defer wal.Stop()
chain, commits, err := makeBlockchainFromWAL(wal)
if err != nil {
t.Fatalf(err.Error())
}
require.NoError(t, err)
stateDB, state, store := stateAndStore(config, privVal.GetPubKey())
stateDB, state, store := stateAndStore(config, privVal.GetPubKey(), kvstore.ProtocolVersion)
store.chain = chain
store.commits = commits
@@ -359,19 +345,22 @@ func testHandshakeReplay(t *testing.T, nBlocks int, mode uint) {
if nBlocks > 0 {
// run nBlocks against a new client to build up the app state.
// use a throwaway tendermint state
proxyApp := proxy.NewAppConns(clientCreator2, nil)
stateDB, state, _ := stateAndStore(config, privVal.GetPubKey())
proxyApp := proxy.NewAppConns(clientCreator2)
stateDB, state, _ := stateAndStore(config, privVal.GetPubKey(), kvstore.ProtocolVersion)
buildAppStateFromChain(proxyApp, stateDB, state, chain, nBlocks, mode)
}
// now start the app using the handshake - it should sync
genDoc, _ := sm.MakeGenesisDocFromFile(config.GenesisFile())
handshaker := NewHandshaker(stateDB, state, store, genDoc)
proxyApp := proxy.NewAppConns(clientCreator2, handshaker)
proxyApp := proxy.NewAppConns(clientCreator2)
if err := proxyApp.Start(); err != nil {
t.Fatalf("Error starting proxy app connections: %v", err)
}
defer proxyApp.Stop()
if err := handshaker.Handshake(proxyApp); err != nil {
t.Fatalf("Error on abci handshake: %v", err)
}
// get the latest app hash from the app
res, err := proxyApp.Query().InfoSync(abci.RequestInfo{Version: ""})
@@ -397,7 +386,7 @@ func testHandshakeReplay(t *testing.T, nBlocks int, mode uint) {
}
func applyBlock(stateDB dbm.DB, st sm.State, blk *types.Block, proxyApp proxy.AppConns) sm.State {
testPartSize := st.ConsensusParams.BlockPartSizeBytes
testPartSize := types.BlockPartSizeBytes
blockExec := sm.NewBlockExecutor(stateDB, log.TestingLogger(), proxyApp.Consensus(), mempool, evpool)
blkID := types.BlockID{blk.Hash(), blk.MakePartSet(testPartSize).Header()}
@@ -447,7 +436,7 @@ func buildAppStateFromChain(proxyApp proxy.AppConns, stateDB dbm.DB,
func buildTMStateFromChain(config *cfg.Config, stateDB dbm.DB, state sm.State, chain []*types.Block, mode uint) sm.State {
// run the whole chain against this client to build up the tendermint state
clientCreator := proxy.NewLocalClientCreator(kvstore.NewPersistentKVStoreApplication(path.Join(config.DBDir(), "1")))
proxyApp := proxy.NewAppConns(clientCreator, nil) // sm.NewHandshaker(config, state, store, ReplayLastBlock))
proxyApp := proxy.NewAppConns(clientCreator)
if err := proxyApp.Start(); err != nil {
panic(err)
}
@@ -524,7 +513,7 @@ func makeBlockchainFromWAL(wal WAL) ([]*types.Block, []*types.Commit, error) {
// if its not the first one, we have a full block
if thisBlockParts != nil {
var block = new(types.Block)
_, err = cdc.UnmarshalBinaryReader(thisBlockParts.GetReader(), block, 0)
_, err = cdc.UnmarshalBinaryLengthPrefixedReader(thisBlockParts.GetReader(), block, 0)
if err != nil {
panic(err)
}
@@ -547,7 +536,7 @@ func makeBlockchainFromWAL(wal WAL) ([]*types.Block, []*types.Commit, error) {
return nil, nil, err
}
case *types.Vote:
if p.Type == types.VoteTypePrecommit {
if p.Type == types.PrecommitType {
thisBlockCommit = &types.Commit{
BlockID: p.BlockID,
Precommits: []*types.Vote{p},
@@ -557,7 +546,7 @@ func makeBlockchainFromWAL(wal WAL) ([]*types.Block, []*types.Commit, error) {
}
// grab the last block too
var block = new(types.Block)
_, err = cdc.UnmarshalBinaryReader(thisBlockParts.GetReader(), block, 0)
_, err = cdc.UnmarshalBinaryLengthPrefixedReader(thisBlockParts.GetReader(), block, 0)
if err != nil {
panic(err)
}
@@ -579,7 +568,7 @@ func readPieceFromWAL(msg *TimedWALMessage) interface{} {
case msgInfo:
switch msg := m.Msg.(type) {
case *ProposalMessage:
return &msg.Proposal.BlockPartsHeader
return &msg.Proposal.BlockID.PartsHeader
case *BlockPartMessage:
return msg.Part
case *VoteMessage:
@@ -593,9 +582,10 @@ func readPieceFromWAL(msg *TimedWALMessage) interface{} {
}
// fresh state and mock store
func stateAndStore(config *cfg.Config, pubKey crypto.PubKey) (dbm.DB, sm.State, *mockBlockStore) {
func stateAndStore(config *cfg.Config, pubKey crypto.PubKey, appVersion version.Protocol) (dbm.DB, sm.State, *mockBlockStore) {
stateDB := dbm.NewMemDB()
state, _ := sm.MakeGenesisStateFromFile(config.GenesisFile())
state.Version.Consensus.App = appVersion
store := NewMockBlockStore(config, state.ConsensusParams)
return stateDB, state, store
}
@@ -620,7 +610,7 @@ func (bs *mockBlockStore) LoadBlock(height int64) *types.Block { return bs.chain
func (bs *mockBlockStore) LoadBlockMeta(height int64) *types.BlockMeta {
block := bs.chain[height-1]
return &types.BlockMeta{
BlockID: types.BlockID{block.Hash(), block.MakePartSet(bs.params.BlockPartSizeBytes).Header()},
BlockID: types.BlockID{block.Hash(), block.MakePartSet(types.BlockPartSizeBytes).Header()},
Header: block.Header,
}
}
@@ -644,18 +634,21 @@ func TestInitChainUpdateValidators(t *testing.T) {
config := ResetConfig("proxy_test_")
privVal := privval.LoadFilePV(config.PrivValidatorFile())
stateDB, state, store := stateAndStore(config, privVal.GetPubKey())
stateDB, state, store := stateAndStore(config, privVal.GetPubKey(), 0x0)
oldValAddr := state.Validators.Validators[0].Address
// now start the app using the handshake - it should sync
genDoc, _ := sm.MakeGenesisDocFromFile(config.GenesisFile())
handshaker := NewHandshaker(stateDB, state, store, genDoc)
proxyApp := proxy.NewAppConns(clientCreator, handshaker)
proxyApp := proxy.NewAppConns(clientCreator)
if err := proxyApp.Start(); err != nil {
t.Fatalf("Error starting proxy app connections: %v", err)
}
defer proxyApp.Stop()
if err := handshaker.Handshake(proxyApp); err != nil {
t.Fatalf("Error on abci handshake: %v", err)
}
// reload the state, check the validator set was updated
state = sm.LoadState(stateDB)

View File

@@ -9,8 +9,8 @@ import (
"sync"
"time"
fail "github.com/ebuchman/fail-test"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/libs/fail"
"github.com/tendermint/tendermint/libs/log"
tmtime "github.com/tendermint/tendermint/types/time"
@@ -22,13 +22,6 @@ import (
"github.com/tendermint/tendermint/types"
)
//-----------------------------------------------------------------------------
// Config
const (
proposalHeartbeatIntervalSeconds = 2
)
//-----------------------------------------------------------------------------
// Errors
@@ -83,7 +76,8 @@ type ConsensusState struct {
// internal state
mtx sync.RWMutex
cstypes.RoundState
state sm.State // State until height-1.
triggeredTimeoutPrecommit bool
state sm.State // State until height-1.
// state changes may be triggered by: msgs from peers,
// msgs from ourself, or by timeouts
@@ -91,6 +85,10 @@ type ConsensusState struct {
internalMsgQueue chan msgInfo
timeoutTicker TimeoutTicker
// information about about added votes and block parts are written on this channel
// so statistics can be computed by reactor
statsMsgQueue chan msgInfo
// we use eventBus to trigger msg broadcasts in the reactor,
// and to notify external subscribers, eg. through a websocket
eventBus *types.EventBus
@@ -113,15 +111,15 @@ type ConsensusState struct {
done chan struct{}
// synchronous pubsub between consensus state and reactor.
// state only emits EventNewRoundStep, EventVote and EventProposalHeartbeat
// state only emits EventNewRoundStep and EventVote
evsw tmevents.EventSwitch
// for reporting metrics
metrics *Metrics
}
// CSOption sets an optional parameter on the ConsensusState.
type CSOption func(*ConsensusState)
// StateOption sets an optional parameter on the ConsensusState.
type StateOption func(*ConsensusState)
// NewConsensusState returns a new ConsensusState.
func NewConsensusState(
@@ -131,7 +129,7 @@ func NewConsensusState(
blockStore sm.BlockStore,
mempool sm.Mempool,
evpool sm.EvidencePool,
options ...CSOption,
options ...StateOption,
) *ConsensusState {
cs := &ConsensusState{
config: config,
@@ -141,6 +139,7 @@ func NewConsensusState(
peerMsgQueue: make(chan msgInfo, msgQueueSize),
internalMsgQueue: make(chan msgInfo, msgQueueSize),
timeoutTicker: NewTimeoutTicker(),
statsMsgQueue: make(chan msgInfo, msgQueueSize),
done: make(chan struct{}),
doWALCatchup: true,
wal: nilWAL{},
@@ -180,8 +179,8 @@ func (cs *ConsensusState) SetEventBus(b *types.EventBus) {
cs.blockExec.SetEventBus(b)
}
// WithMetrics sets the metrics.
func WithMetrics(metrics *Metrics) CSOption {
// StateMetrics sets the metrics.
func StateMetrics(metrics *Metrics) StateOption {
return func(cs *ConsensusState) { cs.metrics = metrics }
}
@@ -201,18 +200,16 @@ func (cs *ConsensusState) GetState() sm.State {
// GetLastHeight returns the last height committed.
// If there were no blocks, returns 0.
func (cs *ConsensusState) GetLastHeight() int64 {
cs.mtx.Lock()
defer cs.mtx.Unlock()
cs.mtx.RLock()
defer cs.mtx.RUnlock()
return cs.RoundState.Height - 1
}
// GetRoundState returns a shallow copy of the internal consensus state.
func (cs *ConsensusState) GetRoundState() *cstypes.RoundState {
cs.mtx.RLock()
defer cs.mtx.RUnlock()
rs := cs.RoundState // copy
cs.mtx.RUnlock()
return &rs
}
@@ -220,7 +217,6 @@ func (cs *ConsensusState) GetRoundState() *cstypes.RoundState {
func (cs *ConsensusState) GetRoundStateJSON() ([]byte, error) {
cs.mtx.RLock()
defer cs.mtx.RUnlock()
return cdc.MarshalJSON(cs.RoundState)
}
@@ -228,7 +224,6 @@ func (cs *ConsensusState) GetRoundStateJSON() ([]byte, error) {
func (cs *ConsensusState) GetRoundStateSimpleJSON() ([]byte, error) {
cs.mtx.RLock()
defer cs.mtx.RUnlock()
return cdc.MarshalJSON(cs.RoundState.RoundStateSimple())
}
@@ -242,15 +237,15 @@ func (cs *ConsensusState) GetValidators() (int64, []*types.Validator) {
// SetPrivValidator sets the private validator account for signing votes.
func (cs *ConsensusState) SetPrivValidator(priv types.PrivValidator) {
cs.mtx.Lock()
defer cs.mtx.Unlock()
cs.privValidator = priv
cs.mtx.Unlock()
}
// SetTimeoutTicker sets the local timer. It may be useful to overwrite for testing.
func (cs *ConsensusState) SetTimeoutTicker(timeoutTicker TimeoutTicker) {
cs.mtx.Lock()
defer cs.mtx.Unlock()
cs.timeoutTicker = timeoutTicker
cs.mtx.Unlock()
}
// LoadCommit loads the commit for a given height.
@@ -322,10 +317,11 @@ func (cs *ConsensusState) startRoutines(maxSteps int) {
go cs.receiveRoutine(maxSteps)
}
// OnStop implements cmn.Service. It stops all routines and waits for the WAL to finish.
// OnStop implements cmn.Service.
func (cs *ConsensusState) OnStop() {
cs.evsw.Stop()
cs.timeoutTicker.Stop()
// WAL is stopped in receiveRoutine.
}
// Wait waits for the the main routine to return.
@@ -454,7 +450,7 @@ func (cs *ConsensusState) reconstructLastCommit(state sm.State) {
return
}
seenCommit := cs.blockStore.LoadSeenCommit(state.LastBlockHeight)
lastPrecommits := types.NewVoteSet(state.ChainID, state.LastBlockHeight, seenCommit.Round(), types.VoteTypePrecommit, state.LastValidators)
lastPrecommits := types.NewVoteSet(state.ChainID, state.LastBlockHeight, seenCommit.Round(), types.PrecommitType, state.LastValidators)
for _, precommit := range seenCommit.Precommits {
if precommit == nil {
continue
@@ -526,10 +522,10 @@ func (cs *ConsensusState) updateToState(state sm.State) {
cs.Proposal = nil
cs.ProposalBlock = nil
cs.ProposalBlockParts = nil
cs.LockedRound = 0
cs.LockedRound = -1
cs.LockedBlock = nil
cs.LockedBlockParts = nil
cs.ValidRound = 0
cs.ValidRound = -1
cs.ValidBlock = nil
cs.ValidBlockParts = nil
cs.Votes = cstypes.NewHeightVoteSet(state.ChainID, height, validators)
@@ -639,7 +635,11 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
err = cs.setProposal(msg.Proposal)
case *BlockPartMessage:
// if the proposal is complete, we'll enterPrevote or tryFinalizeCommit
_, err = cs.addProposalBlockPart(msg, peerID)
added, err := cs.addProposalBlockPart(msg, peerID)
if added {
cs.statsMsgQueue <- mi
}
if err != nil && msg.Round != cs.Round {
cs.Logger.Debug("Received block part from wrong round", "height", cs.Height, "csRound", cs.Round, "blockRound", msg.Round)
err = nil
@@ -647,7 +647,11 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
case *VoteMessage:
// attempt to add the vote and dupeout the validator if its a duplicate signature
// if the vote gives us a 2/3-any or 2/3-one, we transition
err := cs.tryAddVote(msg.Vote, peerID)
added, err := cs.tryAddVote(msg.Vote, peerID)
if added {
cs.statsMsgQueue <- mi
}
if err == ErrAddingVote {
// TODO: punish peer
// We probably don't want to stop the peer here. The vote does not
@@ -698,6 +702,7 @@ func (cs *ConsensusState) handleTimeout(ti timeoutInfo, rs cstypes.RoundState) {
cs.enterPrecommit(ti.Height, ti.Round)
case cstypes.RoundStepPrecommitWait:
cs.eventBus.PublishEventTimeoutWait(cs.RoundStateEvent())
cs.enterPrecommit(ti.Height, ti.Round)
cs.enterNewRound(ti.Height, ti.Round+1)
default:
panic(fmt.Sprintf("Invalid timeout step: %v", ti.Step))
@@ -740,7 +745,7 @@ func (cs *ConsensusState) enterNewRound(height int64, round int) {
validators := cs.Validators
if cs.Round < round {
validators = validators.Copy()
validators.IncrementAccum(round - cs.Round)
validators.IncrementProposerPriority(round - cs.Round)
}
// Setup new round
@@ -759,8 +764,9 @@ func (cs *ConsensusState) enterNewRound(height int64, round int) {
cs.ProposalBlockParts = nil
}
cs.Votes.SetRound(round + 1) // also track next round (round+1) to allow round-skipping
cs.triggeredTimeoutPrecommit = false
cs.eventBus.PublishEventNewRound(cs.RoundStateEvent())
cs.eventBus.PublishEventNewRound(cs.NewRoundEvent())
cs.metrics.Rounds.Set(float64(round))
// Wait for txs to be available in the mempool
@@ -769,9 +775,9 @@ func (cs *ConsensusState) enterNewRound(height int64, round int) {
waitForTxs := cs.config.WaitForTxs() && round == 0 && !cs.needProofBlock(height)
if waitForTxs {
if cs.config.CreateEmptyBlocksInterval > 0 {
cs.scheduleTimeout(cs.config.EmptyBlocksInterval(), height, round, cstypes.RoundStepNewRound)
cs.scheduleTimeout(cs.config.CreateEmptyBlocksInterval, height, round,
cstypes.RoundStepNewRound)
}
go cs.proposalHeartbeat(height, round)
} else {
cs.enterPropose(height, round)
}
@@ -788,32 +794,6 @@ func (cs *ConsensusState) needProofBlock(height int64) bool {
return !bytes.Equal(cs.state.AppHash, lastBlockMeta.Header.AppHash)
}
func (cs *ConsensusState) proposalHeartbeat(height int64, round int) {
counter := 0
addr := cs.privValidator.GetAddress()
valIndex, _ := cs.Validators.GetByAddress(addr)
chainID := cs.state.ChainID
for {
rs := cs.GetRoundState()
// if we've already moved on, no need to send more heartbeats
if rs.Step > cstypes.RoundStepNewRound || rs.Round > round || rs.Height > height {
return
}
heartbeat := &types.Heartbeat{
Height: rs.Height,
Round: rs.Round,
Sequence: counter,
ValidatorAddress: addr,
ValidatorIndex: valIndex,
}
cs.privValidator.SignHeartbeat(chainID, heartbeat)
cs.eventBus.PublishEventProposalHeartbeat(types.EventDataProposalHeartbeat{heartbeat})
cs.evsw.FireEvent(types.EventProposalHeartbeat, heartbeat)
counter++
time.Sleep(proposalHeartbeatIntervalSeconds * time.Second)
}
}
// Enter (CreateEmptyBlocks): from enterNewRound(height,round)
// Enter (CreateEmptyBlocks, CreateEmptyBlocksInterval > 0 ): after enterNewRound(height,round), after timeout of CreateEmptyBlocksInterval
// Enter (!CreateEmptyBlocks) : after enterNewRound(height,round), once txs are in the mempool
@@ -872,10 +852,7 @@ func (cs *ConsensusState) defaultDecideProposal(height int64, round int) {
var blockParts *types.PartSet
// Decide on block
if cs.LockedBlock != nil {
// If we're locked onto a block, just choose that.
block, blockParts = cs.LockedBlock, cs.LockedBlockParts
} else if cs.ValidBlock != nil {
if cs.ValidBlock != nil {
// If there is valid block, choose that.
block, blockParts = cs.ValidBlock, cs.ValidBlockParts
} else {
@@ -887,15 +864,9 @@ func (cs *ConsensusState) defaultDecideProposal(height int64, round int) {
}
// Make proposal
polRound, polBlockID := cs.Votes.POLInfo()
proposal := types.NewProposal(height, round, blockParts.Header(), polRound, polBlockID)
propBlockId := types.BlockID{block.Hash(), blockParts.Header()}
proposal := types.NewProposal(height, round, cs.ValidRound, propBlockId)
if err := cs.privValidator.SignProposal(cs.state.ChainID, proposal); err == nil {
// Set fields
/* fields set by setProposal and addBlockPart
cs.Proposal = proposal
cs.ProposalBlock = block
cs.ProposalBlockParts = blockParts
*/
// send proposal and block parts on internal msg queue
cs.sendInternalMessage(msgInfo{&ProposalMessage{proposal}, ""})
@@ -949,27 +920,23 @@ func (cs *ConsensusState) createProposalBlock() (block *types.Block, blockParts
}
maxBytes := cs.state.ConsensusParams.BlockSize.MaxBytes
maxGas := cs.state.ConsensusParams.BlockSize.MaxGas
// bound evidence to 1/10th of the block
evidence := cs.evpool.PendingEvidence(maxBytes / 10)
evidence := cs.evpool.PendingEvidence(types.MaxEvidenceBytesPerBlock(maxBytes))
// Mempool validated transactions
txs := cs.mempool.ReapMaxBytes(maxDataBytes(maxBytes, cs.state.Validators.Size(), len(evidence)))
txs := cs.mempool.ReapMaxBytesMaxGas(types.MaxDataBytes(
maxBytes,
cs.state.Validators.Size(),
len(evidence),
), maxGas)
proposerAddr := cs.privValidator.GetAddress()
block, parts := cs.state.MakeBlock(cs.Height, txs, commit, evidence, proposerAddr)
return block, parts
}
func maxDataBytes(maxBytes, valsCount, evidenceCount int) int {
return maxBytes -
types.MaxAminoOverheadForBlock -
types.MaxHeaderBytes -
(valsCount * types.MaxVoteBytes) -
(evidenceCount * types.MaxEvidenceBytes)
}
// Enter: `timeoutPropose` after entering Propose.
// Enter: proposal block and POL is ready.
// Enter: any +2/3 prevotes for future round.
// Prevote for LockedBlock if we're locked, or ProposalBlock if valid.
// Otherwise vote nil.
func (cs *ConsensusState) enterPrevote(height int64, round int) {
@@ -984,14 +951,6 @@ func (cs *ConsensusState) enterPrevote(height int64, round int) {
cs.newStep()
}()
// fire event for how we got here
if cs.isProposalComplete() {
cs.eventBus.PublishEventCompleteProposal(cs.RoundStateEvent())
} else {
// we received +2/3 prevotes for a future round
// TODO: catchup event?
}
cs.Logger.Info(fmt.Sprintf("enterPrevote(%v/%v). Current: %v/%v/%v", height, round, cs.Height, cs.Round, cs.Step))
// Sign and broadcast vote as necessary
@@ -1003,17 +962,18 @@ func (cs *ConsensusState) enterPrevote(height int64, round int) {
func (cs *ConsensusState) defaultDoPrevote(height int64, round int) {
logger := cs.Logger.With("height", height, "round", round)
// If a block is locked, prevote that.
if cs.LockedBlock != nil {
logger.Info("enterPrevote: Block was locked")
cs.signAddVote(types.VoteTypePrevote, cs.LockedBlock.Hash(), cs.LockedBlockParts.Header())
cs.signAddVote(types.PrevoteType, cs.LockedBlock.Hash(), cs.LockedBlockParts.Header())
return
}
// If ProposalBlock is nil, prevote nil.
if cs.ProposalBlock == nil {
logger.Info("enterPrevote: ProposalBlock is nil")
cs.signAddVote(types.VoteTypePrevote, nil, types.PartSetHeader{})
cs.signAddVote(types.PrevoteType, nil, types.PartSetHeader{})
return
}
@@ -1022,7 +982,7 @@ func (cs *ConsensusState) defaultDoPrevote(height int64, round int) {
if err != nil {
// ProposalBlock is invalid, prevote nil.
logger.Error("enterPrevote: ProposalBlock is invalid", "err", err)
cs.signAddVote(types.VoteTypePrevote, nil, types.PartSetHeader{})
cs.signAddVote(types.PrevoteType, nil, types.PartSetHeader{})
return
}
@@ -1030,7 +990,7 @@ func (cs *ConsensusState) defaultDoPrevote(height int64, round int) {
// NOTE: the proposal signature is validated when it is received,
// and the proposal block parts are validated as they are received (against the merkle hash in the proposal)
logger.Info("enterPrevote: ProposalBlock is valid")
cs.signAddVote(types.VoteTypePrevote, cs.ProposalBlock.Hash(), cs.ProposalBlockParts.Header())
cs.signAddVote(types.PrevoteType, cs.ProposalBlock.Hash(), cs.ProposalBlockParts.Header())
}
// Enter: any +2/3 prevotes at next round.
@@ -1057,8 +1017,8 @@ func (cs *ConsensusState) enterPrevoteWait(height int64, round int) {
}
// Enter: `timeoutPrevote` after any +2/3 prevotes.
// Enter: `timeoutPrecommit` after any +2/3 precommits.
// Enter: +2/3 precomits for block or nil.
// Enter: any +2/3 precommits for next round.
// Lock & precommit the ProposalBlock if we have enough prevotes for it (a POL in this round)
// else, unlock an existing lock and precommit nil if +2/3 of prevotes were nil,
// else, precommit nil otherwise.
@@ -1088,7 +1048,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
} else {
logger.Info("enterPrecommit: No +2/3 prevotes during enterPrecommit. Precommitting nil.")
}
cs.signAddVote(types.VoteTypePrecommit, nil, types.PartSetHeader{})
cs.signAddVote(types.PrecommitType, nil, types.PartSetHeader{})
return
}
@@ -1107,12 +1067,12 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
logger.Info("enterPrecommit: +2/3 prevoted for nil.")
} else {
logger.Info("enterPrecommit: +2/3 prevoted for nil. Unlocking")
cs.LockedRound = 0
cs.LockedRound = -1
cs.LockedBlock = nil
cs.LockedBlockParts = nil
cs.eventBus.PublishEventUnlock(cs.RoundStateEvent())
}
cs.signAddVote(types.VoteTypePrecommit, nil, types.PartSetHeader{})
cs.signAddVote(types.PrecommitType, nil, types.PartSetHeader{})
return
}
@@ -1123,7 +1083,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
logger.Info("enterPrecommit: +2/3 prevoted locked block. Relocking")
cs.LockedRound = round
cs.eventBus.PublishEventRelock(cs.RoundStateEvent())
cs.signAddVote(types.VoteTypePrecommit, blockID.Hash, blockID.PartsHeader)
cs.signAddVote(types.PrecommitType, blockID.Hash, blockID.PartsHeader)
return
}
@@ -1138,7 +1098,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
cs.LockedBlock = cs.ProposalBlock
cs.LockedBlockParts = cs.ProposalBlockParts
cs.eventBus.PublishEventLock(cs.RoundStateEvent())
cs.signAddVote(types.VoteTypePrecommit, blockID.Hash, blockID.PartsHeader)
cs.signAddVote(types.PrecommitType, blockID.Hash, blockID.PartsHeader)
return
}
@@ -1146,7 +1106,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
// Fetch that block, unlock, and precommit nil.
// The +2/3 prevotes for this round is the POL for our unlock.
// TODO: In the future save the POL prevotes for justification.
cs.LockedRound = 0
cs.LockedRound = -1
cs.LockedBlock = nil
cs.LockedBlockParts = nil
if !cs.ProposalBlockParts.HasHeader(blockID.PartsHeader) {
@@ -1154,15 +1114,19 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
cs.ProposalBlockParts = types.NewPartSetFromHeader(blockID.PartsHeader)
}
cs.eventBus.PublishEventUnlock(cs.RoundStateEvent())
cs.signAddVote(types.VoteTypePrecommit, nil, types.PartSetHeader{})
cs.signAddVote(types.PrecommitType, nil, types.PartSetHeader{})
}
// Enter: any +2/3 precommits for next round.
func (cs *ConsensusState) enterPrecommitWait(height int64, round int) {
logger := cs.Logger.With("height", height, "round", round)
if cs.Height != height || round < cs.Round || (cs.Round == round && cstypes.RoundStepPrecommitWait <= cs.Step) {
logger.Debug(fmt.Sprintf("enterPrecommitWait(%v/%v): Invalid args. Current step: %v/%v/%v", height, round, cs.Height, cs.Round, cs.Step))
if cs.Height != height || round < cs.Round || (cs.Round == round && cs.triggeredTimeoutPrecommit) {
logger.Debug(
fmt.Sprintf(
"enterPrecommitWait(%v/%v): Invalid args. "+
"Current state is Height/Round: %v/%v/, triggeredTimeoutPrecommit:%v",
height, round, cs.Height, cs.Round, cs.triggeredTimeoutPrecommit))
return
}
if !cs.Votes.Precommits(round).HasTwoThirdsAny() {
@@ -1172,7 +1136,7 @@ func (cs *ConsensusState) enterPrecommitWait(height int64, round int) {
defer func() {
// Done enterPrecommitWait:
cs.updateRoundStep(round, cstypes.RoundStepPrecommitWait)
cs.triggeredTimeoutPrecommit = true
cs.newStep()
}()
@@ -1225,6 +1189,8 @@ func (cs *ConsensusState) enterCommit(height int64, commitRound int) {
// Set up ProposalBlockParts and keep waiting.
cs.ProposalBlock = nil
cs.ProposalBlockParts = types.NewPartSetFromHeader(blockID.PartsHeader)
cs.eventBus.PublishEventValidBlock(cs.RoundStateEvent())
cs.evsw.FireEvent(types.EventValidBlock, &cs.RoundState)
} else {
// We just need to keep waiting.
}
@@ -1379,7 +1345,7 @@ func (cs *ConsensusState) recordMetrics(height int64, block *types.Block) {
if height > 1 {
lastBlockMeta := cs.blockStore.LoadBlockMeta(height - 1)
cs.metrics.BlockIntervalSeconds.Observe(
cs.metrics.BlockIntervalSeconds.Set(
block.Time.Sub(lastBlockMeta.Header.Time).Seconds(),
)
}
@@ -1387,6 +1353,8 @@ func (cs *ConsensusState) recordMetrics(height int64, block *types.Block) {
cs.metrics.NumTxs.Set(float64(block.NumTxs))
cs.metrics.BlockSizeBytes.Set(float64(block.Size()))
cs.metrics.TotalTxs.Set(float64(block.TotalTxs))
cs.metrics.CommittedHeight.Set(float64(block.Height))
}
//-----------------------------------------------------------------------------
@@ -1403,14 +1371,9 @@ func (cs *ConsensusState) defaultSetProposal(proposal *types.Proposal) error {
return nil
}
// We don't care about the proposal if we're already in cstypes.RoundStepCommit.
if cstypes.RoundStepCommit <= cs.Step {
return nil
}
// Verify POLRound, which must be -1 or between 0 and proposal.Round exclusive.
if proposal.POLRound != -1 &&
(proposal.POLRound < 0 || proposal.Round <= proposal.POLRound) {
// Verify POLRound, which must be -1 or in range [0, proposal.Round).
if proposal.POLRound < -1 ||
(proposal.POLRound >= 0 && proposal.POLRound >= proposal.Round) {
return ErrInvalidProposalPOLRound
}
@@ -1420,7 +1383,12 @@ func (cs *ConsensusState) defaultSetProposal(proposal *types.Proposal) error {
}
cs.Proposal = proposal
cs.ProposalBlockParts = types.NewPartSetFromHeader(proposal.BlockPartsHeader)
// We don't update cs.ProposalBlockParts if it is already set.
// This happens if we're already in cstypes.RoundStepCommit or if there is a valid block in the current round.
// TODO: We can check if Proposal is for a different block as this is a sign of misbehavior!
if cs.ProposalBlockParts == nil {
cs.ProposalBlockParts = types.NewPartSetFromHeader(proposal.BlockID.PartsHeader)
}
cs.Logger.Info("Received proposal", "proposal", proposal)
return nil
}
@@ -1451,16 +1419,17 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
}
if added && cs.ProposalBlockParts.IsComplete() {
// Added and completed!
_, err = cdc.UnmarshalBinaryReader(
_, err = cdc.UnmarshalBinaryLengthPrefixedReader(
cs.ProposalBlockParts.GetReader(),
&cs.ProposalBlock,
int64(cs.state.ConsensusParams.BlockSize.MaxBytes),
)
if err != nil {
return true, err
return added, err
}
// NOTE: it's possible to receive complete proposal blocks for future rounds without having the proposal
cs.Logger.Info("Received complete proposal block", "height", cs.ProposalBlock.Height, "hash", cs.ProposalBlock.Hash())
cs.eventBus.PublishEventCompleteProposal(cs.CompleteProposalEvent())
// Update Valid* if we can.
prevotes := cs.Votes.Prevotes(cs.Round)
@@ -1483,39 +1452,42 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
if cs.Step <= cstypes.RoundStepPropose && cs.isProposalComplete() {
// Move onto the next step
cs.enterPrevote(height, cs.Round)
if hasTwoThirds { // this is optimisation as this will be triggered when prevote is added
cs.enterPrecommit(height, cs.Round)
}
} else if cs.Step == cstypes.RoundStepCommit {
// If we're waiting on the proposal block...
cs.tryFinalizeCommit(height)
}
return true, nil
return added, nil
}
return added, nil
}
// Attempt to add the vote. if its a duplicate signature, dupeout the validator
func (cs *ConsensusState) tryAddVote(vote *types.Vote, peerID p2p.ID) error {
_, err := cs.addVote(vote, peerID)
func (cs *ConsensusState) tryAddVote(vote *types.Vote, peerID p2p.ID) (bool, error) {
added, err := cs.addVote(vote, peerID)
if err != nil {
// If the vote height is off, we'll just ignore it,
// But if it's a conflicting sig, add it to the cs.evpool.
// If it's otherwise invalid, punish peer.
if err == ErrVoteHeightMismatch {
return err
return added, err
} else if voteErr, ok := err.(*types.ErrVoteConflictingVotes); ok {
if bytes.Equal(vote.ValidatorAddress, cs.privValidator.GetAddress()) {
cs.Logger.Error("Found conflicting vote from ourselves. Did you unsafe_reset a validator?", "height", vote.Height, "round", vote.Round, "type", vote.Type)
return err
return added, err
}
cs.evpool.AddEvidence(voteErr.DuplicateVoteEvidence)
return err
return added, err
} else {
// Probably an invalid signature / Bad peer.
// Seems this can also err sometimes with "Unexpected step" - perhaps not from a bad peer ?
cs.Logger.Error("Error attempting to add vote", "err", err)
return ErrAddingVote
return added, ErrAddingVote
}
}
return nil
return added, nil
}
//-----------------------------------------------------------------------------
@@ -1526,7 +1498,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
// A precommit for the previous height?
// These come in while we wait timeoutCommit
if vote.Height+1 == cs.Height {
if !(cs.Step == cstypes.RoundStepNewHeight && vote.Type == types.VoteTypePrecommit) {
if !(cs.Step == cstypes.RoundStepNewHeight && vote.Type == types.PrecommitType) {
// TODO: give the reason ..
// fmt.Errorf("tryAddVote: Wrong height, not a LastCommit straggler commit.")
return added, ErrVoteHeightMismatch
@@ -1569,7 +1541,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
cs.evsw.FireEvent(types.EventVote, vote)
switch vote.Type {
case types.VoteTypePrevote:
case types.PrevoteType:
prevotes := cs.Votes.Prevotes(vote.Round)
cs.Logger.Info("Added to prevote", "vote", vote, "prevotes", prevotes.StringShort())
@@ -1588,7 +1560,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
!cs.LockedBlock.HashesTo(blockID.Hash) {
cs.Logger.Info("Unlocking because of POL.", "lockedRound", cs.LockedRound, "POLRound", vote.Round)
cs.LockedRound = 0
cs.LockedRound = -1
cs.LockedBlock = nil
cs.LockedBlockParts = nil
cs.eventBus.PublishEventUnlock(cs.RoundStateEvent())
@@ -1596,27 +1568,38 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
// Update Valid* if we can.
// NOTE: our proposal block may be nil or not what received a polka..
// TODO: we may want to still update the ValidBlock and obtain it via gossipping
if !blockID.IsZero() &&
(cs.ValidRound < vote.Round) &&
(vote.Round <= cs.Round) &&
cs.ProposalBlock.HashesTo(blockID.Hash) {
if len(blockID.Hash) != 0 && (cs.ValidRound < vote.Round) && (vote.Round == cs.Round) {
cs.Logger.Info("Updating ValidBlock because of POL.", "validRound", cs.ValidRound, "POLRound", vote.Round)
cs.ValidRound = vote.Round
cs.ValidBlock = cs.ProposalBlock
cs.ValidBlockParts = cs.ProposalBlockParts
if cs.ProposalBlock.HashesTo(blockID.Hash) {
cs.Logger.Info(
"Updating ValidBlock because of POL.", "validRound", cs.ValidRound, "POLRound", vote.Round)
cs.ValidRound = vote.Round
cs.ValidBlock = cs.ProposalBlock
cs.ValidBlockParts = cs.ProposalBlockParts
} else {
cs.Logger.Info(
"Valid block we don't know about. Set ProposalBlock=nil",
"proposal", cs.ProposalBlock.Hash(), "blockId", blockID.Hash)
// We're getting the wrong block.
cs.ProposalBlock = nil
}
if !cs.ProposalBlockParts.HasHeader(blockID.PartsHeader) {
cs.ProposalBlockParts = types.NewPartSetFromHeader(blockID.PartsHeader)
}
cs.evsw.FireEvent(types.EventValidBlock, &cs.RoundState)
cs.eventBus.PublishEventValidBlock(cs.RoundStateEvent())
}
}
// If +2/3 prevotes for *anything* for this or future round:
if cs.Round <= vote.Round && prevotes.HasTwoThirdsAny() {
// Round-skip over to PrevoteWait or goto Precommit.
cs.enterNewRound(height, vote.Round) // if the vote is ahead of us
if prevotes.HasTwoThirdsMajority() {
// If +2/3 prevotes for *anything* for future round:
if cs.Round < vote.Round && prevotes.HasTwoThirdsAny() {
// Round-skip if there is any 2/3+ of votes ahead of us
cs.enterNewRound(height, vote.Round)
} else if cs.Round == vote.Round && cstypes.RoundStepPrevote <= cs.Step { // current round
blockID, ok := prevotes.TwoThirdsMajority()
if ok && (cs.isProposalComplete() || len(blockID.Hash) == 0) {
cs.enterPrecommit(height, vote.Round)
} else {
cs.enterPrevote(height, vote.Round) // if the vote is ahead of us
} else if prevotes.HasTwoThirdsAny() {
cs.enterPrevoteWait(height, vote.Round)
}
} else if cs.Proposal != nil && 0 <= cs.Proposal.POLRound && cs.Proposal.POLRound == vote.Round {
@@ -1626,31 +1609,28 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
}
}
case types.VoteTypePrecommit:
case types.PrecommitType:
precommits := cs.Votes.Precommits(vote.Round)
cs.Logger.Info("Added to precommit", "vote", vote, "precommits", precommits.StringShort())
blockID, ok := precommits.TwoThirdsMajority()
if ok {
if len(blockID.Hash) == 0 {
cs.enterNewRound(height, vote.Round+1)
} else {
cs.enterNewRound(height, vote.Round)
cs.enterPrecommit(height, vote.Round)
// Executed as TwoThirdsMajority could be from a higher round
cs.enterNewRound(height, vote.Round)
cs.enterPrecommit(height, vote.Round)
if len(blockID.Hash) != 0 {
cs.enterCommit(height, vote.Round)
if cs.config.SkipTimeoutCommit && precommits.HasAll() {
// if we have all the votes now,
// go straight to new round (skip timeout commit)
// cs.scheduleTimeout(time.Duration(0), cs.Height, 0, cstypes.RoundStepNewHeight)
cs.enterNewRound(cs.Height, 0)
}
} else {
cs.enterPrecommitWait(height, vote.Round)
}
} else if cs.Round <= vote.Round && precommits.HasTwoThirdsAny() {
cs.enterNewRound(height, vote.Round)
cs.enterPrecommit(height, vote.Round)
cs.enterPrecommitWait(height, vote.Round)
}
default:
panic(fmt.Sprintf("Unexpected vote type %X", vote.Type)) // go-wire should prevent this.
}
@@ -1658,7 +1638,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
return
}
func (cs *ConsensusState) signVote(type_ byte, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
func (cs *ConsensusState) signVote(type_ types.SignedMsgType, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
addr := cs.privValidator.GetAddress()
valIndex, _ := cs.Validators.GetByAddress(addr)
@@ -1693,7 +1673,7 @@ func (cs *ConsensusState) voteTime() time.Time {
}
// sign the vote and publish on internalMsgQueue
func (cs *ConsensusState) signAddVote(type_ byte, hash []byte, header types.PartSetHeader) *types.Vote {
func (cs *ConsensusState) signAddVote(type_ types.SignedMsgType, hash []byte, header types.PartSetHeader) *types.Vote {
// if we don't have a key or we're not in the validator set, do nothing
if cs.privValidator == nil || !cs.Validators.HasAddress(cs.privValidator.GetAddress()) {
return nil

File diff suppressed because it is too large Load Diff

View File

@@ -99,8 +99,8 @@ func (hvs *HeightVoteSet) addRound(round int) {
cmn.PanicSanity("addRound() for an existing round")
}
// log.Debug("addRound(round)", "round", round)
prevotes := types.NewVoteSet(hvs.chainID, hvs.height, round, types.VoteTypePrevote, hvs.valSet)
precommits := types.NewVoteSet(hvs.chainID, hvs.height, round, types.VoteTypePrecommit, hvs.valSet)
prevotes := types.NewVoteSet(hvs.chainID, hvs.height, round, types.PrevoteType, hvs.valSet)
precommits := types.NewVoteSet(hvs.chainID, hvs.height, round, types.PrecommitType, hvs.valSet)
hvs.roundVoteSets[round] = RoundVoteSet{
Prevotes: prevotes,
Precommits: precommits,
@@ -134,13 +134,13 @@ func (hvs *HeightVoteSet) AddVote(vote *types.Vote, peerID p2p.ID) (added bool,
func (hvs *HeightVoteSet) Prevotes(round int) *types.VoteSet {
hvs.mtx.Lock()
defer hvs.mtx.Unlock()
return hvs.getVoteSet(round, types.VoteTypePrevote)
return hvs.getVoteSet(round, types.PrevoteType)
}
func (hvs *HeightVoteSet) Precommits(round int) *types.VoteSet {
hvs.mtx.Lock()
defer hvs.mtx.Unlock()
return hvs.getVoteSet(round, types.VoteTypePrecommit)
return hvs.getVoteSet(round, types.PrecommitType)
}
// Last round and blockID that has +2/3 prevotes for a particular block or nil.
@@ -149,7 +149,7 @@ func (hvs *HeightVoteSet) POLInfo() (polRound int, polBlockID types.BlockID) {
hvs.mtx.Lock()
defer hvs.mtx.Unlock()
for r := hvs.round; r >= 0; r-- {
rvs := hvs.getVoteSet(r, types.VoteTypePrevote)
rvs := hvs.getVoteSet(r, types.PrevoteType)
polBlockID, ok := rvs.TwoThirdsMajority()
if ok {
return r, polBlockID
@@ -158,15 +158,15 @@ func (hvs *HeightVoteSet) POLInfo() (polRound int, polBlockID types.BlockID) {
return -1, types.BlockID{}
}
func (hvs *HeightVoteSet) getVoteSet(round int, type_ byte) *types.VoteSet {
func (hvs *HeightVoteSet) getVoteSet(round int, type_ types.SignedMsgType) *types.VoteSet {
rvs, ok := hvs.roundVoteSets[round]
if !ok {
return nil
}
switch type_ {
case types.VoteTypePrevote:
case types.PrevoteType:
return rvs.Prevotes
case types.VoteTypePrecommit:
case types.PrecommitType:
return rvs.Precommits
default:
cmn.PanicSanity(fmt.Sprintf("Unexpected vote type %X", type_))
@@ -178,7 +178,7 @@ func (hvs *HeightVoteSet) getVoteSet(round int, type_ byte) *types.VoteSet {
// NOTE: if there are too many peers, or too much peer churn,
// this can cause memory issues.
// TODO: implement ability to remove peers too
func (hvs *HeightVoteSet) SetPeerMaj23(round int, type_ byte, peerID p2p.ID, blockID types.BlockID) error {
func (hvs *HeightVoteSet) SetPeerMaj23(round int, type_ types.SignedMsgType, peerID p2p.ID, blockID types.BlockID) error {
hvs.mtx.Lock()
defer hvs.mtx.Unlock()
if !types.IsVoteTypeValid(type_) {

View File

@@ -56,7 +56,7 @@ func makeVoteHR(t *testing.T, height int64, round int, privVals []types.PrivVali
Height: height,
Round: round,
Timestamp: tmtime.Now(),
Type: types.VoteTypePrecommit,
Type: types.PrecommitType,
BlockID: types.BlockID{[]byte("fakehash"), types.PartSetHeader{}},
}
chainID := config.ChainID()

View File

@@ -55,3 +55,31 @@ func (prs PeerRoundState) StringIndented(indent string) string {
indent, prs.CatchupCommit, prs.CatchupCommitRound,
indent)
}
//-----------------------------------------------------------
// These methods are for Protobuf Compatibility
// Size returns the size of the amino encoding, in bytes.
func (ps *PeerRoundState) Size() int {
bs, _ := ps.Marshal()
return len(bs)
}
// Marshal returns the amino encoding.
func (ps *PeerRoundState) Marshal() ([]byte, error) {
return cdc.MarshalBinaryBare(ps)
}
// MarshalTo calls Marshal and copies to the given buffer.
func (ps *PeerRoundState) MarshalTo(data []byte) (int, error) {
bs, err := ps.Marshal()
if err != nil {
return -1, err
}
return copy(data, bs), nil
}
// Unmarshal deserializes from amino encoded form.
func (ps *PeerRoundState) Unmarshal(bs []byte) error {
return cdc.UnmarshalBinaryBare(bs, ps)
}

View File

@@ -26,8 +26,15 @@ const (
RoundStepPrecommitWait = RoundStepType(0x07) // Did receive any +2/3 precommits, start timeout
RoundStepCommit = RoundStepType(0x08) // Entered commit state machine
// NOTE: RoundStepNewHeight acts as RoundStepCommitWait.
// NOTE: Update IsValid method if you change this!
)
// IsValid returns true if the step is valid, false if unknown/undefined.
func (rs RoundStepType) IsValid() bool {
return uint8(rs) >= 0x01 && uint8(rs) <= 0x08
}
// String returns a string
func (rs RoundStepType) String() string {
switch rs {
@@ -105,18 +112,50 @@ func (rs *RoundState) RoundStateSimple() RoundStateSimple {
}
}
// NewRoundEvent returns the RoundState with proposer information as an event.
func (rs *RoundState) NewRoundEvent() types.EventDataNewRound {
addr := rs.Validators.GetProposer().Address
idx, _ := rs.Validators.GetByAddress(addr)
return types.EventDataNewRound{
Height: rs.Height,
Round: rs.Round,
Step: rs.Step.String(),
Proposer: types.ValidatorInfo{
Address: addr,
Index: idx,
},
}
}
// CompleteProposalEvent returns information about a proposed block as an event.
func (rs *RoundState) CompleteProposalEvent() types.EventDataCompleteProposal {
// We must construct BlockID from ProposalBlock and ProposalBlockParts
// cs.Proposal is not guaranteed to be set when this function is called
blockId := types.BlockID{
Hash: rs.ProposalBlock.Hash(),
PartsHeader: rs.ProposalBlockParts.Header(),
}
return types.EventDataCompleteProposal{
Height: rs.Height,
Round: rs.Round,
Step: rs.Step.String(),
BlockID: blockId,
}
}
// RoundStateEvent returns the H/R/S of the RoundState as an event.
func (rs *RoundState) RoundStateEvent() types.EventDataRoundState {
// XXX: copy the RoundState
// if we want to avoid this, we may need synchronous events after all
// copy the RoundState.
// TODO: if we want to avoid this, we may need synchronous events after all
rsCopy := *rs
edrs := types.EventDataRoundState{
return types.EventDataRoundState{
Height: rs.Height,
Round: rs.Round,
Step: rs.Step.String(),
RoundState: &rsCopy,
}
return edrs
}
// String returns a string
@@ -162,3 +201,31 @@ func (rs *RoundState) StringShort() string {
return fmt.Sprintf(`RoundState{H:%v R:%v S:%v ST:%v}`,
rs.Height, rs.Round, rs.Step, rs.StartTime)
}
//-----------------------------------------------------------
// These methods are for Protobuf Compatibility
// Size returns the size of the amino encoding, in bytes.
func (rs *RoundStateSimple) Size() int {
bs, _ := rs.Marshal()
return len(bs)
}
// Marshal returns the amino encoding.
func (rs *RoundStateSimple) Marshal() ([]byte, error) {
return cdc.MarshalBinaryBare(rs)
}
// MarshalTo calls Marshal and copies to the given buffer.
func (rs *RoundStateSimple) MarshalTo(data []byte) (int, error) {
bs, err := rs.Marshal()
if err != nil {
return -1, err
}
return copy(data, bs), nil
}
// Unmarshal deserializes from amino encoded form.
func (rs *RoundStateSimple) Unmarshal(bs []byte) error {
return cdc.UnmarshalBinaryBare(bs, rs)
}

View File

@@ -63,11 +63,8 @@ func BenchmarkRoundStateDeepCopy(b *testing.B) {
// Random Proposal
proposal := &types.Proposal{
Timestamp: tmtime.Now(),
BlockPartsHeader: types.PartSetHeader{
Hash: cmn.RandBytes(20),
},
POLBlockID: blockID,
Signature: sig,
BlockID: blockID,
Signature: sig,
}
// Random HeightVoteSet
// TODO: hvs :=

View File

@@ -1,7 +1,7 @@
package types
import (
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/tendermint/types"
)

View File

@@ -1,11 +0,0 @@
package consensus
import "fmt"
// kind of arbitrary
var Spec = "1" // async
var Major = "0" //
var Minor = "2" // replay refactor
var Revision = "2" // validation -> commit
var Version = fmt.Sprintf("v%s/%s.%s.%s", Spec, Major, Minor, Revision)

View File

@@ -13,12 +13,13 @@ import (
amino "github.com/tendermint/go-amino"
auto "github.com/tendermint/tendermint/libs/autofile"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/libs/log"
"github.com/tendermint/tendermint/types"
tmtime "github.com/tendermint/tendermint/types/time"
)
const (
// must be greater than params.BlockGossip.BlockPartSizeBytes + a few bytes
// must be greater than types.BlockPartSizeBytes + a few bytes
maxMsgSizeBytes = 1024 * 1024 // 1MB
)
@@ -73,13 +74,13 @@ type baseWAL struct {
enc *WALEncoder
}
func NewWAL(walFile string) (*baseWAL, error) {
func NewWAL(walFile string, groupOptions ...func(*auto.Group)) (*baseWAL, error) {
err := cmn.EnsureDir(filepath.Dir(walFile), 0700)
if err != nil {
return nil, errors.Wrap(err, "failed to ensure WAL directory is in place")
}
group, err := auto.OpenGroup(walFile)
group, err := auto.OpenGroup(walFile, groupOptions...)
if err != nil {
return nil, err
}
@@ -95,6 +96,11 @@ func (wal *baseWAL) Group() *auto.Group {
return wal.group
}
func (wal *baseWAL) SetLogger(l log.Logger) {
wal.BaseService.Logger = l
wal.group.SetLogger(l)
}
func (wal *baseWAL) OnStart() error {
size, err := wal.group.Head.Size()
if err != nil {

View File

@@ -4,6 +4,7 @@ import (
"bufio"
"bytes"
"fmt"
"io"
"os"
"path/filepath"
"strings"
@@ -23,12 +24,11 @@ import (
"github.com/tendermint/tendermint/types"
)
// WALWithNBlocks generates a consensus WAL. It does this by spining up a
// WALGenerateNBlocks generates a consensus WAL. It does this by spining up a
// stripped down version of node (proxy app, event bus, consensus state) with a
// persistent kvstore application and special consensus wal instance
// (byteBufferWAL) and waits until numBlocks are created. Then it returns a WAL
// content. If the node fails to produce given numBlocks, it returns an error.
func WALWithNBlocks(numBlocks int) (data []byte, err error) {
// (byteBufferWAL) and waits until numBlocks are created. If the node fails to produce given numBlocks, it returns an error.
func WALGenerateNBlocks(wr io.Writer, numBlocks int) (err error) {
config := getConfig()
app := kvstore.NewPersistentKVStoreApplication(filepath.Join(config.DBDir(), "wal_generator"))
@@ -38,31 +38,33 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
/////////////////////////////////////////////////////////////////////////////
// COPY PASTE FROM node.go WITH A FEW MODIFICATIONS
// NOTE: we can't import node package because of circular dependency
// NOTE: we can't import node package because of circular dependency.
// NOTE: we don't do handshake so need to set state.Version.Consensus.App directly.
privValidatorFile := config.PrivValidatorFile()
privValidator := privval.LoadOrGenFilePV(privValidatorFile)
genDoc, err := types.GenesisDocFromFile(config.GenesisFile())
if err != nil {
return nil, errors.Wrap(err, "failed to read genesis file")
return errors.Wrap(err, "failed to read genesis file")
}
stateDB := db.NewMemDB()
blockStoreDB := db.NewMemDB()
state, err := sm.MakeGenesisState(genDoc)
if err != nil {
return nil, errors.Wrap(err, "failed to make genesis state")
return errors.Wrap(err, "failed to make genesis state")
}
state.Version.Consensus.App = kvstore.ProtocolVersion
blockStore := bc.NewBlockStore(blockStoreDB)
handshaker := NewHandshaker(stateDB, state, blockStore, genDoc)
proxyApp := proxy.NewAppConns(proxy.NewLocalClientCreator(app), handshaker)
proxyApp := proxy.NewAppConns(proxy.NewLocalClientCreator(app))
proxyApp.SetLogger(logger.With("module", "proxy"))
if err := proxyApp.Start(); err != nil {
return nil, errors.Wrap(err, "failed to start proxy app connections")
return errors.Wrap(err, "failed to start proxy app connections")
}
defer proxyApp.Stop()
eventBus := types.NewEventBus()
eventBus.SetLogger(logger.With("module", "events"))
if err := eventBus.Start(); err != nil {
return nil, errors.Wrap(err, "failed to start event bus")
return errors.Wrap(err, "failed to start event bus")
}
defer eventBus.Stop()
mempool := sm.MockMempool{}
@@ -78,8 +80,6 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
/////////////////////////////////////////////////////////////////////////////
// set consensus wal to buffered WAL, which will write all incoming msgs to buffer
var b bytes.Buffer
wr := bufio.NewWriter(&b)
numBlocksWritten := make(chan struct{})
wal := newByteBufferWAL(logger, NewWALEncoder(wr), int64(numBlocks), numBlocksWritten)
// see wal.go#103
@@ -87,20 +87,32 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
consensusState.wal = wal
if err := consensusState.Start(); err != nil {
return nil, errors.Wrap(err, "failed to start consensus state")
return errors.Wrap(err, "failed to start consensus state")
}
select {
case <-numBlocksWritten:
consensusState.Stop()
wr.Flush()
return b.Bytes(), nil
return nil
case <-time.After(1 * time.Minute):
consensusState.Stop()
return []byte{}, fmt.Errorf("waited too long for tendermint to produce %d blocks (grep logs for `wal_generator`)", numBlocks)
return fmt.Errorf("waited too long for tendermint to produce %d blocks (grep logs for `wal_generator`)", numBlocks)
}
}
//WALWithNBlocks returns a WAL content with numBlocks.
func WALWithNBlocks(numBlocks int) (data []byte, err error) {
var b bytes.Buffer
wr := bufio.NewWriter(&b)
if err := WALGenerateNBlocks(wr, numBlocks); err != nil {
return []byte{}, err
}
wr.Flush()
return b.Bytes(), nil
}
// f**ing long, but unique for each test
func makePathname() string {
// get path

View File

@@ -4,11 +4,16 @@ import (
"bytes"
"crypto/rand"
"fmt"
"io/ioutil"
"os"
"path/filepath"
// "sync"
"testing"
"time"
"github.com/tendermint/tendermint/consensus/types"
"github.com/tendermint/tendermint/libs/autofile"
"github.com/tendermint/tendermint/libs/log"
tmtypes "github.com/tendermint/tendermint/types"
tmtime "github.com/tendermint/tendermint/types/time"
@@ -16,6 +21,49 @@ import (
"github.com/stretchr/testify/require"
)
func TestWALTruncate(t *testing.T) {
walDir, err := ioutil.TempDir("", "wal")
require.NoError(t, err)
defer os.RemoveAll(walDir)
walFile := filepath.Join(walDir, "wal")
//this magic number 4K can truncate the content when RotateFile. defaultHeadSizeLimit(10M) is hard to simulate.
//this magic number 1 * time.Millisecond make RotateFile check frequently. defaultGroupCheckDuration(5s) is hard to simulate.
wal, err := NewWAL(walFile,
autofile.GroupHeadSizeLimit(4096),
autofile.GroupCheckDuration(1*time.Millisecond),
)
require.NoError(t, err)
wal.SetLogger(log.TestingLogger())
err = wal.Start()
require.NoError(t, err)
defer wal.Stop()
//60 block's size nearly 70K, greater than group's headBuf size(4096 * 10), when headBuf is full, truncate content will Flush to the file.
//at this time, RotateFile is called, truncate content exist in each file.
err = WALGenerateNBlocks(wal.Group(), 60)
require.NoError(t, err)
time.Sleep(1 * time.Millisecond) //wait groupCheckDuration, make sure RotateFile run
wal.Group().Flush()
h := int64(50)
gr, found, err := wal.SearchForEndHeight(h, &WALSearchOptions{})
assert.NoError(t, err, fmt.Sprintf("expected not to err on height %d", h))
assert.True(t, found, fmt.Sprintf("expected to find end height for %d", h))
assert.NotNil(t, gr, "expected group not to be nil")
defer gr.Close()
dec := NewWALDecoder(gr)
msg, err := dec.Decode()
assert.NoError(t, err, "expected to decode a message")
rs, ok := msg.Msg.(tmtypes.EventDataRoundState)
assert.True(t, ok, "expected message of type EventDataRoundState")
assert.Equal(t, rs.Height, h+1, fmt.Sprintf("wrong height"))
}
func TestWALEncoderDecoder(t *testing.T) {
now := tmtime.Now()
msgs := []TimedWALMessage{
@@ -49,9 +97,8 @@ func TestWALSearchForEndHeight(t *testing.T) {
walFile := tempWALWithData(walBody)
wal, err := NewWAL(walFile)
if err != nil {
t.Fatal(err)
}
require.NoError(t, err)
wal.SetLogger(log.TestingLogger())
h := int64(3)
gr, found, err := wal.SearchForEndHeight(h, &WALSearchOptions{})

View File

@@ -5,7 +5,7 @@ import (
"fmt"
"io/ioutil"
"golang.org/x/crypto/openpgp/armor"
"golang.org/x/crypto/openpgp/armor" // forked to github.com/tendermint/crypto
)
func EncodeArmor(blockType string, headers map[string]string, data []byte) string {

View File

@@ -1,21 +1,24 @@
package crypto
import (
"github.com/tendermint/tendermint/crypto/tmhash"
cmn "github.com/tendermint/tendermint/libs/common"
)
type PrivKey interface {
Bytes() []byte
Sign(msg []byte) ([]byte, error)
PubKey() PubKey
Equals(PrivKey) bool
}
const (
// AddressSize is the size of a pubkey address.
AddressSize = tmhash.TruncatedSize
)
// An address is a []byte, but hex-encoded even in JSON.
// []byte leaves us the option to change the address length.
// Use an alias so Unmarshal methods (with ptr receivers) are available too.
type Address = cmn.HexBytes
func AddressHash(bz []byte) Address {
return Address(tmhash.SumTruncated(bz))
}
type PubKey interface {
Address() Address
Bytes() []byte
@@ -23,6 +26,13 @@ type PubKey interface {
Equals(PubKey) bool
}
type PrivKey interface {
Bytes() []byte
Sign(msg []byte) ([]byte, error)
PubKey() PubKey
Equals(PrivKey) bool
}
type Symmetric interface {
Keygen() []byte
Encrypt(plaintext []byte, secret []byte) (ciphertext []byte)

View File

@@ -6,9 +6,9 @@ import (
"fmt"
"io"
"github.com/tendermint/ed25519"
"github.com/tendermint/ed25519/extra25519"
amino "github.com/tendermint/go-amino"
"golang.org/x/crypto/ed25519" // forked to github.com/tendermint/crypto
"github.com/tendermint/tendermint/crypto"
"github.com/tendermint/tendermint/crypto/tmhash"
)
@@ -46,9 +46,14 @@ func (privKey PrivKeyEd25519) Bytes() []byte {
}
// Sign produces a signature on the provided message.
// This assumes the privkey is wellformed in the golang format.
// The first 32 bytes should be random,
// corresponding to the normal ed25519 private key.
// The latter 32 bytes should be the compressed public key.
// If these conditions aren't met, Sign will panic or produce an
// incorrect signature.
func (privKey PrivKeyEd25519) Sign(msg []byte) ([]byte, error) {
privKeyBytes := [64]byte(privKey)
signatureBytes := ed25519.Sign(&privKeyBytes, msg)
signatureBytes := ed25519.Sign(privKey[:], msg)
return signatureBytes[:], nil
}
@@ -65,14 +70,14 @@ func (privKey PrivKeyEd25519) PubKey() crypto.PubKey {
break
}
}
if initialized {
var pubkeyBytes [PubKeyEd25519Size]byte
copy(pubkeyBytes[:], privKeyBytes[32:])
return PubKeyEd25519(pubkeyBytes)
if !initialized {
panic("Expected PrivKeyEd25519 to include concatenated pubkey bytes")
}
pubBytes := *ed25519.MakePublicKey(&privKeyBytes)
return PubKeyEd25519(pubBytes)
var pubkeyBytes [PubKeyEd25519Size]byte
copy(pubkeyBytes[:], privKeyBytes[32:])
return PubKeyEd25519(pubkeyBytes)
}
// Equals - you probably don't need to use this.
@@ -85,17 +90,6 @@ func (privKey PrivKeyEd25519) Equals(other crypto.PrivKey) bool {
}
}
// ToCurve25519 takes a private key and returns its representation on
// Curve25519. Curve25519 is birationally equivalent to Edwards25519,
// which Ed25519 uses internally. This method is intended for use in
// an X25519 Diffie Hellman key exchange.
func (privKey PrivKeyEd25519) ToCurve25519() *[PubKeyEd25519Size]byte {
keyCurve25519 := new([32]byte)
privKeyBytes := [64]byte(privKey)
extra25519.PrivateKeyToCurve25519(keyCurve25519, &privKeyBytes)
return keyCurve25519
}
// GenPrivKey generates a new ed25519 private key.
// It uses OS randomness in conjunction with the current global random seed
// in tendermint/libs/common to generate the private key.
@@ -105,16 +99,16 @@ func GenPrivKey() PrivKeyEd25519 {
// genPrivKey generates a new ed25519 private key using the provided reader.
func genPrivKey(rand io.Reader) PrivKeyEd25519 {
privKey := new([64]byte)
_, err := io.ReadFull(rand, privKey[:32])
seed := make([]byte, 32)
_, err := io.ReadFull(rand, seed[:])
if err != nil {
panic(err)
}
// ed25519.MakePublicKey(privKey) alters the last 32 bytes of privKey.
// It places the pubkey in the last 32 bytes of privKey, and returns the
// public key.
ed25519.MakePublicKey(privKey)
return PrivKeyEd25519(*privKey)
privKey := ed25519.NewKeyFromSeed(seed)
var privKeyEd PrivKeyEd25519
copy(privKeyEd[:], privKey)
return privKeyEd
}
// GenPrivKeyFromSecret hashes the secret with SHA2, and uses
@@ -122,14 +116,12 @@ func genPrivKey(rand io.Reader) PrivKeyEd25519 {
// NOTE: secret should be the output of a KDF like bcrypt,
// if it's derived from user input.
func GenPrivKeyFromSecret(secret []byte) PrivKeyEd25519 {
privKey32 := crypto.Sha256(secret) // Not Ripemd160 because we want 32 bytes.
privKey := new([64]byte)
copy(privKey[:32], privKey32)
// ed25519.MakePublicKey(privKey) alters the last 32 bytes of privKey.
// It places the pubkey in the last 32 bytes of privKey, and returns the
// public key.
ed25519.MakePublicKey(privKey)
return PrivKeyEd25519(*privKey)
seed := crypto.Sha256(secret) // Not Ripemd160 because we want 32 bytes.
privKey := ed25519.NewKeyFromSeed(seed)
var privKeyEd PrivKeyEd25519
copy(privKeyEd[:], privKey)
return privKeyEd
}
//-------------------------------------
@@ -144,7 +136,7 @@ type PubKeyEd25519 [PubKeyEd25519Size]byte
// Address is the SHA256-20 of the raw pubkey bytes.
func (pubKey PubKeyEd25519) Address() crypto.Address {
return crypto.Address(tmhash.Sum(pubKey[:]))
return crypto.Address(tmhash.SumTruncated(pubKey[:]))
}
// Bytes marshals the PubKey using amino encoding.
@@ -156,30 +148,12 @@ func (pubKey PubKeyEd25519) Bytes() []byte {
return bz
}
func (pubKey PubKeyEd25519) VerifyBytes(msg []byte, sig_ []byte) bool {
func (pubKey PubKeyEd25519) VerifyBytes(msg []byte, sig []byte) bool {
// make sure we use the same algorithm to sign
if len(sig_) != SignatureSize {
if len(sig) != SignatureSize {
return false
}
sig := new([SignatureSize]byte)
copy(sig[:], sig_)
pubKeyBytes := [PubKeyEd25519Size]byte(pubKey)
return ed25519.Verify(&pubKeyBytes, msg, sig)
}
// ToCurve25519 takes a public key and returns its representation on
// Curve25519. Curve25519 is birationally equivalent to Edwards25519,
// which Ed25519 uses internally. This method is intended for use in
// an X25519 Diffie Hellman key exchange.
//
// If there is an error, then this function returns nil.
func (pubKey PubKeyEd25519) ToCurve25519() *[PubKeyEd25519Size]byte {
keyCurve25519, pubKeyBytes := new([PubKeyEd25519Size]byte), [PubKeyEd25519Size]byte(pubKey)
ok := extra25519.PublicKeyToCurve25519(keyCurve25519, &pubKeyBytes)
if !ok {
return nil
}
return keyCurve25519
return ed25519.Verify(pubKey[:], msg, sig)
}
func (pubKey PubKeyEd25519) String() string {

View File

@@ -1,8 +1,9 @@
package cryptoAmino
import (
amino "github.com/tendermint/go-amino"
"reflect"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/tendermint/crypto"
"github.com/tendermint/tendermint/crypto/ed25519"
"github.com/tendermint/tendermint/crypto/multisig"
@@ -11,6 +12,12 @@ import (
var cdc = amino.NewCodec()
// routeTable is used to map public key concrete types back
// to their amino routes. This should eventually be handled
// by amino. Example usage:
// routeTable[reflect.TypeOf(ed25519.PubKeyEd25519{})] = ed25519.PubKeyAminoRoute
var routeTable = make(map[reflect.Type]string, 3)
func init() {
// NOTE: It's important that there be no conflicts here,
// as that would change the canonical representations,
@@ -19,6 +26,20 @@ func init() {
// https://github.com/tendermint/go-amino/issues/9
// is resolved
RegisterAmino(cdc)
// TODO: Have amino provide a way to go from concrete struct to route directly.
// Its currently a private API
routeTable[reflect.TypeOf(ed25519.PubKeyEd25519{})] = ed25519.PubKeyAminoRoute
routeTable[reflect.TypeOf(secp256k1.PubKeySecp256k1{})] = secp256k1.PubKeyAminoRoute
routeTable[reflect.TypeOf(&multisig.PubKeyMultisigThreshold{})] = multisig.PubKeyMultisigThresholdAminoRoute
}
// PubkeyAminoRoute returns the amino route of a pubkey
// cdc is currently passed in, as eventually this will not be using
// a package level codec.
func PubkeyAminoRoute(cdc *amino.Codec, key crypto.PubKey) (string, bool) {
route, found := routeTable[reflect.TypeOf(key)]
return route, found
}
// RegisterAmino registers all crypto related types in the given (amino) codec.

View File

@@ -8,6 +8,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/tendermint/tendermint/crypto"
"github.com/tendermint/tendermint/crypto/ed25519"
"github.com/tendermint/tendermint/crypto/multisig"
"github.com/tendermint/tendermint/crypto/secp256k1"
)
@@ -119,11 +120,29 @@ func TestNilEncodings(t *testing.T) {
var e, f crypto.PrivKey
checkAminoJSON(t, &e, &f, true)
assert.EqualValues(t, e, f)
}
func TestPubKeyInvalidDataProperReturnsEmpty(t *testing.T) {
pk, err := PubKeyFromBytes([]byte("foo"))
require.NotNil(t, err, "expecting a non-nil error")
require.Nil(t, pk, "expecting an empty public key on error")
require.NotNil(t, err)
require.Nil(t, pk)
}
func TestPubkeyAminoRoute(t *testing.T) {
tests := []struct {
key crypto.PubKey
want string
found bool
}{
{ed25519.PubKeyEd25519{}, ed25519.PubKeyAminoRoute, true},
{secp256k1.PubKeySecp256k1{}, secp256k1.PubKeyAminoRoute, true},
{&multisig.PubKeyMultisigThreshold{}, multisig.PubKeyMultisigThresholdAminoRoute, true},
}
for i, tc := range tests {
got, found := PubkeyAminoRoute(cdc, tc.key)
require.Equal(t, tc.found, found, "not equal on tc %d", i)
if tc.found {
require.Equal(t, tc.want, got, "not equal on tc %d", i)
}
}
}

View File

@@ -3,7 +3,7 @@ package crypto
import (
"crypto/sha256"
"golang.org/x/crypto/ripemd160"
"golang.org/x/crypto/ripemd160" // forked to github.com/tendermint/crypto
)
func Sha256(bytes []byte) []byte {

6
crypto/merkle/compile.sh Normal file
View File

@@ -0,0 +1,6 @@
#! /bin/bash
protoc --gogo_out=. -I $GOPATH/src/ -I . -I $GOPATH/src/github.com/gogo/protobuf/protobuf merkle.proto
echo "--> adding nolint declarations to protobuf generated files"
awk '/package merkle/ { print "//nolint: gas"; print; next }1' merkle.pb.go > merkle.pb.go.new
mv merkle.pb.go.new merkle.pb.go

792
crypto/merkle/merkle.pb.go Normal file
View File

@@ -0,0 +1,792 @@
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: crypto/merkle/merkle.proto
package merkle
import proto "github.com/gogo/protobuf/proto"
import fmt "fmt"
import math "math"
import _ "github.com/gogo/protobuf/gogoproto"
import bytes "bytes"
import io "io"
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion2 // please upgrade the proto package
// ProofOp defines an operation used for calculating Merkle root
// The data could be arbitrary format, providing nessecary data
// for example neighbouring node hash
type ProofOp struct {
Type string `protobuf:"bytes,1,opt,name=type,proto3" json:"type,omitempty"`
Key []byte `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty"`
Data []byte `protobuf:"bytes,3,opt,name=data,proto3" json:"data,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *ProofOp) Reset() { *m = ProofOp{} }
func (m *ProofOp) String() string { return proto.CompactTextString(m) }
func (*ProofOp) ProtoMessage() {}
func (*ProofOp) Descriptor() ([]byte, []int) {
return fileDescriptor_merkle_5d3f6051907285da, []int{0}
}
func (m *ProofOp) XXX_Unmarshal(b []byte) error {
return m.Unmarshal(b)
}
func (m *ProofOp) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
if deterministic {
return xxx_messageInfo_ProofOp.Marshal(b, m, deterministic)
} else {
b = b[:cap(b)]
n, err := m.MarshalTo(b)
if err != nil {
return nil, err
}
return b[:n], nil
}
}
func (dst *ProofOp) XXX_Merge(src proto.Message) {
xxx_messageInfo_ProofOp.Merge(dst, src)
}
func (m *ProofOp) XXX_Size() int {
return m.Size()
}
func (m *ProofOp) XXX_DiscardUnknown() {
xxx_messageInfo_ProofOp.DiscardUnknown(m)
}
var xxx_messageInfo_ProofOp proto.InternalMessageInfo
func (m *ProofOp) GetType() string {
if m != nil {
return m.Type
}
return ""
}
func (m *ProofOp) GetKey() []byte {
if m != nil {
return m.Key
}
return nil
}
func (m *ProofOp) GetData() []byte {
if m != nil {
return m.Data
}
return nil
}
// Proof is Merkle proof defined by the list of ProofOps
type Proof struct {
Ops []ProofOp `protobuf:"bytes,1,rep,name=ops" json:"ops"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Proof) Reset() { *m = Proof{} }
func (m *Proof) String() string { return proto.CompactTextString(m) }
func (*Proof) ProtoMessage() {}
func (*Proof) Descriptor() ([]byte, []int) {
return fileDescriptor_merkle_5d3f6051907285da, []int{1}
}
func (m *Proof) XXX_Unmarshal(b []byte) error {
return m.Unmarshal(b)
}
func (m *Proof) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
if deterministic {
return xxx_messageInfo_Proof.Marshal(b, m, deterministic)
} else {
b = b[:cap(b)]
n, err := m.MarshalTo(b)
if err != nil {
return nil, err
}
return b[:n], nil
}
}
func (dst *Proof) XXX_Merge(src proto.Message) {
xxx_messageInfo_Proof.Merge(dst, src)
}
func (m *Proof) XXX_Size() int {
return m.Size()
}
func (m *Proof) XXX_DiscardUnknown() {
xxx_messageInfo_Proof.DiscardUnknown(m)
}
var xxx_messageInfo_Proof proto.InternalMessageInfo
func (m *Proof) GetOps() []ProofOp {
if m != nil {
return m.Ops
}
return nil
}
func init() {
proto.RegisterType((*ProofOp)(nil), "merkle.ProofOp")
proto.RegisterType((*Proof)(nil), "merkle.Proof")
}
func (this *ProofOp) Equal(that interface{}) bool {
if that == nil {
return this == nil
}
that1, ok := that.(*ProofOp)
if !ok {
that2, ok := that.(ProofOp)
if ok {
that1 = &that2
} else {
return false
}
}
if that1 == nil {
return this == nil
} else if this == nil {
return false
}
if this.Type != that1.Type {
return false
}
if !bytes.Equal(this.Key, that1.Key) {
return false
}
if !bytes.Equal(this.Data, that1.Data) {
return false
}
if !bytes.Equal(this.XXX_unrecognized, that1.XXX_unrecognized) {
return false
}
return true
}
func (this *Proof) Equal(that interface{}) bool {
if that == nil {
return this == nil
}
that1, ok := that.(*Proof)
if !ok {
that2, ok := that.(Proof)
if ok {
that1 = &that2
} else {
return false
}
}
if that1 == nil {
return this == nil
} else if this == nil {
return false
}
if len(this.Ops) != len(that1.Ops) {
return false
}
for i := range this.Ops {
if !this.Ops[i].Equal(&that1.Ops[i]) {
return false
}
}
if !bytes.Equal(this.XXX_unrecognized, that1.XXX_unrecognized) {
return false
}
return true
}
func (m *ProofOp) Marshal() (dAtA []byte, err error) {
size := m.Size()
dAtA = make([]byte, size)
n, err := m.MarshalTo(dAtA)
if err != nil {
return nil, err
}
return dAtA[:n], nil
}
func (m *ProofOp) MarshalTo(dAtA []byte) (int, error) {
var i int
_ = i
var l int
_ = l
if len(m.Type) > 0 {
dAtA[i] = 0xa
i++
i = encodeVarintMerkle(dAtA, i, uint64(len(m.Type)))
i += copy(dAtA[i:], m.Type)
}
if len(m.Key) > 0 {
dAtA[i] = 0x12
i++
i = encodeVarintMerkle(dAtA, i, uint64(len(m.Key)))
i += copy(dAtA[i:], m.Key)
}
if len(m.Data) > 0 {
dAtA[i] = 0x1a
i++
i = encodeVarintMerkle(dAtA, i, uint64(len(m.Data)))
i += copy(dAtA[i:], m.Data)
}
if m.XXX_unrecognized != nil {
i += copy(dAtA[i:], m.XXX_unrecognized)
}
return i, nil
}
func (m *Proof) Marshal() (dAtA []byte, err error) {
size := m.Size()
dAtA = make([]byte, size)
n, err := m.MarshalTo(dAtA)
if err != nil {
return nil, err
}
return dAtA[:n], nil
}
func (m *Proof) MarshalTo(dAtA []byte) (int, error) {
var i int
_ = i
var l int
_ = l
if len(m.Ops) > 0 {
for _, msg := range m.Ops {
dAtA[i] = 0xa
i++
i = encodeVarintMerkle(dAtA, i, uint64(msg.Size()))
n, err := msg.MarshalTo(dAtA[i:])
if err != nil {
return 0, err
}
i += n
}
}
if m.XXX_unrecognized != nil {
i += copy(dAtA[i:], m.XXX_unrecognized)
}
return i, nil
}
func encodeVarintMerkle(dAtA []byte, offset int, v uint64) int {
for v >= 1<<7 {
dAtA[offset] = uint8(v&0x7f | 0x80)
v >>= 7
offset++
}
dAtA[offset] = uint8(v)
return offset + 1
}
func NewPopulatedProofOp(r randyMerkle, easy bool) *ProofOp {
this := &ProofOp{}
this.Type = string(randStringMerkle(r))
v1 := r.Intn(100)
this.Key = make([]byte, v1)
for i := 0; i < v1; i++ {
this.Key[i] = byte(r.Intn(256))
}
v2 := r.Intn(100)
this.Data = make([]byte, v2)
for i := 0; i < v2; i++ {
this.Data[i] = byte(r.Intn(256))
}
if !easy && r.Intn(10) != 0 {
this.XXX_unrecognized = randUnrecognizedMerkle(r, 4)
}
return this
}
func NewPopulatedProof(r randyMerkle, easy bool) *Proof {
this := &Proof{}
if r.Intn(10) != 0 {
v3 := r.Intn(5)
this.Ops = make([]ProofOp, v3)
for i := 0; i < v3; i++ {
v4 := NewPopulatedProofOp(r, easy)
this.Ops[i] = *v4
}
}
if !easy && r.Intn(10) != 0 {
this.XXX_unrecognized = randUnrecognizedMerkle(r, 2)
}
return this
}
type randyMerkle interface {
Float32() float32
Float64() float64
Int63() int64
Int31() int32
Uint32() uint32
Intn(n int) int
}
func randUTF8RuneMerkle(r randyMerkle) rune {
ru := r.Intn(62)
if ru < 10 {
return rune(ru + 48)
} else if ru < 36 {
return rune(ru + 55)
}
return rune(ru + 61)
}
func randStringMerkle(r randyMerkle) string {
v5 := r.Intn(100)
tmps := make([]rune, v5)
for i := 0; i < v5; i++ {
tmps[i] = randUTF8RuneMerkle(r)
}
return string(tmps)
}
func randUnrecognizedMerkle(r randyMerkle, maxFieldNumber int) (dAtA []byte) {
l := r.Intn(5)
for i := 0; i < l; i++ {
wire := r.Intn(4)
if wire == 3 {
wire = 5
}
fieldNumber := maxFieldNumber + r.Intn(100)
dAtA = randFieldMerkle(dAtA, r, fieldNumber, wire)
}
return dAtA
}
func randFieldMerkle(dAtA []byte, r randyMerkle, fieldNumber int, wire int) []byte {
key := uint32(fieldNumber)<<3 | uint32(wire)
switch wire {
case 0:
dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
v6 := r.Int63()
if r.Intn(2) == 0 {
v6 *= -1
}
dAtA = encodeVarintPopulateMerkle(dAtA, uint64(v6))
case 1:
dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
dAtA = append(dAtA, byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)))
case 2:
dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
ll := r.Intn(100)
dAtA = encodeVarintPopulateMerkle(dAtA, uint64(ll))
for j := 0; j < ll; j++ {
dAtA = append(dAtA, byte(r.Intn(256)))
}
default:
dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
dAtA = append(dAtA, byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)))
}
return dAtA
}
func encodeVarintPopulateMerkle(dAtA []byte, v uint64) []byte {
for v >= 1<<7 {
dAtA = append(dAtA, uint8(uint64(v)&0x7f|0x80))
v >>= 7
}
dAtA = append(dAtA, uint8(v))
return dAtA
}
func (m *ProofOp) Size() (n int) {
var l int
_ = l
l = len(m.Type)
if l > 0 {
n += 1 + l + sovMerkle(uint64(l))
}
l = len(m.Key)
if l > 0 {
n += 1 + l + sovMerkle(uint64(l))
}
l = len(m.Data)
if l > 0 {
n += 1 + l + sovMerkle(uint64(l))
}
if m.XXX_unrecognized != nil {
n += len(m.XXX_unrecognized)
}
return n
}
func (m *Proof) Size() (n int) {
var l int
_ = l
if len(m.Ops) > 0 {
for _, e := range m.Ops {
l = e.Size()
n += 1 + l + sovMerkle(uint64(l))
}
}
if m.XXX_unrecognized != nil {
n += len(m.XXX_unrecognized)
}
return n
}
func sovMerkle(x uint64) (n int) {
for {
n++
x >>= 7
if x == 0 {
break
}
}
return n
}
func sozMerkle(x uint64) (n int) {
return sovMerkle(uint64((x << 1) ^ uint64((int64(x) >> 63))))
}
func (m *ProofOp) Unmarshal(dAtA []byte) error {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
preIndex := iNdEx
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowMerkle
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
fieldNum := int32(wire >> 3)
wireType := int(wire & 0x7)
if wireType == 4 {
return fmt.Errorf("proto: ProofOp: wiretype end group for non-group")
}
if fieldNum <= 0 {
return fmt.Errorf("proto: ProofOp: illegal tag %d (wire type %d)", fieldNum, wire)
}
switch fieldNum {
case 1:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Type", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowMerkle
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return ErrInvalidLengthMerkle
}
postIndex := iNdEx + intStringLen
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Type = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
case 2:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Key", wireType)
}
var byteLen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowMerkle
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
byteLen |= (int(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
if byteLen < 0 {
return ErrInvalidLengthMerkle
}
postIndex := iNdEx + byteLen
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Key = append(m.Key[:0], dAtA[iNdEx:postIndex]...)
if m.Key == nil {
m.Key = []byte{}
}
iNdEx = postIndex
case 3:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Data", wireType)
}
var byteLen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowMerkle
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
byteLen |= (int(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
if byteLen < 0 {
return ErrInvalidLengthMerkle
}
postIndex := iNdEx + byteLen
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Data = append(m.Data[:0], dAtA[iNdEx:postIndex]...)
if m.Data == nil {
m.Data = []byte{}
}
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := skipMerkle(dAtA[iNdEx:])
if err != nil {
return err
}
if skippy < 0 {
return ErrInvalidLengthMerkle
}
if (iNdEx + skippy) > l {
return io.ErrUnexpectedEOF
}
m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...)
iNdEx += skippy
}
}
if iNdEx > l {
return io.ErrUnexpectedEOF
}
return nil
}
func (m *Proof) Unmarshal(dAtA []byte) error {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
preIndex := iNdEx
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowMerkle
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
fieldNum := int32(wire >> 3)
wireType := int(wire & 0x7)
if wireType == 4 {
return fmt.Errorf("proto: Proof: wiretype end group for non-group")
}
if fieldNum <= 0 {
return fmt.Errorf("proto: Proof: illegal tag %d (wire type %d)", fieldNum, wire)
}
switch fieldNum {
case 1:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Ops", wireType)
}
var msglen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowMerkle
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
msglen |= (int(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
if msglen < 0 {
return ErrInvalidLengthMerkle
}
postIndex := iNdEx + msglen
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Ops = append(m.Ops, ProofOp{})
if err := m.Ops[len(m.Ops)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil {
return err
}
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := skipMerkle(dAtA[iNdEx:])
if err != nil {
return err
}
if skippy < 0 {
return ErrInvalidLengthMerkle
}
if (iNdEx + skippy) > l {
return io.ErrUnexpectedEOF
}
m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...)
iNdEx += skippy
}
}
if iNdEx > l {
return io.ErrUnexpectedEOF
}
return nil
}
func skipMerkle(dAtA []byte) (n int, err error) {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return 0, ErrIntOverflowMerkle
}
if iNdEx >= l {
return 0, io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
wireType := int(wire & 0x7)
switch wireType {
case 0:
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return 0, ErrIntOverflowMerkle
}
if iNdEx >= l {
return 0, io.ErrUnexpectedEOF
}
iNdEx++
if dAtA[iNdEx-1] < 0x80 {
break
}
}
return iNdEx, nil
case 1:
iNdEx += 8
return iNdEx, nil
case 2:
var length int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return 0, ErrIntOverflowMerkle
}
if iNdEx >= l {
return 0, io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
length |= (int(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
iNdEx += length
if length < 0 {
return 0, ErrInvalidLengthMerkle
}
return iNdEx, nil
case 3:
for {
var innerWire uint64
var start int = iNdEx
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return 0, ErrIntOverflowMerkle
}
if iNdEx >= l {
return 0, io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
innerWire |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
innerWireType := int(innerWire & 0x7)
if innerWireType == 4 {
break
}
next, err := skipMerkle(dAtA[start:])
if err != nil {
return 0, err
}
iNdEx = start + next
}
return iNdEx, nil
case 4:
return iNdEx, nil
case 5:
iNdEx += 4
return iNdEx, nil
default:
return 0, fmt.Errorf("proto: illegal wireType %d", wireType)
}
}
panic("unreachable")
}
var (
ErrInvalidLengthMerkle = fmt.Errorf("proto: negative length found during unmarshaling")
ErrIntOverflowMerkle = fmt.Errorf("proto: integer overflow")
)
func init() { proto.RegisterFile("crypto/merkle/merkle.proto", fileDescriptor_merkle_5d3f6051907285da) }
var fileDescriptor_merkle_5d3f6051907285da = []byte{
// 200 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x4a, 0x2e, 0xaa, 0x2c,
0x28, 0xc9, 0xd7, 0xcf, 0x4d, 0x2d, 0xca, 0xce, 0x49, 0x85, 0x52, 0x7a, 0x05, 0x45, 0xf9, 0x25,
0xf9, 0x42, 0x6c, 0x10, 0x9e, 0x94, 0x6e, 0x7a, 0x66, 0x49, 0x46, 0x69, 0x92, 0x5e, 0x72, 0x7e,
0xae, 0x7e, 0x7a, 0x7e, 0x7a, 0xbe, 0x3e, 0x58, 0x3a, 0xa9, 0x34, 0x0d, 0xcc, 0x03, 0x73, 0xc0,
0x2c, 0x88, 0x36, 0x25, 0x67, 0x2e, 0xf6, 0x80, 0xa2, 0xfc, 0xfc, 0x34, 0xff, 0x02, 0x21, 0x21,
0x2e, 0x96, 0x92, 0xca, 0x82, 0x54, 0x09, 0x46, 0x05, 0x46, 0x0d, 0xce, 0x20, 0x30, 0x5b, 0x48,
0x80, 0x8b, 0x39, 0x3b, 0xb5, 0x52, 0x82, 0x49, 0x81, 0x51, 0x83, 0x27, 0x08, 0xc4, 0x04, 0xa9,
0x4a, 0x49, 0x2c, 0x49, 0x94, 0x60, 0x06, 0x0b, 0x81, 0xd9, 0x4a, 0x06, 0x5c, 0xac, 0x60, 0x43,
0x84, 0xd4, 0xb9, 0x98, 0xf3, 0x0b, 0x8a, 0x25, 0x18, 0x15, 0x98, 0x35, 0xb8, 0x8d, 0xf8, 0xf5,
0xa0, 0x0e, 0x84, 0x5a, 0xe0, 0xc4, 0x72, 0xe2, 0x9e, 0x3c, 0x43, 0x10, 0x48, 0x85, 0x93, 0xc8,
0x8f, 0x87, 0x72, 0x8c, 0x2b, 0x1e, 0xc9, 0x31, 0x9e, 0x78, 0x24, 0xc7, 0x78, 0xe1, 0x91, 0x1c,
0xe3, 0x83, 0x47, 0x72, 0x8c, 0x49, 0x6c, 0x60, 0x37, 0x19, 0x03, 0x02, 0x00, 0x00, 0xff, 0xff,
0xb9, 0x2b, 0x0f, 0xd1, 0xe8, 0x00, 0x00, 0x00,
}

View File

@@ -0,0 +1,30 @@
syntax = "proto3";
package merkle;
// For more information on gogo.proto, see:
// https://github.com/gogo/protobuf/blob/master/extensions.md
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.sizer_all) = true;
option (gogoproto.populate_all) = true;
option (gogoproto.equal_all) = true;
//----------------------------------------
// Message types
// ProofOp defines an operation used for calculating Merkle root
// The data could be arbitrary format, providing nessecary data
// for example neighbouring node hash
message ProofOp {
string type = 1;
bytes key = 2;
bytes data = 3;
}
// Proof is Merkle proof defined by the list of ProofOps
message Proof {
repeated ProofOp ops = 1 [(gogoproto.nullable)=false];
}

138
crypto/merkle/proof.go Normal file
View File

@@ -0,0 +1,138 @@
package merkle
import (
"bytes"
cmn "github.com/tendermint/tendermint/libs/common"
)
//----------------------------------------
// ProofOp gets converted to an instance of ProofOperator:
// ProofOperator is a layer for calculating intermediate Merkle roots
// when a series of Merkle trees are chained together.
// Run() takes leaf values from a tree and returns the Merkle
// root for the corresponding tree. It takes and returns a list of bytes
// to allow multiple leaves to be part of a single proof, for instance in a range proof.
// ProofOp() encodes the ProofOperator in a generic way so it can later be
// decoded with OpDecoder.
type ProofOperator interface {
Run([][]byte) ([][]byte, error)
GetKey() []byte
ProofOp() ProofOp
}
//----------------------------------------
// Operations on a list of ProofOperators
// ProofOperators is a slice of ProofOperator(s).
// Each operator will be applied to the input value sequentially
// and the last Merkle root will be verified with already known data
type ProofOperators []ProofOperator
func (poz ProofOperators) VerifyValue(root []byte, keypath string, value []byte) (err error) {
return poz.Verify(root, keypath, [][]byte{value})
}
func (poz ProofOperators) Verify(root []byte, keypath string, args [][]byte) (err error) {
keys, err := KeyPathToKeys(keypath)
if err != nil {
return
}
for i, op := range poz {
key := op.GetKey()
if len(key) != 0 {
if len(keys) == 0 {
return cmn.NewError("Key path has insufficient # of parts: expected no more keys but got %+v", string(key))
}
lastKey := keys[len(keys)-1]
if !bytes.Equal(lastKey, key) {
return cmn.NewError("Key mismatch on operation #%d: expected %+v but got %+v", i, string(lastKey), string(key))
}
keys = keys[:len(keys)-1]
}
args, err = op.Run(args)
if err != nil {
return
}
}
if !bytes.Equal(root, args[0]) {
return cmn.NewError("Calculated root hash is invalid: expected %+v but got %+v", root, args[0])
}
if len(keys) != 0 {
return cmn.NewError("Keypath not consumed all")
}
return nil
}
//----------------------------------------
// ProofRuntime - main entrypoint
type OpDecoder func(ProofOp) (ProofOperator, error)
type ProofRuntime struct {
decoders map[string]OpDecoder
}
func NewProofRuntime() *ProofRuntime {
return &ProofRuntime{
decoders: make(map[string]OpDecoder),
}
}
func (prt *ProofRuntime) RegisterOpDecoder(typ string, dec OpDecoder) {
_, ok := prt.decoders[typ]
if ok {
panic("already registered for type " + typ)
}
prt.decoders[typ] = dec
}
func (prt *ProofRuntime) Decode(pop ProofOp) (ProofOperator, error) {
decoder := prt.decoders[pop.Type]
if decoder == nil {
return nil, cmn.NewError("unrecognized proof type %v", pop.Type)
}
return decoder(pop)
}
func (prt *ProofRuntime) DecodeProof(proof *Proof) (ProofOperators, error) {
var poz ProofOperators
for _, pop := range proof.Ops {
operator, err := prt.Decode(pop)
if err != nil {
return nil, cmn.ErrorWrap(err, "decoding a proof operator")
}
poz = append(poz, operator)
}
return poz, nil
}
func (prt *ProofRuntime) VerifyValue(proof *Proof, root []byte, keypath string, value []byte) (err error) {
return prt.Verify(proof, root, keypath, [][]byte{value})
}
// TODO In the long run we'll need a method of classifcation of ops,
// whether existence or absence or perhaps a third?
func (prt *ProofRuntime) VerifyAbsence(proof *Proof, root []byte, keypath string) (err error) {
return prt.Verify(proof, root, keypath, nil)
}
func (prt *ProofRuntime) Verify(proof *Proof, root []byte, keypath string, args [][]byte) (err error) {
poz, err := prt.DecodeProof(proof)
if err != nil {
return cmn.ErrorWrap(err, "decoding proof")
}
return poz.Verify(root, keypath, args)
}
// DefaultProofRuntime only knows about Simple value
// proofs.
// To use e.g. IAVL proofs, register op-decoders as
// defined in the IAVL package.
func DefaultProofRuntime() (prt *ProofRuntime) {
prt = NewProofRuntime()
prt.RegisterOpDecoder(ProofOpSimpleValue, SimpleValueOpDecoder)
return
}

View File

@@ -0,0 +1,111 @@
package merkle
import (
"encoding/hex"
"fmt"
"net/url"
"strings"
cmn "github.com/tendermint/tendermint/libs/common"
)
/*
For generalized Merkle proofs, each layer of the proof may require an
optional key. The key may be encoded either by URL-encoding or
(upper-case) hex-encoding.
TODO: In the future, more encodings may be supported, like base32 (e.g.
/32:)
For example, for a Cosmos-SDK application where the first two proof layers
are SimpleValueOps, and the third proof layer is an IAVLValueOp, the keys
might look like:
0: []byte("App")
1: []byte("IBC")
2: []byte{0x01, 0x02, 0x03}
Assuming that we know that the first two layers are always ASCII texts, we
probably want to use URLEncoding for those, whereas the third layer will
require HEX encoding for efficient representation.
kp := new(KeyPath)
kp.AppendKey([]byte("App"), KeyEncodingURL)
kp.AppendKey([]byte("IBC"), KeyEncodingURL)
kp.AppendKey([]byte{0x01, 0x02, 0x03}, KeyEncodingURL)
kp.String() // Should return "/App/IBC/x:010203"
NOTE: Key paths must begin with a `/`.
NOTE: All encodings *MUST* work compatibly, such that you can choose to use
whatever encoding, and the decoded keys will always be the same. In other
words, it's just as good to encode all three keys using URL encoding or HEX
encoding... it just wouldn't be optimal in terms of readability or space
efficiency.
NOTE: Punycode will never be supported here, because not all values can be
decoded. For example, no string decodes to the string "xn--blah" in
Punycode.
*/
type keyEncoding int
const (
KeyEncodingURL keyEncoding = iota
KeyEncodingHex
KeyEncodingMax // Number of known encodings. Used for testing
)
type Key struct {
name []byte
enc keyEncoding
}
type KeyPath []Key
func (pth KeyPath) AppendKey(key []byte, enc keyEncoding) KeyPath {
return append(pth, Key{key, enc})
}
func (pth KeyPath) String() string {
res := ""
for _, key := range pth {
switch key.enc {
case KeyEncodingURL:
res += "/" + url.PathEscape(string(key.name))
case KeyEncodingHex:
res += "/x:" + fmt.Sprintf("%X", key.name)
default:
panic("unexpected key encoding type")
}
}
return res
}
// Decode a path to a list of keys. Path must begin with `/`.
// Each key must use a known encoding.
func KeyPathToKeys(path string) (keys [][]byte, err error) {
if path == "" || path[0] != '/' {
return nil, cmn.NewError("key path string must start with a forward slash '/'")
}
parts := strings.Split(path[1:], "/")
keys = make([][]byte, len(parts))
for i, part := range parts {
if strings.HasPrefix(part, "x:") {
hexPart := part[2:]
key, err := hex.DecodeString(hexPart)
if err != nil {
return nil, cmn.ErrorWrap(err, "decoding hex-encoded part #%d: /%s", i, part)
}
keys[i] = key
} else {
key, err := url.PathUnescape(part)
if err != nil {
return nil, cmn.ErrorWrap(err, "decoding url-encoded part #%d: /%s", i, part)
}
keys[i] = []byte(key) // TODO Test this with random bytes, I'm not sure that it works for arbitrary bytes...
}
}
return keys, nil
}

View File

@@ -0,0 +1,41 @@
package merkle
import (
"math/rand"
"testing"
"github.com/stretchr/testify/require"
)
func TestKeyPath(t *testing.T) {
var path KeyPath
keys := make([][]byte, 10)
alphanum := "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
for d := 0; d < 1e4; d++ {
path = nil
for i := range keys {
enc := keyEncoding(rand.Intn(int(KeyEncodingMax)))
keys[i] = make([]byte, rand.Uint32()%20)
switch enc {
case KeyEncodingURL:
for j := range keys[i] {
keys[i][j] = alphanum[rand.Intn(len(alphanum))]
}
case KeyEncodingHex:
rand.Read(keys[i])
default:
panic("Unexpected encoding")
}
path = path.AppendKey(keys[i], enc)
}
res, err := KeyPathToKeys(path.String())
require.Nil(t, err)
for i, key := range keys {
require.Equal(t, key, res[i])
}
}
}

View File

@@ -0,0 +1,91 @@
package merkle
import (
"bytes"
"fmt"
"github.com/tendermint/tendermint/crypto/tmhash"
cmn "github.com/tendermint/tendermint/libs/common"
)
const ProofOpSimpleValue = "simple:v"
// SimpleValueOp takes a key and a single value as argument and
// produces the root hash. The corresponding tree structure is
// the SimpleMap tree. SimpleMap takes a Hasher, and currently
// Tendermint uses aminoHasher. SimpleValueOp should support
// the hash function as used in aminoHasher. TODO support
// additional hash functions here as options/args to this
// operator.
//
// If the produced root hash matches the expected hash, the
// proof is good.
type SimpleValueOp struct {
// Encoded in ProofOp.Key.
key []byte
// To encode in ProofOp.Data
Proof *SimpleProof `json:"simple_proof"`
}
var _ ProofOperator = SimpleValueOp{}
func NewSimpleValueOp(key []byte, proof *SimpleProof) SimpleValueOp {
return SimpleValueOp{
key: key,
Proof: proof,
}
}
func SimpleValueOpDecoder(pop ProofOp) (ProofOperator, error) {
if pop.Type != ProofOpSimpleValue {
return nil, cmn.NewError("unexpected ProofOp.Type; got %v, want %v", pop.Type, ProofOpSimpleValue)
}
var op SimpleValueOp // a bit strange as we'll discard this, but it works.
err := cdc.UnmarshalBinaryLengthPrefixed(pop.Data, &op)
if err != nil {
return nil, cmn.ErrorWrap(err, "decoding ProofOp.Data into SimpleValueOp")
}
return NewSimpleValueOp(pop.Key, op.Proof), nil
}
func (op SimpleValueOp) ProofOp() ProofOp {
bz := cdc.MustMarshalBinaryLengthPrefixed(op)
return ProofOp{
Type: ProofOpSimpleValue,
Key: op.key,
Data: bz,
}
}
func (op SimpleValueOp) String() string {
return fmt.Sprintf("SimpleValueOp{%v}", op.GetKey())
}
func (op SimpleValueOp) Run(args [][]byte) ([][]byte, error) {
if len(args) != 1 {
return nil, cmn.NewError("expected 1 arg, got %v", len(args))
}
value := args[0]
hasher := tmhash.New()
hasher.Write(value) // does not error
vhash := hasher.Sum(nil)
// Wrap <op.Key, vhash> to hash the KVPair.
hasher = tmhash.New()
encodeByteSlice(hasher, []byte(op.key)) // does not error
encodeByteSlice(hasher, []byte(vhash)) // does not error
kvhash := hasher.Sum(nil)
if !bytes.Equal(kvhash, op.Proof.LeafHash) {
return nil, cmn.NewError("leaf hash mismatch: want %X got %X", op.Proof.LeafHash, kvhash)
}
return [][]byte{
op.Proof.ComputeRootHash(),
}, nil
}
func (op SimpleValueOp) GetKey() []byte {
return op.key
}

140
crypto/merkle/proof_test.go Normal file
View File

@@ -0,0 +1,140 @@
package merkle
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tendermint/go-amino"
cmn "github.com/tendermint/tendermint/libs/common"
)
const ProofOpDomino = "test:domino"
// Expects given input, produces given output.
// Like the game dominos.
type DominoOp struct {
key string // unexported, may be empty
Input string
Output string
}
func NewDominoOp(key, input, output string) DominoOp {
return DominoOp{
key: key,
Input: input,
Output: output,
}
}
func DominoOpDecoder(pop ProofOp) (ProofOperator, error) {
if pop.Type != ProofOpDomino {
panic("unexpected proof op type")
}
var op DominoOp // a bit strange as we'll discard this, but it works.
err := amino.UnmarshalBinaryLengthPrefixed(pop.Data, &op)
if err != nil {
return nil, cmn.ErrorWrap(err, "decoding ProofOp.Data into SimpleValueOp")
}
return NewDominoOp(string(pop.Key), op.Input, op.Output), nil
}
func (dop DominoOp) ProofOp() ProofOp {
bz := amino.MustMarshalBinaryLengthPrefixed(dop)
return ProofOp{
Type: ProofOpDomino,
Key: []byte(dop.key),
Data: bz,
}
}
func (dop DominoOp) Run(input [][]byte) (output [][]byte, err error) {
if len(input) != 1 {
return nil, cmn.NewError("Expected input of length 1")
}
if string(input[0]) != dop.Input {
return nil, cmn.NewError("Expected input %v, got %v",
dop.Input, string(input[0]))
}
return [][]byte{[]byte(dop.Output)}, nil
}
func (dop DominoOp) GetKey() []byte {
return []byte(dop.key)
}
//----------------------------------------
func TestProofOperators(t *testing.T) {
var err error
// ProofRuntime setup
// TODO test this somehow.
// prt := NewProofRuntime()
// prt.RegisterOpDecoder(ProofOpDomino, DominoOpDecoder)
// ProofOperators setup
op1 := NewDominoOp("KEY1", "INPUT1", "INPUT2")
op2 := NewDominoOp("KEY2", "INPUT2", "INPUT3")
op3 := NewDominoOp("", "INPUT3", "INPUT4")
op4 := NewDominoOp("KEY4", "INPUT4", "OUTPUT4")
// Good
popz := ProofOperators([]ProofOperator{op1, op2, op3, op4})
err = popz.Verify(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.Nil(t, err)
err = popz.VerifyValue(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", bz("INPUT1"))
assert.Nil(t, err)
// BAD INPUT
err = popz.Verify(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1_WRONG")})
assert.NotNil(t, err)
err = popz.VerifyValue(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", bz("INPUT1_WRONG"))
assert.NotNil(t, err)
// BAD KEY 1
err = popz.Verify(bz("OUTPUT4"), "/KEY3/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD KEY 2
err = popz.Verify(bz("OUTPUT4"), "KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD KEY 3
err = popz.Verify(bz("OUTPUT4"), "/KEY4/KEY2/KEY1/", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD KEY 4
err = popz.Verify(bz("OUTPUT4"), "//KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD KEY 5
err = popz.Verify(bz("OUTPUT4"), "/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD OUTPUT 1
err = popz.Verify(bz("OUTPUT4_WRONG"), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD OUTPUT 2
err = popz.Verify(bz(""), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD POPZ 1
popz = []ProofOperator{op1, op2, op4}
err = popz.Verify(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD POPZ 2
popz = []ProofOperator{op4, op3, op2, op1}
err = popz.Verify(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
// BAD POPZ 3
popz = []ProofOperator{}
err = popz.Verify(bz("OUTPUT4"), "/KEY4/KEY2/KEY1", [][]byte{bz("INPUT1")})
assert.NotNil(t, err)
}
func bz(s string) []byte {
return []byte(s)
}

View File

@@ -1,6 +1,9 @@
package merkle
import (
"bytes"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/tendermint/crypto/tmhash"
cmn "github.com/tendermint/tendermint/libs/common"
)
@@ -20,14 +23,15 @@ func newSimpleMap() *simpleMap {
}
}
// Set hashes the key and value and appends it to the kv pairs.
func (sm *simpleMap) Set(key string, value Hasher) {
// Set creates a kv pair of the key and the hash of the value,
// and then appends it to simpleMap's kv pairs.
func (sm *simpleMap) Set(key string, value []byte) {
sm.sorted = false
// The value is hashed, so you can
// check for equality with a cached value (say)
// and make a determination to fetch or not.
vhash := value.Hash()
vhash := tmhash.Sum(value)
sm.kvs = append(sm.kvs, cmn.KVPair{
Key: []byte(key),
@@ -66,23 +70,25 @@ func (sm *simpleMap) KVPairs() cmn.KVPairs {
// then hashed.
type KVPair cmn.KVPair
func (kv KVPair) Hash() []byte {
hasher := tmhash.New()
err := encodeByteSlice(hasher, kv.Key)
// Bytes returns key || value, with both the
// key and value length prefixed.
func (kv KVPair) Bytes() []byte {
var b bytes.Buffer
err := amino.EncodeByteSlice(&b, kv.Key)
if err != nil {
panic(err)
}
err = encodeByteSlice(hasher, kv.Value)
err = amino.EncodeByteSlice(&b, kv.Value)
if err != nil {
panic(err)
}
return hasher.Sum(nil)
return b.Bytes()
}
func hashKVPairs(kvs cmn.KVPairs) []byte {
kvsH := make([]Hasher, len(kvs))
kvsH := make([][]byte, len(kvs))
for i, kvp := range kvs {
kvsH[i] = KVPair(kvp)
kvsH[i] = KVPair(kvp).Bytes()
}
return SimpleHashFromHashers(kvsH)
return SimpleHashFromByteSlices(kvsH)
}

View File

@@ -5,50 +5,29 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tendermint/tendermint/crypto/tmhash"
)
type strHasher string
func (str strHasher) Hash() []byte {
return tmhash.Sum([]byte(str))
}
func TestSimpleMap(t *testing.T) {
{
db := newSimpleMap()
db.Set("key1", strHasher("value1"))
assert.Equal(t, "fa9bc106ffd932d919bee935ceb6cf2b3dd72d8f", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
tests := []struct {
keys []string
values []string // each string gets converted to []byte in test
want string
}{
{[]string{"key1"}, []string{"value1"}, "321d150de16dceb51c72981b432b115045383259b1a550adf8dc80f927508967"},
{[]string{"key1"}, []string{"value2"}, "2a9e4baf321eac99f6eecc3406603c14bc5e85bb7b80483cbfc75b3382d24a2f"},
// swap order with 2 keys
{[]string{"key1", "key2"}, []string{"value1", "value2"}, "c4d8913ab543ba26aa970646d4c99a150fd641298e3367cf68ca45fb45a49881"},
{[]string{"key2", "key1"}, []string{"value2", "value1"}, "c4d8913ab543ba26aa970646d4c99a150fd641298e3367cf68ca45fb45a49881"},
// swap order with 3 keys
{[]string{"key1", "key2", "key3"}, []string{"value1", "value2", "value3"}, "b23cef00eda5af4548a213a43793f2752d8d9013b3f2b64bc0523a4791196268"},
{[]string{"key1", "key3", "key2"}, []string{"value1", "value3", "value2"}, "b23cef00eda5af4548a213a43793f2752d8d9013b3f2b64bc0523a4791196268"},
}
{
for i, tc := range tests {
db := newSimpleMap()
db.Set("key1", strHasher("value2"))
assert.Equal(t, "e00e7dcfe54e9fafef5111e813a587f01ba9c3e8", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
}
{
db := newSimpleMap()
db.Set("key1", strHasher("value1"))
db.Set("key2", strHasher("value2"))
assert.Equal(t, "eff12d1c703a1022ab509287c0f196130123d786", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
}
{
db := newSimpleMap()
db.Set("key2", strHasher("value2")) // NOTE: out of order
db.Set("key1", strHasher("value1"))
assert.Equal(t, "eff12d1c703a1022ab509287c0f196130123d786", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
}
{
db := newSimpleMap()
db.Set("key1", strHasher("value1"))
db.Set("key2", strHasher("value2"))
db.Set("key3", strHasher("value3"))
assert.Equal(t, "b2c62a277c08dbd2ad73ca53cd1d6bfdf5830d26", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
}
{
db := newSimpleMap()
db.Set("key2", strHasher("value2")) // NOTE: out of order
db.Set("key1", strHasher("value1"))
db.Set("key3", strHasher("value3"))
assert.Equal(t, "b2c62a277c08dbd2ad73ca53cd1d6bfdf5830d26", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
for i := 0; i < len(tc.keys); i++ {
db.Set(tc.keys[i], []byte(tc.values[i]))
}
got := db.Hash()
assert.Equal(t, tc.want, fmt.Sprintf("%x", got), "Hash didn't match on tc %d", i)
}
}

View File

@@ -2,23 +2,39 @@ package merkle
import (
"bytes"
"errors"
"fmt"
"github.com/tendermint/tendermint/crypto/tmhash"
cmn "github.com/tendermint/tendermint/libs/common"
)
// SimpleProof represents a simple merkle proof.
// SimpleProof represents a simple Merkle proof.
// NOTE: The convention for proofs is to include leaf hashes but to
// exclude the root hash.
// This convention is implemented across IAVL range proofs as well.
// Keep this consistent unless there's a very good reason to change
// everything. This also affects the generalized proof system as
// well.
type SimpleProof struct {
Aunts [][]byte `json:"aunts"` // Hashes from leaf's sibling to a root's child.
Total int `json:"total"` // Total number of items.
Index int `json:"index"` // Index of item to prove.
LeafHash []byte `json:"leaf_hash"` // Hash of item value.
Aunts [][]byte `json:"aunts"` // Hashes from leaf's sibling to a root's child.
}
// SimpleProofsFromHashers computes inclusion proof for given items.
// SimpleProofsFromByteSlices computes inclusion proof for given items.
// proofs[0] is the proof for items[0].
func SimpleProofsFromHashers(items []Hasher) (rootHash []byte, proofs []*SimpleProof) {
trails, rootSPN := trailsFromHashers(items)
func SimpleProofsFromByteSlices(items [][]byte) (rootHash []byte, proofs []*SimpleProof) {
trails, rootSPN := trailsFromByteSlices(items)
rootHash = rootSPN.Hash
proofs = make([]*SimpleProof, len(items))
for i, trail := range trails {
proofs[i] = &SimpleProof{
Aunts: trail.FlattenAunts(),
Total: len(items),
Index: i,
LeafHash: trail.Hash,
Aunts: trail.FlattenAunts(),
}
}
return
@@ -27,19 +43,19 @@ func SimpleProofsFromHashers(items []Hasher) (rootHash []byte, proofs []*SimpleP
// SimpleProofsFromMap generates proofs from a map. The keys/values of the map will be used as the keys/values
// in the underlying key-value pairs.
// The keys are sorted before the proofs are computed.
func SimpleProofsFromMap(m map[string]Hasher) (rootHash []byte, proofs map[string]*SimpleProof, keys []string) {
func SimpleProofsFromMap(m map[string][]byte) (rootHash []byte, proofs map[string]*SimpleProof, keys []string) {
sm := newSimpleMap()
for k, v := range m {
sm.Set(k, v)
}
sm.Sort()
kvs := sm.kvs
kvsH := make([]Hasher, 0, len(kvs))
for _, kvp := range kvs {
kvsH = append(kvsH, KVPair(kvp))
kvsBytes := make([][]byte, len(kvs))
for i, kvp := range kvs {
kvsBytes[i] = KVPair(kvp).Bytes()
}
rootHash, proofList := SimpleProofsFromHashers(kvsH)
rootHash, proofList := SimpleProofsFromByteSlices(kvsBytes)
proofs = make(map[string]*SimpleProof)
keys = make([]string, len(proofList))
for i, kvp := range kvs {
@@ -49,11 +65,33 @@ func SimpleProofsFromMap(m map[string]Hasher) (rootHash []byte, proofs map[strin
return
}
// Verify that leafHash is a leaf hash of the simple-merkle-tree
// which hashes to rootHash.
func (sp *SimpleProof) Verify(index int, total int, leafHash []byte, rootHash []byte) bool {
computedHash := computeHashFromAunts(index, total, leafHash, sp.Aunts)
return computedHash != nil && bytes.Equal(computedHash, rootHash)
// Verify that the SimpleProof proves the root hash.
// Check sp.Index/sp.Total manually if needed
func (sp *SimpleProof) Verify(rootHash []byte, leafHash []byte) error {
if sp.Total < 0 {
return errors.New("Proof total must be positive")
}
if sp.Index < 0 {
return errors.New("Proof index cannot be negative")
}
if !bytes.Equal(sp.LeafHash, leafHash) {
return cmn.NewError("invalid leaf hash: wanted %X got %X", leafHash, sp.LeafHash)
}
computedHash := sp.ComputeRootHash()
if !bytes.Equal(computedHash, rootHash) {
return cmn.NewError("invalid root hash: wanted %X got %X", rootHash, computedHash)
}
return nil
}
// Compute the root hash given a leaf hash. Does not verify the result.
func (sp *SimpleProof) ComputeRootHash() []byte {
return computeHashFromAunts(
sp.Index,
sp.Total,
sp.LeafHash,
sp.Aunts,
)
}
// String implements the stringer interface for SimpleProof.
@@ -96,13 +134,13 @@ func computeHashFromAunts(index int, total int, leafHash []byte, innerHashes [][
if leftHash == nil {
return nil
}
return SimpleHashFromTwoHashes(leftHash, innerHashes[len(innerHashes)-1])
return simpleHashFromTwoHashes(leftHash, innerHashes[len(innerHashes)-1])
}
rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1])
if rightHash == nil {
return nil
}
return SimpleHashFromTwoHashes(innerHashes[len(innerHashes)-1], rightHash)
return simpleHashFromTwoHashes(innerHashes[len(innerHashes)-1], rightHash)
}
}
@@ -138,18 +176,18 @@ func (spn *SimpleProofNode) FlattenAunts() [][]byte {
// trails[0].Hash is the leaf hash for items[0].
// trails[i].Parent.Parent....Parent == root for all i.
func trailsFromHashers(items []Hasher) (trails []*SimpleProofNode, root *SimpleProofNode) {
func trailsFromByteSlices(items [][]byte) (trails []*SimpleProofNode, root *SimpleProofNode) {
// Recursive impl.
switch len(items) {
case 0:
return nil, nil
case 1:
trail := &SimpleProofNode{items[0].Hash(), nil, nil, nil}
trail := &SimpleProofNode{tmhash.Sum(items[0]), nil, nil, nil}
return []*SimpleProofNode{trail}, trail
default:
lefts, leftRoot := trailsFromHashers(items[:(len(items)+1)/2])
rights, rightRoot := trailsFromHashers(items[(len(items)+1)/2:])
rootHash := SimpleHashFromTwoHashes(leftRoot.Hash, rightRoot.Hash)
lefts, leftRoot := trailsFromByteSlices(items[:(len(items)+1)/2])
rights, rightRoot := trailsFromByteSlices(items[(len(items)+1)/2:])
rootHash := simpleHashFromTwoHashes(leftRoot.Hash, rightRoot.Hash)
root := &SimpleProofNode{rootHash, nil, nil, nil}
leftRoot.Parent = root
leftRoot.Right = rightRoot

View File

@@ -4,8 +4,8 @@ import (
"github.com/tendermint/tendermint/crypto/tmhash"
)
// SimpleHashFromTwoHashes is the basic operation of the Merkle tree: Hash(left | right).
func SimpleHashFromTwoHashes(left, right []byte) []byte {
// simpleHashFromTwoHashes is the basic operation of the Merkle tree: Hash(left | right).
func simpleHashFromTwoHashes(left, right []byte) []byte {
var hasher = tmhash.New()
err := encodeByteSlice(hasher, left)
if err != nil {
@@ -18,41 +18,29 @@ func SimpleHashFromTwoHashes(left, right []byte) []byte {
return hasher.Sum(nil)
}
// SimpleHashFromHashers computes a Merkle tree from items that can be hashed.
func SimpleHashFromHashers(items []Hasher) []byte {
hashes := make([][]byte, len(items))
for i, item := range items {
hash := item.Hash()
hashes[i] = hash
// SimpleHashFromByteSlices computes a Merkle tree where the leaves are the byte slice,
// in the provided order.
func SimpleHashFromByteSlices(items [][]byte) []byte {
switch len(items) {
case 0:
return nil
case 1:
return tmhash.Sum(items[0])
default:
left := SimpleHashFromByteSlices(items[:(len(items)+1)/2])
right := SimpleHashFromByteSlices(items[(len(items)+1)/2:])
return simpleHashFromTwoHashes(left, right)
}
return simpleHashFromHashes(hashes)
}
// SimpleHashFromMap computes a Merkle tree from sorted map.
// Like calling SimpleHashFromHashers with
// `item = []byte(Hash(key) | Hash(value))`,
// sorted by `item`.
func SimpleHashFromMap(m map[string]Hasher) []byte {
func SimpleHashFromMap(m map[string][]byte) []byte {
sm := newSimpleMap()
for k, v := range m {
sm.Set(k, v)
}
return sm.Hash()
}
//----------------------------------------------------------------
// Expects hashes!
func simpleHashFromHashes(hashes [][]byte) []byte {
// Recursive impl.
switch len(hashes) {
case 0:
return nil
case 1:
return hashes[0]
default:
left := simpleHashFromHashes(hashes[:(len(hashes)+1)/2])
right := simpleHashFromHashes(hashes[(len(hashes)+1)/2:])
return SimpleHashFromTwoHashes(left, right)
}
}

View File

@@ -1,13 +1,13 @@
package merkle
import (
"bytes"
"testing"
"github.com/stretchr/testify/require"
cmn "github.com/tendermint/tendermint/libs/common"
. "github.com/tendermint/tendermint/libs/test"
"testing"
"github.com/tendermint/tendermint/crypto/tmhash"
)
@@ -21,69 +21,52 @@ func TestSimpleProof(t *testing.T) {
total := 100
items := make([]Hasher, total)
items := make([][]byte, total)
for i := 0; i < total; i++ {
items[i] = testItem(cmn.RandBytes(tmhash.Size))
}
rootHash := SimpleHashFromHashers(items)
rootHash := SimpleHashFromByteSlices(items)
rootHash2, proofs := SimpleProofsFromHashers(items)
rootHash2, proofs := SimpleProofsFromByteSlices(items)
if !bytes.Equal(rootHash, rootHash2) {
t.Errorf("Unmatched root hashes: %X vs %X", rootHash, rootHash2)
}
require.Equal(t, rootHash, rootHash2, "Unmatched root hashes: %X vs %X", rootHash, rootHash2)
// For each item, check the trail.
for i, item := range items {
itemHash := item.Hash()
itemHash := tmhash.Sum(item)
proof := proofs[i]
// Verify success
ok := proof.Verify(i, total, itemHash, rootHash)
if !ok {
t.Errorf("Verification failed for index %v.", i)
}
// Check total/index
require.Equal(t, proof.Index, i, "Unmatched indicies: %d vs %d", proof.Index, i)
// Wrong item index should make it fail
{
ok = proof.Verify((i+1)%total, total, itemHash, rootHash)
if ok {
t.Errorf("Expected verification to fail for wrong index %v.", i)
}
}
require.Equal(t, proof.Total, total, "Unmatched totals: %d vs %d", proof.Total, total)
// Verify success
err := proof.Verify(rootHash, itemHash)
require.NoError(t, err, "Verificatior failed: %v.", err)
// Trail too long should make it fail
origAunts := proof.Aunts
proof.Aunts = append(proof.Aunts, cmn.RandBytes(32))
{
ok = proof.Verify(i, total, itemHash, rootHash)
if ok {
t.Errorf("Expected verification to fail for wrong trail length.")
}
}
err = proof.Verify(rootHash, itemHash)
require.Error(t, err, "Expected verification to fail for wrong trail length")
proof.Aunts = origAunts
// Trail too short should make it fail
proof.Aunts = proof.Aunts[0 : len(proof.Aunts)-1]
{
ok = proof.Verify(i, total, itemHash, rootHash)
if ok {
t.Errorf("Expected verification to fail for wrong trail length.")
}
}
err = proof.Verify(rootHash, itemHash)
require.Error(t, err, "Expected verification to fail for wrong trail length")
proof.Aunts = origAunts
// Mutating the itemHash should make it fail.
ok = proof.Verify(i, total, MutateByteSlice(itemHash), rootHash)
if ok {
t.Errorf("Expected verification to fail for mutated leaf hash")
}
err = proof.Verify(rootHash, MutateByteSlice(itemHash))
require.Error(t, err, "Expected verification to fail for mutated leaf hash")
// Mutating the rootHash should make it fail.
ok = proof.Verify(i, total, itemHash, MutateByteSlice(rootHash))
if ok {
t.Errorf("Expected verification to fail for mutated root hash")
}
err = proof.Verify(MutateByteSlice(rootHash), itemHash)
require.Error(t, err, "Expected verification to fail for mutated root hash")
}
}

View File

@@ -25,11 +25,6 @@ type Tree interface {
IterateRange(start []byte, end []byte, ascending bool, fx func(key []byte, value []byte) (stop bool)) (stopped bool)
}
// Hasher represents a hashable piece of data which can be hashed in the Tree.
type Hasher interface {
Hash() []byte
}
//-----------------------------------------------------------------------
// Uvarint length prefixed byteslice

12
crypto/merkle/wire.go Normal file
View File

@@ -0,0 +1,12 @@
package merkle
import (
"github.com/tendermint/go-amino"
)
var cdc *amino.Codec
func init() {
cdc = amino.NewCodec()
cdc.Seal()
}

View File

@@ -1,7 +1,6 @@
package crypto
import (
"crypto/aes"
"crypto/cipher"
crand "crypto/rand"
"crypto/sha256"
@@ -9,16 +8,29 @@ import (
"io"
"sync"
. "github.com/tendermint/tendermint/libs/common"
"golang.org/x/crypto/chacha20poly1305"
)
// NOTE: This is ignored for now until we have time
// to properly review the MixEntropy function - https://github.com/tendermint/tendermint/issues/2099.
//
// The randomness here is derived from xoring a chacha20 keystream with
// output from crypto/rand's OS Entropy Reader. (Due to fears of the OS'
// entropy being backdoored)
//
// For forward secrecy of produced randomness, the internal chacha key is hashed
// and thereby rotated after each call.
var gRandInfo *randInfo
func init() {
gRandInfo = &randInfo{}
gRandInfo.MixEntropy(randBytes(32)) // Init
// TODO: uncomment after reviewing MixEntropy -
// https://github.com/tendermint/tendermint/issues/2099
// gRandInfo.MixEntropy(randBytes(32)) // Init
}
// WARNING: This function needs review - https://github.com/tendermint/tendermint/issues/2099.
// Mix additional bytes of randomness, e.g. from hardware, user-input, etc.
// It is OK to call it multiple times. It does not diminish security.
func MixEntropy(seedBytes []byte) {
@@ -30,20 +42,28 @@ func randBytes(numBytes int) []byte {
b := make([]byte, numBytes)
_, err := crand.Read(b)
if err != nil {
PanicCrisis(err)
panic(err)
}
return b
}
// This only uses the OS's randomness
func CRandBytes(numBytes int) []byte {
return randBytes(numBytes)
}
/* TODO: uncomment after reviewing MixEntropy - https://github.com/tendermint/tendermint/issues/2099
// This uses the OS and the Seed(s).
func CRandBytes(numBytes int) []byte {
b := make([]byte, numBytes)
_, err := gRandInfo.Read(b)
if err != nil {
PanicCrisis(err)
}
return b
return randBytes(numBytes)
b := make([]byte, numBytes)
_, err := gRandInfo.Read(b)
if err != nil {
panic(err)
}
return b
}
*/
// CRandHex returns a hex encoded string that's floor(numDigits/2) * 2 long.
//
@@ -53,23 +73,29 @@ func CRandHex(numDigits int) string {
return hex.EncodeToString(CRandBytes(numDigits / 2))
}
// Returns a crand.Reader.
func CReader() io.Reader {
return crand.Reader
}
/* TODO: uncomment after reviewing MixEntropy - https://github.com/tendermint/tendermint/issues/2099
// Returns a crand.Reader mixed with user-supplied entropy
func CReader() io.Reader {
return gRandInfo
}
*/
//--------------------------------------------------------------------------------
type randInfo struct {
mtx sync.Mutex
seedBytes [32]byte
cipherAES256 cipher.Block
streamAES256 cipher.Stream
reader io.Reader
mtx sync.Mutex
seedBytes [chacha20poly1305.KeySize]byte
chacha cipher.AEAD
reader io.Reader
}
// You can call this as many times as you'd like.
// XXX TODO review
// XXX/TODO: review - https://github.com/tendermint/tendermint/issues/2099
func (ri *randInfo) MixEntropy(seedBytes []byte) {
ri.mtx.Lock()
defer ri.mtx.Unlock()
@@ -79,30 +105,35 @@ func (ri *randInfo) MixEntropy(seedBytes []byte) {
h.Write(seedBytes)
h.Write(ri.seedBytes[:])
hashBytes := h.Sum(nil)
hashBytes32 := [32]byte{}
copy(hashBytes32[:], hashBytes)
ri.seedBytes = xorBytes32(ri.seedBytes, hashBytes32)
// Create new cipher.Block
var err error
ri.cipherAES256, err = aes.NewCipher(ri.seedBytes[:])
copy(ri.seedBytes[:], hashBytes)
chacha, err := chacha20poly1305.New(ri.seedBytes[:])
if err != nil {
PanicSanity("Error creating AES256 cipher: " + err.Error())
panic("Initializing chacha20 failed")
}
// Create new stream
ri.streamAES256 = cipher.NewCTR(ri.cipherAES256, randBytes(aes.BlockSize))
ri.chacha = chacha
// Create new reader
ri.reader = &cipher.StreamReader{S: ri.streamAES256, R: crand.Reader}
ri.reader = &cipher.StreamReader{S: ri, R: crand.Reader}
}
func (ri *randInfo) XORKeyStream(dst, src []byte) {
// nonce being 0 is safe due to never re-using a key.
emptyNonce := make([]byte, 12)
tmpDst := ri.chacha.Seal([]byte{}, emptyNonce, src, []byte{0})
// this removes the poly1305 tag as well, since chacha is a stream cipher
// and we truncate at input length.
copy(dst, tmpDst[:len(src)])
// hash seedBytes for forward secrecy, and initialize new chacha instance
newSeed := sha256.Sum256(ri.seedBytes[:])
chacha, err := chacha20poly1305.New(newSeed[:])
if err != nil {
panic("Initializing chacha20 failed")
}
ri.chacha = chacha
}
func (ri *randInfo) Read(b []byte) (n int, err error) {
ri.mtx.Lock()
defer ri.mtx.Unlock()
return ri.reader.Read(b)
}
func xorBytes32(bytesA [32]byte, bytesB [32]byte) (res [32]byte) {
for i, b := range bytesA {
res[i] = b ^ bytesB[i]
}
return res
n, err = ri.reader.Read(b)
ri.mtx.Unlock()
return
}

23
crypto/random_test.go Normal file
View File

@@ -0,0 +1,23 @@
package crypto_test
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/tendermint/tendermint/crypto"
)
// the purpose of this test is primarily to ensure that the randomness
// generation won't error.
func TestRandomConsistency(t *testing.T) {
x1 := crypto.CRandBytes(256)
x2 := crypto.CRandBytes(256)
x3 := crypto.CRandBytes(256)
x4 := crypto.CRandBytes(256)
x5 := crypto.CRandBytes(256)
require.NotEqual(t, x1, x2)
require.NotEqual(t, x3, x4)
require.NotEqual(t, x4, x5)
require.NotEqual(t, x1, x5)
}

View File

@@ -9,8 +9,9 @@ import (
secp256k1 "github.com/tendermint/btcd/btcec"
amino "github.com/tendermint/go-amino"
"golang.org/x/crypto/ripemd160" // forked to github.com/tendermint/crypto
"github.com/tendermint/tendermint/crypto"
"golang.org/x/crypto/ripemd160"
)
//-------------------------------------

View File

@@ -6,10 +6,27 @@ import (
)
const (
Size = 20
Size = sha256.Size
BlockSize = sha256.BlockSize
)
// New returns a new hash.Hash.
func New() hash.Hash {
return sha256.New()
}
// Sum returns the SHA256 of the bz.
func Sum(bz []byte) []byte {
h := sha256.Sum256(bz)
return h[:]
}
//-------------------------------------------------------------
const (
TruncatedSize = 20
)
type sha256trunc struct {
sha256 hash.Hash
}
@@ -19,7 +36,7 @@ func (h sha256trunc) Write(p []byte) (n int, err error) {
}
func (h sha256trunc) Sum(b []byte) []byte {
shasum := h.sha256.Sum(b)
return shasum[:Size]
return shasum[:TruncatedSize]
}
func (h sha256trunc) Reset() {
@@ -27,22 +44,22 @@ func (h sha256trunc) Reset() {
}
func (h sha256trunc) Size() int {
return Size
return TruncatedSize
}
func (h sha256trunc) BlockSize() int {
return h.sha256.BlockSize()
}
// New returns a new hash.Hash.
func New() hash.Hash {
// NewTruncated returns a new hash.Hash.
func NewTruncated() hash.Hash {
return sha256trunc{
sha256: sha256.New(),
}
}
// Sum returns the first 20 bytes of SHA256 of the bz.
func Sum(bz []byte) []byte {
// SumTruncated returns the first 20 bytes of SHA256 of the bz.
func SumTruncated(bz []byte) []byte {
hash := sha256.Sum256(bz)
return hash[:Size]
return hash[:TruncatedSize]
}

View File

@@ -14,10 +14,29 @@ func TestHash(t *testing.T) {
hasher.Write(testVector)
bz := hasher.Sum(nil)
bz2 := tmhash.Sum(testVector)
hasher = sha256.New()
hasher.Write(testVector)
bz2 := hasher.Sum(nil)
bz2 = bz2[:20]
bz3 := hasher.Sum(nil)
assert.Equal(t, bz, bz2)
assert.Equal(t, bz, bz3)
}
func TestHashTruncated(t *testing.T) {
testVector := []byte("abc")
hasher := tmhash.NewTruncated()
hasher.Write(testVector)
bz := hasher.Sum(nil)
bz2 := tmhash.SumTruncated(testVector)
hasher = sha256.New()
hasher.Write(testVector)
bz3 := hasher.Sum(nil)
bz3 = bz3[:tmhash.TruncatedSize]
assert.Equal(t, bz, bz2)
assert.Equal(t, bz, bz3)
}

View File

@@ -8,7 +8,7 @@ import (
"errors"
"fmt"
"golang.org/x/crypto/chacha20poly1305"
"golang.org/x/crypto/chacha20poly1305" // forked to github.com/tendermint/crypto
)
// Implements crypto.AEAD

View File

@@ -4,9 +4,10 @@ import (
"errors"
"fmt"
"golang.org/x/crypto/nacl/secretbox" // forked to github.com/tendermint/crypto
"github.com/tendermint/tendermint/crypto"
cmn "github.com/tendermint/tendermint/libs/common"
"golang.org/x/crypto/nacl/secretbox"
)
// TODO, make this into a struct that implements crypto.Symmetric.

View File

@@ -6,8 +6,9 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/crypto/bcrypt" // forked to github.com/tendermint/crypto
"github.com/tendermint/tendermint/crypto"
"golang.org/x/crypto/bcrypt"
)
func TestSimple(t *testing.T) {
@@ -29,7 +30,9 @@ func TestSimpleWithKDF(t *testing.T) {
plaintext := []byte("sometext")
secretPass := []byte("somesecret")
secret, err := bcrypt.GenerateFromPassword(secretPass, 12)
salt := []byte("somesaltsomesalt") // len 16
// NOTE: we use a fork of x/crypto so we can inject our own randomness for salt
secret, err := bcrypt.GenerateFromPassword(salt, secretPass, 12)
if err != nil {
t.Error(err)
}

118
docs/.vuepress/config.js Normal file
View File

@@ -0,0 +1,118 @@
module.exports = {
title: "Tendermint Documentation",
description: "Documentation for Tendermint Core.",
ga: "UA-51029217-1",
dest: "./dist/docs",
base: "/docs/",
markdown: {
lineNumbers: true
},
themeConfig: {
lastUpdated: "Last Updated",
nav: [{ text: "Back to Tendermint", link: "https://tendermint.com" }],
sidebar: [
{
title: "Introduction",
collapsable: false,
children: [
"/introduction/",
"/introduction/quick-start",
"/introduction/install",
"/introduction/what-is-tendermint"
]
},
{
title: "Tendermint Core",
collapsable: false,
children: [
"/tendermint-core/",
"/tendermint-core/using-tendermint",
"/tendermint-core/configuration",
"/tendermint-core/rpc",
"/tendermint-core/running-in-production",
"/tendermint-core/fast-sync",
"/tendermint-core/how-to-read-logs",
"/tendermint-core/block-structure",
"/tendermint-core/light-client-protocol",
"/tendermint-core/metrics",
"/tendermint-core/secure-p2p",
"/tendermint-core/validators"
]
},
{
title: "Tools",
collapsable: false,
children: [
"/tools/",
"/tools/benchmarking",
"/tools/monitoring"
]
},
{
title: "Networks",
collapsable: false,
children: [
"/networks/",
"/networks/docker-compose",
"/networks/terraform-and-ansible",
]
},
{
title: "Apps",
collapsable: false,
children: [
"/app-dev/getting-started",
"/app-dev/abci-cli",
"/app-dev/app-architecture",
"/app-dev/app-development",
"/app-dev/subscribing-to-events-via-websocket",
"/app-dev/indexing-transactions",
"/app-dev/abci-spec",
"/app-dev/ecosystem"
]
},
{
title: "Tendermint Spec",
collapsable: true,
children: [
"/spec/",
"/spec/blockchain/blockchain",
"/spec/blockchain/encoding",
"/spec/blockchain/state",
"/spec/software/abci",
"/spec/consensus/bft-time",
"/spec/consensus/consensus",
"/spec/consensus/light-client",
"/spec/software/wal",
"/spec/p2p/config",
"/spec/p2p/connection",
"/spec/p2p/node",
"/spec/p2p/peer",
"/spec/reactors/block_sync/reactor",
"/spec/reactors/block_sync/impl",
"/spec/reactors/consensus/consensus",
"/spec/reactors/consensus/consensus-reactor",
"/spec/reactors/consensus/proposer-selection",
"/spec/reactors/evidence/reactor",
"/spec/reactors/mempool/concurrency",
"/spec/reactors/mempool/config",
"/spec/reactors/mempool/functionality",
"/spec/reactors/mempool/messages",
"/spec/reactors/mempool/reactor",
"/spec/reactors/pex/pex",
"/spec/reactors/pex/reactor",
]
},
{
title: "ABCI Spec",
collapsable: false,
children: [
"/spec/abci/",
"/spec/abci/abci",
"/spec/abci/apps",
"/spec/abci/client-server"
]
}
]
}
};

17
docs/.vuepress/dist/404.html vendored Normal file
View File

@@ -0,0 +1,17 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>VuePress</title>
<meta name="description" content="">
<link rel="preload" href="/assets/css/1.styles.c01b7ee3.css" as="style"><link rel="preload" href="/assets/js/app.48f1ff5f.js" as="script"><link rel="prefetch" href="/assets/js/0.7c2695bf.js">
<link rel="stylesheet" href="/assets/css/1.styles.c01b7ee3.css">
</head>
<body>
<div id="app" data-server-rendered="true"><div class="theme-container"><div class="content"><h1>404</h1><blockquote>Looks like we've got some broken links.</blockquote><a href="/" class="router-link-active">Take me home.</a></div></div></div>
<script src="/assets/js/app.48f1ff5f.js" defer></script>
</body>
</html>

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><svg xmlns="http://www.w3.org/2000/svg" width="12" height="13"><g stroke-width="2" stroke="#aaa" fill="none"><path d="M11.29 11.71l-4-4"/><circle cx="5" cy="5" r="4"/></g></svg>

After

Width:  |  Height:  |  Size: 216 B

View File

@@ -0,0 +1 @@
(window.webpackJsonp=window.webpackJsonp||[]).push([[0],{136:function(e,t,s){"use strict";s.r(t);var n=s(0),r=Object(n.a)({},function(){this.$createElement;this._self._c;return this._m(0)},[function(){var e=this.$createElement,t=this._self._c||e;return t("div",{staticClass:"content"},[t("h1",{attrs:{id:"hello-vuepress"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#hello-vuepress","aria-hidden":"true"}},[this._v("#")]),this._v(" Hello VuePress")])])}],!1,null,null,null);t.default=r.exports}}]);

File diff suppressed because one or more lines are too long

17
docs/.vuepress/dist/index.html vendored Normal file
View File

@@ -0,0 +1,17 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Hello VuePress</title>
<meta name="description" content="">
<link rel="preload" href="/assets/css/1.styles.c01b7ee3.css" as="style"><link rel="preload" href="/assets/js/app.48f1ff5f.js" as="script"><link rel="preload" href="/assets/js/0.7c2695bf.js" as="script">
<link rel="stylesheet" href="/assets/css/1.styles.c01b7ee3.css">
</head>
<body>
<div id="app" data-server-rendered="true"><div class="theme-container no-sidebar"><header class="navbar"><div class="sidebar-button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"><path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path></svg></div><a href="/" class="home-link router-link-exact-active router-link-active"></a><div class="links"><div class="search-box"><input aria-label="Search" autocomplete="off" spellcheck="false" value=""><!----></div><!----></div></header><div class="sidebar-mask"></div><div class="sidebar"><!----><!----></div><div class="page"><div class="content"><h1 id="hello-vuepress"><a href="#hello-vuepress" aria-hidden="true" class="header-anchor">#</a> Hello VuePress</h1></div><div class="page-edit"><!----><!----></div><!----></div></div></div>
<script src="/assets/js/0.7c2695bf.js" defer></script><script src="/assets/js/app.48f1ff5f.js" defer></script>
</body>
</html>

View File

@@ -1,17 +1,99 @@
# Documentation Maintenance Overview
# Docs Build Workflow
The documentation found in this directory is hosted at:
The documentation for Tendermint Core is hosted at:
- https://tendermint.com/docs/
- https://tendermint.com/docs/ and
- https://tendermint-staging.interblock.io/docs/
and built using [VuePress](https://vuepress.vuejs.org/) from the tendermint website repo:
built from the files in this (`/docs`) directory for
[master](https://github.com/tendermint/tendermint/tree/master/docs)
and [develop](https://github.com/tendermint/tendermint/tree/develop/docs),
respectively.
- https://github.com/tendermint/tendermint.com
## How It Works
Under the hood, Jenkins listens for changes (on develop or master) in ./docs then rebuilds
either the staging or production site depending on which branch the changes were made.
There is a Jenkins job listening for changes in the `/docs` directory, on both
the `master` and `develop` branches. Any updates to files in this directory
on those branches will automatically trigger a website deployment. Under the hood,
a private website repository has make targets consumed by a standard Jenkins task.
To update the Table of Contents (layout of the documentation sidebar), edit the
`config.js` in this directory, while the `README.md` is the landing page for the
website documentation.
## README
The [README.md](./README.md) is also the landing page for the documentation
on the website. During the Jenkins build, the current commit is added to the bottom
of the README.
## Config.js
The [config.js](./.vuepress/config.js) generates the sidebar and Table of Contents
on the website docs. Note the use of relative links and the omission of
file extensions. Additional features are available to improve the look
of the sidebar.
## Links
**NOTE:** Strongly consider the existing links - both within this directory
and to the website docs - when moving or deleting files.
Links to directories *MUST* end in a `/`.
Relative links should be used nearly everywhere, having discovered and weighed the following:
### Relative
Where is the other file, relative to the current one?
- works both on GitHub and for the VuePress build
- confusing / annoying to have things like: `../../../../myfile.md`
- requires more updates when files are re-shuffled
### Absolute
Where is the other file, given the root of the repo?
- works on GitHub, doesn't work for the VuePress build
- this is much nicer: `/docs/hereitis/myfile.md`
- if you move that file around, the links inside it are preserved (but not to it, of course)
### Full
The full GitHub URL to a file or directory. Used occasionally when it makes sense
to send users to the GitHub.
## Building Locally
To build and serve the documentation locally, run:
```
# from this directory
npm install
npm install -g vuepress
```
then change the following line in the `config.js`:
```
base: "/docs/",
```
to:
```
base: "/",
```
Finally, go up one directory to the root of the repo and run:
```
# from root of repo
vuepress build docs
cd dist/docs
python -m SimpleHTTPServer 8080
```
then navigate to localhost:8080 in your browser.
## Consistency
Because the build processes are identical (as is the information contained herein), this file should be kept in sync as
much as possible with its [counterpart in the Cosmos SDK repo](https://github.com/cosmos/cosmos-sdk/blob/develop/docs/DOCS_README.md).

View File

@@ -1,27 +1,29 @@
# Tendermint
Welcome to the Tendermint Core documentation! The introduction below provides
an overview to help you navigate to your area of interest.
Welcome to the Tendermint Core documentation!
## Introduction
Tendermint Core is a blockchain application platform; it provides the equivalent
of a web-server, database, and supporting libraries for blockchain applications
written in any programming language. Like a web-server serving web applications,
Tendermint serves blockchain applications.
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state
transition machine - written in any programming language - and securely
replicates it on many machines. In other words, a blockchain.
More formally, Tendermint Core performs Byzantine Fault Tolerant (BFT)
State Machine Replication (SMR) for arbitrary deterministic, finite state machines.
For more background, see [What is
Tendermint?](introduction/what-is-tendermint.md).
Tendermint requires an application running over the Application Blockchain
Interface (ABCI) - and comes packaged with an example application to do so.
Follow the [installation instructions](./introduction/install.md) to get up and running
quickly. For more details on [using tendermint](./tendermint-core/using-tendermint.md) see that
and the following sections.
To get started quickly with an example application, see the [quick start guide](introduction/quick-start.md).
## Networks
To learn about application development on Tendermint, see the [Application Blockchain Interface](spec/abci/).
Testnets can be setup manually on one or more machines, or automatically on one
or more machine, using a variety of methods described in the [deploy testnets
section](./networks/deploy-testnets.md).
For more details on using Tendermint, see the respective documentation for
[Tendermint Core](tendermint-core/), [benchmarking and monitoring](tools/), and [network deployments](networks/).
## Application Development
## Contribute
The first step to building application on Tendermint is to [install
ABCI-CLI](./app-dev/getting-started.md) and play with the example applications.
To contribute to the documentation, see [this file](https://github.com/tendermint/tendermint/blob/master/docs/DOCS_README.md) for details of the build process and
considerations when making changes.
## Version
This documentation is built from the following commit:

Some files were not shown because too many files have changed in this diff Show More