set next validators along with validators while replay (#2637)

Closes #2634

.circleci/config.yml

@@ -301,6 +301,8 @@ jobs:
       - run: mkdir -p $GOPATH/src/github.com/tendermint
       - run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
       - run: bash test/p2p/circleci.sh
+      - store_artifacts:
+          path: /home/circleci/project/test/p2p/logs
 
   upload_coverage:
     <<: *defaults

CHANGELOG.md

@@ -1,5 +1,60 @@
 # Changelog
 
+## v0.25.0
+
+*September 22, 2018*
+
+Special thanks to external contributors on this release:
+@scriptionist, @bradyjoestar, @WALL-E
+
+This release is mostly about the ConsensusParams - removing fields and enforcing MaxGas.
+It also addresses some issues found via security audit, removes various unused
+functions from `libs/common`, and implements
+[ADR-012](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-012-peer-transport.md).
+
+Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint).
+
+BREAKING CHANGES:
+
+* CLI/RPC/Config
+  * [rpc] [\#2391](https://github.com/tendermint/tendermint/issues/2391) /status `result.node_info.other` became a map
+  * [types] [\#2364](https://github.com/tendermint/tendermint/issues/2364) Remove `TxSize` and `BlockGossip` from `ConsensusParams`
+    * Maximum tx size is now set implicitly via the `BlockSize.MaxBytes`
+    * The size of block parts in the consensus is now fixed to 64kB
+
+* Apps
+  * [mempool] [\#2360](https://github.com/tendermint/tendermint/issues/2360) Mempool tracks the `ResponseCheckTx.GasWanted` and
+    `ConsensusParams.BlockSize.MaxGas` and enforces:
+    - `GasWanted <= MaxGas` for every tx
+    - `(sum of GasWanted in block) <= MaxGas` for block proposal
+
+* Go API
+  * [libs/common] [\#2431](https://github.com/tendermint/tendermint/issues/2431) Remove Word256 due to lack of use
+  * [libs/common] [\#2452](https://github.com/tendermint/tendermint/issues/2452) Remove the following functions due to lack of use:
+    * byteslice.go: cmn.IsZeros, cmn.RightPadBytes, cmn.LeftPadBytes, cmn.PrefixEndBytes
+    * strings.go: cmn.IsHex, cmn.StripHex
+    * int.go: Uint64Slice, all put/get int64 methods
+
+FEATURES:
+- [rpc] [\#2415](https://github.com/tendermint/tendermint/issues/2415) New `/consensus_params?height=X` endpoint to query the consensus
+  params at any height (@scriptonist)
+- [types] [\#1714](https://github.com/tendermint/tendermint/issues/1714) Add Address to GenesisValidator
+- [metrics] [\#2337](https://github.com/tendermint/tendermint/issues/2337) `consensus.block_interval_metrics` is now gauge, not histogram (you will be able to see spikes, if any)
+- [libs] [\#2286](https://github.com/tendermint/tendermint/issues/2286) Panic if `autofile` or `db/fsdb` permissions change from 0600.
+
+IMPROVEMENTS:
+- [libs/db] [\#2371](https://github.com/tendermint/tendermint/issues/2371) Output error instead of panic when the given `db_backend` is not initialised (@bradyjoestar)
+- [mempool] [\#2399](https://github.com/tendermint/tendermint/issues/2399) Make mempool cache a proper LRU (@bradyjoestar)
+- [p2p] [\#2126](https://github.com/tendermint/tendermint/issues/2126) Introduce PeerTransport interface to improve isolation of concerns
+- [libs/common] [\#2326](https://github.com/tendermint/tendermint/issues/2326) Service returns ErrNotStarted
+
+BUG FIXES:
+- [node] [\#2294](https://github.com/tendermint/tendermint/issues/2294) Delay starting node until Genesis time
+- [consensus] [\#2048](https://github.com/tendermint/tendermint/issues/2048) Correct peer statistics for marking peer as good
+- [rpc] [\#2460](https://github.com/tendermint/tendermint/issues/2460) StartHTTPAndTLSServer() now passes StartTLS() errors back to the caller rather than hanging forever.
+- [p2p] [\#2047](https://github.com/tendermint/tendermint/issues/2047) Accept new connections asynchronously
+- [tm-bench] [\#2410](https://github.com/tendermint/tendermint/issues/2410) Enforce minimum transaction size (@WALL-E)
+
 ## 0.24.0
 
 *September 6th, 2018*

CHANGELOG_PENDING.md

@@ -1,35 +1,69 @@
 # Pending
 
-Special thanks to external contributors with PRs included in this release:
+Special thanks to external contributors on this release:
+@goolAdapter, @bradyjoestar
 
 BREAKING CHANGES:
 
 * CLI/RPC/Config
-  * [rpc] [\#2391](https://github.com/tendermint/tendermint/issues/2391) /status `result.node_info.other` became a map
+  * [config] \#2232 timeouts as time.Duration, not ints
+  * [config] \#2505 Remove Mempool.RecheckEmpty (it was effectively useless anyways)
+  * [config] `mempool.wal` is disabled by default
+  * [rpc] \#2298 `/abci_query` takes `prove` argument instead of `trusted` and switches the default
+    behaviour to `prove=false`
+  * [privval] \#2459 Split `SocketPVMsg`s implementations into Request and Response, where the Response may contain a error message (returned by the remote signer)
 
 * Apps
-  * [mempool] \#2310 Mempool tracks the `ResponseCheckTx.GasWanted` and enforces `ConsensusParams.BlockSize.MaxGas` on proposals.
+  * [abci] \#2298 ResponseQuery.Proof is now a structured merkle.Proof, not just
+    arbitrary bytes
 
 * Go API
-  * [libs/common] \#2431 Remove Word256 due to lack of use
-  * [libs/common] \#2452 Remove the following functions due to lack of use:
-    * byteslice.go: cmn.IsZeros, cmn.RightPadBytes, cmn.LeftPadBytes, cmn.PrefixEndBytes
-    * strings.go: cmn.IsHex, cmn.StripHex
-    * int.go: Uint64Slice, all put/get int64 methods
+  * [node] Remove node.RunForever
+  * [config] \#2232 timeouts as time.Duration, not ints
+  * [rpc/client] \#2298 `ABCIQueryOptions.Trusted` -> `ABCIQueryOptions.Prove`
+  * [types] \#2298 Remove `Index` and `Total` fields from `TxProof`.
+  * [crypto/merkle & lite] \#2298 Various changes to accomodate General Merkle trees
+  * [crypto/merkle] \#2595 Remove all Hasher objects in favor of byte slices
+  * [types] \#2598 `VoteTypeXxx` are now
 
 * Blockchain Protocol
+  * [types] Update SignBytes for `Vote`/`Proposal`/`Heartbeat`:
+    * \#2459 Use amino encoding instead of JSON in `SignBytes`.
+    * \#2598 Reorder fields and use fixed sized encoding.
+    * \#2598 Change `Type` field fromt `string` to `byte` and use new
+      `SignedMsgType` to enumerate.
+  * [types] \#2512 Remove the pubkey field from the validator hash
+  * [state] \#2587 Require block.Time of the fist block to be genesis time
 
 * P2P Protocol
 
-
 FEATURES:
+- [crypto/merkle] \#2298 General Merkle Proof scheme for chaining various types of Merkle trees together
+- [abci] \#2557 Add `Codespace` field to `Response{CheckTx, DeliverTx, Query}`
 
 IMPROVEMENTS:
-- [libs/db] \#2371 Output error instead of panic when the given db_backend is not initialised (@bradyjoestar)
-- [mempool] [\#2399](https://github.com/tendermint/tendermint/issues/2399) Make mempool cache a proper LRU (@bradyjoestar)
-- [types] [\#1714](https://github.com/tendermint/tendermint/issues/1714) Add Address to GenesisValidator
-- [metrics] `consensus.block_interval_metrics` is now gauge, not histogram (you will be able to see spikes, if any)
-- [p2p] \#2126 Introduce PeerTransport interface to improve isolation of concerns
+- Additional Metrics
+    - [consensus] [\#2169](https://github.com/cosmos/cosmos-sdk/issues/2169)
+    - [p2p] [\#2169](https://github.com/cosmos/cosmos-sdk/issues/2169)
+- [config] \#2232 Added ValidateBasic method, which performs basic checks
+- [crypto/ed25519] \#2558 Switch to use latest `golang.org/x/crypto` through our fork at
+  github.com/tendermint/crypto
+- [tools] \#2238 Binary dependencies are now locked to a specific git commit
+- [crypto] \#2099 make crypto random use chacha, and have forward secrecy of generated randomness
 
 BUG FIXES:
-- [node] \#2294 Delay starting node until Genesis time
+- [autofile] \#2428 Group.RotateFile need call Flush() before rename (@goolAdapter)
+- [node] \#2434 Make node respond to signal interrupts while sleeping for genesis time
+- [consensus] [\#1690](https://github.com/tendermint/tendermint/issues/1690) wait for
+timeoutPrecommit before starting next round
+- [consensus] [\#1745](https://github.com/tendermint/tendermint/issues/1745) wait for
+Proposal or timeoutProposal before entering prevote
+- [evidence] \#2515 fix db iter leak (@goolAdapter)
+- [common/bit_array] Fixed a bug in the `Or` function
+- [common/bit_array] Fixed a bug in the `Sub` function (@james-ray)
+- [common] \#2534 Make bit array's PickRandom choose uniformly from true bits
+- [consensus] \#1637 Limit the amount of evidence that can be included in a
+  block
+- [p2p] \#2555 fix p2p switch FlushThrottle value (@goolAdapter)
+- [libs/event] \#2518 fix event concurrency flaw (@goolAdapter)
+- [state] \#2616 Pass nil to NewValidatorSet() when genesis file's Validators field is nil

Gopkg.lock

@@ -364,18 +364,6 @@
   pruneopts = "UT"
   revision = "e5840949ff4fff0c56f9b6a541e22b63581ea9df"
 
-[[projects]]
-  branch = "master"
-  digest = "1:087aaa7920e5d0bf79586feb57ce01c35c830396ab4392798112e8aae8c47722"
-  name = "github.com/tendermint/ed25519"
-  packages = [
-    ".",
-    "edwards25519",
-    "extra25519",
-  ]
-  pruneopts = "UT"
-  revision = "d8387025d2b9d158cf4efb07e7ebf814bcce2057"
-
 [[projects]]
   digest = "1:e0a2a4be1e20c305badc2b0a7a9ab7fef6da500763bec23ab81df3b5f9eec9ee"
   name = "github.com/tendermint/go-amino"
@@ -385,14 +373,15 @@
   version = "v0.12.0-rc0"
 
 [[projects]]
-  branch = "master"
-  digest = "1:c31a37cafc12315b8bd745c8ad6a006ac25350472488162a821e557b3e739d67"
+  digest = "1:72b71e3a29775e5752ed7a8012052a3dee165e27ec18cedddae5288058f09acf"
   name = "golang.org/x/crypto"
   packages = [
     "bcrypt",
     "blowfish",
     "chacha20poly1305",
     "curve25519",
+    "ed25519",
+    "ed25519/internal/edwards25519",
     "hkdf",
     "internal/chacha20",
     "internal/subtle",
@@ -405,7 +394,8 @@
     "salsa20/salsa",
   ]
   pruneopts = "UT"
-  revision = "56440b844dfe139a8ac053f4ecac0b20b79058f4"
+  revision = "3764759f34a542a3aef74d6b02e35be7ab893bba"
+  source = "github.com/tendermint/crypto"
 
 [[projects]]
   digest = "1:d36f55a999540d29b6ea3c2ea29d71c76b1d9853fdcd3e5c5cb4836f2ba118f1"
@@ -543,12 +533,11 @@
     "github.com/syndtr/goleveldb/leveldb/iterator",
     "github.com/syndtr/goleveldb/leveldb/opt",
     "github.com/tendermint/btcd/btcec",
-    "github.com/tendermint/ed25519",
-    "github.com/tendermint/ed25519/extra25519",
     "github.com/tendermint/go-amino",
     "golang.org/x/crypto/bcrypt",
     "golang.org/x/crypto/chacha20poly1305",
     "golang.org/x/crypto/curve25519",
+    "golang.org/x/crypto/ed25519",
     "golang.org/x/crypto/hkdf",
     "golang.org/x/crypto/nacl/box",
     "golang.org/x/crypto/nacl/secretbox",

Gopkg.toml

@@ -72,6 +72,11 @@
 ## Some repos dont have releases.
 ## Pin to revision
 
+[[constraint]]
+  name = "golang.org/x/crypto"
+  source = "github.com/tendermint/crypto"
+  revision = "3764759f34a542a3aef74d6b02e35be7ab893bba"
+
 [[override]]
   name = "github.com/jmhodges/levigo"
   revision = "c42d9e0ca023e2198120196f842701bb4c55d7b9"

Makefile

@@ -1,16 +1,17 @@
 GOTOOLS = \
 	github.com/mitchellh/gox \
 	github.com/golang/dep/cmd/dep \
-	gopkg.in/alecthomas/gometalinter.v2 \
+	github.com/alecthomas/gometalinter \
 	github.com/gogo/protobuf/protoc-gen-gogo \
 	github.com/square/certstrap
+GOBIN?=${GOPATH}/bin
 PACKAGES=$(shell go list ./...)
 
 INCLUDE = -I=. -I=${GOPATH}/src -I=${GOPATH}/src/github.com/gogo/protobuf/protobuf
 BUILD_TAGS?='tendermint'
 BUILD_FLAGS = -ldflags "-X github.com/tendermint/tendermint/version.GitCommit=`git rev-parse --short=8 HEAD`"
 
-LINT_FLAGS = --exclude '.*\.pb\.go' --vendor --deadline=600s
+LINT_FLAGS = --exclude '.*\.pb\.go' --exclude 'vendor/*' --vendor --deadline=600s
 
 all: check build test install
 
@@ -23,16 +24,19 @@ check: check_tools get_vendor_deps
 build:
 	CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint/
 
+build_c:
+	CGO_ENABLED=1 go build $(BUILD_FLAGS) -tags "$(BUILD_TAGS) gcc" -o build/tendermint ./cmd/tendermint/
+
 build_race:
 	CGO_ENABLED=0 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint
 
 install:
-	CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
+	CGO_ENABLED=0 go install  $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
 
 ########################################
 ### Protobuf
 
-protoc_all: protoc_libs protoc_abci protoc_grpc
+protoc_all: protoc_libs protoc_merkle protoc_abci protoc_grpc
 
 %.pb.go: %.proto
 	## If you get the following error,
@@ -72,12 +76,13 @@ check_tools:
 
 get_tools:
 	@echo "--> Installing tools"
-	go get -u -v $(GOTOOLS)
-	@gometalinter.v2 --install
+	./scripts/get_tools.sh
+	@echo "--> Downloading linters (this may take awhile)"
+	$(GOPATH)/src/github.com/alecthomas/gometalinter/scripts/install.sh -b $(GOBIN)
 
 update_tools:
 	@echo "--> Updating tools"
-	go get -u -v $(GOTOOLS)
+	./scripts/get_tools.sh
 
 #Update dependencies
 get_vendor_deps:
@@ -134,6 +139,8 @@ grpc_dbserver:
 
 protoc_grpc: rpc/grpc/types.pb.go
 
+protoc_merkle: crypto/merkle/merkle.pb.go
+
 ########################################
 ### Testing
 
@@ -177,6 +184,9 @@ test_p2p:
 	cd ..
 	# requires 'tester' the image from above
 	bash test/p2p/test.sh tester
+	# the `docker cp` takes a really long time; uncomment for debugging
+	#
+	# mkdir -p test/p2p/logs && docker cp rsyslog:/var/log test/p2p/logs
 
 test_integrations:
 	make build_docker_test_image
@@ -219,7 +229,7 @@ fmt:
 
 metalinter:
 	@echo "--> Running linter"
-	@gometalinter.v2 $(LINT_FLAGS) --disable-all  \
+	@gometalinter $(LINT_FLAGS) --disable-all  \
 		--enable=deadcode \
 		--enable=gosimple \
 	 	--enable=misspell \
@@ -248,7 +258,7 @@ metalinter:
 
 metalinter_all:
 	@echo "--> Running linter (all)"
-	gometalinter.v2 $(LINT_FLAGS) --enable-all --disable=lll ./...
+	gometalinter $(LINT_FLAGS) --enable-all --disable=lll ./...
 
 DESTINATION = ./index.html.md
 

README.md

@@ -118,11 +118,12 @@ CHANGELOG even if they don't lead to MINOR version bumps:
 - rpc/client
 - config
 - node
-- libs/bech32
-- libs/common
-- libs/db
-- libs/errors
-- libs/log
+- libs
+  - bech32
+  - common
+  - db
+  - errors
+  - log
 
 Exported objects in these packages that are not covered by the versioning scheme
 are explicitly marked by `// UNSTABLE` in their go doc comment and may change at any

UPGRADING.md

@@ -3,6 +3,12 @@
 This guide provides steps to be followed when you upgrade your applications to
 a newer version of Tendermint Core.
 
+## v0.25.0
+
+This release has minimal impact.
+
+If you use GasWanted in ABCI and want to enforce it, set the MaxGas in the genesis file (default is no max).
+
 ## v0.24.0
 
 New 0.24.0 release contains a lot of changes to the state and types. It's not

abci/cmd/abci-cli/abci-cli.go

@@ -22,6 +22,7 @@ import (
 	servertest "github.com/tendermint/tendermint/abci/tests/server"
 	"github.com/tendermint/tendermint/abci/types"
 	"github.com/tendermint/tendermint/abci/version"
+	"github.com/tendermint/tendermint/crypto/merkle"
 )
 
 // client is a global variable so it can be reused by the console
@@ -100,7 +101,7 @@ type queryResponse struct {
 	Key    []byte
 	Value  []byte
 	Height int64
-	Proof  []byte
+	Proof  *merkle.Proof
 }
 
 func Execute() error {
@@ -748,7 +749,7 @@ func printResponse(cmd *cobra.Command, args []string, rsp response) {
 			fmt.Printf("-> value.hex: %X\n", rsp.Query.Value)
 		}
 		if rsp.Query.Proof != nil {
-			fmt.Printf("-> proof: %X\n", rsp.Query.Proof)
+			fmt.Printf("-> proof: %#v\n", rsp.Query.Proof)
 		}
 	}
 }

abci/example/code/code.go

@@ -6,4 +6,5 @@ const (
 	CodeTypeEncodingError uint32 = 1
 	CodeTypeBadNonce      uint32 = 2
 	CodeTypeUnauthorized  uint32 = 3
+	CodeTypeUnknownError  uint32 = 4
 )

abci/example/kvstore/kvstore.go

@@ -81,7 +81,7 @@ func (app *KVStoreApplication) DeliverTx(tx []byte) types.ResponseDeliverTx {
 	app.state.Size += 1
 
 	tags := []cmn.KVPair{
-		{Key: []byte("app.creator"), Value: []byte("jae")},
+		{Key: []byte("app.creator"), Value: []byte("Cosmoshi Netowoko")},
 		{Key: []byte("app.key"), Value: key},
 	}
 	return types.ResponseDeliverTx{Code: code.CodeTypeOK, Tags: tags}
@@ -114,6 +114,7 @@ func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery type
 		}
 		return
 	} else {
+		resQuery.Key = reqQuery.Data
 		value := app.state.db.Get(prefixKey(reqQuery.Data))
 		resQuery.Value = value
 		if value != nil {

abci/tests/test_cli/ex1.abci.out

@@ -28,6 +28,8 @@
 -> code: OK
 -> log: exists
 -> height: 0
+-> key: abc
+-> key.hex: 616263
 -> value: abc
 -> value.hex: 616263
 
@@ -42,6 +44,8 @@
 -> code: OK
 -> log: exists
 -> height: 0
+-> key: def
+-> key.hex: 646566
 -> value: xyz
 -> value.hex: 78797A
 

abci/types/types.proto

@@ -6,6 +6,7 @@ package types;
 import "github.com/gogo/protobuf/gogoproto/gogo.proto";
 import "google/protobuf/timestamp.proto";
 import "github.com/tendermint/tendermint/libs/common/types.proto";
+import "github.com/tendermint/tendermint/crypto/merkle/merkle.proto";
 
 // This file is copied from http://github.com/tendermint/abci
 // NOTE: When using custom types, mind the warnings.
@@ -154,8 +155,9 @@ message ResponseQuery {
   int64 index = 5;
   bytes key = 6;
   bytes value = 7;
-  bytes proof = 8;
+  merkle.Proof proof = 8;
   int64 height = 9;
+  string codespace = 10;
 }
 
 message ResponseBeginBlock {
@@ -170,6 +172,7 @@ message ResponseCheckTx {
   int64 gas_wanted  = 5;
   int64 gas_used = 6;
   repeated common.KVPair tags = 7 [(gogoproto.nullable)=false, (gogoproto.jsontag)="tags,omitempty"];
+  string codespace = 8;
 }
 
 message ResponseDeliverTx {
@@ -180,6 +183,7 @@ message ResponseDeliverTx {
   int64 gas_wanted = 5;
   int64 gas_used = 6;
   repeated common.KVPair tags = 7 [(gogoproto.nullable)=false, (gogoproto.jsontag)="tags,omitempty"];
+  string codespace = 8;
 }
 
 message ResponseEndBlock {

abci/types/typespb_test.go

@@ -14,6 +14,7 @@ import fmt "fmt"
 import math "math"
 import _ "github.com/gogo/protobuf/gogoproto"
 import _ "github.com/golang/protobuf/ptypes/timestamp"
+import _ "github.com/tendermint/tendermint/crypto/merkle"
 import _ "github.com/tendermint/tendermint/libs/common"
 
 // Reference imports to suppress errors if they are not otherwise used.

benchmarks/codec_test.go

@@ -12,23 +12,27 @@ import (
 	ctypes "github.com/tendermint/tendermint/rpc/core/types"
 )
 
+func testNodeInfo(id p2p.ID) p2p.DefaultNodeInfo {
+	return p2p.DefaultNodeInfo{
+		ID_:        id,
+		Moniker:    "SOMENAME",
+		Network:    "SOMENAME",
+		ListenAddr: "SOMEADDR",
+		Version:    "SOMEVER",
+		Other: p2p.DefaultNodeInfoOther{
+			AminoVersion: "SOMESTRING",
+			P2PVersion:   "OTHERSTRING",
+		},
+	}
+}
+
 func BenchmarkEncodeStatusWire(b *testing.B) {
 	b.StopTimer()
 	cdc := amino.NewCodec()
 	ctypes.RegisterAmino(cdc)
 	nodeKey := p2p.NodeKey{PrivKey: ed25519.GenPrivKey()}
 	status := &ctypes.ResultStatus{
-		NodeInfo: p2p.NodeInfo{
-			ID:         nodeKey.ID(),
-			Moniker:    "SOMENAME",
-			Network:    "SOMENAME",
-			ListenAddr: "SOMEADDR",
-			Version:    "SOMEVER",
-			Other: p2p.NodeInfoOther{
-				AminoVersion: "SOMESTRING",
-				P2PVersion:   "OTHERSTRING",
-			},
-		},
+		NodeInfo: testNodeInfo(nodeKey.ID()),
 		SyncInfo: ctypes.SyncInfo{
 			LatestBlockHash:   []byte("SOMEBYTES"),
 			LatestBlockHeight: 123,
@@ -56,17 +60,7 @@ func BenchmarkEncodeNodeInfoWire(b *testing.B) {
 	cdc := amino.NewCodec()
 	ctypes.RegisterAmino(cdc)
 	nodeKey := p2p.NodeKey{PrivKey: ed25519.GenPrivKey()}
-	nodeInfo := p2p.NodeInfo{
-		ID:         nodeKey.ID(),
-		Moniker:    "SOMENAME",
-		Network:    "SOMENAME",
-		ListenAddr: "SOMEADDR",
-		Version:    "SOMEVER",
-		Other: p2p.NodeInfoOther{
-			AminoVersion: "SOMESTRING",
-			P2PVersion:   "OTHERSTRING",
-		},
-	}
+	nodeInfo := testNodeInfo(nodeKey.ID())
 	b.StartTimer()
 
 	counter := 0
@@ -84,17 +78,7 @@ func BenchmarkEncodeNodeInfoBinary(b *testing.B) {
 	cdc := amino.NewCodec()
 	ctypes.RegisterAmino(cdc)
 	nodeKey := p2p.NodeKey{PrivKey: ed25519.GenPrivKey()}
-	nodeInfo := p2p.NodeInfo{
-		ID:         nodeKey.ID(),
-		Moniker:    "SOMENAME",
-		Network:    "SOMENAME",
-		ListenAddr: "SOMEADDR",
-		Version:    "SOMEVER",
-		Other: p2p.NodeInfoOther{
-			AminoVersion: "SOMESTRING",
-			P2PVersion:   "OTHERSTRING",
-		},
-	}
+	nodeInfo := testNodeInfo(nodeKey.ID())
 	b.StartTimer()
 
 	counter := 0

blockchain/reactor_test.go

@@ -198,7 +198,7 @@ func (tp *bcrTestPeer) TrySend(chID byte, msgBytes []byte) bool {
 }
 
 func (tp *bcrTestPeer) Send(chID byte, msgBytes []byte) bool { return tp.TrySend(chID, msgBytes) }
-func (tp *bcrTestPeer) NodeInfo() p2p.NodeInfo               { return p2p.NodeInfo{} }
+func (tp *bcrTestPeer) NodeInfo() p2p.NodeInfo               { return p2p.DefaultNodeInfo{} }
 func (tp *bcrTestPeer) Status() p2p.ConnectionStatus         { return p2p.ConnectionStatus{} }
 func (tp *bcrTestPeer) ID() p2p.ID                           { return tp.id }
 func (tp *bcrTestPeer) IsOutbound() bool                     { return false }
@@ -206,4 +206,3 @@ func (tp *bcrTestPeer) IsPersistent() bool                   { return true }
 func (tp *bcrTestPeer) Get(s string) interface{}             { return s }
 func (tp *bcrTestPeer) Set(string, interface{})              {}
 func (tp *bcrTestPeer) RemoteIP() net.IP                     { return []byte{127, 0, 0, 1} }
-func (tp *bcrTestPeer) OriginalAddr() *p2p.NetAddress        { return nil }

cmd/tendermint/commands/init.go

@@ -59,8 +59,8 @@ func initFilesWithConfig(config *cfg.Config) error {
 		}
 		genDoc.Validators = []types.GenesisValidator{{
 			Address: pv.GetPubKey().Address(),
-			PubKey: pv.GetPubKey(),
-			Power:  10,
+			PubKey:  pv.GetPubKey(),
+			Power:   10,
 		}}
 
 		if err := genDoc.SaveAs(genFile); err != nil {

cmd/tendermint/commands/lite.go

@@ -26,10 +26,12 @@ just with added trust and running locally.`,
 }
 
 var (
-	listenAddr string
-	nodeAddr   string
-	chainID    string
-	home       string
+	listenAddr         string
+	nodeAddr           string
+	chainID            string
+	home               string
+	maxOpenConnections int
+	cacheSize          int
 )
 
 func init() {
@@ -37,6 +39,8 @@ func init() {
 	LiteCmd.Flags().StringVar(&nodeAddr, "node", "tcp://localhost:26657", "Connect to a Tendermint node at this address")
 	LiteCmd.Flags().StringVar(&chainID, "chain-id", "tendermint", "Specify the Tendermint chain ID")
 	LiteCmd.Flags().StringVar(&home, "home-dir", ".tendermint-lite", "Specify the home directory")
+	LiteCmd.Flags().IntVar(&maxOpenConnections, "max-open-connections", 900, "Maximum number of simultaneous connections (including WebSocket).")
+	LiteCmd.Flags().IntVar(&cacheSize, "cache-size", 10, "Specify the memory trust store cache size")
 }
 
 func ensureAddrHasSchemeOrDefaultToTCP(addr string) (string, error) {
@@ -69,7 +73,7 @@ func runProxy(cmd *cobra.Command, args []string) error {
 	node := rpcclient.NewHTTP(nodeAddr, "/websocket")
 
 	logger.Info("Constructing Verifier...")
-	cert, err := proxy.NewVerifier(chainID, home, node, logger)
+	cert, err := proxy.NewVerifier(chainID, home, node, logger, cacheSize)
 	if err != nil {
 		return cmn.ErrorWrap(err, "constructing Verifier")
 	}
@@ -77,7 +81,7 @@ func runProxy(cmd *cobra.Command, args []string) error {
 	sc := proxy.SecureClient(node, cert)
 
 	logger.Info("Starting proxy...")
-	err = proxy.StartProxy(sc, listenAddr, logger)
+	err = proxy.StartProxy(sc, listenAddr, logger, maxOpenConnections)
 	if err != nil {
 		return cmn.ErrorWrap(err, "starting proxy")
 	}

cmd/tendermint/commands/root.go

@@ -1,6 +1,7 @@
 package commands
 
 import (
+	"fmt"
 	"os"
 
 	"github.com/spf13/cobra"
@@ -35,6 +36,9 @@ func ParseConfig() (*cfg.Config, error) {
 	}
 	conf.SetRoot(conf.RootDir)
 	cfg.EnsureRoot(conf.RootDir)
+	if err = conf.ValidateBasic(); err != nil {
+		return nil, fmt.Errorf("Error in config file: %v", err)
+	}
 	return conf, err
 }
 

cmd/tendermint/commands/run_node.go

@@ -2,6 +2,9 @@ package commands
 
 import (
 	"fmt"
+	"os"
+	"os/signal"
+	"syscall"
 
 	"github.com/spf13/cobra"
 
@@ -49,19 +52,31 @@ func NewRunNodeCmd(nodeProvider nm.NodeProvider) *cobra.Command {
 		Use:   "node",
 		Short: "Run the tendermint node",
 		RunE: func(cmd *cobra.Command, args []string) error {
-			// Create & start node
 			n, err := nodeProvider(config, logger)
 			if err != nil {
 				return fmt.Errorf("Failed to create node: %v", err)
 			}
 
+			// Stop upon receiving SIGTERM or CTRL-C
+			c := make(chan os.Signal, 1)
+			signal.Notify(c, os.Interrupt, syscall.SIGTERM)
+			go func() {
+				for sig := range c {
+					logger.Error(fmt.Sprintf("captured %v, exiting...", sig))
+					if n.IsRunning() {
+						n.Stop()
+					}
+					os.Exit(1)
+				}
+			}()
+
 			if err := n.Start(); err != nil {
 				return fmt.Errorf("Failed to start node: %v", err)
 			}
 			logger.Info("Started node", "nodeInfo", n.Switch().NodeInfo())
 
-			// Trap signal, run forever.
-			n.RunForever()
+			// Run forever
+			select {}
 
 			return nil
 		},

cmd/tendermint/commands/testnet.go

@@ -92,9 +92,9 @@ func testnetFiles(cmd *cobra.Command, args []string) error {
 		pv := privval.LoadFilePV(pvFile)
 		genVals[i] = types.GenesisValidator{
 			Address: pv.GetPubKey().Address(),
-			PubKey: pv.GetPubKey(),
-			Power:  1,
-			Name:   nodeDirName,
+			PubKey:  pv.GetPubKey(),
+			Power:   1,
+			Name:    nodeDirName,
 		}
 	}
 

config/config.go

@@ -5,6 +5,8 @@ import (
 	"os"
 	"path/filepath"
 	"time"
+
+	"github.com/pkg/errors"
 )
 
 const (
@@ -19,7 +21,7 @@ const (
 // generate the config.toml. Please reflect any changes
 // made here in the defaultConfigTemplate constant in
 // config/toml.go
-// NOTE: tmlibs/cli must know to look in the config dir!
+// NOTE: libs/cli must know to look in the config dir!
 var (
 	DefaultTendermintDir = ".tendermint"
 	defaultConfigDir     = "config"
@@ -89,6 +91,27 @@ func (cfg *Config) SetRoot(root string) *Config {
 	return cfg
 }
 
+// ValidateBasic performs basic validation (checking param bounds, etc.) and
+// returns an error if any check fails.
+func (cfg *Config) ValidateBasic() error {
+	if err := cfg.RPC.ValidateBasic(); err != nil {
+		return errors.Wrap(err, "Error in [rpc] section")
+	}
+	if err := cfg.P2P.ValidateBasic(); err != nil {
+		return errors.Wrap(err, "Error in [p2p] section")
+	}
+	if err := cfg.Mempool.ValidateBasic(); err != nil {
+		return errors.Wrap(err, "Error in [mempool] section")
+	}
+	if err := cfg.Consensus.ValidateBasic(); err != nil {
+		return errors.Wrap(err, "Error in [consensus] section")
+	}
+	return errors.Wrap(
+		cfg.Instrumentation.ValidateBasic(),
+		"Error in [instrumentation] section",
+	)
+}
+
 //-----------------------------------------------------------------------------
 // BaseConfig
 
@@ -265,6 +288,18 @@ func TestRPCConfig() *RPCConfig {
 	return cfg
 }
 
+// ValidateBasic performs basic validation (checking param bounds, etc.) and
+// returns an error if any check fails.
+func (cfg *RPCConfig) ValidateBasic() error {
+	if cfg.GRPCMaxOpenConnections < 0 {
+		return errors.New("grpc_max_open_connections can't be negative")
+	}
+	if cfg.MaxOpenConnections < 0 {
+		return errors.New("max_open_connections can't be negative")
+	}
+	return nil
+}
+
 //-----------------------------------------------------------------------------
 // P2PConfig
 
@@ -301,8 +336,8 @@ type P2PConfig struct {
 	// Maximum number of outbound peers to connect to, excluding persistent peers
 	MaxNumOutboundPeers int `mapstructure:"max_num_outbound_peers"`
 
-	// Time to wait before flushing messages out on the connection, in ms
-	FlushThrottleTimeout int `mapstructure:"flush_throttle_timeout"`
+	// Time to wait before flushing messages out on the connection
+	FlushThrottleTimeout time.Duration `mapstructure:"flush_throttle_timeout"`
 
 	// Maximum size of a message packet payload, in bytes
 	MaxPacketMsgPayloadSize int `mapstructure:"max_packet_msg_payload_size"`
@@ -351,7 +386,7 @@ func DefaultP2PConfig() *P2PConfig {
 		AddrBookStrict:          true,
 		MaxNumInboundPeers:      40,
 		MaxNumOutboundPeers:     10,
-		FlushThrottleTimeout:    100,
+		FlushThrottleTimeout:    100 * time.Millisecond,
 		MaxPacketMsgPayloadSize: 1024,    // 1 kB
 		SendRate:                5120000, // 5 mB/s
 		RecvRate:                5120000, // 5 mB/s
@@ -370,7 +405,7 @@ func DefaultP2PConfig() *P2PConfig {
 func TestP2PConfig() *P2PConfig {
 	cfg := DefaultP2PConfig()
 	cfg.ListenAddress = "tcp://0.0.0.0:36656"
-	cfg.FlushThrottleTimeout = 10
+	cfg.FlushThrottleTimeout = 10 * time.Millisecond
 	cfg.AllowDuplicateIP = true
 	return cfg
 }
@@ -380,6 +415,30 @@ func (cfg *P2PConfig) AddrBookFile() string {
 	return rootify(cfg.AddrBook, cfg.RootDir)
 }
 
+// ValidateBasic performs basic validation (checking param bounds, etc.) and
+// returns an error if any check fails.
+func (cfg *P2PConfig) ValidateBasic() error {
+	if cfg.MaxNumInboundPeers < 0 {
+		return errors.New("max_num_inbound_peers can't be negative")
+	}
+	if cfg.MaxNumOutboundPeers < 0 {
+		return errors.New("max_num_outbound_peers can't be negative")
+	}
+	if cfg.FlushThrottleTimeout < 0 {
+		return errors.New("flush_throttle_timeout can't be negative")
+	}
+	if cfg.MaxPacketMsgPayloadSize < 0 {
+		return errors.New("max_packet_msg_payload_size can't be negative")
+	}
+	if cfg.SendRate < 0 {
+		return errors.New("send_rate can't be negative")
+	}
+	if cfg.RecvRate < 0 {
+		return errors.New("recv_rate can't be negative")
+	}
+	return nil
+}
+
 // FuzzConnConfig is a FuzzedConnection configuration.
 type FuzzConnConfig struct {
 	Mode         int
@@ -405,22 +464,20 @@ func DefaultFuzzConnConfig() *FuzzConnConfig {
 
 // MempoolConfig defines the configuration options for the Tendermint mempool
 type MempoolConfig struct {
-	RootDir      string `mapstructure:"home"`
-	Recheck      bool   `mapstructure:"recheck"`
-	RecheckEmpty bool   `mapstructure:"recheck_empty"`
-	Broadcast    bool   `mapstructure:"broadcast"`
-	WalPath      string `mapstructure:"wal_dir"`
-	Size         int    `mapstructure:"size"`
-	CacheSize    int    `mapstructure:"cache_size"`
+	RootDir   string `mapstructure:"home"`
+	Recheck   bool   `mapstructure:"recheck"`
+	Broadcast bool   `mapstructure:"broadcast"`
+	WalPath   string `mapstructure:"wal_dir"`
+	Size      int    `mapstructure:"size"`
+	CacheSize int    `mapstructure:"cache_size"`
 }
 
 // DefaultMempoolConfig returns a default configuration for the Tendermint mempool
 func DefaultMempoolConfig() *MempoolConfig {
 	return &MempoolConfig{
-		Recheck:      true,
-		RecheckEmpty: true,
-		Broadcast:    true,
-		WalPath:      filepath.Join(defaultDataDir, "mempool.wal"),
+		Recheck:   true,
+		Broadcast: true,
+		WalPath:   "",
 		// Each signature verification takes .5ms, size reduced until we implement
 		// ABCI Recheck
 		Size:      5000,
@@ -440,6 +497,18 @@ func (cfg *MempoolConfig) WalDir() string {
 	return rootify(cfg.WalPath, cfg.RootDir)
 }
 
+// ValidateBasic performs basic validation (checking param bounds, etc.) and
+// returns an error if any check fails.
+func (cfg *MempoolConfig) ValidateBasic() error {
+	if cfg.Size < 0 {
+		return errors.New("size can't be negative")
+	}
+	if cfg.CacheSize < 0 {
+		return errors.New("cache_size can't be negative")
+	}
+	return nil
+}
+
 //-----------------------------------------------------------------------------
 // ConsensusConfig
 
@@ -450,72 +519,70 @@ type ConsensusConfig struct {
 	WalPath string `mapstructure:"wal_file"`
 	walFile string // overrides WalPath if set
 
-	// All timeouts are in milliseconds
-	TimeoutPropose        int `mapstructure:"timeout_propose"`
-	TimeoutProposeDelta   int `mapstructure:"timeout_propose_delta"`
-	TimeoutPrevote        int `mapstructure:"timeout_prevote"`
-	TimeoutPrevoteDelta   int `mapstructure:"timeout_prevote_delta"`
-	TimeoutPrecommit      int `mapstructure:"timeout_precommit"`
-	TimeoutPrecommitDelta int `mapstructure:"timeout_precommit_delta"`
-	TimeoutCommit         int `mapstructure:"timeout_commit"`
+	TimeoutPropose        time.Duration `mapstructure:"timeout_propose"`
+	TimeoutProposeDelta   time.Duration `mapstructure:"timeout_propose_delta"`
+	TimeoutPrevote        time.Duration `mapstructure:"timeout_prevote"`
+	TimeoutPrevoteDelta   time.Duration `mapstructure:"timeout_prevote_delta"`
+	TimeoutPrecommit      time.Duration `mapstructure:"timeout_precommit"`
+	TimeoutPrecommitDelta time.Duration `mapstructure:"timeout_precommit_delta"`
+	TimeoutCommit         time.Duration `mapstructure:"timeout_commit"`
 
 	// Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
 	SkipTimeoutCommit bool `mapstructure:"skip_timeout_commit"`
 
-	// EmptyBlocks mode and possible interval between empty blocks in seconds
-	CreateEmptyBlocks         bool `mapstructure:"create_empty_blocks"`
-	CreateEmptyBlocksInterval int  `mapstructure:"create_empty_blocks_interval"`
+	// EmptyBlocks mode and possible interval between empty blocks
+	CreateEmptyBlocks         bool          `mapstructure:"create_empty_blocks"`
+	CreateEmptyBlocksInterval time.Duration `mapstructure:"create_empty_blocks_interval"`
 
-	// Reactor sleep duration parameters are in milliseconds
-	PeerGossipSleepDuration     int `mapstructure:"peer_gossip_sleep_duration"`
-	PeerQueryMaj23SleepDuration int `mapstructure:"peer_query_maj23_sleep_duration"`
+	// Reactor sleep duration parameters
+	PeerGossipSleepDuration     time.Duration `mapstructure:"peer_gossip_sleep_duration"`
+	PeerQueryMaj23SleepDuration time.Duration `mapstructure:"peer_query_maj23_sleep_duration"`
 
-	// Block time parameters in milliseconds. Corresponds to the minimum time increment between consecutive blocks.
-	BlockTimeIota int `mapstructure:"blocktime_iota"`
+	// Block time parameters. Corresponds to the minimum time increment between consecutive blocks.
+	BlockTimeIota time.Duration `mapstructure:"blocktime_iota"`
 }
 
 // DefaultConsensusConfig returns a default configuration for the consensus service
 func DefaultConsensusConfig() *ConsensusConfig {
 	return &ConsensusConfig{
 		WalPath:                     filepath.Join(defaultDataDir, "cs.wal", "wal"),
-		TimeoutPropose:              3000,
-		TimeoutProposeDelta:         500,
-		TimeoutPrevote:              1000,
-		TimeoutPrevoteDelta:         500,
-		TimeoutPrecommit:            1000,
-		TimeoutPrecommitDelta:       500,
-		TimeoutCommit:               1000,
+		TimeoutPropose:              3000 * time.Millisecond,
+		TimeoutProposeDelta:         500 * time.Millisecond,
+		TimeoutPrevote:              1000 * time.Millisecond,
+		TimeoutPrevoteDelta:         500 * time.Millisecond,
+		TimeoutPrecommit:            1000 * time.Millisecond,
+		TimeoutPrecommitDelta:       500 * time.Millisecond,
+		TimeoutCommit:               1000 * time.Millisecond,
 		SkipTimeoutCommit:           false,
 		CreateEmptyBlocks:           true,
-		CreateEmptyBlocksInterval:   0,
-		PeerGossipSleepDuration:     100,
-		PeerQueryMaj23SleepDuration: 2000,
-		BlockTimeIota:               1000,
+		CreateEmptyBlocksInterval:   0 * time.Second,
+		PeerGossipSleepDuration:     100 * time.Millisecond,
+		PeerQueryMaj23SleepDuration: 2000 * time.Millisecond,
+		BlockTimeIota:               1000 * time.Millisecond,
 	}
 }
 
 // TestConsensusConfig returns a configuration for testing the consensus service
 func TestConsensusConfig() *ConsensusConfig {
 	cfg := DefaultConsensusConfig()
-	cfg.TimeoutPropose = 100
-	cfg.TimeoutProposeDelta = 1
-	cfg.TimeoutPrevote = 10
-	cfg.TimeoutPrevoteDelta = 1
-	cfg.TimeoutPrecommit = 10
-	cfg.TimeoutPrecommitDelta = 1
-	cfg.TimeoutCommit = 10
+	cfg.TimeoutPropose = 40 * time.Millisecond
+	cfg.TimeoutProposeDelta = 1 * time.Millisecond
+	cfg.TimeoutPrevote = 10 * time.Millisecond
+	cfg.TimeoutPrevoteDelta = 1 * time.Millisecond
+	cfg.TimeoutPrecommit = 10 * time.Millisecond
+	cfg.TimeoutPrecommitDelta = 1 * time.Millisecond
+	cfg.TimeoutCommit = 10 * time.Millisecond
 	cfg.SkipTimeoutCommit = true
-	cfg.PeerGossipSleepDuration = 5
-	cfg.PeerQueryMaj23SleepDuration = 250
-	cfg.BlockTimeIota = 10
+	cfg.PeerGossipSleepDuration = 5 * time.Millisecond
+	cfg.PeerQueryMaj23SleepDuration = 250 * time.Millisecond
+	cfg.BlockTimeIota = 10 * time.Millisecond
 	return cfg
 }
 
 // MinValidVoteTime returns the minimum acceptable block time.
 // See the [BFT time spec](https://godoc.org/github.com/tendermint/tendermint/docs/spec/consensus/bft-time.md).
 func (cfg *ConsensusConfig) MinValidVoteTime(lastBlockTime time.Time) time.Time {
-	return lastBlockTime.
-		Add(time.Duration(cfg.BlockTimeIota) * time.Millisecond)
+	return lastBlockTime.Add(cfg.BlockTimeIota)
 }
 
 // WaitForTxs returns true if the consensus should wait for transactions before entering the propose step
@@ -523,39 +590,30 @@ func (cfg *ConsensusConfig) WaitForTxs() bool {
 	return !cfg.CreateEmptyBlocks || cfg.CreateEmptyBlocksInterval > 0
 }
 
-// EmptyBlocks returns the amount of time to wait before proposing an empty block or starting the propose timer if there are no txs available
-func (cfg *ConsensusConfig) EmptyBlocksInterval() time.Duration {
-	return time.Duration(cfg.CreateEmptyBlocksInterval) * time.Second
-}
-
 // Propose returns the amount of time to wait for a proposal
 func (cfg *ConsensusConfig) Propose(round int) time.Duration {
-	return time.Duration(cfg.TimeoutPropose+cfg.TimeoutProposeDelta*round) * time.Millisecond
+	return time.Duration(
+		cfg.TimeoutPropose.Nanoseconds()+cfg.TimeoutProposeDelta.Nanoseconds()*int64(round),
+	) * time.Nanosecond
 }
 
 // Prevote returns the amount of time to wait for straggler votes after receiving any +2/3 prevotes
 func (cfg *ConsensusConfig) Prevote(round int) time.Duration {
-	return time.Duration(cfg.TimeoutPrevote+cfg.TimeoutPrevoteDelta*round) * time.Millisecond
+	return time.Duration(
+		cfg.TimeoutPrevote.Nanoseconds()+cfg.TimeoutPrevoteDelta.Nanoseconds()*int64(round),
+	) * time.Nanosecond
 }
 
 // Precommit returns the amount of time to wait for straggler votes after receiving any +2/3 precommits
 func (cfg *ConsensusConfig) Precommit(round int) time.Duration {
-	return time.Duration(cfg.TimeoutPrecommit+cfg.TimeoutPrecommitDelta*round) * time.Millisecond
+	return time.Duration(
+		cfg.TimeoutPrecommit.Nanoseconds()+cfg.TimeoutPrecommitDelta.Nanoseconds()*int64(round),
+	) * time.Nanosecond
 }
 
 // Commit returns the amount of time to wait for straggler votes after receiving +2/3 precommits for a single block (ie. a commit).
 func (cfg *ConsensusConfig) Commit(t time.Time) time.Time {
-	return t.Add(time.Duration(cfg.TimeoutCommit) * time.Millisecond)
-}
-
-// PeerGossipSleep returns the amount of time to sleep if there is nothing to send from the ConsensusReactor
-func (cfg *ConsensusConfig) PeerGossipSleep() time.Duration {
-	return time.Duration(cfg.PeerGossipSleepDuration) * time.Millisecond
-}
-
-// PeerQueryMaj23Sleep returns the amount of time to sleep after each VoteSetMaj23Message is sent in the ConsensusReactor
-func (cfg *ConsensusConfig) PeerQueryMaj23Sleep() time.Duration {
-	return time.Duration(cfg.PeerQueryMaj23SleepDuration) * time.Millisecond
+	return t.Add(cfg.TimeoutCommit)
 }
 
 // WalFile returns the full path to the write-ahead log file
@@ -571,6 +629,45 @@ func (cfg *ConsensusConfig) SetWalFile(walFile string) {
 	cfg.walFile = walFile
 }
 
+// ValidateBasic performs basic validation (checking param bounds, etc.) and
+// returns an error if any check fails.
+func (cfg *ConsensusConfig) ValidateBasic() error {
+	if cfg.TimeoutPropose < 0 {
+		return errors.New("timeout_propose can't be negative")
+	}
+	if cfg.TimeoutProposeDelta < 0 {
+		return errors.New("timeout_propose_delta can't be negative")
+	}
+	if cfg.TimeoutPrevote < 0 {
+		return errors.New("timeout_prevote can't be negative")
+	}
+	if cfg.TimeoutPrevoteDelta < 0 {
+		return errors.New("timeout_prevote_delta can't be negative")
+	}
+	if cfg.TimeoutPrecommit < 0 {
+		return errors.New("timeout_precommit can't be negative")
+	}
+	if cfg.TimeoutPrecommitDelta < 0 {
+		return errors.New("timeout_precommit_delta can't be negative")
+	}
+	if cfg.TimeoutCommit < 0 {
+		return errors.New("timeout_commit can't be negative")
+	}
+	if cfg.CreateEmptyBlocksInterval < 0 {
+		return errors.New("create_empty_blocks_interval can't be negative")
+	}
+	if cfg.PeerGossipSleepDuration < 0 {
+		return errors.New("peer_gossip_sleep_duration can't be negative")
+	}
+	if cfg.PeerQueryMaj23SleepDuration < 0 {
+		return errors.New("peer_query_maj23_sleep_duration can't be negative")
+	}
+	if cfg.BlockTimeIota < 0 {
+		return errors.New("blocktime_iota can't be negative")
+	}
+	return nil
+}
+
 //-----------------------------------------------------------------------------
 // TxIndexConfig
 
@@ -634,6 +731,9 @@ type InstrumentationConfig struct {
 	// you increase your OS limits.
 	// 0 - unlimited.
 	MaxOpenConnections int `mapstructure:"max_open_connections"`
+
+	// Tendermint instrumentation namespace.
+	Namespace string `mapstructure:"namespace"`
 }
 
 // DefaultInstrumentationConfig returns a default configuration for metrics
@@ -643,6 +743,7 @@ func DefaultInstrumentationConfig() *InstrumentationConfig {
 		Prometheus:           false,
 		PrometheusListenAddr: ":26660",
 		MaxOpenConnections:   3,
+		Namespace:            "tendermint",
 	}
 }
 
@@ -652,6 +753,15 @@ func TestInstrumentationConfig() *InstrumentationConfig {
 	return DefaultInstrumentationConfig()
 }
 
+// ValidateBasic performs basic validation (checking param bounds, etc.) and
+// returns an error if any check fails.
+func (cfg *InstrumentationConfig) ValidateBasic() error {
+	if cfg.MaxOpenConnections < 0 {
+		return errors.New("max_open_connections can't be negative")
+	}
+	return nil
+}
+
 //-----------------------------------------------------------------------------
 // Utils
 

config/config_test.go

@@ -2,6 +2,7 @@ package config
 
 import (
 	"testing"
+	"time"
 
 	"github.com/stretchr/testify/assert"
 )
@@ -26,3 +27,12 @@ func TestDefaultConfig(t *testing.T) {
 	assert.Equal("/foo/wal/mem", cfg.Mempool.WalDir())
 
 }
+
+func TestConfigValidateBasic(t *testing.T) {
+	cfg := DefaultConfig()
+	assert.NoError(t, cfg.ValidateBasic())
+
+	// tamper with timeout_propose
+	cfg.Consensus.TimeoutPropose = -10 * time.Second
+	assert.Error(t, cfg.ValidateBasic())
+}

config/toml.go

@@ -99,7 +99,7 @@ priv_validator_file = "{{ js .BaseConfig.PrivValidator }}"
 priv_validator_laddr = "{{ .BaseConfig.PrivValidatorListenAddr }}"
 
 # Path to the JSON file containing the private key to use for node authentication in the p2p protocol
-node_key_file = "{{ js .BaseConfig.NodeKey}}"
+node_key_file = "{{ js .BaseConfig.NodeKey }}"
 
 # Mechanism to connect to the ABCI application: socket | grpc
 abci = "{{ .BaseConfig.ABCI }}"
@@ -172,15 +172,15 @@ addr_book_file = "{{ js .P2P.AddrBook }}"
 # Set false for private or local networks
 addr_book_strict = {{ .P2P.AddrBookStrict }}
 
-# Time to wait before flushing messages out on the connection, in ms
-flush_throttle_timeout = {{ .P2P.FlushThrottleTimeout }}
-
 # Maximum number of inbound peers
 max_num_inbound_peers = {{ .P2P.MaxNumInboundPeers }}
 
 # Maximum number of outbound peers to connect to, excluding persistent peers
 max_num_outbound_peers = {{ .P2P.MaxNumOutboundPeers }}
 
+# Time to wait before flushing messages out on the connection
+flush_throttle_timeout = "{{ .P2P.FlushThrottleTimeout }}"
+
 # Maximum size of a message packet payload, in bytes
 max_packet_msg_payload_size = {{ .P2P.MaxPacketMsgPayloadSize }}
 
@@ -202,11 +202,17 @@ seed_mode = {{ .P2P.SeedMode }}
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = "{{ .P2P.PrivatePeerIDs }}"
 
+# Toggle to disable guard against peers connecting from the same ip.
+allow_duplicate_ip = {{ .P2P.AllowDuplicateIP }}
+
+# Peer connection configuration.
+handshake_timeout = "{{ .P2P.HandshakeTimeout }}"
+dial_timeout = "{{ .P2P.DialTimeout }}"
+
 ##### mempool configuration options #####
 [mempool]
 
 recheck = {{ .Mempool.Recheck }}
-recheck_empty = {{ .Mempool.RecheckEmpty }}
 broadcast = {{ .Mempool.Broadcast }}
 wal_dir = "{{ js .Mempool.WalPath }}"
 
@@ -221,25 +227,24 @@ cache_size = {{ .Mempool.CacheSize }}
 
 wal_file = "{{ js .Consensus.WalPath }}"
 
-# All timeouts are in milliseconds
-timeout_propose = {{ .Consensus.TimeoutPropose }}
-timeout_propose_delta = {{ .Consensus.TimeoutProposeDelta }}
-timeout_prevote = {{ .Consensus.TimeoutPrevote }}
-timeout_prevote_delta = {{ .Consensus.TimeoutPrevoteDelta }}
-timeout_precommit = {{ .Consensus.TimeoutPrecommit }}
-timeout_precommit_delta = {{ .Consensus.TimeoutPrecommitDelta }}
-timeout_commit = {{ .Consensus.TimeoutCommit }}
+timeout_propose = "{{ .Consensus.TimeoutPropose }}"
+timeout_propose_delta = "{{ .Consensus.TimeoutProposeDelta }}"
+timeout_prevote = "{{ .Consensus.TimeoutPrevote }}"
+timeout_prevote_delta = "{{ .Consensus.TimeoutPrevoteDelta }}"
+timeout_precommit = "{{ .Consensus.TimeoutPrecommit }}"
+timeout_precommit_delta = "{{ .Consensus.TimeoutPrecommitDelta }}"
+timeout_commit = "{{ .Consensus.TimeoutCommit }}"
 
 # Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
 skip_timeout_commit = {{ .Consensus.SkipTimeoutCommit }}
 
-# EmptyBlocks mode and possible interval between empty blocks in seconds
+# EmptyBlocks mode and possible interval between empty blocks
 create_empty_blocks = {{ .Consensus.CreateEmptyBlocks }}
-create_empty_blocks_interval = {{ .Consensus.CreateEmptyBlocksInterval }}
+create_empty_blocks_interval = "{{ .Consensus.CreateEmptyBlocksInterval }}"
 
-# Reactor sleep duration parameters are in milliseconds
-peer_gossip_sleep_duration = {{ .Consensus.PeerGossipSleepDuration }}
-peer_query_maj23_sleep_duration = {{ .Consensus.PeerQueryMaj23SleepDuration }}
+# Reactor sleep duration parameters
+peer_gossip_sleep_duration = "{{ .Consensus.PeerGossipSleepDuration }}"
+peer_query_maj23_sleep_duration = "{{ .Consensus.PeerQueryMaj23SleepDuration }}"
 
 ##### transactions indexer configuration options #####
 [tx_index]
@@ -284,6 +289,9 @@ prometheus_listen_addr = "{{ .Instrumentation.PrometheusListenAddr }}"
 # you increase your OS limits.
 # 0 - unlimited.
 max_open_connections = {{ .Instrumentation.MaxOpenConnections }}
+
+# Instrumentation namespace
+namespace = "{{ .Instrumentation.Namespace }}"
 `
 
 /****** these are for test settings ***********/
@@ -334,7 +342,7 @@ func ResetTestRoot(testName string) *Config {
 }
 
 var testGenesis = `{
-  "genesis_time": "0001-01-01T00:00:00.000Z",
+  "genesis_time": "2017-10-10T08:20:13.695936996Z",
   "chain_id": "tendermint_test",
   "validators": [
     {

consensus/byzantine_test.go

@@ -226,8 +226,8 @@ func sendProposalAndParts(height int64, round int, cs *ConsensusState, peer p2p.
 
 	// votes
 	cs.mtx.Lock()
-	prevote, _ := cs.signVote(types.VoteTypePrevote, blockHash, parts.Header())
-	precommit, _ := cs.signVote(types.VoteTypePrecommit, blockHash, parts.Header())
+	prevote, _ := cs.signVote(types.PrevoteType, blockHash, parts.Header())
+	precommit, _ := cs.signVote(types.PrecommitType, blockHash, parts.Header())
 	cs.mtx.Unlock()
 
 	peer.Send(VoteChannel, cdc.MustMarshalBinaryBare(&VoteMessage{prevote}))

consensus/common_test.go

@@ -7,6 +7,7 @@ import (
 	"io/ioutil"
 	"os"
 	"path"
+	"reflect"
 	"sort"
 	"sync"
 	"testing"
@@ -38,8 +39,8 @@ const (
 )
 
 // genesis, chain_id, priv_val
-var config *cfg.Config              // NOTE: must be reset for each _test.go file
-var ensureTimeout = time.Second * 1 // must be in seconds because CreateEmptyBlocksInterval is
+var config *cfg.Config // NOTE: must be reset for each _test.go file
+var ensureTimeout = time.Millisecond * 100
 
 func ensureDir(dir string, mode os.FileMode) {
 	if err := cmn.EnsureDir(dir, mode); err != nil {
@@ -70,7 +71,7 @@ func NewValidatorStub(privValidator types.PrivValidator, valIndex int) *validato
 	}
 }
 
-func (vs *validatorStub) signVote(voteType byte, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
+func (vs *validatorStub) signVote(voteType types.SignedMsgType, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
 	vote := &types.Vote{
 		ValidatorIndex:   vs.Index,
 		ValidatorAddress: vs.PrivValidator.GetAddress(),
@@ -85,7 +86,7 @@ func (vs *validatorStub) signVote(voteType byte, hash []byte, header types.PartS
 }
 
 // Sign vote for type/hash/header
-func signVote(vs *validatorStub, voteType byte, hash []byte, header types.PartSetHeader) *types.Vote {
+func signVote(vs *validatorStub, voteType types.SignedMsgType, hash []byte, header types.PartSetHeader) *types.Vote {
 	v, err := vs.signVote(voteType, hash, header)
 	if err != nil {
 		panic(fmt.Errorf("failed to sign vote: %v", err))
@@ -93,7 +94,7 @@ func signVote(vs *validatorStub, voteType byte, hash []byte, header types.PartSe
 	return v
 }
 
-func signVotes(voteType byte, hash []byte, header types.PartSetHeader, vss ...*validatorStub) []*types.Vote {
+func signVotes(voteType types.SignedMsgType, hash []byte, header types.PartSetHeader, vss ...*validatorStub) []*types.Vote {
 	votes := make([]*types.Vote, len(vss))
 	for i, vs := range vss {
 		votes[i] = signVote(vs, voteType, hash, header)
@@ -143,7 +144,7 @@ func addVotes(to *ConsensusState, votes ...*types.Vote) {
 	}
 }
 
-func signAddVotes(to *ConsensusState, voteType byte, hash []byte, header types.PartSetHeader, vss ...*validatorStub) {
+func signAddVotes(to *ConsensusState, voteType types.SignedMsgType, hash []byte, header types.PartSetHeader, vss ...*validatorStub) {
 	votes := signVotes(voteType, hash, header, vss...)
 	addVotes(to, votes...)
 }
@@ -306,23 +307,191 @@ func randConsensusState(nValidators int) (*ConsensusState, []*validatorStub) {
 
 //-------------------------------------------------------------------------------
 
-func ensureNoNewStep(stepCh <-chan interface{}) {
-	timer := time.NewTimer(ensureTimeout)
+func ensureNoNewEvent(ch <-chan interface{}, timeout time.Duration,
+	errorMessage string) {
 	select {
-	case <-timer.C:
+	case <-time.After(timeout):
 		break
-	case <-stepCh:
-		panic("We should be stuck waiting, not moving to the next step")
+	case <-ch:
+		panic(errorMessage)
 	}
 }
 
-func ensureNewStep(stepCh <-chan interface{}) {
-	timer := time.NewTimer(ensureTimeout)
+func ensureNoNewEventOnChannel(ch <-chan interface{}) {
+	ensureNoNewEvent(
+		ch,
+		ensureTimeout,
+		"We should be stuck waiting, not receiving new event on the channel")
+}
+
+func ensureNoNewRoundStep(stepCh <-chan interface{}) {
+	ensureNoNewEvent(
+		stepCh,
+		ensureTimeout,
+		"We should be stuck waiting, not receiving NewRoundStep event")
+}
+
+func ensureNoNewUnlock(unlockCh <-chan interface{}) {
+	ensureNoNewEvent(
+		unlockCh,
+		ensureTimeout,
+		"We should be stuck waiting, not receiving Unlock event")
+}
+
+func ensureNoNewTimeout(stepCh <-chan interface{}, timeout int64) {
+	timeoutDuration := time.Duration(timeout*5) * time.Nanosecond
+	ensureNoNewEvent(
+		stepCh,
+		timeoutDuration,
+		"We should be stuck waiting, not receiving NewTimeout event")
+}
+
+func ensureNewEvent(
+	ch <-chan interface{},
+	height int64,
+	round int,
+	timeout time.Duration,
+	errorMessage string) {
+
 	select {
-	case <-timer.C:
-		panic("We shouldnt be stuck waiting")
-	case <-stepCh:
+	case <-time.After(timeout):
+		panic(errorMessage)
+	case ev := <-ch:
+		rs, ok := ev.(types.EventDataRoundState)
+		if !ok {
+			panic(
+				fmt.Sprintf(
+					"expected a EventDataRoundState, got %v.Wrong subscription channel?",
+					reflect.TypeOf(rs)))
+		}
+		if rs.Height != height {
+			panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
+		}
+		if rs.Round != round {
+			panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
+		}
+		// TODO: We could check also for a step at this point!
+	}
+}
+
+func ensureNewRoundStep(stepCh <-chan interface{}, height int64, round int) {
+	ensureNewEvent(
+		stepCh,
+		height,
+		round,
+		ensureTimeout,
+		"Timeout expired while waiting for NewStep event")
+}
+
+func ensureNewVote(voteCh <-chan interface{}, height int64, round int) {
+	select {
+	case <-time.After(ensureTimeout):
 		break
+	case v := <-voteCh:
+		edv, ok := v.(types.EventDataVote)
+		if !ok {
+			panic(fmt.Sprintf("expected a *types.Vote, "+
+				"got %v. wrong subscription channel?",
+				reflect.TypeOf(v)))
+		}
+		vote := edv.Vote
+		if vote.Height != height {
+			panic(fmt.Sprintf("expected height %v, got %v", height, vote.Height))
+		}
+		if vote.Round != round {
+			panic(fmt.Sprintf("expected round %v, got %v", round, vote.Round))
+		}
+	}
+}
+
+func ensureNewRound(roundCh <-chan interface{}, height int64, round int) {
+	ensureNewEvent(roundCh, height, round, ensureTimeout,
+		"Timeout expired while waiting for NewRound event")
+}
+
+func ensureNewTimeout(timeoutCh <-chan interface{}, height int64, round int, timeout int64) {
+	timeoutDuration := time.Duration(timeout*3) * time.Nanosecond
+	ensureNewEvent(timeoutCh, height, round, timeoutDuration,
+		"Timeout expired while waiting for NewTimeout event")
+}
+
+func ensureNewProposal(proposalCh <-chan interface{}, height int64, round int) {
+	ensureNewEvent(proposalCh, height, round, ensureTimeout,
+		"Timeout expired while waiting for NewProposal event")
+}
+
+func ensureNewBlock(blockCh <-chan interface{}, height int64) {
+	select {
+	case <-time.After(ensureTimeout):
+		panic("Timeout expired while waiting for NewBlock event")
+	case ev := <-blockCh:
+		block, ok := ev.(types.EventDataNewBlock)
+		if !ok {
+			panic(fmt.Sprintf("expected a *types.EventDataNewBlock, "+
+				"got %v. wrong subscription channel?",
+				reflect.TypeOf(block)))
+		}
+		if block.Block.Height != height {
+			panic(fmt.Sprintf("expected height %v, got %v", height, block.Block.Height))
+		}
+	}
+}
+
+func ensureNewBlockHeader(blockCh <-chan interface{}, height int64, blockHash cmn.HexBytes) {
+	select {
+	case <-time.After(ensureTimeout):
+		panic("Timeout expired while waiting for NewBlockHeader event")
+	case ev := <-blockCh:
+		blockHeader, ok := ev.(types.EventDataNewBlockHeader)
+		if !ok {
+			panic(fmt.Sprintf("expected a *types.EventDataNewBlockHeader, "+
+				"got %v. wrong subscription channel?",
+				reflect.TypeOf(blockHeader)))
+		}
+		if blockHeader.Header.Height != height {
+			panic(fmt.Sprintf("expected height %v, got %v", height, blockHeader.Header.Height))
+		}
+		if !bytes.Equal(blockHeader.Header.Hash(), blockHash) {
+			panic(fmt.Sprintf("expected header %X, got %X", blockHash, blockHeader.Header.Hash()))
+		}
+	}
+}
+
+func ensureNewUnlock(unlockCh <-chan interface{}, height int64, round int) {
+	ensureNewEvent(unlockCh, height, round, ensureTimeout,
+		"Timeout expired while waiting for NewUnlock event")
+}
+
+func ensureVote(voteCh <-chan interface{}, height int64, round int,
+	voteType types.SignedMsgType) {
+	select {
+	case <-time.After(ensureTimeout):
+		panic("Timeout expired while waiting for NewVote event")
+	case v := <-voteCh:
+		edv, ok := v.(types.EventDataVote)
+		if !ok {
+			panic(fmt.Sprintf("expected a *types.Vote, "+
+				"got %v. wrong subscription channel?",
+				reflect.TypeOf(v)))
+		}
+		vote := edv.Vote
+		if vote.Height != height {
+			panic(fmt.Sprintf("expected height %v, got %v", height, vote.Height))
+		}
+		if vote.Round != round {
+			panic(fmt.Sprintf("expected round %v, got %v", round, vote.Round))
+		}
+		if vote.Type != voteType {
+			panic(fmt.Sprintf("expected type %v, got %v", voteType, vote.Type))
+		}
+	}
+}
+
+func ensureNewEventOnChannel(ch <-chan interface{}) {
+	select {
+	case <-time.After(ensureTimeout):
+		panic("Timeout expired while waiting for new activity on the channel")
+	case <-ch:
 	}
 }
 
@@ -399,7 +568,7 @@ func randConsensusNetWithPeers(nValidators, nPeers int, testName string, tickerF
 
 func getSwitchIndex(switches []*p2p.Switch, peer p2p.Peer) int {
 	for i, s := range switches {
-		if peer.NodeInfo().ID == s.NodeInfo().ID {
+		if peer.NodeInfo().ID() == s.NodeInfo().ID() {
 			return i
 		}
 	}

consensus/mempool_test.go

@@ -28,17 +28,17 @@ func TestMempoolNoProgressUntilTxsAvailable(t *testing.T) {
 	newBlockCh := subscribe(cs.eventBus, types.EventQueryNewBlock)
 	startTestRound(cs, height, round)
 
-	ensureNewStep(newBlockCh) // first block gets committed
-	ensureNoNewStep(newBlockCh)
+	ensureNewEventOnChannel(newBlockCh) // first block gets committed
+	ensureNoNewEventOnChannel(newBlockCh)
 	deliverTxsRange(cs, 0, 1)
-	ensureNewStep(newBlockCh) // commit txs
-	ensureNewStep(newBlockCh) // commit updated app hash
-	ensureNoNewStep(newBlockCh)
+	ensureNewEventOnChannel(newBlockCh) // commit txs
+	ensureNewEventOnChannel(newBlockCh) // commit updated app hash
+	ensureNoNewEventOnChannel(newBlockCh)
 }
 
 func TestMempoolProgressAfterCreateEmptyBlocksInterval(t *testing.T) {
 	config := ResetConfig("consensus_mempool_txs_available_test")
-	config.Consensus.CreateEmptyBlocksInterval = int(ensureTimeout.Seconds())
+	config.Consensus.CreateEmptyBlocksInterval = ensureTimeout
 	state, privVals := randGenesisState(1, false, 10)
 	cs := newConsensusStateWithConfig(config, state, privVals[0], NewCounterApplication())
 	cs.mempool.EnableTxsAvailable()
@@ -46,9 +46,9 @@ func TestMempoolProgressAfterCreateEmptyBlocksInterval(t *testing.T) {
 	newBlockCh := subscribe(cs.eventBus, types.EventQueryNewBlock)
 	startTestRound(cs, height, round)
 
-	ensureNewStep(newBlockCh)   // first block gets committed
-	ensureNoNewStep(newBlockCh) // then we dont make a block ...
-	ensureNewStep(newBlockCh)   // until the CreateEmptyBlocksInterval has passed
+	ensureNewEventOnChannel(newBlockCh)   // first block gets committed
+	ensureNoNewEventOnChannel(newBlockCh) // then we dont make a block ...
+	ensureNewEventOnChannel(newBlockCh)   // until the CreateEmptyBlocksInterval has passed
 }
 
 func TestMempoolProgressInHigherRound(t *testing.T) {
@@ -72,13 +72,19 @@ func TestMempoolProgressInHigherRound(t *testing.T) {
 	}
 	startTestRound(cs, height, round)
 
-	ensureNewStep(newRoundCh) // first round at first height
-	ensureNewStep(newBlockCh) // first block gets committed
-	ensureNewStep(newRoundCh) // first round at next height
-	deliverTxsRange(cs, 0, 1) // we deliver txs, but dont set a proposal so we get the next round
-	<-timeoutCh
-	ensureNewStep(newRoundCh) // wait for the next round
-	ensureNewStep(newBlockCh) // now we can commit the block
+	ensureNewRoundStep(newRoundCh, height, round) // first round at first height
+	ensureNewEventOnChannel(newBlockCh)           // first block gets committed
+
+	height = height + 1 // moving to the next height
+	round = 0
+
+	ensureNewRoundStep(newRoundCh, height, round) // first round at next height
+	deliverTxsRange(cs, 0, 1)                     // we deliver txs, but dont set a proposal so we get the next round
+	ensureNewTimeout(timeoutCh, height, round, cs.config.TimeoutPropose.Nanoseconds())
+
+	round = round + 1                             // moving to the next round
+	ensureNewRoundStep(newRoundCh, height, round) // wait for the next round
+	ensureNewEventOnChannel(newBlockCh)           // now we can commit the block
 }
 
 func deliverTxsRange(cs *ConsensusState, start, end int) {

consensus/metrics.go

@@ -8,6 +8,8 @@ import (
 	stdprometheus "github.com/prometheus/client_golang/prometheus"
 )
 
+const MetricsSubsystem = "consensus"
+
 // Metrics contains metrics exposed by this package.
 type Metrics struct {
 	// Height of the chain.
@@ -38,75 +40,111 @@ type Metrics struct {
 	BlockSizeBytes metrics.Gauge
 	// Total number of transactions.
 	TotalTxs metrics.Gauge
+	// The latest block height.
+	CommittedHeight metrics.Gauge
+	// Whether or not a node is fast syncing. 1 if yes, 0 if no.
+	FastSyncing metrics.Gauge
+
+	// Number of blockparts transmitted by peer.
+	BlockParts metrics.Counter
 }
 
 // PrometheusMetrics returns Metrics build using Prometheus client library.
-func PrometheusMetrics() *Metrics {
+func PrometheusMetrics(namespace string) *Metrics {
 	return &Metrics{
 		Height: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "height",
 			Help:      "Height of the chain.",
 		}, []string{}),
 		Rounds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "rounds",
 			Help:      "Number of rounds.",
 		}, []string{}),
 
 		Validators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "validators",
 			Help:      "Number of validators.",
 		}, []string{}),
 		ValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "validators_power",
 			Help:      "Total power of all validators.",
 		}, []string{}),
 		MissingValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "missing_validators",
 			Help:      "Number of validators who did not sign.",
 		}, []string{}),
 		MissingValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "missing_validators_power",
 			Help:      "Total power of the missing validators.",
 		}, []string{}),
 		ByzantineValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "byzantine_validators",
 			Help:      "Number of validators who tried to double sign.",
 		}, []string{}),
 		ByzantineValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "byzantine_validators_power",
 			Help:      "Total power of the byzantine validators.",
 		}, []string{}),
 
-
 		BlockIntervalSeconds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "block_interval_seconds",
 			Help:      "Time between this and the last block.",
 		}, []string{}),
 
 		NumTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "num_txs",
 			Help:      "Number of transactions.",
 		}, []string{}),
 		BlockSizeBytes: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "block_size_bytes",
 			Help:      "Size of the block.",
 		}, []string{}),
 		TotalTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
-			Subsystem: "consensus",
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
 			Name:      "total_txs",
 			Help:      "Total number of transactions.",
 		}, []string{}),
+		CommittedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
+			Name:      "latest_block_height",
+			Help:      "The latest block height.",
+		}, []string{}),
+		FastSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
+			Name:      "fast_syncing",
+			Help:      "Whether or not a node is fast syncing. 1 if yes, 0 if no.",
+		}, []string{}),
+		BlockParts: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
+			Namespace: namespace,
+			Subsystem: MetricsSubsystem,
+			Name:      "block_parts",
+			Help:      "Number of blockparts transmitted by peer.",
+		}, []string{"peer_id"}),
 	}
 }
 
@@ -126,8 +164,11 @@ func NopMetrics() *Metrics {
 
 		BlockIntervalSeconds: discard.NewGauge(),
 
-		NumTxs:         discard.NewGauge(),
-		BlockSizeBytes: discard.NewGauge(),
-		TotalTxs:       discard.NewGauge(),
+		NumTxs:          discard.NewGauge(),
+		BlockSizeBytes:  discard.NewGauge(),
+		TotalTxs:        discard.NewGauge(),
+		CommittedHeight: discard.NewGauge(),
+		FastSyncing:     discard.NewGauge(),
+		BlockParts:      discard.NewCounter(),
 	}
 }

consensus/reactor.go

@@ -8,7 +8,7 @@ import (
 
 	"github.com/pkg/errors"
 
-	amino "github.com/tendermint/go-amino"
+	"github.com/tendermint/go-amino"
 
 	cstypes "github.com/tendermint/tendermint/consensus/types"
 	cmn "github.com/tendermint/tendermint/libs/common"
@@ -29,6 +29,7 @@ const (
 	maxMsgSize = 1048576 // 1MB; NOTE/TODO: keep in sync with types.PartSet sizes.
 
 	blocksToContributeToBecomeGoodPeer = 10000
+	votesToContributeToBecomeGoodPeer  = 10000
 )
 
 //-----------------------------------------------------------------------------
@@ -42,16 +43,27 @@ type ConsensusReactor struct {
 	mtx      sync.RWMutex
 	fastSync bool
 	eventBus *types.EventBus
+
+	metrics *Metrics
 }
 
+type ReactorOption func(*ConsensusReactor)
+
 // NewConsensusReactor returns a new ConsensusReactor with the given
 // consensusState.
-func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *ConsensusReactor {
+func NewConsensusReactor(consensusState *ConsensusState, fastSync bool, options ...ReactorOption) *ConsensusReactor {
 	conR := &ConsensusReactor{
 		conS:     consensusState,
 		fastSync: fastSync,
+		metrics:  NopMetrics(),
 	}
+	conR.updateFastSyncingMetric()
 	conR.BaseReactor = *p2p.NewBaseReactor("ConsensusReactor", conR)
+
+	for _, option := range options {
+		option(conR)
+	}
+
 	return conR
 }
 
@@ -60,6 +72,9 @@ func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *Consens
 func (conR *ConsensusReactor) OnStart() error {
 	conR.Logger.Info("ConsensusReactor ", "fastSync", conR.FastSync())
 
+	// start routine that computes peer statistics for evaluating peer quality
+	go conR.peerStatsRoutine()
+
 	conR.subscribeToBroadcastEvents()
 
 	if !conR.FastSync() {
@@ -94,6 +109,7 @@ func (conR *ConsensusReactor) SwitchToConsensus(state sm.State, blocksSynced int
 	conR.mtx.Lock()
 	conR.fastSync = false
 	conR.mtx.Unlock()
+	conR.metrics.FastSyncing.Set(0)
 
 	if blocksSynced > 0 {
 		// dont bother with the WAL if we fast synced
@@ -221,9 +237,9 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
 			// (and consequently shows which we don't have)
 			var ourVotes *cmn.BitArray
 			switch msg.Type {
-			case types.VoteTypePrevote:
+			case types.PrevoteType:
 				ourVotes = votes.Prevotes(msg.Round).BitArrayByBlockID(msg.BlockID)
-			case types.VoteTypePrecommit:
+			case types.PrecommitType:
 				ourVotes = votes.Precommits(msg.Round).BitArrayByBlockID(msg.BlockID)
 			default:
 				conR.Logger.Error("Bad VoteSetBitsMessage field Type")
@@ -258,9 +274,7 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
 			ps.ApplyProposalPOLMessage(msg)
 		case *BlockPartMessage:
 			ps.SetHasProposalBlockPart(msg.Height, msg.Round, msg.Part.Index)
-			if numBlocks := ps.RecordBlockPart(msg); numBlocks%blocksToContributeToBecomeGoodPeer == 0 {
-				conR.Switch.MarkPeerAsGood(src)
-			}
+			conR.metrics.BlockParts.With("peer_id", string(src.ID())).Add(1)
 			conR.conS.peerMsgQueue <- msgInfo{msg, src.ID()}
 		default:
 			conR.Logger.Error(fmt.Sprintf("Unknown message type %v", reflect.TypeOf(msg)))
@@ -280,9 +294,6 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
 			ps.EnsureVoteBitArrays(height, valSize)
 			ps.EnsureVoteBitArrays(height-1, lastCommitSize)
 			ps.SetHasVote(msg.Vote)
-			if blocks := ps.RecordVote(msg.Vote); blocks%blocksToContributeToBecomeGoodPeer == 0 {
-				conR.Switch.MarkPeerAsGood(src)
-			}
 
 			cs.peerMsgQueue <- msgInfo{msg, src.ID()}
 
@@ -306,9 +317,9 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
 			if height == msg.Height {
 				var ourVotes *cmn.BitArray
 				switch msg.Type {
-				case types.VoteTypePrevote:
+				case types.PrevoteType:
 					ourVotes = votes.Prevotes(msg.Round).BitArrayByBlockID(msg.BlockID)
-				case types.VoteTypePrecommit:
+				case types.PrecommitType:
 					ourVotes = votes.Precommits(msg.Round).BitArrayByBlockID(msg.BlockID)
 				default:
 					conR.Logger.Error("Bad VoteSetBitsMessage field Type")
@@ -497,7 +508,7 @@ OUTER_LOOP:
 		// If height and round don't match, sleep.
 		if (rs.Height != prs.Height) || (rs.Round != prs.Round) {
 			//logger.Info("Peer Height|Round mismatch, sleeping", "peerHeight", prs.Height, "peerRound", prs.Round, "peer", peer)
-			time.Sleep(conR.conS.config.PeerGossipSleep())
+			time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 			continue OUTER_LOOP
 		}
 
@@ -533,7 +544,7 @@ OUTER_LOOP:
 		}
 
 		// Nothing to do. Sleep.
-		time.Sleep(conR.conS.config.PeerGossipSleep())
+		time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 		continue OUTER_LOOP
 	}
 }
@@ -547,12 +558,12 @@ func (conR *ConsensusReactor) gossipDataForCatchup(logger log.Logger, rs *cstype
 		if blockMeta == nil {
 			logger.Error("Failed to load block meta",
 				"ourHeight", rs.Height, "blockstoreHeight", conR.conS.blockStore.Height())
-			time.Sleep(conR.conS.config.PeerGossipSleep())
+			time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 			return
 		} else if !blockMeta.BlockID.PartsHeader.Equals(prs.ProposalBlockPartsHeader) {
 			logger.Info("Peer ProposalBlockPartsHeader mismatch, sleeping",
 				"blockPartsHeader", blockMeta.BlockID.PartsHeader, "peerBlockPartsHeader", prs.ProposalBlockPartsHeader)
-			time.Sleep(conR.conS.config.PeerGossipSleep())
+			time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 			return
 		}
 		// Load the part
@@ -560,7 +571,7 @@ func (conR *ConsensusReactor) gossipDataForCatchup(logger log.Logger, rs *cstype
 		if part == nil {
 			logger.Error("Could not load part", "index", index,
 				"blockPartsHeader", blockMeta.BlockID.PartsHeader, "peerBlockPartsHeader", prs.ProposalBlockPartsHeader)
-			time.Sleep(conR.conS.config.PeerGossipSleep())
+			time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 			return
 		}
 		// Send the part
@@ -578,7 +589,7 @@ func (conR *ConsensusReactor) gossipDataForCatchup(logger log.Logger, rs *cstype
 		return
 	}
 	//logger.Info("No parts to send in catch-up, sleeping")
-	time.Sleep(conR.conS.config.PeerGossipSleep())
+	time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 }
 
 func (conR *ConsensusReactor) gossipVotesRoutine(peer p2p.Peer, ps *PeerState) {
@@ -647,7 +658,7 @@ OUTER_LOOP:
 			sleeping = 1
 		}
 
-		time.Sleep(conR.conS.config.PeerGossipSleep())
+		time.Sleep(conR.conS.config.PeerGossipSleepDuration)
 		continue OUTER_LOOP
 	}
 }
@@ -728,10 +739,10 @@ OUTER_LOOP:
 					peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
 						Height:  prs.Height,
 						Round:   prs.Round,
-						Type:    types.VoteTypePrevote,
+						Type:    types.PrevoteType,
 						BlockID: maj23,
 					}))
-					time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
+					time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
 				}
 			}
 		}
@@ -745,10 +756,10 @@ OUTER_LOOP:
 					peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
 						Height:  prs.Height,
 						Round:   prs.Round,
-						Type:    types.VoteTypePrecommit,
+						Type:    types.PrecommitType,
 						BlockID: maj23,
 					}))
-					time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
+					time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
 				}
 			}
 		}
@@ -762,10 +773,10 @@ OUTER_LOOP:
 					peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
 						Height:  prs.Height,
 						Round:   prs.ProposalPOLRound,
-						Type:    types.VoteTypePrevote,
+						Type:    types.PrevoteType,
 						BlockID: maj23,
 					}))
-					time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
+					time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
 				}
 			}
 		}
@@ -781,19 +792,56 @@ OUTER_LOOP:
 				peer.TrySend(StateChannel, cdc.MustMarshalBinaryBare(&VoteSetMaj23Message{
 					Height:  prs.Height,
 					Round:   commit.Round(),
-					Type:    types.VoteTypePrecommit,
+					Type:    types.PrecommitType,
 					BlockID: commit.BlockID,
 				}))
-				time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
+				time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
 			}
 		}
 
-		time.Sleep(conR.conS.config.PeerQueryMaj23Sleep())
+		time.Sleep(conR.conS.config.PeerQueryMaj23SleepDuration)
 
 		continue OUTER_LOOP
 	}
 }
 
+func (conR *ConsensusReactor) peerStatsRoutine() {
+	for {
+		if !conR.IsRunning() {
+			conR.Logger.Info("Stopping peerStatsRoutine")
+			return
+		}
+
+		select {
+		case msg := <-conR.conS.statsMsgQueue:
+			// Get peer
+			peer := conR.Switch.Peers().Get(msg.PeerID)
+			if peer == nil {
+				conR.Logger.Debug("Attempt to update stats for non-existent peer",
+					"peer", msg.PeerID)
+				continue
+			}
+			// Get peer state
+			ps := peer.Get(types.PeerStateKey).(*PeerState)
+			switch msg.Msg.(type) {
+			case *VoteMessage:
+				if numVotes := ps.RecordVote(); numVotes%votesToContributeToBecomeGoodPeer == 0 {
+					conR.Switch.MarkPeerAsGood(peer)
+				}
+			case *BlockPartMessage:
+				if numParts := ps.RecordBlockPart(); numParts%blocksToContributeToBecomeGoodPeer == 0 {
+					conR.Switch.MarkPeerAsGood(peer)
+				}
+			}
+		case <-conR.conS.Quit():
+			return
+
+		case <-conR.Quit():
+			return
+		}
+	}
+}
+
 // String returns a string representation of the ConsensusReactor.
 // NOTE: For now, it is just a hard-coded string to avoid accessing unprotected shared variables.
 // TODO: improve!
@@ -814,6 +862,21 @@ func (conR *ConsensusReactor) StringIndented(indent string) string {
 	return s
 }
 
+func (conR *ConsensusReactor) updateFastSyncingMetric() {
+	var fastSyncing float64
+	if conR.fastSync {
+		fastSyncing = 1
+	} else {
+		fastSyncing = 0
+	}
+	conR.metrics.FastSyncing.Set(fastSyncing)
+}
+
+// ReactorMetrics sets the metrics
+func ReactorMetrics(metrics *Metrics) ReactorOption {
+	return func(conR *ConsensusReactor) { conR.metrics = metrics }
+}
+
 //-----------------------------------------------------------------------------
 
 var (
@@ -836,15 +899,13 @@ type PeerState struct {
 
 // peerStateStats holds internal statistics for a peer.
 type peerStateStats struct {
-	LastVoteHeight      int64 `json:"last_vote_height"`
-	Votes               int   `json:"votes"`
-	LastBlockPartHeight int64 `json:"last_block_part_height"`
-	BlockParts          int   `json:"block_parts"`
+	Votes      int `json:"votes"`
+	BlockParts int `json:"block_parts"`
 }
 
 func (pss peerStateStats) String() string {
-	return fmt.Sprintf("peerStateStats{lvh: %d, votes: %d, lbph: %d, blockParts: %d}",
-		pss.LastVoteHeight, pss.Votes, pss.LastBlockPartHeight, pss.BlockParts)
+	return fmt.Sprintf("peerStateStats{votes: %d, blockParts: %d}",
+		pss.Votes, pss.BlockParts)
 }
 
 // NewPeerState returns a new PeerState for the given Peer
@@ -961,7 +1022,7 @@ func (ps *PeerState) PickVoteToSend(votes types.VoteSetReader) (vote *types.Vote
 		return nil, false
 	}
 
-	height, round, type_, size := votes.Height(), votes.Round(), votes.Type(), votes.Size()
+	height, round, type_, size := votes.Height(), votes.Round(), types.SignedMsgType(votes.Type()), votes.Size()
 
 	// Lazily set data using 'votes'.
 	if votes.IsCommit() {
@@ -980,7 +1041,7 @@ func (ps *PeerState) PickVoteToSend(votes types.VoteSetReader) (vote *types.Vote
 	return nil, false
 }
 
-func (ps *PeerState) getVoteBitArray(height int64, round int, type_ byte) *cmn.BitArray {
+func (ps *PeerState) getVoteBitArray(height int64, round int, type_ types.SignedMsgType) *cmn.BitArray {
 	if !types.IsVoteTypeValid(type_) {
 		return nil
 	}
@@ -988,25 +1049,25 @@ func (ps *PeerState) getVoteBitArray(height int64, round int, type_ byte) *cmn.B
 	if ps.PRS.Height == height {
 		if ps.PRS.Round == round {
 			switch type_ {
-			case types.VoteTypePrevote:
+			case types.PrevoteType:
 				return ps.PRS.Prevotes
-			case types.VoteTypePrecommit:
+			case types.PrecommitType:
 				return ps.PRS.Precommits
 			}
 		}
 		if ps.PRS.CatchupCommitRound == round {
 			switch type_ {
-			case types.VoteTypePrevote:
+			case types.PrevoteType:
 				return nil
-			case types.VoteTypePrecommit:
+			case types.PrecommitType:
 				return ps.PRS.CatchupCommit
 			}
 		}
 		if ps.PRS.ProposalPOLRound == round {
 			switch type_ {
-			case types.VoteTypePrevote:
+			case types.PrevoteType:
 				return ps.PRS.ProposalPOL
-			case types.VoteTypePrecommit:
+			case types.PrecommitType:
 				return nil
 			}
 		}
@@ -1015,9 +1076,9 @@ func (ps *PeerState) getVoteBitArray(height int64, round int, type_ byte) *cmn.B
 	if ps.PRS.Height == height+1 {
 		if ps.PRS.LastCommitRound == round {
 			switch type_ {
-			case types.VoteTypePrevote:
+			case types.PrevoteType:
 				return nil
-			case types.VoteTypePrecommit:
+			case types.PrecommitType:
 				return ps.PRS.LastCommit
 			}
 		}
@@ -1080,18 +1141,14 @@ func (ps *PeerState) ensureVoteBitArrays(height int64, numValidators int) {
 	}
 }
 
-// RecordVote updates internal statistics for this peer by recording the vote.
-// It returns the total number of votes (1 per block). This essentially means
-// the number of blocks for which peer has been sending us votes.
-func (ps *PeerState) RecordVote(vote *types.Vote) int {
+// RecordVote increments internal votes related statistics for this peer.
+// It returns the total number of added votes.
+func (ps *PeerState) RecordVote() int {
 	ps.mtx.Lock()
 	defer ps.mtx.Unlock()
 
-	if ps.Stats.LastVoteHeight >= vote.Height {
-		return ps.Stats.Votes
-	}
-	ps.Stats.LastVoteHeight = vote.Height
 	ps.Stats.Votes++
+
 	return ps.Stats.Votes
 }
 
@@ -1104,25 +1161,17 @@ func (ps *PeerState) VotesSent() int {
 	return ps.Stats.Votes
 }
 
-// RecordBlockPart updates internal statistics for this peer by recording the
-// block part. It returns the total number of block parts (1 per block). This
-// essentially means the number of blocks for which peer has been sending us
-// block parts.
-func (ps *PeerState) RecordBlockPart(bp *BlockPartMessage) int {
+// RecordBlockPart increments internal block part related statistics for this peer.
+// It returns the total number of added block parts.
+func (ps *PeerState) RecordBlockPart() int {
 	ps.mtx.Lock()
 	defer ps.mtx.Unlock()
 
-	if ps.Stats.LastBlockPartHeight >= bp.Height {
-		return ps.Stats.BlockParts
-	}
-
-	ps.Stats.LastBlockPartHeight = bp.Height
 	ps.Stats.BlockParts++
 	return ps.Stats.BlockParts
 }
 
-// BlockPartsSent returns the number of blocks for which peer has been sending
-// us block parts.
+// BlockPartsSent returns the number of useful block parts the peer has sent us.
 func (ps *PeerState) BlockPartsSent() int {
 	ps.mtx.Lock()
 	defer ps.mtx.Unlock()
@@ -1138,7 +1187,7 @@ func (ps *PeerState) SetHasVote(vote *types.Vote) {
 	ps.setHasVote(vote.Height, vote.Round, vote.Type, vote.ValidatorIndex)
 }
 
-func (ps *PeerState) setHasVote(height int64, round int, type_ byte, index int) {
+func (ps *PeerState) setHasVote(height int64, round int, type_ types.SignedMsgType, index int) {
 	logger := ps.logger.With("peerH/R", fmt.Sprintf("%d/%d", ps.PRS.Height, ps.PRS.Round), "H/R", fmt.Sprintf("%d/%d", height, round))
 	logger.Debug("setHasVote", "type", type_, "index", index)
 
@@ -1404,7 +1453,7 @@ func (m *VoteMessage) String() string {
 type HasVoteMessage struct {
 	Height int64
 	Round  int
-	Type   byte
+	Type   types.SignedMsgType
 	Index  int
 }
 
@@ -1419,7 +1468,7 @@ func (m *HasVoteMessage) String() string {
 type VoteSetMaj23Message struct {
 	Height  int64
 	Round   int
-	Type    byte
+	Type    types.SignedMsgType
 	BlockID types.BlockID
 }
 
@@ -1434,7 +1483,7 @@ func (m *VoteSetMaj23Message) String() string {
 type VoteSetBitsMessage struct {
 	Height  int64
 	Round   int
-	Type    byte
+	Type    types.SignedMsgType
 	BlockID types.BlockID
 	Votes   *cmn.BitArray
 }

consensus/reactor_test.go

@@ -11,24 +11,20 @@ import (
 	"testing"
 	"time"
 
-	abcicli "github.com/tendermint/tendermint/abci/client"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/tendermint/tendermint/abci/client"
 	"github.com/tendermint/tendermint/abci/example/kvstore"
 	abci "github.com/tendermint/tendermint/abci/types"
 	bc "github.com/tendermint/tendermint/blockchain"
-	cmn "github.com/tendermint/tendermint/libs/common"
+	cfg "github.com/tendermint/tendermint/config"
 	dbm "github.com/tendermint/tendermint/libs/db"
 	"github.com/tendermint/tendermint/libs/log"
 	mempl "github.com/tendermint/tendermint/mempool"
-	sm "github.com/tendermint/tendermint/state"
-	tmtime "github.com/tendermint/tendermint/types/time"
-
-	cfg "github.com/tendermint/tendermint/config"
 	"github.com/tendermint/tendermint/p2p"
-	p2pdummy "github.com/tendermint/tendermint/p2p/dummy"
+	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
-
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
 )
 
 func init() {
@@ -101,6 +97,9 @@ func TestReactorBasic(t *testing.T) {
 
 // Ensure we can process blocks with evidence
 func TestReactorWithEvidence(t *testing.T) {
+	types.RegisterMockEvidences(cdc)
+	types.RegisterMockEvidences(types.GetCodec())
+
 	nValidators := 4
 	testName := "consensus_reactor_test"
 	tickerFunc := newMockTickerFunc(true)
@@ -246,110 +245,25 @@ func TestReactorProposalHeartbeats(t *testing.T) {
 	}, css)
 }
 
-// Test we record block parts from other peers
-func TestReactorRecordsBlockParts(t *testing.T) {
-	// create dummy peer
-	peer := p2pdummy.NewPeer()
-	ps := NewPeerState(peer).SetLogger(log.TestingLogger())
-	peer.Set(types.PeerStateKey, ps)
+// Test we record stats about votes and block parts from other peers.
+func TestReactorRecordsVotesAndBlockParts(t *testing.T) {
+	N := 4
+	css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
+	reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
+	defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
 
-	// create reactor
-	css := randConsensusNet(1, "consensus_reactor_records_block_parts_test", newMockTickerFunc(true), newPersistentKVStore)
-	reactor := NewConsensusReactor(css[0], false) // so we dont start the consensus states
-	reactor.SetEventBus(css[0].eventBus)
-	reactor.SetLogger(log.TestingLogger())
-	sw := p2p.MakeSwitch(cfg.DefaultP2PConfig(), 1, "testing", "123.123.123", func(i int, sw *p2p.Switch) *p2p.Switch { return sw })
-	reactor.SetSwitch(sw)
-	err := reactor.Start()
-	require.NoError(t, err)
-	defer reactor.Stop()
+	// wait till everyone makes the first new block
+	timeoutWaitGroup(t, N, func(j int) {
+		<-eventChans[j]
+	}, css)
 
-	// 1) new block part
-	parts := types.NewPartSetFromData(cmn.RandBytes(100), 10)
-	msg := &BlockPartMessage{
-		Height: 2,
-		Round:  0,
-		Part:   parts.GetPart(0),
-	}
-	bz, err := cdc.MarshalBinaryBare(msg)
-	require.NoError(t, err)
+	// Get peer
+	peer := reactors[1].Switch.Peers().List()[0]
+	// Get peer state
+	ps := peer.Get(types.PeerStateKey).(*PeerState)
 
-	reactor.Receive(DataChannel, peer, bz)
-	require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should have increased by 1")
-
-	// 2) block part with the same height, but different round
-	msg.Round = 1
-
-	bz, err = cdc.MarshalBinaryBare(msg)
-	require.NoError(t, err)
-
-	reactor.Receive(DataChannel, peer, bz)
-	require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should stay the same")
-
-	// 3) block part from earlier height
-	msg.Height = 1
-	msg.Round = 0
-
-	bz, err = cdc.MarshalBinaryBare(msg)
-	require.NoError(t, err)
-
-	reactor.Receive(DataChannel, peer, bz)
-	require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should stay the same")
-}
-
-// Test we record votes from other peers.
-func TestReactorRecordsVotes(t *testing.T) {
-	// Create dummy peer.
-	peer := p2pdummy.NewPeer()
-	ps := NewPeerState(peer).SetLogger(log.TestingLogger())
-	peer.Set(types.PeerStateKey, ps)
-
-	// Create reactor.
-	css := randConsensusNet(1, "consensus_reactor_records_votes_test", newMockTickerFunc(true), newPersistentKVStore)
-	reactor := NewConsensusReactor(css[0], false) // so we dont start the consensus states
-	reactor.SetEventBus(css[0].eventBus)
-	reactor.SetLogger(log.TestingLogger())
-	sw := p2p.MakeSwitch(cfg.DefaultP2PConfig(), 1, "testing", "123.123.123", func(i int, sw *p2p.Switch) *p2p.Switch { return sw })
-	reactor.SetSwitch(sw)
-	err := reactor.Start()
-	require.NoError(t, err)
-	defer reactor.Stop()
-	_, val := css[0].state.Validators.GetByIndex(0)
-
-	// 1) new vote
-	vote := &types.Vote{
-		ValidatorIndex:   0,
-		ValidatorAddress: val.Address,
-		Height:           2,
-		Round:            0,
-		Timestamp:        tmtime.Now(),
-		Type:             types.VoteTypePrevote,
-		BlockID:          types.BlockID{},
-	}
-	bz, err := cdc.MarshalBinaryBare(&VoteMessage{vote})
-	require.NoError(t, err)
-
-	reactor.Receive(VoteChannel, peer, bz)
-	assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should have increased by 1")
-
-	// 2) vote with the same height, but different round
-	vote.Round = 1
-
-	bz, err = cdc.MarshalBinaryBare(&VoteMessage{vote})
-	require.NoError(t, err)
-
-	reactor.Receive(VoteChannel, peer, bz)
-	assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should stay the same")
-
-	// 3) vote from earlier height
-	vote.Height = 1
-	vote.Round = 0
-
-	bz, err = cdc.MarshalBinaryBare(&VoteMessage{vote})
-	require.NoError(t, err)
-
-	reactor.Receive(VoteChannel, peer, bz)
-	assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should stay the same")
+	assert.Equal(t, true, ps.VotesSent() > 0, "number of votes sent should have increased")
+	assert.Equal(t, true, ps.BlockPartsSent() > 0, "number of votes sent should have increased")
 }
 
 //-------------------------------------------------------------

consensus/replay.go

@@ -287,6 +287,7 @@ func (h *Handshaker) ReplayBlocks(state sm.State, appHash []byte, appBlockHeight
 				return nil, err
 			}
 			state.Validators = types.NewValidatorSet(vals)
+			state.NextValidators = types.NewValidatorSet(vals)
 		}
 		if res.ConsensusParams != nil {
 			state.ConsensusParams = types.PB2TM.ConsensusParams(res.ConsensusParams)

consensus/replay_test.go

@@ -542,7 +542,7 @@ func makeBlockchainFromWAL(wal WAL) ([]*types.Block, []*types.Commit, error) {
 				return nil, nil, err
 			}
 		case *types.Vote:
-			if p.Type == types.VoteTypePrecommit {
+			if p.Type == types.PrecommitType {
 				thisBlockCommit = &types.Commit{
 					BlockID:    p.BlockID,
 					Precommits: []*types.Vote{p},

consensus/state.go

@@ -83,7 +83,8 @@ type ConsensusState struct {
 	// internal state
 	mtx sync.RWMutex
 	cstypes.RoundState
-	state sm.State // State until height-1.
+	triggeredTimeoutPrecommit bool
+	state                     sm.State // State until height-1.
 
 	// state changes may be triggered by: msgs from peers,
 	// msgs from ourself, or by timeouts
@@ -91,6 +92,10 @@ type ConsensusState struct {
 	internalMsgQueue chan msgInfo
 	timeoutTicker    TimeoutTicker
 
+	// information about about added votes and block parts are written on this channel
+	// so statistics can be computed by reactor
+	statsMsgQueue chan msgInfo
+
 	// we use eventBus to trigger msg broadcasts in the reactor,
 	// and to notify external subscribers, eg. through a websocket
 	eventBus *types.EventBus
@@ -120,8 +125,8 @@ type ConsensusState struct {
 	metrics *Metrics
 }
 
-// CSOption sets an optional parameter on the ConsensusState.
-type CSOption func(*ConsensusState)
+// StateOption sets an optional parameter on the ConsensusState.
+type StateOption func(*ConsensusState)
 
 // NewConsensusState returns a new ConsensusState.
 func NewConsensusState(
@@ -131,7 +136,7 @@ func NewConsensusState(
 	blockStore sm.BlockStore,
 	mempool sm.Mempool,
 	evpool sm.EvidencePool,
-	options ...CSOption,
+	options ...StateOption,
 ) *ConsensusState {
 	cs := &ConsensusState{
 		config:           config,
@@ -141,6 +146,7 @@ func NewConsensusState(
 		peerMsgQueue:     make(chan msgInfo, msgQueueSize),
 		internalMsgQueue: make(chan msgInfo, msgQueueSize),
 		timeoutTicker:    NewTimeoutTicker(),
+		statsMsgQueue:    make(chan msgInfo, msgQueueSize),
 		done:             make(chan struct{}),
 		doWALCatchup:     true,
 		wal:              nilWAL{},
@@ -180,8 +186,8 @@ func (cs *ConsensusState) SetEventBus(b *types.EventBus) {
 	cs.blockExec.SetEventBus(b)
 }
 
-// WithMetrics sets the metrics.
-func WithMetrics(metrics *Metrics) CSOption {
+// StateMetrics sets the metrics.
+func StateMetrics(metrics *Metrics) StateOption {
 	return func(cs *ConsensusState) { cs.metrics = metrics }
 }
 
@@ -454,7 +460,7 @@ func (cs *ConsensusState) reconstructLastCommit(state sm.State) {
 		return
 	}
 	seenCommit := cs.blockStore.LoadSeenCommit(state.LastBlockHeight)
-	lastPrecommits := types.NewVoteSet(state.ChainID, state.LastBlockHeight, seenCommit.Round(), types.VoteTypePrecommit, state.LastValidators)
+	lastPrecommits := types.NewVoteSet(state.ChainID, state.LastBlockHeight, seenCommit.Round(), types.PrecommitType, state.LastValidators)
 	for _, precommit := range seenCommit.Precommits {
 		if precommit == nil {
 			continue
@@ -639,7 +645,11 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
 		err = cs.setProposal(msg.Proposal)
 	case *BlockPartMessage:
 		// if the proposal is complete, we'll enterPrevote or tryFinalizeCommit
-		_, err = cs.addProposalBlockPart(msg, peerID)
+		added, err := cs.addProposalBlockPart(msg, peerID)
+		if added {
+			cs.statsMsgQueue <- mi
+		}
+
 		if err != nil && msg.Round != cs.Round {
 			cs.Logger.Debug("Received block part from wrong round", "height", cs.Height, "csRound", cs.Round, "blockRound", msg.Round)
 			err = nil
@@ -647,7 +657,11 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
 	case *VoteMessage:
 		// attempt to add the vote and dupeout the validator if its a duplicate signature
 		// if the vote gives us a 2/3-any or 2/3-one, we transition
-		err := cs.tryAddVote(msg.Vote, peerID)
+		added, err := cs.tryAddVote(msg.Vote, peerID)
+		if added {
+			cs.statsMsgQueue <- mi
+		}
+
 		if err == ErrAddingVote {
 			// TODO: punish peer
 			// We probably don't want to stop the peer here. The vote does not
@@ -698,6 +712,7 @@ func (cs *ConsensusState) handleTimeout(ti timeoutInfo, rs cstypes.RoundState) {
 		cs.enterPrecommit(ti.Height, ti.Round)
 	case cstypes.RoundStepPrecommitWait:
 		cs.eventBus.PublishEventTimeoutWait(cs.RoundStateEvent())
+		cs.enterPrecommit(ti.Height, ti.Round)
 		cs.enterNewRound(ti.Height, ti.Round+1)
 	default:
 		panic(fmt.Sprintf("Invalid timeout step: %v", ti.Step))
@@ -759,6 +774,7 @@ func (cs *ConsensusState) enterNewRound(height int64, round int) {
 		cs.ProposalBlockParts = nil
 	}
 	cs.Votes.SetRound(round + 1) // also track next round (round+1) to allow round-skipping
+	cs.triggeredTimeoutPrecommit = false
 
 	cs.eventBus.PublishEventNewRound(cs.RoundStateEvent())
 	cs.metrics.Rounds.Set(float64(round))
@@ -769,7 +785,8 @@ func (cs *ConsensusState) enterNewRound(height int64, round int) {
 	waitForTxs := cs.config.WaitForTxs() && round == 0 && !cs.needProofBlock(height)
 	if waitForTxs {
 		if cs.config.CreateEmptyBlocksInterval > 0 {
-			cs.scheduleTimeout(cs.config.EmptyBlocksInterval(), height, round, cstypes.RoundStepNewRound)
+			cs.scheduleTimeout(cs.config.CreateEmptyBlocksInterval, height, round,
+				cstypes.RoundStepNewRound)
 		}
 		go cs.proposalHeartbeat(height, round)
 	} else {
@@ -1000,17 +1017,18 @@ func (cs *ConsensusState) enterPrevote(height int64, round int) {
 
 func (cs *ConsensusState) defaultDoPrevote(height int64, round int) {
 	logger := cs.Logger.With("height", height, "round", round)
+
 	// If a block is locked, prevote that.
 	if cs.LockedBlock != nil {
 		logger.Info("enterPrevote: Block was locked")
-		cs.signAddVote(types.VoteTypePrevote, cs.LockedBlock.Hash(), cs.LockedBlockParts.Header())
+		cs.signAddVote(types.PrevoteType, cs.LockedBlock.Hash(), cs.LockedBlockParts.Header())
 		return
 	}
 
 	// If ProposalBlock is nil, prevote nil.
 	if cs.ProposalBlock == nil {
 		logger.Info("enterPrevote: ProposalBlock is nil")
-		cs.signAddVote(types.VoteTypePrevote, nil, types.PartSetHeader{})
+		cs.signAddVote(types.PrevoteType, nil, types.PartSetHeader{})
 		return
 	}
 
@@ -1019,7 +1037,7 @@ func (cs *ConsensusState) defaultDoPrevote(height int64, round int) {
 	if err != nil {
 		// ProposalBlock is invalid, prevote nil.
 		logger.Error("enterPrevote: ProposalBlock is invalid", "err", err)
-		cs.signAddVote(types.VoteTypePrevote, nil, types.PartSetHeader{})
+		cs.signAddVote(types.PrevoteType, nil, types.PartSetHeader{})
 		return
 	}
 
@@ -1027,7 +1045,7 @@ func (cs *ConsensusState) defaultDoPrevote(height int64, round int) {
 	// NOTE: the proposal signature is validated when it is received,
 	// and the proposal block parts are validated as they are received (against the merkle hash in the proposal)
 	logger.Info("enterPrevote: ProposalBlock is valid")
-	cs.signAddVote(types.VoteTypePrevote, cs.ProposalBlock.Hash(), cs.ProposalBlockParts.Header())
+	cs.signAddVote(types.PrevoteType, cs.ProposalBlock.Hash(), cs.ProposalBlockParts.Header())
 }
 
 // Enter: any +2/3 prevotes at next round.
@@ -1085,7 +1103,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
 		} else {
 			logger.Info("enterPrecommit: No +2/3 prevotes during enterPrecommit. Precommitting nil.")
 		}
-		cs.signAddVote(types.VoteTypePrecommit, nil, types.PartSetHeader{})
+		cs.signAddVote(types.PrecommitType, nil, types.PartSetHeader{})
 		return
 	}
 
@@ -1109,7 +1127,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
 			cs.LockedBlockParts = nil
 			cs.eventBus.PublishEventUnlock(cs.RoundStateEvent())
 		}
-		cs.signAddVote(types.VoteTypePrecommit, nil, types.PartSetHeader{})
+		cs.signAddVote(types.PrecommitType, nil, types.PartSetHeader{})
 		return
 	}
 
@@ -1120,7 +1138,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
 		logger.Info("enterPrecommit: +2/3 prevoted locked block. Relocking")
 		cs.LockedRound = round
 		cs.eventBus.PublishEventRelock(cs.RoundStateEvent())
-		cs.signAddVote(types.VoteTypePrecommit, blockID.Hash, blockID.PartsHeader)
+		cs.signAddVote(types.PrecommitType, blockID.Hash, blockID.PartsHeader)
 		return
 	}
 
@@ -1135,7 +1153,7 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
 		cs.LockedBlock = cs.ProposalBlock
 		cs.LockedBlockParts = cs.ProposalBlockParts
 		cs.eventBus.PublishEventLock(cs.RoundStateEvent())
-		cs.signAddVote(types.VoteTypePrecommit, blockID.Hash, blockID.PartsHeader)
+		cs.signAddVote(types.PrecommitType, blockID.Hash, blockID.PartsHeader)
 		return
 	}
 
@@ -1151,15 +1169,19 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
 		cs.ProposalBlockParts = types.NewPartSetFromHeader(blockID.PartsHeader)
 	}
 	cs.eventBus.PublishEventUnlock(cs.RoundStateEvent())
-	cs.signAddVote(types.VoteTypePrecommit, nil, types.PartSetHeader{})
+	cs.signAddVote(types.PrecommitType, nil, types.PartSetHeader{})
 }
 
 // Enter: any +2/3 precommits for next round.
 func (cs *ConsensusState) enterPrecommitWait(height int64, round int) {
 	logger := cs.Logger.With("height", height, "round", round)
 
-	if cs.Height != height || round < cs.Round || (cs.Round == round && cstypes.RoundStepPrecommitWait <= cs.Step) {
-		logger.Debug(fmt.Sprintf("enterPrecommitWait(%v/%v): Invalid args. Current step: %v/%v/%v", height, round, cs.Height, cs.Round, cs.Step))
+	if cs.Height != height || round < cs.Round || (cs.Round == round && cs.triggeredTimeoutPrecommit) {
+		logger.Debug(
+			fmt.Sprintf(
+				"enterPrecommitWait(%v/%v): Invalid args. "+
+					"Current state is Height/Round: %v/%v/, triggeredTimeoutPrecommit:%v",
+				height, round, cs.Height, cs.Round, cs.triggeredTimeoutPrecommit))
 		return
 	}
 	if !cs.Votes.Precommits(round).HasTwoThirdsAny() {
@@ -1169,7 +1191,7 @@ func (cs *ConsensusState) enterPrecommitWait(height int64, round int) {
 
 	defer func() {
 		// Done enterPrecommitWait:
-		cs.updateRoundStep(round, cstypes.RoundStepPrecommitWait)
+		cs.triggeredTimeoutPrecommit = true
 		cs.newStep()
 	}()
 
@@ -1384,6 +1406,8 @@ func (cs *ConsensusState) recordMetrics(height int64, block *types.Block) {
 	cs.metrics.NumTxs.Set(float64(block.NumTxs))
 	cs.metrics.BlockSizeBytes.Set(float64(block.Size()))
 	cs.metrics.TotalTxs.Set(float64(block.TotalTxs))
+	cs.metrics.CommittedHeight.Set(float64(block.Height))
+
 }
 
 //-----------------------------------------------------------------------------
@@ -1454,7 +1478,7 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
 			int64(cs.state.ConsensusParams.BlockSize.MaxBytes),
 		)
 		if err != nil {
-			return true, err
+			return added, err
 		}
 		// NOTE: it's possible to receive complete proposal blocks for future rounds without having the proposal
 		cs.Logger.Info("Received complete proposal block", "height", cs.ProposalBlock.Height, "hash", cs.ProposalBlock.Hash())
@@ -1480,39 +1504,42 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
 		if cs.Step <= cstypes.RoundStepPropose && cs.isProposalComplete() {
 			// Move onto the next step
 			cs.enterPrevote(height, cs.Round)
+			if hasTwoThirds { // this is optimisation as this will be triggered when prevote is added
+				cs.enterPrecommit(height, cs.Round)
+			}
 		} else if cs.Step == cstypes.RoundStepCommit {
 			// If we're waiting on the proposal block...
 			cs.tryFinalizeCommit(height)
 		}
-		return true, nil
+		return added, nil
 	}
 	return added, nil
 }
 
 // Attempt to add the vote. if its a duplicate signature, dupeout the validator
-func (cs *ConsensusState) tryAddVote(vote *types.Vote, peerID p2p.ID) error {
-	_, err := cs.addVote(vote, peerID)
+func (cs *ConsensusState) tryAddVote(vote *types.Vote, peerID p2p.ID) (bool, error) {
+	added, err := cs.addVote(vote, peerID)
 	if err != nil {
 		// If the vote height is off, we'll just ignore it,
 		// But if it's a conflicting sig, add it to the cs.evpool.
 		// If it's otherwise invalid, punish peer.
 		if err == ErrVoteHeightMismatch {
-			return err
+			return added, err
 		} else if voteErr, ok := err.(*types.ErrVoteConflictingVotes); ok {
 			if bytes.Equal(vote.ValidatorAddress, cs.privValidator.GetAddress()) {
 				cs.Logger.Error("Found conflicting vote from ourselves. Did you unsafe_reset a validator?", "height", vote.Height, "round", vote.Round, "type", vote.Type)
-				return err
+				return added, err
 			}
 			cs.evpool.AddEvidence(voteErr.DuplicateVoteEvidence)
-			return err
+			return added, err
 		} else {
 			// Probably an invalid signature / Bad peer.
 			// Seems this can also err sometimes with "Unexpected step" - perhaps not from a bad peer ?
 			cs.Logger.Error("Error attempting to add vote", "err", err)
-			return ErrAddingVote
+			return added, ErrAddingVote
 		}
 	}
-	return nil
+	return added, nil
 }
 
 //-----------------------------------------------------------------------------
@@ -1523,7 +1550,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 	// A precommit for the previous height?
 	// These come in while we wait timeoutCommit
 	if vote.Height+1 == cs.Height {
-		if !(cs.Step == cstypes.RoundStepNewHeight && vote.Type == types.VoteTypePrecommit) {
+		if !(cs.Step == cstypes.RoundStepNewHeight && vote.Type == types.PrecommitType) {
 			// TODO: give the reason ..
 			// fmt.Errorf("tryAddVote: Wrong height, not a LastCommit straggler commit.")
 			return added, ErrVoteHeightMismatch
@@ -1566,7 +1593,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 	cs.evsw.FireEvent(types.EventVote, vote)
 
 	switch vote.Type {
-	case types.VoteTypePrevote:
+	case types.PrevoteType:
 		prevotes := cs.Votes.Prevotes(vote.Round)
 		cs.Logger.Info("Added to prevote", "vote", vote, "prevotes", prevotes.StringShort())
 
@@ -1594,7 +1621,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 			// Update Valid* if we can.
 			// NOTE: our proposal block may be nil or not what received a polka..
 			// TODO: we may want to still update the ValidBlock and obtain it via gossipping
-			if !blockID.IsZero() &&
+			if len(blockID.Hash) != 0 &&
 				(cs.ValidRound < vote.Round) &&
 				(vote.Round <= cs.Round) &&
 				cs.ProposalBlock.HashesTo(blockID.Hash) {
@@ -1606,14 +1633,14 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 			}
 		}
 
-		// If +2/3 prevotes for *anything* for this or future round:
-		if cs.Round <= vote.Round && prevotes.HasTwoThirdsAny() {
-			// Round-skip over to PrevoteWait or goto Precommit.
-			cs.enterNewRound(height, vote.Round) // if the vote is ahead of us
+		// If +2/3 prevotes for *anything* for future round:
+		if cs.Round < vote.Round && prevotes.HasTwoThirdsAny() {
+			// Round-skip if there is any 2/3+ of votes ahead of us
+			cs.enterNewRound(height, vote.Round)
+		} else if cs.Round == vote.Round && cstypes.RoundStepPrevote <= cs.Step { // current round
 			if prevotes.HasTwoThirdsMajority() {
 				cs.enterPrecommit(height, vote.Round)
-			} else {
-				cs.enterPrevote(height, vote.Round) // if the vote is ahead of us
+			} else if prevotes.HasTwoThirdsAny() {
 				cs.enterPrevoteWait(height, vote.Round)
 			}
 		} else if cs.Proposal != nil && 0 <= cs.Proposal.POLRound && cs.Proposal.POLRound == vote.Round {
@@ -1623,31 +1650,28 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 			}
 		}
 
-	case types.VoteTypePrecommit:
+	case types.PrecommitType:
 		precommits := cs.Votes.Precommits(vote.Round)
 		cs.Logger.Info("Added to precommit", "vote", vote, "precommits", precommits.StringShort())
+
 		blockID, ok := precommits.TwoThirdsMajority()
 		if ok {
-			if len(blockID.Hash) == 0 {
-				cs.enterNewRound(height, vote.Round+1)
-			} else {
-				cs.enterNewRound(height, vote.Round)
-				cs.enterPrecommit(height, vote.Round)
+			// Executed as TwoThirdsMajority could be from a higher round
+			cs.enterNewRound(height, vote.Round)
+			cs.enterPrecommit(height, vote.Round)
+			if len(blockID.Hash) != 0 {
 				cs.enterCommit(height, vote.Round)
-
 				if cs.config.SkipTimeoutCommit && precommits.HasAll() {
-					// if we have all the votes now,
-					// go straight to new round (skip timeout commit)
-					// cs.scheduleTimeout(time.Duration(0), cs.Height, 0, cstypes.RoundStepNewHeight)
 					cs.enterNewRound(cs.Height, 0)
 				}
-
+			} else {
+				cs.enterPrecommitWait(height, vote.Round)
 			}
 		} else if cs.Round <= vote.Round && precommits.HasTwoThirdsAny() {
 			cs.enterNewRound(height, vote.Round)
-			cs.enterPrecommit(height, vote.Round)
 			cs.enterPrecommitWait(height, vote.Round)
 		}
+
 	default:
 		panic(fmt.Sprintf("Unexpected vote type %X", vote.Type)) // go-wire should prevent this.
 	}
@@ -1655,7 +1679,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 	return
 }
 
-func (cs *ConsensusState) signVote(type_ byte, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
+func (cs *ConsensusState) signVote(type_ types.SignedMsgType, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
 	addr := cs.privValidator.GetAddress()
 	valIndex, _ := cs.Validators.GetByAddress(addr)
 
@@ -1690,7 +1714,7 @@ func (cs *ConsensusState) voteTime() time.Time {
 }
 
 // sign the vote and publish on internalMsgQueue
-func (cs *ConsensusState) signAddVote(type_ byte, hash []byte, header types.PartSetHeader) *types.Vote {
+func (cs *ConsensusState) signAddVote(type_ types.SignedMsgType, hash []byte, header types.PartSetHeader) *types.Vote {
 	// if we don't have a key or we're not in the validator set, do nothing
 	if cs.privValidator == nil || !cs.Validators.HasAddress(cs.privValidator.GetAddress()) {
 		return nil

consensus/types/height_vote_set.go

@@ -99,8 +99,8 @@ func (hvs *HeightVoteSet) addRound(round int) {
 		cmn.PanicSanity("addRound() for an existing round")
 	}
 	// log.Debug("addRound(round)", "round", round)
-	prevotes := types.NewVoteSet(hvs.chainID, hvs.height, round, types.VoteTypePrevote, hvs.valSet)
-	precommits := types.NewVoteSet(hvs.chainID, hvs.height, round, types.VoteTypePrecommit, hvs.valSet)
+	prevotes := types.NewVoteSet(hvs.chainID, hvs.height, round, types.PrevoteType, hvs.valSet)
+	precommits := types.NewVoteSet(hvs.chainID, hvs.height, round, types.PrecommitType, hvs.valSet)
 	hvs.roundVoteSets[round] = RoundVoteSet{
 		Prevotes:   prevotes,
 		Precommits: precommits,
@@ -134,13 +134,13 @@ func (hvs *HeightVoteSet) AddVote(vote *types.Vote, peerID p2p.ID) (added bool,
 func (hvs *HeightVoteSet) Prevotes(round int) *types.VoteSet {
 	hvs.mtx.Lock()
 	defer hvs.mtx.Unlock()
-	return hvs.getVoteSet(round, types.VoteTypePrevote)
+	return hvs.getVoteSet(round, types.PrevoteType)
 }
 
 func (hvs *HeightVoteSet) Precommits(round int) *types.VoteSet {
 	hvs.mtx.Lock()
 	defer hvs.mtx.Unlock()
-	return hvs.getVoteSet(round, types.VoteTypePrecommit)
+	return hvs.getVoteSet(round, types.PrecommitType)
 }
 
 // Last round and blockID that has +2/3 prevotes for a particular block or nil.
@@ -149,7 +149,7 @@ func (hvs *HeightVoteSet) POLInfo() (polRound int, polBlockID types.BlockID) {
 	hvs.mtx.Lock()
 	defer hvs.mtx.Unlock()
 	for r := hvs.round; r >= 0; r-- {
-		rvs := hvs.getVoteSet(r, types.VoteTypePrevote)
+		rvs := hvs.getVoteSet(r, types.PrevoteType)
 		polBlockID, ok := rvs.TwoThirdsMajority()
 		if ok {
 			return r, polBlockID
@@ -158,15 +158,15 @@ func (hvs *HeightVoteSet) POLInfo() (polRound int, polBlockID types.BlockID) {
 	return -1, types.BlockID{}
 }
 
-func (hvs *HeightVoteSet) getVoteSet(round int, type_ byte) *types.VoteSet {
+func (hvs *HeightVoteSet) getVoteSet(round int, type_ types.SignedMsgType) *types.VoteSet {
 	rvs, ok := hvs.roundVoteSets[round]
 	if !ok {
 		return nil
 	}
 	switch type_ {
-	case types.VoteTypePrevote:
+	case types.PrevoteType:
 		return rvs.Prevotes
-	case types.VoteTypePrecommit:
+	case types.PrecommitType:
 		return rvs.Precommits
 	default:
 		cmn.PanicSanity(fmt.Sprintf("Unexpected vote type %X", type_))
@@ -178,7 +178,7 @@ func (hvs *HeightVoteSet) getVoteSet(round int, type_ byte) *types.VoteSet {
 // NOTE: if there are too many peers, or too much peer churn,
 // this can cause memory issues.
 // TODO: implement ability to remove peers too
-func (hvs *HeightVoteSet) SetPeerMaj23(round int, type_ byte, peerID p2p.ID, blockID types.BlockID) error {
+func (hvs *HeightVoteSet) SetPeerMaj23(round int, type_ types.SignedMsgType, peerID p2p.ID, blockID types.BlockID) error {
 	hvs.mtx.Lock()
 	defer hvs.mtx.Unlock()
 	if !types.IsVoteTypeValid(type_) {

consensus/types/height_vote_set_test.go

@@ -56,7 +56,7 @@ func makeVoteHR(t *testing.T, height int64, round int, privVals []types.PrivVali
 		Height:           height,
 		Round:            round,
 		Timestamp:        tmtime.Now(),
-		Type:             types.VoteTypePrecommit,
+		Type:             types.PrecommitType,
 		BlockID:          types.BlockID{[]byte("fakehash"), types.PartSetHeader{}},
 	}
 	chainID := config.ChainID()

consensus/types/round_state.go

@@ -107,8 +107,8 @@ func (rs *RoundState) RoundStateSimple() RoundStateSimple {
 
 // RoundStateEvent returns the H/R/S of the RoundState as an event.
 func (rs *RoundState) RoundStateEvent() types.EventDataRoundState {
-	// XXX: copy the RoundState
-	// if we want to avoid this, we may need synchronous events after all
+	// copy the RoundState.
+	// TODO: if we want to avoid this, we may need synchronous events after all
 	rsCopy := *rs
 	edrs := types.EventDataRoundState{
 		Height:     rs.Height,

consensus/types/wire.go

@@ -1,7 +1,7 @@
 package types
 
 import (
-	"github.com/tendermint/go-amino"
+	amino "github.com/tendermint/go-amino"
 	"github.com/tendermint/tendermint/types"
 )
 

consensus/wal.go

@@ -73,13 +73,13 @@ type baseWAL struct {
 	enc *WALEncoder
 }
 
-func NewWAL(walFile string) (*baseWAL, error) {
+func NewWAL(walFile string, groupOptions ...func(*auto.Group)) (*baseWAL, error) {
 	err := cmn.EnsureDir(filepath.Dir(walFile), 0700)
 	if err != nil {
 		return nil, errors.Wrap(err, "failed to ensure WAL directory is in place")
 	}
 
-	group, err := auto.OpenGroup(walFile)
+	group, err := auto.OpenGroup(walFile, groupOptions...)
 	if err != nil {
 		return nil, err
 	}

consensus/wal_generator.go

@@ -4,6 +4,7 @@ import (
 	"bufio"
 	"bytes"
 	"fmt"
+	"io"
 	"os"
 	"path/filepath"
 	"strings"
@@ -23,12 +24,11 @@ import (
 	"github.com/tendermint/tendermint/types"
 )
 
-// WALWithNBlocks generates a consensus WAL. It does this by spining up a
+// WALGenerateNBlocks generates a consensus WAL. It does this by spining up a
 // stripped down version of node (proxy app, event bus, consensus state) with a
 // persistent kvstore application and special consensus wal instance
-// (byteBufferWAL) and waits until numBlocks are created. Then it returns a WAL
-// content. If the node fails to produce given numBlocks, it returns an error.
-func WALWithNBlocks(numBlocks int) (data []byte, err error) {
+// (byteBufferWAL) and waits until numBlocks are created. If the node fails to produce given numBlocks, it returns an error.
+func WALGenerateNBlocks(wr io.Writer, numBlocks int) (err error) {
 	config := getConfig()
 
 	app := kvstore.NewPersistentKVStoreApplication(filepath.Join(config.DBDir(), "wal_generator"))
@@ -43,26 +43,26 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
 	privValidator := privval.LoadOrGenFilePV(privValidatorFile)
 	genDoc, err := types.GenesisDocFromFile(config.GenesisFile())
 	if err != nil {
-		return nil, errors.Wrap(err, "failed to read genesis file")
+		return errors.Wrap(err, "failed to read genesis file")
 	}
 	stateDB := db.NewMemDB()
 	blockStoreDB := db.NewMemDB()
 	state, err := sm.MakeGenesisState(genDoc)
 	if err != nil {
-		return nil, errors.Wrap(err, "failed to make genesis state")
+		return errors.Wrap(err, "failed to make genesis state")
 	}
 	blockStore := bc.NewBlockStore(blockStoreDB)
 	proxyApp := proxy.NewAppConns(proxy.NewLocalClientCreator(app))
 	proxyApp.SetLogger(logger.With("module", "proxy"))
 	if err := proxyApp.Start(); err != nil {
-		return nil, errors.Wrap(err, "failed to start proxy app connections")
+		return errors.Wrap(err, "failed to start proxy app connections")
 	}
 	defer proxyApp.Stop()
 
 	eventBus := types.NewEventBus()
 	eventBus.SetLogger(logger.With("module", "events"))
 	if err := eventBus.Start(); err != nil {
-		return nil, errors.Wrap(err, "failed to start event bus")
+		return errors.Wrap(err, "failed to start event bus")
 	}
 	defer eventBus.Stop()
 	mempool := sm.MockMempool{}
@@ -78,8 +78,6 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
 	/////////////////////////////////////////////////////////////////////////////
 
 	// set consensus wal to buffered WAL, which will write all incoming msgs to buffer
-	var b bytes.Buffer
-	wr := bufio.NewWriter(&b)
 	numBlocksWritten := make(chan struct{})
 	wal := newByteBufferWAL(logger, NewWALEncoder(wr), int64(numBlocks), numBlocksWritten)
 	// see wal.go#103
@@ -87,20 +85,32 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
 	consensusState.wal = wal
 
 	if err := consensusState.Start(); err != nil {
-		return nil, errors.Wrap(err, "failed to start consensus state")
+		return errors.Wrap(err, "failed to start consensus state")
 	}
 
 	select {
 	case <-numBlocksWritten:
 		consensusState.Stop()
-		wr.Flush()
-		return b.Bytes(), nil
+		return nil
 	case <-time.After(1 * time.Minute):
 		consensusState.Stop()
-		return []byte{}, fmt.Errorf("waited too long for tendermint to produce %d blocks (grep logs for `wal_generator`)", numBlocks)
+		return fmt.Errorf("waited too long for tendermint to produce %d blocks (grep logs for `wal_generator`)", numBlocks)
 	}
 }
 
+//WALWithNBlocks returns a WAL content with numBlocks.
+func WALWithNBlocks(numBlocks int) (data []byte, err error) {
+	var b bytes.Buffer
+	wr := bufio.NewWriter(&b)
+
+	if err := WALGenerateNBlocks(wr, numBlocks); err != nil {
+		return []byte{}, err
+	}
+
+	wr.Flush()
+	return b.Bytes(), nil
+}
+
 // f**ing long, but unique for each test
 func makePathname() string {
 	// get path

consensus/wal_test.go

@@ -4,11 +4,16 @@ import (
 	"bytes"
 	"crypto/rand"
 	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+
 	// "sync"
 	"testing"
 	"time"
 
 	"github.com/tendermint/tendermint/consensus/types"
+	"github.com/tendermint/tendermint/libs/autofile"
 	tmtypes "github.com/tendermint/tendermint/types"
 	tmtime "github.com/tendermint/tendermint/types/time"
 
@@ -16,6 +21,51 @@ import (
 	"github.com/stretchr/testify/require"
 )
 
+func TestWALTruncate(t *testing.T) {
+	walDir, err := ioutil.TempDir("", "wal")
+	if err != nil {
+		panic(fmt.Errorf("failed to create temp WAL file: %v", err))
+	}
+	defer os.RemoveAll(walDir)
+
+	walFile := filepath.Join(walDir, "wal")
+
+	//this magic number 4K can truncate the content when RotateFile. defaultHeadSizeLimit(10M) is hard to simulate.
+	//this magic number 1 * time.Millisecond make RotateFile check frequently. defaultGroupCheckDuration(5s) is hard to simulate.
+	wal, err := NewWAL(walFile, autofile.GroupHeadSizeLimit(4096), autofile.GroupCheckDuration(1*time.Millisecond))
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	wal.Start()
+	defer wal.Stop()
+
+	//60 block's size nearly 70K, greater than group's headBuf size(4096 * 10), when headBuf is full, truncate content will Flush to the file.
+	//at this time, RotateFile is called, truncate content exist in each file.
+	err = WALGenerateNBlocks(wal.Group(), 60)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	time.Sleep(1 * time.Millisecond) //wait groupCheckDuration, make sure RotateFile run
+
+	wal.Group().Flush()
+
+	h := int64(50)
+	gr, found, err := wal.SearchForEndHeight(h, &WALSearchOptions{})
+	assert.NoError(t, err, fmt.Sprintf("expected not to err on height %d", h))
+	assert.True(t, found, fmt.Sprintf("expected to find end height for %d", h))
+	assert.NotNil(t, gr, "expected group not to be nil")
+	defer gr.Close()
+
+	dec := NewWALDecoder(gr)
+	msg, err := dec.Decode()
+	assert.NoError(t, err, "expected to decode a message")
+	rs, ok := msg.Msg.(tmtypes.EventDataRoundState)
+	assert.True(t, ok, "expected message of type EventDataRoundState")
+	assert.Equal(t, rs.Height, h+1, fmt.Sprintf("wrong height"))
+}
+
 func TestWALEncoderDecoder(t *testing.T) {
 	now := tmtime.Now()
 	msgs := []TimedWALMessage{

crypto/armor/armor.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 	"io/ioutil"
 
-	"golang.org/x/crypto/openpgp/armor"
+	"golang.org/x/crypto/openpgp/armor" // forked to github.com/tendermint/crypto
 )
 
 func EncodeArmor(blockType string, headers map[string]string, data []byte) string {

crypto/ed25519/ed25519.go

@@ -6,9 +6,9 @@ import (
 	"fmt"
 	"io"
 
-	"github.com/tendermint/ed25519"
-	"github.com/tendermint/ed25519/extra25519"
 	amino "github.com/tendermint/go-amino"
+	"golang.org/x/crypto/ed25519" // forked to github.com/tendermint/crypto
+
 	"github.com/tendermint/tendermint/crypto"
 	"github.com/tendermint/tendermint/crypto/tmhash"
 )
@@ -46,9 +46,14 @@ func (privKey PrivKeyEd25519) Bytes() []byte {
 }
 
 // Sign produces a signature on the provided message.
+// This assumes the privkey is wellformed in the golang format.
+// The first 32 bytes should be random,
+// corresponding to the normal ed25519 private key.
+// The latter 32 bytes should be the compressed public key.
+// If these conditions aren't met, Sign will panic or produce an
+// incorrect signature.
 func (privKey PrivKeyEd25519) Sign(msg []byte) ([]byte, error) {
-	privKeyBytes := [64]byte(privKey)
-	signatureBytes := ed25519.Sign(&privKeyBytes, msg)
+	signatureBytes := ed25519.Sign(privKey[:], msg)
 	return signatureBytes[:], nil
 }
 
@@ -65,14 +70,14 @@ func (privKey PrivKeyEd25519) PubKey() crypto.PubKey {
 			break
 		}
 	}
-	if initialized {
-		var pubkeyBytes [PubKeyEd25519Size]byte
-		copy(pubkeyBytes[:], privKeyBytes[32:])
-		return PubKeyEd25519(pubkeyBytes)
+
+	if !initialized {
+		panic("Expected PrivKeyEd25519 to include concatenated pubkey bytes")
 	}
 
-	pubBytes := *ed25519.MakePublicKey(&privKeyBytes)
-	return PubKeyEd25519(pubBytes)
+	var pubkeyBytes [PubKeyEd25519Size]byte
+	copy(pubkeyBytes[:], privKeyBytes[32:])
+	return PubKeyEd25519(pubkeyBytes)
 }
 
 // Equals - you probably don't need to use this.
@@ -85,17 +90,6 @@ func (privKey PrivKeyEd25519) Equals(other crypto.PrivKey) bool {
 	}
 }
 
-// ToCurve25519 takes a private key and returns its representation on
-// Curve25519. Curve25519 is birationally equivalent to Edwards25519,
-// which Ed25519 uses internally. This method is intended for use in
-// an X25519 Diffie Hellman key exchange.
-func (privKey PrivKeyEd25519) ToCurve25519() *[PubKeyEd25519Size]byte {
-	keyCurve25519 := new([32]byte)
-	privKeyBytes := [64]byte(privKey)
-	extra25519.PrivateKeyToCurve25519(keyCurve25519, &privKeyBytes)
-	return keyCurve25519
-}
-
 // GenPrivKey generates a new ed25519 private key.
 // It uses OS randomness in conjunction with the current global random seed
 // in tendermint/libs/common to generate the private key.
@@ -105,16 +99,16 @@ func GenPrivKey() PrivKeyEd25519 {
 
 // genPrivKey generates a new ed25519 private key using the provided reader.
 func genPrivKey(rand io.Reader) PrivKeyEd25519 {
-	privKey := new([64]byte)
-	_, err := io.ReadFull(rand, privKey[:32])
+	seed := make([]byte, 32)
+	_, err := io.ReadFull(rand, seed[:])
 	if err != nil {
 		panic(err)
 	}
-	// ed25519.MakePublicKey(privKey) alters the last 32 bytes of privKey.
-	// It places the pubkey in the last 32 bytes of privKey, and returns the
-	// public key.
-	ed25519.MakePublicKey(privKey)
-	return PrivKeyEd25519(*privKey)
+
+	privKey := ed25519.NewKeyFromSeed(seed)
+	var privKeyEd PrivKeyEd25519
+	copy(privKeyEd[:], privKey)
+	return privKeyEd
 }
 
 // GenPrivKeyFromSecret hashes the secret with SHA2, and uses
@@ -122,14 +116,12 @@ func genPrivKey(rand io.Reader) PrivKeyEd25519 {
 // NOTE: secret should be the output of a KDF like bcrypt,
 // if it's derived from user input.
 func GenPrivKeyFromSecret(secret []byte) PrivKeyEd25519 {
-	privKey32 := crypto.Sha256(secret) // Not Ripemd160 because we want 32 bytes.
-	privKey := new([64]byte)
-	copy(privKey[:32], privKey32)
-	// ed25519.MakePublicKey(privKey) alters the last 32 bytes of privKey.
-	// It places the pubkey in the last 32 bytes of privKey, and returns the
-	// public key.
-	ed25519.MakePublicKey(privKey)
-	return PrivKeyEd25519(*privKey)
+	seed := crypto.Sha256(secret) // Not Ripemd160 because we want 32 bytes.
+
+	privKey := ed25519.NewKeyFromSeed(seed)
+	var privKeyEd PrivKeyEd25519
+	copy(privKeyEd[:], privKey)
+	return privKeyEd
 }
 
 //-------------------------------------
@@ -156,30 +148,12 @@ func (pubKey PubKeyEd25519) Bytes() []byte {
 	return bz
 }
 
-func (pubKey PubKeyEd25519) VerifyBytes(msg []byte, sig_ []byte) bool {
+func (pubKey PubKeyEd25519) VerifyBytes(msg []byte, sig []byte) bool {
 	// make sure we use the same algorithm to sign
-	if len(sig_) != SignatureSize {
+	if len(sig) != SignatureSize {
 		return false
 	}
-	sig := new([SignatureSize]byte)
-	copy(sig[:], sig_)
-	pubKeyBytes := [PubKeyEd25519Size]byte(pubKey)
-	return ed25519.Verify(&pubKeyBytes, msg, sig)
-}
-
-// ToCurve25519 takes a public key and returns its representation on
-// Curve25519. Curve25519 is birationally equivalent to Edwards25519,
-// which Ed25519 uses internally. This method is intended for use in
-// an X25519 Diffie Hellman key exchange.
-//
-// If there is an error, then this function returns nil.
-func (pubKey PubKeyEd25519) ToCurve25519() *[PubKeyEd25519Size]byte {
-	keyCurve25519, pubKeyBytes := new([PubKeyEd25519Size]byte), [PubKeyEd25519Size]byte(pubKey)
-	ok := extra25519.PublicKeyToCurve25519(keyCurve25519, &pubKeyBytes)
-	if !ok {
-		return nil
-	}
-	return keyCurve25519
+	return ed25519.Verify(pubKey[:], msg, sig)
 }
 
 func (pubKey PubKeyEd25519) String() string {

crypto/encoding/amino/amino.go

@@ -1,8 +1,9 @@
 package cryptoAmino
 
 import (
-	amino "github.com/tendermint/go-amino"
+	"reflect"
 
+	amino "github.com/tendermint/go-amino"
 	"github.com/tendermint/tendermint/crypto"
 	"github.com/tendermint/tendermint/crypto/ed25519"
 	"github.com/tendermint/tendermint/crypto/multisig"
@@ -11,6 +12,12 @@ import (
 
 var cdc = amino.NewCodec()
 
+// routeTable is used to map public key concrete types back
+// to their amino routes. This should eventually be handled
+// by amino. Example usage:
+// routeTable[reflect.TypeOf(ed25519.PubKeyEd25519{})] = ed25519.PubKeyAminoRoute
+var routeTable = make(map[reflect.Type]string, 3)
+
 func init() {
 	// NOTE: It's important that there be no conflicts here,
 	// as that would change the canonical representations,
@@ -19,6 +26,20 @@ func init() {
 	// https://github.com/tendermint/go-amino/issues/9
 	// is resolved
 	RegisterAmino(cdc)
+
+	// TODO: Have amino provide a way to go from concrete struct to route directly.
+	// Its currently a private API
+	routeTable[reflect.TypeOf(ed25519.PubKeyEd25519{})] = ed25519.PubKeyAminoRoute
+	routeTable[reflect.TypeOf(secp256k1.PubKeySecp256k1{})] = secp256k1.PubKeyAminoRoute
+	routeTable[reflect.TypeOf(&multisig.PubKeyMultisigThreshold{})] = multisig.PubKeyMultisigThresholdAminoRoute
+}
+
+// PubkeyAminoRoute returns the amino route of a pubkey
+// cdc is currently passed in, as eventually this will not be using
+// a package level codec.
+func PubkeyAminoRoute(cdc *amino.Codec, key crypto.PubKey) (string, bool) {
+	route, found := routeTable[reflect.TypeOf(key)]
+	return route, found
 }
 
 // RegisterAmino registers all crypto related types in the given (amino) codec.

crypto/encoding/amino/encode_test.go

@@ -8,6 +8,7 @@ import (
 	"github.com/stretchr/testify/require"
 	"github.com/tendermint/tendermint/crypto"
 	"github.com/tendermint/tendermint/crypto/ed25519"
+	"github.com/tendermint/tendermint/crypto/multisig"
 	"github.com/tendermint/tendermint/crypto/secp256k1"
 )
 
@@ -119,11 +120,29 @@ func TestNilEncodings(t *testing.T) {
 	var e, f crypto.PrivKey
 	checkAminoJSON(t, &e, &f, true)
 	assert.EqualValues(t, e, f)
-
 }
 
 func TestPubKeyInvalidDataProperReturnsEmpty(t *testing.T) {
 	pk, err := PubKeyFromBytes([]byte("foo"))
-	require.NotNil(t, err, "expecting a non-nil error")
-	require.Nil(t, pk, "expecting an empty public key on error")
+	require.NotNil(t, err)
+	require.Nil(t, pk)
+}
+
+func TestPubkeyAminoRoute(t *testing.T) {
+	tests := []struct {
+		key   crypto.PubKey
+		want  string
+		found bool
+	}{
+		{ed25519.PubKeyEd25519{}, ed25519.PubKeyAminoRoute, true},
+		{secp256k1.PubKeySecp256k1{}, secp256k1.PubKeyAminoRoute, true},
+		{&multisig.PubKeyMultisigThreshold{}, multisig.PubKeyMultisigThresholdAminoRoute, true},
+	}
+	for i, tc := range tests {
+		got, found := PubkeyAminoRoute(cdc, tc.key)
+		require.Equal(t, tc.found, found, "not equal on tc %d", i)
+		if tc.found {
+			require.Equal(t, tc.want, got, "not equal on tc %d", i)
+		}
+	}
 }

crypto/hash.go

@@ -3,7 +3,7 @@ package crypto
 import (
 	"crypto/sha256"
 
-	"golang.org/x/crypto/ripemd160"
+	"golang.org/x/crypto/ripemd160" // forked to github.com/tendermint/crypto
 )
 
 func Sha256(bytes []byte) []byte {

crypto/merkle/compile.sh

@@ -0,0 +1,6 @@
+#! /bin/bash
+
+protoc --gogo_out=. -I $GOPATH/src/ -I . -I $GOPATH/src/github.com/gogo/protobuf/protobuf merkle.proto
+echo "--> adding nolint declarations to protobuf generated files"
+awk '/package merkle/ { print "//nolint: gas"; print; next }1' merkle.pb.go > merkle.pb.go.new
+mv merkle.pb.go.new merkle.pb.go

crypto/merkle/merkle.pb.go

@@ -0,0 +1,792 @@
+// Code generated by protoc-gen-gogo. DO NOT EDIT.
+// source: crypto/merkle/merkle.proto
+
+package merkle
+
+import proto "github.com/gogo/protobuf/proto"
+import fmt "fmt"
+import math "math"
+import _ "github.com/gogo/protobuf/gogoproto"
+
+import bytes "bytes"
+
+import io "io"
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ = proto.Marshal
+var _ = fmt.Errorf
+var _ = math.Inf
+
+// This is a compile-time assertion to ensure that this generated file
+// is compatible with the proto package it is being compiled against.
+// A compilation error at this line likely means your copy of the
+// proto package needs to be updated.
+const _ = proto.GoGoProtoPackageIsVersion2 // please upgrade the proto package
+
+// ProofOp defines an operation used for calculating Merkle root
+// The data could be arbitrary format, providing nessecary data
+// for example neighbouring node hash
+type ProofOp struct {
+	Type                 string   `protobuf:"bytes,1,opt,name=type,proto3" json:"type,omitempty"`
+	Key                  []byte   `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty"`
+	Data                 []byte   `protobuf:"bytes,3,opt,name=data,proto3" json:"data,omitempty"`
+	XXX_NoUnkeyedLiteral struct{} `json:"-"`
+	XXX_unrecognized     []byte   `json:"-"`
+	XXX_sizecache        int32    `json:"-"`
+}
+
+func (m *ProofOp) Reset()         { *m = ProofOp{} }
+func (m *ProofOp) String() string { return proto.CompactTextString(m) }
+func (*ProofOp) ProtoMessage()    {}
+func (*ProofOp) Descriptor() ([]byte, []int) {
+	return fileDescriptor_merkle_5d3f6051907285da, []int{0}
+}
+func (m *ProofOp) XXX_Unmarshal(b []byte) error {
+	return m.Unmarshal(b)
+}
+func (m *ProofOp) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
+	if deterministic {
+		return xxx_messageInfo_ProofOp.Marshal(b, m, deterministic)
+	} else {
+		b = b[:cap(b)]
+		n, err := m.MarshalTo(b)
+		if err != nil {
+			return nil, err
+		}
+		return b[:n], nil
+	}
+}
+func (dst *ProofOp) XXX_Merge(src proto.Message) {
+	xxx_messageInfo_ProofOp.Merge(dst, src)
+}
+func (m *ProofOp) XXX_Size() int {
+	return m.Size()
+}
+func (m *ProofOp) XXX_DiscardUnknown() {
+	xxx_messageInfo_ProofOp.DiscardUnknown(m)
+}
+
+var xxx_messageInfo_ProofOp proto.InternalMessageInfo
+
+func (m *ProofOp) GetType() string {
+	if m != nil {
+		return m.Type
+	}
+	return ""
+}
+
+func (m *ProofOp) GetKey() []byte {
+	if m != nil {
+		return m.Key
+	}
+	return nil
+}
+
+func (m *ProofOp) GetData() []byte {
+	if m != nil {
+		return m.Data
+	}
+	return nil
+}
+
+// Proof is Merkle proof defined by the list of ProofOps
+type Proof struct {
+	Ops                  []ProofOp `protobuf:"bytes,1,rep,name=ops" json:"ops"`
+	XXX_NoUnkeyedLiteral struct{}  `json:"-"`
+	XXX_unrecognized     []byte    `json:"-"`
+	XXX_sizecache        int32     `json:"-"`
+}
+
+func (m *Proof) Reset()         { *m = Proof{} }
+func (m *Proof) String() string { return proto.CompactTextString(m) }
+func (*Proof) ProtoMessage()    {}
+func (*Proof) Descriptor() ([]byte, []int) {
+	return fileDescriptor_merkle_5d3f6051907285da, []int{1}
+}
+func (m *Proof) XXX_Unmarshal(b []byte) error {
+	return m.Unmarshal(b)
+}
+func (m *Proof) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
+	if deterministic {
+		return xxx_messageInfo_Proof.Marshal(b, m, deterministic)
+	} else {
+		b = b[:cap(b)]
+		n, err := m.MarshalTo(b)
+		if err != nil {
+			return nil, err
+		}
+		return b[:n], nil
+	}
+}
+func (dst *Proof) XXX_Merge(src proto.Message) {
+	xxx_messageInfo_Proof.Merge(dst, src)
+}
+func (m *Proof) XXX_Size() int {
+	return m.Size()
+}
+func (m *Proof) XXX_DiscardUnknown() {
+	xxx_messageInfo_Proof.DiscardUnknown(m)
+}
+
+var xxx_messageInfo_Proof proto.InternalMessageInfo
+
+func (m *Proof) GetOps() []ProofOp {
+	if m != nil {
+		return m.Ops
+	}
+	return nil
+}
+
+func init() {
+	proto.RegisterType((*ProofOp)(nil), "merkle.ProofOp")
+	proto.RegisterType((*Proof)(nil), "merkle.Proof")
+}
+func (this *ProofOp) Equal(that interface{}) bool {
+	if that == nil {
+		return this == nil
+	}
+
+	that1, ok := that.(*ProofOp)
+	if !ok {
+		that2, ok := that.(ProofOp)
+		if ok {
+			that1 = &that2
+		} else {
+			return false
+		}
+	}
+	if that1 == nil {
+		return this == nil
+	} else if this == nil {
+		return false
+	}
+	if this.Type != that1.Type {
+		return false
+	}
+	if !bytes.Equal(this.Key, that1.Key) {
+		return false
+	}
+	if !bytes.Equal(this.Data, that1.Data) {
+		return false
+	}
+	if !bytes.Equal(this.XXX_unrecognized, that1.XXX_unrecognized) {
+		return false
+	}
+	return true
+}
+func (this *Proof) Equal(that interface{}) bool {
+	if that == nil {
+		return this == nil
+	}
+
+	that1, ok := that.(*Proof)
+	if !ok {
+		that2, ok := that.(Proof)
+		if ok {
+			that1 = &that2
+		} else {
+			return false
+		}
+	}
+	if that1 == nil {
+		return this == nil
+	} else if this == nil {
+		return false
+	}
+	if len(this.Ops) != len(that1.Ops) {
+		return false
+	}
+	for i := range this.Ops {
+		if !this.Ops[i].Equal(&that1.Ops[i]) {
+			return false
+		}
+	}
+	if !bytes.Equal(this.XXX_unrecognized, that1.XXX_unrecognized) {
+		return false
+	}
+	return true
+}
+func (m *ProofOp) Marshal() (dAtA []byte, err error) {
+	size := m.Size()
+	dAtA = make([]byte, size)
+	n, err := m.MarshalTo(dAtA)
+	if err != nil {
+		return nil, err
+	}
+	return dAtA[:n], nil
+}
+
+func (m *ProofOp) MarshalTo(dAtA []byte) (int, error) {
+	var i int
+	_ = i
+	var l int
+	_ = l
+	if len(m.Type) > 0 {
+		dAtA[i] = 0xa
+		i++
+		i = encodeVarintMerkle(dAtA, i, uint64(len(m.Type)))
+		i += copy(dAtA[i:], m.Type)
+	}
+	if len(m.Key) > 0 {
+		dAtA[i] = 0x12
+		i++
+		i = encodeVarintMerkle(dAtA, i, uint64(len(m.Key)))
+		i += copy(dAtA[i:], m.Key)
+	}
+	if len(m.Data) > 0 {
+		dAtA[i] = 0x1a
+		i++
+		i = encodeVarintMerkle(dAtA, i, uint64(len(m.Data)))
+		i += copy(dAtA[i:], m.Data)
+	}
+	if m.XXX_unrecognized != nil {
+		i += copy(dAtA[i:], m.XXX_unrecognized)
+	}
+	return i, nil
+}
+
+func (m *Proof) Marshal() (dAtA []byte, err error) {
+	size := m.Size()
+	dAtA = make([]byte, size)
+	n, err := m.MarshalTo(dAtA)
+	if err != nil {
+		return nil, err
+	}
+	return dAtA[:n], nil
+}
+
+func (m *Proof) MarshalTo(dAtA []byte) (int, error) {
+	var i int
+	_ = i
+	var l int
+	_ = l
+	if len(m.Ops) > 0 {
+		for _, msg := range m.Ops {
+			dAtA[i] = 0xa
+			i++
+			i = encodeVarintMerkle(dAtA, i, uint64(msg.Size()))
+			n, err := msg.MarshalTo(dAtA[i:])
+			if err != nil {
+				return 0, err
+			}
+			i += n
+		}
+	}
+	if m.XXX_unrecognized != nil {
+		i += copy(dAtA[i:], m.XXX_unrecognized)
+	}
+	return i, nil
+}
+
+func encodeVarintMerkle(dAtA []byte, offset int, v uint64) int {
+	for v >= 1<<7 {
+		dAtA[offset] = uint8(v&0x7f | 0x80)
+		v >>= 7
+		offset++
+	}
+	dAtA[offset] = uint8(v)
+	return offset + 1
+}
+func NewPopulatedProofOp(r randyMerkle, easy bool) *ProofOp {
+	this := &ProofOp{}
+	this.Type = string(randStringMerkle(r))
+	v1 := r.Intn(100)
+	this.Key = make([]byte, v1)
+	for i := 0; i < v1; i++ {
+		this.Key[i] = byte(r.Intn(256))
+	}
+	v2 := r.Intn(100)
+	this.Data = make([]byte, v2)
+	for i := 0; i < v2; i++ {
+		this.Data[i] = byte(r.Intn(256))
+	}
+	if !easy && r.Intn(10) != 0 {
+		this.XXX_unrecognized = randUnrecognizedMerkle(r, 4)
+	}
+	return this
+}
+
+func NewPopulatedProof(r randyMerkle, easy bool) *Proof {
+	this := &Proof{}
+	if r.Intn(10) != 0 {
+		v3 := r.Intn(5)
+		this.Ops = make([]ProofOp, v3)
+		for i := 0; i < v3; i++ {
+			v4 := NewPopulatedProofOp(r, easy)
+			this.Ops[i] = *v4
+		}
+	}
+	if !easy && r.Intn(10) != 0 {
+		this.XXX_unrecognized = randUnrecognizedMerkle(r, 2)
+	}
+	return this
+}
+
+type randyMerkle interface {
+	Float32() float32
+	Float64() float64
+	Int63() int64
+	Int31() int32
+	Uint32() uint32
+	Intn(n int) int
+}
+
+func randUTF8RuneMerkle(r randyMerkle) rune {
+	ru := r.Intn(62)
+	if ru < 10 {
+		return rune(ru + 48)
+	} else if ru < 36 {
+		return rune(ru + 55)
+	}
+	return rune(ru + 61)
+}
+func randStringMerkle(r randyMerkle) string {
+	v5 := r.Intn(100)
+	tmps := make([]rune, v5)
+	for i := 0; i < v5; i++ {
+		tmps[i] = randUTF8RuneMerkle(r)
+	}
+	return string(tmps)
+}
+func randUnrecognizedMerkle(r randyMerkle, maxFieldNumber int) (dAtA []byte) {
+	l := r.Intn(5)
+	for i := 0; i < l; i++ {
+		wire := r.Intn(4)
+		if wire == 3 {
+			wire = 5
+		}
+		fieldNumber := maxFieldNumber + r.Intn(100)
+		dAtA = randFieldMerkle(dAtA, r, fieldNumber, wire)
+	}
+	return dAtA
+}
+func randFieldMerkle(dAtA []byte, r randyMerkle, fieldNumber int, wire int) []byte {
+	key := uint32(fieldNumber)<<3 | uint32(wire)
+	switch wire {
+	case 0:
+		dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
+		v6 := r.Int63()
+		if r.Intn(2) == 0 {
+			v6 *= -1
+		}
+		dAtA = encodeVarintPopulateMerkle(dAtA, uint64(v6))
+	case 1:
+		dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
+		dAtA = append(dAtA, byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)))
+	case 2:
+		dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
+		ll := r.Intn(100)
+		dAtA = encodeVarintPopulateMerkle(dAtA, uint64(ll))
+		for j := 0; j < ll; j++ {
+			dAtA = append(dAtA, byte(r.Intn(256)))
+		}
+	default:
+		dAtA = encodeVarintPopulateMerkle(dAtA, uint64(key))
+		dAtA = append(dAtA, byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)), byte(r.Intn(256)))
+	}
+	return dAtA
+}
+func encodeVarintPopulateMerkle(dAtA []byte, v uint64) []byte {
+	for v >= 1<<7 {
+		dAtA = append(dAtA, uint8(uint64(v)&0x7f|0x80))
+		v >>= 7
+	}
+	dAtA = append(dAtA, uint8(v))
+	return dAtA
+}
+func (m *ProofOp) Size() (n int) {
+	var l int
+	_ = l
+	l = len(m.Type)
+	if l > 0 {
+		n += 1 + l + sovMerkle(uint64(l))
+	}
+	l = len(m.Key)
+	if l > 0 {
+		n += 1 + l + sovMerkle(uint64(l))
+	}
+	l = len(m.Data)
+	if l > 0 {
+		n += 1 + l + sovMerkle(uint64(l))
+	}
+	if m.XXX_unrecognized != nil {
+		n += len(m.XXX_unrecognized)
+	}
+	return n
+}
+
+func (m *Proof) Size() (n int) {
+	var l int
+	_ = l
+	if len(m.Ops) > 0 {
+		for _, e := range m.Ops {
+			l = e.Size()
+			n += 1 + l + sovMerkle(uint64(l))
+		}
+	}
+	if m.XXX_unrecognized != nil {
+		n += len(m.XXX_unrecognized)
+	}
+	return n
+}
+
+func sovMerkle(x uint64) (n int) {
+	for {
+		n++
+		x >>= 7
+		if x == 0 {
+			break
+		}
+	}
+	return n
+}
+func sozMerkle(x uint64) (n int) {
+	return sovMerkle(uint64((x << 1) ^ uint64((int64(x) >> 63))))
+}
+func (m *ProofOp) Unmarshal(dAtA []byte) error {
+	l := len(dAtA)
+	iNdEx := 0
+	for iNdEx < l {
+		preIndex := iNdEx
+		var wire uint64
+		for shift := uint(0); ; shift += 7 {
+			if shift >= 64 {
+				return ErrIntOverflowMerkle
+			}
+			if iNdEx >= l {
+				return io.ErrUnexpectedEOF
+			}
+			b := dAtA[iNdEx]
+			iNdEx++
+			wire |= (uint64(b) & 0x7F) << shift
+			if b < 0x80 {
+				break
+			}
+		}
+		fieldNum := int32(wire >> 3)
+		wireType := int(wire & 0x7)
+		if wireType == 4 {
+			return fmt.Errorf("proto: ProofOp: wiretype end group for non-group")
+		}
+		if fieldNum <= 0 {
+			return fmt.Errorf("proto: ProofOp: illegal tag %d (wire type %d)", fieldNum, wire)
+		}
+		switch fieldNum {
+		case 1:
+			if wireType != 2 {
+				return fmt.Errorf("proto: wrong wireType = %d for field Type", wireType)
+			}
+			var stringLen uint64
+			for shift := uint(0); ; shift += 7 {
+				if shift >= 64 {
+					return ErrIntOverflowMerkle
+				}
+				if iNdEx >= l {
+					return io.ErrUnexpectedEOF
+				}
+				b := dAtA[iNdEx]
+				iNdEx++
+				stringLen |= (uint64(b) & 0x7F) << shift
+				if b < 0x80 {
+					break
+				}
+			}
+			intStringLen := int(stringLen)
+			if intStringLen < 0 {
+				return ErrInvalidLengthMerkle
+			}
+			postIndex := iNdEx + intStringLen
+			if postIndex > l {
+				return io.ErrUnexpectedEOF
+			}
+			m.Type = string(dAtA[iNdEx:postIndex])
+			iNdEx = postIndex
+		case 2:
+			if wireType != 2 {
+				return fmt.Errorf("proto: wrong wireType = %d for field Key", wireType)
+			}
+			var byteLen int
+			for shift := uint(0); ; shift += 7 {
+				if shift >= 64 {
+					return ErrIntOverflowMerkle
+				}
+				if iNdEx >= l {
+					return io.ErrUnexpectedEOF
+				}
+				b := dAtA[iNdEx]
+				iNdEx++
+				byteLen |= (int(b) & 0x7F) << shift
+				if b < 0x80 {
+					break
+				}
+			}
+			if byteLen < 0 {
+				return ErrInvalidLengthMerkle
+			}
+			postIndex := iNdEx + byteLen
+			if postIndex > l {
+				return io.ErrUnexpectedEOF
+			}
+			m.Key = append(m.Key[:0], dAtA[iNdEx:postIndex]...)
+			if m.Key == nil {
+				m.Key = []byte{}
+			}
+			iNdEx = postIndex
+		case 3:
+			if wireType != 2 {
+				return fmt.Errorf("proto: wrong wireType = %d for field Data", wireType)
+			}
+			var byteLen int
+			for shift := uint(0); ; shift += 7 {
+				if shift >= 64 {
+					return ErrIntOverflowMerkle
+				}
+				if iNdEx >= l {
+					return io.ErrUnexpectedEOF
+				}
+				b := dAtA[iNdEx]
+				iNdEx++
+				byteLen |= (int(b) & 0x7F) << shift
+				if b < 0x80 {
+					break
+				}
+			}
+			if byteLen < 0 {
+				return ErrInvalidLengthMerkle
+			}
+			postIndex := iNdEx + byteLen
+			if postIndex > l {
+				return io.ErrUnexpectedEOF
+			}
+			m.Data = append(m.Data[:0], dAtA[iNdEx:postIndex]...)
+			if m.Data == nil {
+				m.Data = []byte{}
+			}
+			iNdEx = postIndex
+		default:
+			iNdEx = preIndex
+			skippy, err := skipMerkle(dAtA[iNdEx:])
+			if err != nil {
+				return err
+			}
+			if skippy < 0 {
+				return ErrInvalidLengthMerkle
+			}
+			if (iNdEx + skippy) > l {
+				return io.ErrUnexpectedEOF
+			}
+			m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...)
+			iNdEx += skippy
+		}
+	}
+
+	if iNdEx > l {
+		return io.ErrUnexpectedEOF
+	}
+	return nil
+}
+func (m *Proof) Unmarshal(dAtA []byte) error {
+	l := len(dAtA)
+	iNdEx := 0
+	for iNdEx < l {
+		preIndex := iNdEx
+		var wire uint64
+		for shift := uint(0); ; shift += 7 {
+			if shift >= 64 {
+				return ErrIntOverflowMerkle
+			}
+			if iNdEx >= l {
+				return io.ErrUnexpectedEOF
+			}
+			b := dAtA[iNdEx]
+			iNdEx++
+			wire |= (uint64(b) & 0x7F) << shift
+			if b < 0x80 {
+				break
+			}
+		}
+		fieldNum := int32(wire >> 3)
+		wireType := int(wire & 0x7)
+		if wireType == 4 {
+			return fmt.Errorf("proto: Proof: wiretype end group for non-group")
+		}
+		if fieldNum <= 0 {
+			return fmt.Errorf("proto: Proof: illegal tag %d (wire type %d)", fieldNum, wire)
+		}
+		switch fieldNum {
+		case 1:
+			if wireType != 2 {
+				return fmt.Errorf("proto: wrong wireType = %d for field Ops", wireType)
+			}
+			var msglen int
+			for shift := uint(0); ; shift += 7 {
+				if shift >= 64 {
+					return ErrIntOverflowMerkle
+				}
+				if iNdEx >= l {
+					return io.ErrUnexpectedEOF
+				}
+				b := dAtA[iNdEx]
+				iNdEx++
+				msglen |= (int(b) & 0x7F) << shift
+				if b < 0x80 {
+					break
+				}
+			}
+			if msglen < 0 {
+				return ErrInvalidLengthMerkle
+			}
+			postIndex := iNdEx + msglen
+			if postIndex > l {
+				return io.ErrUnexpectedEOF
+			}
+			m.Ops = append(m.Ops, ProofOp{})
+			if err := m.Ops[len(m.Ops)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil {
+				return err
+			}
+			iNdEx = postIndex
+		default:
+			iNdEx = preIndex
+			skippy, err := skipMerkle(dAtA[iNdEx:])
+			if err != nil {
+				return err
+			}
+			if skippy < 0 {
+				return ErrInvalidLengthMerkle
+			}
+			if (iNdEx + skippy) > l {
+				return io.ErrUnexpectedEOF
+			}
+			m.XXX_unrecognized = append(m.XXX_unrecognized, dAtA[iNdEx:iNdEx+skippy]...)
+			iNdEx += skippy
+		}
+	}
+
+	if iNdEx > l {
+		return io.ErrUnexpectedEOF
+	}
+	return nil
+}
+func skipMerkle(dAtA []byte) (n int, err error) {
+	l := len(dAtA)
+	iNdEx := 0
+	for iNdEx < l {
+		var wire uint64
+		for shift := uint(0); ; shift += 7 {
+			if shift >= 64 {
+				return 0, ErrIntOverflowMerkle
+			}
+			if iNdEx >= l {
+				return 0, io.ErrUnexpectedEOF
+			}
+			b := dAtA[iNdEx]
+			iNdEx++
+			wire |= (uint64(b) & 0x7F) << shift
+			if b < 0x80 {
+				break
+			}
+		}
+		wireType := int(wire & 0x7)
+		switch wireType {
+		case 0:
+			for shift := uint(0); ; shift += 7 {
+				if shift >= 64 {
+					return 0, ErrIntOverflowMerkle
+				}
+				if iNdEx >= l {
+					return 0, io.ErrUnexpectedEOF
+				}
+				iNdEx++
+				if dAtA[iNdEx-1] < 0x80 {
+					break
+				}
+			}
+			return iNdEx, nil
+		case 1:
+			iNdEx += 8
+			return iNdEx, nil
+		case 2:
+			var length int
+			for shift := uint(0); ; shift += 7 {
+				if shift >= 64 {
+					return 0, ErrIntOverflowMerkle
+				}
+				if iNdEx >= l {
+					return 0, io.ErrUnexpectedEOF
+				}
+				b := dAtA[iNdEx]
+				iNdEx++
+				length |= (int(b) & 0x7F) << shift
+				if b < 0x80 {
+					break
+				}
+			}
+			iNdEx += length
+			if length < 0 {
+				return 0, ErrInvalidLengthMerkle
+			}
+			return iNdEx, nil
+		case 3:
+			for {
+				var innerWire uint64
+				var start int = iNdEx
+				for shift := uint(0); ; shift += 7 {
+					if shift >= 64 {
+						return 0, ErrIntOverflowMerkle
+					}
+					if iNdEx >= l {
+						return 0, io.ErrUnexpectedEOF
+					}
+					b := dAtA[iNdEx]
+					iNdEx++
+					innerWire |= (uint64(b) & 0x7F) << shift
+					if b < 0x80 {
+						break
+					}
+				}
+				innerWireType := int(innerWire & 0x7)
+				if innerWireType == 4 {
+					break
+				}
+				next, err := skipMerkle(dAtA[start:])
+				if err != nil {
+					return 0, err
+				}
+				iNdEx = start + next
+			}
+			return iNdEx, nil
+		case 4:
+			return iNdEx, nil
+		case 5:
+			iNdEx += 4
+			return iNdEx, nil
+		default:
+			return 0, fmt.Errorf("proto: illegal wireType %d", wireType)
+		}
+	}
+	panic("unreachable")
+}
+
+var (
+	ErrInvalidLengthMerkle = fmt.Errorf("proto: negative length found during unmarshaling")
+	ErrIntOverflowMerkle   = fmt.Errorf("proto: integer overflow")
+)
+
+func init() { proto.RegisterFile("crypto/merkle/merkle.proto", fileDescriptor_merkle_5d3f6051907285da) }
+
+var fileDescriptor_merkle_5d3f6051907285da = []byte{
+	// 200 bytes of a gzipped FileDescriptorProto
+	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x4a, 0x2e, 0xaa, 0x2c,
+	0x28, 0xc9, 0xd7, 0xcf, 0x4d, 0x2d, 0xca, 0xce, 0x49, 0x85, 0x52, 0x7a, 0x05, 0x45, 0xf9, 0x25,
+	0xf9, 0x42, 0x6c, 0x10, 0x9e, 0x94, 0x6e, 0x7a, 0x66, 0x49, 0x46, 0x69, 0x92, 0x5e, 0x72, 0x7e,
+	0xae, 0x7e, 0x7a, 0x7e, 0x7a, 0xbe, 0x3e, 0x58, 0x3a, 0xa9, 0x34, 0x0d, 0xcc, 0x03, 0x73, 0xc0,
+	0x2c, 0x88, 0x36, 0x25, 0x67, 0x2e, 0xf6, 0x80, 0xa2, 0xfc, 0xfc, 0x34, 0xff, 0x02, 0x21, 0x21,
+	0x2e, 0x96, 0x92, 0xca, 0x82, 0x54, 0x09, 0x46, 0x05, 0x46, 0x0d, 0xce, 0x20, 0x30, 0x5b, 0x48,
+	0x80, 0x8b, 0x39, 0x3b, 0xb5, 0x52, 0x82, 0x49, 0x81, 0x51, 0x83, 0x27, 0x08, 0xc4, 0x04, 0xa9,
+	0x4a, 0x49, 0x2c, 0x49, 0x94, 0x60, 0x06, 0x0b, 0x81, 0xd9, 0x4a, 0x06, 0x5c, 0xac, 0x60, 0x43,
+	0x84, 0xd4, 0xb9, 0x98, 0xf3, 0x0b, 0x8a, 0x25, 0x18, 0x15, 0x98, 0x35, 0xb8, 0x8d, 0xf8, 0xf5,
+	0xa0, 0x0e, 0x84, 0x5a, 0xe0, 0xc4, 0x72, 0xe2, 0x9e, 0x3c, 0x43, 0x10, 0x48, 0x85, 0x93, 0xc8,
+	0x8f, 0x87, 0x72, 0x8c, 0x2b, 0x1e, 0xc9, 0x31, 0x9e, 0x78, 0x24, 0xc7, 0x78, 0xe1, 0x91, 0x1c,
+	0xe3, 0x83, 0x47, 0x72, 0x8c, 0x49, 0x6c, 0x60, 0x37, 0x19, 0x03, 0x02, 0x00, 0x00, 0xff, 0xff,
+	0xb9, 0x2b, 0x0f, 0xd1, 0xe8, 0x00, 0x00, 0x00,
+}

crypto/merkle/merkle.proto

@@ -0,0 +1,30 @@
+syntax = "proto3";
+package merkle;
+
+// For more information on gogo.proto, see:
+// https://github.com/gogo/protobuf/blob/master/extensions.md
+import "github.com/gogo/protobuf/gogoproto/gogo.proto";
+
+option (gogoproto.marshaler_all) = true;
+option (gogoproto.unmarshaler_all) = true;
+option (gogoproto.sizer_all) = true;
+
+option (gogoproto.populate_all) = true;
+option (gogoproto.equal_all) = true;
+
+//----------------------------------------
+// Message types
+
+// ProofOp defines an operation used for calculating Merkle root
+// The data could be arbitrary format, providing nessecary data
+// for example neighbouring node hash
+message ProofOp {
+  string type = 1;
+  bytes key = 2;
+  bytes data = 3;
+}
+
+// Proof is Merkle proof defined by the list of ProofOps
+message Proof {
+  repeated ProofOp ops = 1 [(gogoproto.nullable)=false];
+}

crypto/merkle/proof.go

@@ -0,0 +1,134 @@
+package merkle
+
+import (
+	"bytes"
+
+	cmn "github.com/tendermint/tendermint/libs/common"
+)
+
+//----------------------------------------
+// ProofOp gets converted to an instance of ProofOperator:
+
+// ProofOperator is a layer for calculating intermediate Merkle roots
+// when a series of Merkle trees are chained together.
+// Run() takes leaf values from a tree and returns the Merkle
+// root for the corresponding tree. It takes and returns a list of bytes
+// to allow multiple leaves to be part of a single proof, for instance in a range proof.
+// ProofOp() encodes the ProofOperator in a generic way so it can later be
+// decoded with OpDecoder.
+type ProofOperator interface {
+	Run([][]byte) ([][]byte, error)
+	GetKey() []byte
+	ProofOp() ProofOp
+}
+
+//----------------------------------------
+// Operations on a list of ProofOperators
+
+// ProofOperators is a slice of ProofOperator(s).
+// Each operator will be applied to the input value sequentially
+// and the last Merkle root will be verified with already known data
+type ProofOperators []ProofOperator
+
+func (poz ProofOperators) VerifyValue(root []byte, keypath string, value []byte) (err error) {
+	return poz.Verify(root, keypath, [][]byte{value})
+}
+
+func (poz ProofOperators) Verify(root []byte, keypath string, args [][]byte) (err error) {
+	keys, err := KeyPathToKeys(keypath)
+	if err != nil {
+		return
+	}
+
+	for i, op := range poz {
+		key := op.GetKey()
+		if len(key) != 0 {
+			if !bytes.Equal(keys[0], key) {
+				return cmn.NewError("Key mismatch on operation #%d: expected %+v but %+v", i, []byte(keys[0]), []byte(key))
+			}
+			keys = keys[1:]
+		}
+		args, err = op.Run(args)
+		if err != nil {
+			return
+		}
+	}
+	if !bytes.Equal(root, args[0]) {
+		return cmn.NewError("Calculated root hash is invalid: expected %+v but %+v", root, args[0])
+	}
+	if len(keys) != 0 {
+		return cmn.NewError("Keypath not consumed all")
+	}
+	return nil
+}
+
+//----------------------------------------
+// ProofRuntime - main entrypoint
+
+type OpDecoder func(ProofOp) (ProofOperator, error)
+
+type ProofRuntime struct {
+	decoders map[string]OpDecoder
+}
+
+func NewProofRuntime() *ProofRuntime {
+	return &ProofRuntime{
+		decoders: make(map[string]OpDecoder),
+	}
+}
+
+func (prt *ProofRuntime) RegisterOpDecoder(typ string, dec OpDecoder) {
+	_, ok := prt.decoders[typ]
+	if ok {
+		panic("already registered for type " + typ)
+	}
+	prt.decoders[typ] = dec
+}
+
+func (prt *ProofRuntime) Decode(pop ProofOp) (ProofOperator, error) {
+	decoder := prt.decoders[pop.Type]
+	if decoder == nil {
+		return nil, cmn.NewError("unrecognized proof type %v", pop.Type)
+	}
+	return decoder(pop)
+}
+
+func (prt *ProofRuntime) DecodeProof(proof *Proof) (ProofOperators, error) {
+	var poz ProofOperators
+	for _, pop := range proof.Ops {
+		operator, err := prt.Decode(pop)
+		if err != nil {
+			return nil, cmn.ErrorWrap(err, "decoding a proof operator")
+		}
+		poz = append(poz, operator)
+	}
+	return poz, nil
+}
+
+func (prt *ProofRuntime) VerifyValue(proof *Proof, root []byte, keypath string, value []byte) (err error) {
+	return prt.Verify(proof, root, keypath, [][]byte{value})
+}
+
+// TODO In the long run we'll need a method of classifcation of ops,
+// whether existence or absence or perhaps a third?
+func (prt *ProofRuntime) VerifyAbsence(proof *Proof, root []byte, keypath string) (err error) {
+	return prt.Verify(proof, root, keypath, nil)
+}
+
+func (prt *ProofRuntime) Verify(proof *Proof, root []byte, keypath string, args [][]byte) (err error) {
+	poz, err := prt.DecodeProof(proof)
+	if err != nil {
+		return cmn.ErrorWrap(err, "decoding proof")
+	}
+	return poz.Verify(root, keypath, args)
+}
+
+// DefaultProofRuntime only knows about Simple value
+// proofs.
+// To use e.g. IAVL proofs, register op-decoders as
+// defined in the IAVL package.
+func DefaultProofRuntime() (prt *ProofRuntime) {
+	prt = NewProofRuntime()
+	prt.RegisterOpDecoder(ProofOpSimpleValue, SimpleValueOpDecoder)
+	return
+}

crypto/merkle/proof_key_path.go

@@ -0,0 +1,111 @@
+package merkle
+
+import (
+	"encoding/hex"
+	"fmt"
+	"net/url"
+	"strings"
+
+	cmn "github.com/tendermint/tendermint/libs/common"
+)
+
+/*
+
+	For generalized Merkle proofs, each layer of the proof may require an
+	optional key.  The key may be encoded either by URL-encoding or
+	(upper-case) hex-encoding.
+	TODO: In the future, more encodings may be supported, like base32 (e.g.
+	/32:)
+
+	For example, for a Cosmos-SDK application where the first two proof layers
+	are SimpleValueOps, and the third proof layer is an IAVLValueOp, the keys
+	might look like:
+
+	0: []byte("App")
+	1: []byte("IBC")
+	2: []byte{0x01, 0x02, 0x03}
+
+	Assuming that we know that the first two layers are always ASCII texts, we
+	probably want to use URLEncoding for those, whereas the third layer will
+	require HEX encoding for efficient representation.
+
+	kp := new(KeyPath)
+	kp.AppendKey([]byte("App"), KeyEncodingURL)
+	kp.AppendKey([]byte("IBC"), KeyEncodingURL)
+	kp.AppendKey([]byte{0x01, 0x02, 0x03}, KeyEncodingURL)
+	kp.String() // Should return "/App/IBC/x:010203"
+
+	NOTE: Key paths must begin with a `/`.
+
+	NOTE: All encodings *MUST* work compatibly, such that you can choose to use
+	whatever encoding, and the decoded keys will always be the same.  In other
+	words, it's just as good to encode all three keys using URL encoding or HEX
+	encoding... it just wouldn't be optimal in terms of readability or space
+	efficiency.
+
+	NOTE: Punycode will never be supported here, because not all values can be
+	decoded.  For example, no string decodes to the string "xn--blah" in
+	Punycode.
+
+*/
+
+type keyEncoding int
+
+const (
+	KeyEncodingURL keyEncoding = iota
+	KeyEncodingHex
+	KeyEncodingMax // Number of known encodings. Used for testing
+)
+
+type Key struct {
+	name []byte
+	enc  keyEncoding
+}
+
+type KeyPath []Key
+
+func (pth KeyPath) AppendKey(key []byte, enc keyEncoding) KeyPath {
+	return append(pth, Key{key, enc})
+}
+
+func (pth KeyPath) String() string {
+	res := ""
+	for _, key := range pth {
+		switch key.enc {
+		case KeyEncodingURL:
+			res += "/" + url.PathEscape(string(key.name))
+		case KeyEncodingHex:
+			res += "/x:" + fmt.Sprintf("%X", key.name)
+		default:
+			panic("unexpected key encoding type")
+		}
+	}
+	return res
+}
+
+// Decode a path to a list of keys. Path must begin with `/`.
+// Each key must use a known encoding.
+func KeyPathToKeys(path string) (keys [][]byte, err error) {
+	if path == "" || path[0] != '/' {
+		return nil, cmn.NewError("key path string must start with a forward slash '/'")
+	}
+	parts := strings.Split(path[1:], "/")
+	keys = make([][]byte, len(parts))
+	for i, part := range parts {
+		if strings.HasPrefix(part, "x:") {
+			hexPart := part[2:]
+			key, err := hex.DecodeString(hexPart)
+			if err != nil {
+				return nil, cmn.ErrorWrap(err, "decoding hex-encoded part #%d: /%s", i, part)
+			}
+			keys[i] = key
+		} else {
+			key, err := url.PathUnescape(part)
+			if err != nil {
+				return nil, cmn.ErrorWrap(err, "decoding url-encoded part #%d: /%s", i, part)
+			}
+			keys[i] = []byte(key) // TODO Test this with random bytes, I'm not sure that it works for arbitrary bytes...
+		}
+	}
+	return keys, nil
+}

crypto/merkle/proof_key_path_test.go

@@ -0,0 +1,41 @@
+package merkle
+
+import (
+	"math/rand"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+)
+
+func TestKeyPath(t *testing.T) {
+	var path KeyPath
+	keys := make([][]byte, 10)
+	alphanum := "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
+
+	for d := 0; d < 1e4; d++ {
+		path = nil
+
+		for i := range keys {
+			enc := keyEncoding(rand.Intn(int(KeyEncodingMax)))
+			keys[i] = make([]byte, rand.Uint32()%20)
+			switch enc {
+			case KeyEncodingURL:
+				for j := range keys[i] {
+					keys[i][j] = alphanum[rand.Intn(len(alphanum))]
+				}
+			case KeyEncodingHex:
+				rand.Read(keys[i])
+			default:
+				panic("Unexpected encoding")
+			}
+			path = path.AppendKey(keys[i], enc)
+		}
+
+		res, err := KeyPathToKeys(path.String())
+		require.Nil(t, err)
+
+		for i, key := range keys {
+			require.Equal(t, key, res[i])
+		}
+	}
+}

crypto/merkle/proof_simple_value.go

@@ -0,0 +1,91 @@
+package merkle
+
+import (
+	"bytes"
+	"fmt"
+
+	"github.com/tendermint/tendermint/crypto/tmhash"
+	cmn "github.com/tendermint/tendermint/libs/common"
+)
+
+const ProofOpSimpleValue = "simple:v"
+
+// SimpleValueOp takes a key and a single value as argument and
+// produces the root hash.  The corresponding tree structure is
+// the SimpleMap tree.  SimpleMap takes a Hasher, and currently
+// Tendermint uses aminoHasher.  SimpleValueOp should support
+// the hash function as used in aminoHasher.  TODO support
+// additional hash functions here as options/args to this
+// operator.
+//
+// If the produced root hash matches the expected hash, the
+// proof is good.
+type SimpleValueOp struct {
+	// Encoded in ProofOp.Key.
+	key []byte
+
+	// To encode in ProofOp.Data
+	Proof *SimpleProof `json:"simple_proof"`
+}
+
+var _ ProofOperator = SimpleValueOp{}
+
+func NewSimpleValueOp(key []byte, proof *SimpleProof) SimpleValueOp {
+	return SimpleValueOp{
+		key:   key,
+		Proof: proof,
+	}
+}
+
+func SimpleValueOpDecoder(pop ProofOp) (ProofOperator, error) {
+	if pop.Type != ProofOpSimpleValue {
+		return nil, cmn.NewError("unexpected ProofOp.Type; got %v, want %v", pop.Type, ProofOpSimpleValue)
+	}
+	var op SimpleValueOp // a bit strange as we'll discard this, but it works.
+	err := cdc.UnmarshalBinary(pop.Data, &op)
+	if err != nil {
+		return nil, cmn.ErrorWrap(err, "decoding ProofOp.Data into SimpleValueOp")
+	}
+	return NewSimpleValueOp(pop.Key, op.Proof), nil
+}
+
+func (op SimpleValueOp) ProofOp() ProofOp {
+	bz := cdc.MustMarshalBinary(op)
+	return ProofOp{
+		Type: ProofOpSimpleValue,
+		Key:  op.key,
+		Data: bz,
+	}
+}
+
+func (op SimpleValueOp) String() string {
+	return fmt.Sprintf("SimpleValueOp{%v}", op.GetKey())
+}
+
+func (op SimpleValueOp) Run(args [][]byte) ([][]byte, error) {
+	if len(args) != 1 {
+		return nil, cmn.NewError("expected 1 arg, got %v", len(args))
+	}
+	value := args[0]
+	hasher := tmhash.New()
+	hasher.Write(value) // does not error
+	vhash := hasher.Sum(nil)
+
+	// Wrap <op.Key, vhash> to hash the KVPair.
+	hasher = tmhash.New()
+	encodeByteSlice(hasher, []byte(op.key)) // does not error
+	encodeByteSlice(hasher, []byte(vhash))  // does not error
+	kvhash := hasher.Sum(nil)
+
+	if !bytes.Equal(kvhash, op.Proof.LeafHash) {
+		return nil, cmn.NewError("leaf hash mismatch: want %X got %X", op.Proof.LeafHash, kvhash)
+	}
+
+	return [][]byte{
+		op.Proof.ComputeRootHash(),
+	}, nil
+}
+
+func (op SimpleValueOp) GetKey() []byte {
+	return op.key
+}

crypto/merkle/simple_map.go

@@ -1,6 +1,9 @@
 package merkle
 
 import (
+	"bytes"
+
+	amino "github.com/tendermint/go-amino"
 	"github.com/tendermint/tendermint/crypto/tmhash"
 	cmn "github.com/tendermint/tendermint/libs/common"
 )
@@ -20,14 +23,15 @@ func newSimpleMap() *simpleMap {
 	}
 }
 
-// Set hashes the key and value and appends it to the kv pairs.
-func (sm *simpleMap) Set(key string, value Hasher) {
+// Set creates a kv pair of the key and the hash of the value,
+// and then appends it to simpleMap's kv pairs.
+func (sm *simpleMap) Set(key string, value []byte) {
 	sm.sorted = false
 
 	// The value is hashed, so you can
 	// check for equality with a cached value (say)
 	// and make a determination to fetch or not.
-	vhash := value.Hash()
+	vhash := tmhash.Sum(value)
 
 	sm.kvs = append(sm.kvs, cmn.KVPair{
 		Key:   []byte(key),
@@ -66,23 +70,25 @@ func (sm *simpleMap) KVPairs() cmn.KVPairs {
 // then hashed.
 type KVPair cmn.KVPair
 
-func (kv KVPair) Hash() []byte {
-	hasher := tmhash.New()
-	err := encodeByteSlice(hasher, kv.Key)
+// Bytes returns key || value, with both the
+// key and value length prefixed.
+func (kv KVPair) Bytes() []byte {
+	var b bytes.Buffer
+	err := amino.EncodeByteSlice(&b, kv.Key)
 	if err != nil {
 		panic(err)
 	}
-	err = encodeByteSlice(hasher, kv.Value)
+	err = amino.EncodeByteSlice(&b, kv.Value)
 	if err != nil {
 		panic(err)
 	}
-	return hasher.Sum(nil)
+	return b.Bytes()
 }
 
 func hashKVPairs(kvs cmn.KVPairs) []byte {
-	kvsH := make([]Hasher, len(kvs))
+	kvsH := make([][]byte, len(kvs))
 	for i, kvp := range kvs {
-		kvsH[i] = KVPair(kvp)
+		kvsH[i] = KVPair(kvp).Bytes()
 	}
-	return SimpleHashFromHashers(kvsH)
+	return SimpleHashFromByteSlices(kvsH)
 }

crypto/merkle/simple_map_test.go

@@ -5,50 +5,29 @@ import (
 	"testing"
 
 	"github.com/stretchr/testify/assert"
-	"github.com/tendermint/tendermint/crypto/tmhash"
 )
 
-type strHasher string
-
-func (str strHasher) Hash() []byte {
-	return tmhash.Sum([]byte(str))
-}
-
 func TestSimpleMap(t *testing.T) {
-	{
-		db := newSimpleMap()
-		db.Set("key1", strHasher("value1"))
-		assert.Equal(t, "fa9bc106ffd932d919bee935ceb6cf2b3dd72d8f", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
+	tests := []struct {
+		keys   []string
+		values []string // each string gets converted to []byte in test
+		want   string
+	}{
+		{[]string{"key1"}, []string{"value1"}, "fa9bc106ffd932d919bee935ceb6cf2b3dd72d8f"},
+		{[]string{"key1"}, []string{"value2"}, "e00e7dcfe54e9fafef5111e813a587f01ba9c3e8"},
+		// swap order with 2 keys
+		{[]string{"key1", "key2"}, []string{"value1", "value2"}, "eff12d1c703a1022ab509287c0f196130123d786"},
+		{[]string{"key2", "key1"}, []string{"value2", "value1"}, "eff12d1c703a1022ab509287c0f196130123d786"},
+		// swap order with 3 keys
+		{[]string{"key1", "key2", "key3"}, []string{"value1", "value2", "value3"}, "b2c62a277c08dbd2ad73ca53cd1d6bfdf5830d26"},
+		{[]string{"key1", "key3", "key2"}, []string{"value1", "value3", "value2"}, "b2c62a277c08dbd2ad73ca53cd1d6bfdf5830d26"},
 	}
-	{
+	for i, tc := range tests {
 		db := newSimpleMap()
-		db.Set("key1", strHasher("value2"))
-		assert.Equal(t, "e00e7dcfe54e9fafef5111e813a587f01ba9c3e8", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
-	}
-	{
-		db := newSimpleMap()
-		db.Set("key1", strHasher("value1"))
-		db.Set("key2", strHasher("value2"))
-		assert.Equal(t, "eff12d1c703a1022ab509287c0f196130123d786", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
-	}
-	{
-		db := newSimpleMap()
-		db.Set("key2", strHasher("value2")) // NOTE: out of order
-		db.Set("key1", strHasher("value1"))
-		assert.Equal(t, "eff12d1c703a1022ab509287c0f196130123d786", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
-	}
-	{
-		db := newSimpleMap()
-		db.Set("key1", strHasher("value1"))
-		db.Set("key2", strHasher("value2"))
-		db.Set("key3", strHasher("value3"))
-		assert.Equal(t, "b2c62a277c08dbd2ad73ca53cd1d6bfdf5830d26", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
-	}
-	{
-		db := newSimpleMap()
-		db.Set("key2", strHasher("value2")) // NOTE: out of order
-		db.Set("key1", strHasher("value1"))
-		db.Set("key3", strHasher("value3"))
-		assert.Equal(t, "b2c62a277c08dbd2ad73ca53cd1d6bfdf5830d26", fmt.Sprintf("%x", db.Hash()), "Hash didn't match")
+		for i := 0; i < len(tc.keys); i++ {
+			db.Set(tc.keys[i], []byte(tc.values[i]))
+		}
+		got := db.Hash()
+		assert.Equal(t, tc.want, fmt.Sprintf("%x", got), "Hash didn't match on tc %d", i)
 	}
 }

crypto/merkle/simple_proof.go

@@ -2,23 +2,39 @@ package merkle
 
 import (
 	"bytes"
+	"errors"
 	"fmt"
+
+	"github.com/tendermint/tendermint/crypto/tmhash"
+	cmn "github.com/tendermint/tendermint/libs/common"
 )
 
-// SimpleProof represents a simple merkle proof.
+// SimpleProof represents a simple Merkle proof.
+// NOTE: The convention for proofs is to include leaf hashes but to
+// exclude the root hash.
+// This convention is implemented across IAVL range proofs as well.
+// Keep this consistent unless there's a very good reason to change
+// everything.  This also affects the generalized proof system as
+// well.
 type SimpleProof struct {
-	Aunts [][]byte `json:"aunts"` // Hashes from leaf's sibling to a root's child.
+	Total    int      `json:"total"`     // Total number of items.
+	Index    int      `json:"index"`     // Index of item to prove.
+	LeafHash []byte   `json:"leaf_hash"` // Hash of item value.
+	Aunts    [][]byte `json:"aunts"`     // Hashes from leaf's sibling to a root's child.
 }
 
-// SimpleProofsFromHashers computes inclusion proof for given items.
+// SimpleProofsFromByteSlices computes inclusion proof for given items.
 // proofs[0] is the proof for items[0].
-func SimpleProofsFromHashers(items []Hasher) (rootHash []byte, proofs []*SimpleProof) {
-	trails, rootSPN := trailsFromHashers(items)
+func SimpleProofsFromByteSlices(items [][]byte) (rootHash []byte, proofs []*SimpleProof) {
+	trails, rootSPN := trailsFromByteSlices(items)
 	rootHash = rootSPN.Hash
 	proofs = make([]*SimpleProof, len(items))
 	for i, trail := range trails {
 		proofs[i] = &SimpleProof{
-			Aunts: trail.FlattenAunts(),
+			Total:    len(items),
+			Index:    i,
+			LeafHash: trail.Hash,
+			Aunts:    trail.FlattenAunts(),
 		}
 	}
 	return
@@ -27,19 +43,19 @@ func SimpleProofsFromHashers(items []Hasher) (rootHash []byte, proofs []*SimpleP
 // SimpleProofsFromMap generates proofs from a map. The keys/values of the map will be used as the keys/values
 // in the underlying key-value pairs.
 // The keys are sorted before the proofs are computed.
-func SimpleProofsFromMap(m map[string]Hasher) (rootHash []byte, proofs map[string]*SimpleProof, keys []string) {
+func SimpleProofsFromMap(m map[string][]byte) (rootHash []byte, proofs map[string]*SimpleProof, keys []string) {
 	sm := newSimpleMap()
 	for k, v := range m {
 		sm.Set(k, v)
 	}
 	sm.Sort()
 	kvs := sm.kvs
-	kvsH := make([]Hasher, 0, len(kvs))
-	for _, kvp := range kvs {
-		kvsH = append(kvsH, KVPair(kvp))
+	kvsBytes := make([][]byte, len(kvs))
+	for i, kvp := range kvs {
+		kvsBytes[i] = KVPair(kvp).Bytes()
 	}
 
-	rootHash, proofList := SimpleProofsFromHashers(kvsH)
+	rootHash, proofList := SimpleProofsFromByteSlices(kvsBytes)
 	proofs = make(map[string]*SimpleProof)
 	keys = make([]string, len(proofList))
 	for i, kvp := range kvs {
@@ -49,11 +65,33 @@ func SimpleProofsFromMap(m map[string]Hasher) (rootHash []byte, proofs map[strin
 	return
 }
 
-// Verify that leafHash is a leaf hash of the simple-merkle-tree
-// which hashes to rootHash.
-func (sp *SimpleProof) Verify(index int, total int, leafHash []byte, rootHash []byte) bool {
-	computedHash := computeHashFromAunts(index, total, leafHash, sp.Aunts)
-	return computedHash != nil && bytes.Equal(computedHash, rootHash)
+// Verify that the SimpleProof proves the root hash.
+// Check sp.Index/sp.Total manually if needed
+func (sp *SimpleProof) Verify(rootHash []byte, leafHash []byte) error {
+	if sp.Total < 0 {
+		return errors.New("Proof total must be positive")
+	}
+	if sp.Index < 0 {
+		return errors.New("Proof index cannot be negative")
+	}
+	if !bytes.Equal(sp.LeafHash, leafHash) {
+		return cmn.NewError("invalid leaf hash: wanted %X got %X", leafHash, sp.LeafHash)
+	}
+	computedHash := sp.ComputeRootHash()
+	if !bytes.Equal(computedHash, rootHash) {
+		return cmn.NewError("invalid root hash: wanted %X got %X", rootHash, computedHash)
+	}
+	return nil
+}
+
+// Compute the root hash given a leaf hash.  Does not verify the result.
+func (sp *SimpleProof) ComputeRootHash() []byte {
+	return computeHashFromAunts(
+		sp.Index,
+		sp.Total,
+		sp.LeafHash,
+		sp.Aunts,
+	)
 }
 
 // String implements the stringer interface for SimpleProof.
@@ -138,17 +176,17 @@ func (spn *SimpleProofNode) FlattenAunts() [][]byte {
 
 // trails[0].Hash is the leaf hash for items[0].
 // trails[i].Parent.Parent....Parent == root for all i.
-func trailsFromHashers(items []Hasher) (trails []*SimpleProofNode, root *SimpleProofNode) {
+func trailsFromByteSlices(items [][]byte) (trails []*SimpleProofNode, root *SimpleProofNode) {
 	// Recursive impl.
 	switch len(items) {
 	case 0:
 		return nil, nil
 	case 1:
-		trail := &SimpleProofNode{items[0].Hash(), nil, nil, nil}
+		trail := &SimpleProofNode{tmhash.Sum(items[0]), nil, nil, nil}
 		return []*SimpleProofNode{trail}, trail
 	default:
-		lefts, leftRoot := trailsFromHashers(items[:(len(items)+1)/2])
-		rights, rightRoot := trailsFromHashers(items[(len(items)+1)/2:])
+		lefts, leftRoot := trailsFromByteSlices(items[:(len(items)+1)/2])
+		rights, rightRoot := trailsFromByteSlices(items[(len(items)+1)/2:])
 		rootHash := SimpleHashFromTwoHashes(leftRoot.Hash, rightRoot.Hash)
 		root := &SimpleProofNode{rootHash, nil, nil, nil}
 		leftRoot.Parent = root

crypto/merkle/simple_tree.go

@@ -18,41 +18,29 @@ func SimpleHashFromTwoHashes(left, right []byte) []byte {
 	return hasher.Sum(nil)
 }
 
-// SimpleHashFromHashers computes a Merkle tree from items that can be hashed.
-func SimpleHashFromHashers(items []Hasher) []byte {
-	hashes := make([][]byte, len(items))
-	for i, item := range items {
-		hash := item.Hash()
-		hashes[i] = hash
+// SimpleHashFromByteSlices computes a Merkle tree where the leaves are the byte slice,
+// in the provided order.
+func SimpleHashFromByteSlices(items [][]byte) []byte {
+	switch len(items) {
+	case 0:
+		return nil
+	case 1:
+		return tmhash.Sum(items[0])
+	default:
+		left := SimpleHashFromByteSlices(items[:(len(items)+1)/2])
+		right := SimpleHashFromByteSlices(items[(len(items)+1)/2:])
+		return SimpleHashFromTwoHashes(left, right)
 	}
-	return simpleHashFromHashes(hashes)
 }
 
 // SimpleHashFromMap computes a Merkle tree from sorted map.
 // Like calling SimpleHashFromHashers with
 // `item = []byte(Hash(key) | Hash(value))`,
 // sorted by `item`.
-func SimpleHashFromMap(m map[string]Hasher) []byte {
+func SimpleHashFromMap(m map[string][]byte) []byte {
 	sm := newSimpleMap()
 	for k, v := range m {
 		sm.Set(k, v)
 	}
 	return sm.Hash()
 }
-
-//----------------------------------------------------------------
-
-// Expects hashes!
-func simpleHashFromHashes(hashes [][]byte) []byte {
-	// Recursive impl.
-	switch len(hashes) {
-	case 0:
-		return nil
-	case 1:
-		return hashes[0]
-	default:
-		left := simpleHashFromHashes(hashes[:(len(hashes)+1)/2])
-		right := simpleHashFromHashes(hashes[(len(hashes)+1)/2:])
-		return SimpleHashFromTwoHashes(left, right)
-	}
-}

crypto/merkle/simple_tree_test.go

@@ -1,13 +1,13 @@
 package merkle
 
 import (
-	"bytes"
+	"testing"
+
+	"github.com/stretchr/testify/require"
 
 	cmn "github.com/tendermint/tendermint/libs/common"
 	. "github.com/tendermint/tendermint/libs/test"
 
-	"testing"
-
 	"github.com/tendermint/tendermint/crypto/tmhash"
 )
 
@@ -21,69 +21,52 @@ func TestSimpleProof(t *testing.T) {
 
 	total := 100
 
-	items := make([]Hasher, total)
+	items := make([][]byte, total)
 	for i := 0; i < total; i++ {
 		items[i] = testItem(cmn.RandBytes(tmhash.Size))
 	}
 
-	rootHash := SimpleHashFromHashers(items)
+	rootHash := SimpleHashFromByteSlices(items)
 
-	rootHash2, proofs := SimpleProofsFromHashers(items)
+	rootHash2, proofs := SimpleProofsFromByteSlices(items)
 
-	if !bytes.Equal(rootHash, rootHash2) {
-		t.Errorf("Unmatched root hashes: %X vs %X", rootHash, rootHash2)
-	}
+	require.Equal(t, rootHash, rootHash2, "Unmatched root hashes: %X vs %X", rootHash, rootHash2)
 
 	// For each item, check the trail.
 	for i, item := range items {
-		itemHash := item.Hash()
+		itemHash := tmhash.Sum(item)
 		proof := proofs[i]
 
-		// Verify success
-		ok := proof.Verify(i, total, itemHash, rootHash)
-		if !ok {
-			t.Errorf("Verification failed for index %v.", i)
-		}
+		// Check total/index
+		require.Equal(t, proof.Index, i, "Unmatched indicies: %d vs %d", proof.Index, i)
 
-		// Wrong item index should make it fail
-		{
-			ok = proof.Verify((i+1)%total, total, itemHash, rootHash)
-			if ok {
-				t.Errorf("Expected verification to fail for wrong index %v.", i)
-			}
-		}
+		require.Equal(t, proof.Total, total, "Unmatched totals: %d vs %d", proof.Total, total)
+
+		// Verify success
+		err := proof.Verify(rootHash, itemHash)
+		require.NoError(t, err, "Verificatior failed: %v.", err)
 
 		// Trail too long should make it fail
 		origAunts := proof.Aunts
 		proof.Aunts = append(proof.Aunts, cmn.RandBytes(32))
-		{
-			ok = proof.Verify(i, total, itemHash, rootHash)
-			if ok {
-				t.Errorf("Expected verification to fail for wrong trail length.")
-			}
-		}
+		err = proof.Verify(rootHash, itemHash)
+		require.Error(t, err, "Expected verification to fail for wrong trail length")
+
 		proof.Aunts = origAunts
 
 		// Trail too short should make it fail
 		proof.Aunts = proof.Aunts[0 : len(proof.Aunts)-1]
-		{
-			ok = proof.Verify(i, total, itemHash, rootHash)
-			if ok {
-				t.Errorf("Expected verification to fail for wrong trail length.")
-			}
-		}
+		err = proof.Verify(rootHash, itemHash)
+		require.Error(t, err, "Expected verification to fail for wrong trail length")
+
 		proof.Aunts = origAunts
 
 		// Mutating the itemHash should make it fail.
-		ok = proof.Verify(i, total, MutateByteSlice(itemHash), rootHash)
-		if ok {
-			t.Errorf("Expected verification to fail for mutated leaf hash")
-		}
+		err = proof.Verify(rootHash, MutateByteSlice(itemHash))
+		require.Error(t, err, "Expected verification to fail for mutated leaf hash")
 
 		// Mutating the rootHash should make it fail.
-		ok = proof.Verify(i, total, itemHash, MutateByteSlice(rootHash))
-		if ok {
-			t.Errorf("Expected verification to fail for mutated root hash")
-		}
+		err = proof.Verify(MutateByteSlice(rootHash), itemHash)
+		require.Error(t, err, "Expected verification to fail for mutated root hash")
 	}
 }

crypto/merkle/types.go

@@ -25,11 +25,6 @@ type Tree interface {
 	IterateRange(start []byte, end []byte, ascending bool, fx func(key []byte, value []byte) (stop bool)) (stopped bool)
 }
 
-// Hasher represents a hashable piece of data which can be hashed in the Tree.
-type Hasher interface {
-	Hash() []byte
-}
-
 //-----------------------------------------------------------------------
 
 // Uvarint length prefixed byteslice

crypto/merkle/wire.go

@@ -0,0 +1,12 @@
+package merkle
+
+import (
+	"github.com/tendermint/go-amino"
+)
+
+var cdc *amino.Codec
+
+func init() {
+	cdc = amino.NewCodec()
+	cdc.Seal()
+}

crypto/random.go

@@ -1,7 +1,6 @@
 package crypto
 
 import (
-	"crypto/aes"
 	"crypto/cipher"
 	crand "crypto/rand"
 	"crypto/sha256"
@@ -9,9 +8,17 @@ import (
 	"io"
 	"sync"
 
+	"golang.org/x/crypto/chacha20poly1305"
+
 	. "github.com/tendermint/tendermint/libs/common"
 )
 
+// The randomness here is derived from xoring a chacha20 keystream with
+// output from crypto/rand's OS Entropy Reader. (Due to fears of the OS'
+// entropy being backdoored)
+//
+// For forward secrecy of produced randomness, the internal chacha key is hashed
+// and thereby rotated after each call.
 var gRandInfo *randInfo
 
 func init() {
@@ -61,11 +68,10 @@ func CReader() io.Reader {
 //--------------------------------------------------------------------------------
 
 type randInfo struct {
-	mtx          sync.Mutex
-	seedBytes    [32]byte
-	cipherAES256 cipher.Block
-	streamAES256 cipher.Stream
-	reader       io.Reader
+	mtx       sync.Mutex
+	seedBytes [chacha20poly1305.KeySize]byte
+	chacha    cipher.AEAD
+	reader    io.Reader
 }
 
 // You can call this as many times as you'd like.
@@ -79,30 +85,35 @@ func (ri *randInfo) MixEntropy(seedBytes []byte) {
 	h.Write(seedBytes)
 	h.Write(ri.seedBytes[:])
 	hashBytes := h.Sum(nil)
-	hashBytes32 := [32]byte{}
-	copy(hashBytes32[:], hashBytes)
-	ri.seedBytes = xorBytes32(ri.seedBytes, hashBytes32)
-	// Create new cipher.Block
-	var err error
-	ri.cipherAES256, err = aes.NewCipher(ri.seedBytes[:])
+	copy(ri.seedBytes[:], hashBytes)
+	chacha, err := chacha20poly1305.New(ri.seedBytes[:])
 	if err != nil {
-		PanicSanity("Error creating AES256 cipher: " + err.Error())
+		panic("Initializing chacha20 failed")
 	}
-	// Create new stream
-	ri.streamAES256 = cipher.NewCTR(ri.cipherAES256, randBytes(aes.BlockSize))
+	ri.chacha = chacha
 	// Create new reader
-	ri.reader = &cipher.StreamReader{S: ri.streamAES256, R: crand.Reader}
+	ri.reader = &cipher.StreamReader{S: ri, R: crand.Reader}
+}
+
+func (ri *randInfo) XORKeyStream(dst, src []byte) {
+	// nonce being 0 is safe due to never re-using a key.
+	emptyNonce := make([]byte, 12)
+	tmpDst := ri.chacha.Seal([]byte{}, emptyNonce, src, []byte{0})
+	// this removes the poly1305 tag as well, since chacha is a stream cipher
+	// and we truncate at input length.
+	copy(dst, tmpDst[:len(src)])
+	// hash seedBytes for forward secrecy, and initialize new chacha instance
+	newSeed := sha256.Sum256(ri.seedBytes[:])
+	chacha, err := chacha20poly1305.New(newSeed[:])
+	if err != nil {
+		panic("Initializing chacha20 failed")
+	}
+	ri.chacha = chacha
 }
 
 func (ri *randInfo) Read(b []byte) (n int, err error) {
 	ri.mtx.Lock()
-	defer ri.mtx.Unlock()
-	return ri.reader.Read(b)
-}
-
-func xorBytes32(bytesA [32]byte, bytesB [32]byte) (res [32]byte) {
-	for i, b := range bytesA {
-		res[i] = b ^ bytesB[i]
-	}
-	return res
+	n, err = ri.reader.Read(b)
+	ri.mtx.Unlock()
+	return
 }

crypto/random_test.go

@@ -0,0 +1,23 @@
+package crypto_test
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/require"
+
+	"github.com/tendermint/tendermint/crypto"
+)
+
+// the purpose of this test is primarily to ensure that the randomness
+// generation won't error.
+func TestRandomConsistency(t *testing.T) {
+	x1 := crypto.CRandBytes(256)
+	x2 := crypto.CRandBytes(256)
+	x3 := crypto.CRandBytes(256)
+	x4 := crypto.CRandBytes(256)
+	x5 := crypto.CRandBytes(256)
+	require.NotEqual(t, x1, x2)
+	require.NotEqual(t, x3, x4)
+	require.NotEqual(t, x4, x5)
+	require.NotEqual(t, x1, x5)
+}

crypto/secp256k1/secp256k1.go

@@ -9,8 +9,9 @@ import (
 
 	secp256k1 "github.com/tendermint/btcd/btcec"
 	amino "github.com/tendermint/go-amino"
+	"golang.org/x/crypto/ripemd160" // forked to github.com/tendermint/crypto
+
 	"github.com/tendermint/tendermint/crypto"
-	"golang.org/x/crypto/ripemd160"
 )
 
 //-------------------------------------

crypto/xchacha20poly1305/xchachapoly.go

@@ -8,7 +8,7 @@ import (
 	"errors"
 	"fmt"
 
-	"golang.org/x/crypto/chacha20poly1305"
+	"golang.org/x/crypto/chacha20poly1305" // forked to github.com/tendermint/crypto
 )
 
 // Implements crypto.AEAD

crypto/xsalsa20symmetric/symmetric.go

@@ -4,9 +4,10 @@ import (
 	"errors"
 	"fmt"
 
+	"golang.org/x/crypto/nacl/secretbox" // forked to github.com/tendermint/crypto
+
 	"github.com/tendermint/tendermint/crypto"
 	cmn "github.com/tendermint/tendermint/libs/common"
-	"golang.org/x/crypto/nacl/secretbox"
 )
 
 // TODO, make this into a struct that implements crypto.Symmetric.

crypto/xsalsa20symmetric/symmetric_test.go

@@ -6,8 +6,9 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
+	"golang.org/x/crypto/bcrypt" // forked to github.com/tendermint/crypto
+
 	"github.com/tendermint/tendermint/crypto"
-	"golang.org/x/crypto/bcrypt"
 )
 
 func TestSimple(t *testing.T) {
@@ -29,7 +30,9 @@ func TestSimpleWithKDF(t *testing.T) {
 
 	plaintext := []byte("sometext")
 	secretPass := []byte("somesecret")
-	secret, err := bcrypt.GenerateFromPassword(secretPass, 12)
+	salt := []byte("somesaltsomesalt") // len 16
+	// NOTE: we use a fork of x/crypto so we can inject our own randomness for salt
+	secret, err := bcrypt.GenerateFromPassword(salt, secretPass, 12)
 	if err != nil {
 		t.Error(err)
 	}

docs/.vuepress/config.js

@@ -11,18 +11,20 @@ module.exports = {
     nav: [{ text: "Back to Tendermint", link: "https://tendermint.com" }],
     sidebar: [
       {
-        title: "Getting Started",
+        title: "Introduction",
         collapsable: false,
         children: [
+          "/introduction/",
           "/introduction/quick-start",
           "/introduction/install",
-          "/introduction/introduction"
+          "/introduction/what-is-tendermint"
         ]
       },
       {
         title: "Tendermint Core",
         collapsable: false,
         children: [
+          "/tendermint-core/",
           "/tendermint-core/using-tendermint",
           "/tendermint-core/configuration",
           "/tendermint-core/rpc",
@@ -40,15 +42,17 @@ module.exports = {
         title: "Tools",
         collapsable: false,
         children:  [
-	  "tools/benchmarking",
-	  "tools/monitoring"
+	  "/tools/",
+	  "/tools/benchmarking",
+	  "/tools/monitoring"
 	]
       },
       {
         title: "Networks",
         collapsable: false,
         children: [
-          "/networks/deploy-testnets",
+          "/networks/",
+          "/networks/docker-compose",
           "/networks/terraform-and-ansible",
         ]
       },
@@ -99,21 +103,14 @@ module.exports = {
 	]
       },
       {
-        title: "ABCI Specification",
+        title: "ABCI Spec",
         collapsable: false,
         children: [
+          "/spec/abci/",
           "/spec/abci/abci",
           "/spec/abci/apps",
           "/spec/abci/client-server"
         ]
-      },
-      {
-        title: "Research",
-        collapsable: false,
-        children: [
-	  "/research/determinism",
-	  "/research/transactional-semantics"
-	]
       }
     ]
   }

docs/DOCS_README.md

@@ -20,7 +20,8 @@ a private website repository has make targets consumed by a standard Jenkins tas
 ## README
 
 The [README.md](./README.md) is also the landing page for the documentation
-on the website.
+on the website. During the Jenkins build, the current commit is added to the bottom
+of the README.
 
 ## Config.js
 
@@ -34,6 +35,8 @@ of the sidebar.
 **NOTE:** Strongly consider the existing links - both within this directory
 and to the website docs - when moving or deleting files.
 
+Links to directories *MUST* end in a `/`.
+
 Relative links should be used nearly everywhere, having discovered and weighed the following:
 
 ### Relative

docs/README.md

@@ -1,41 +1,29 @@
 # Tendermint
 
-Welcome to the Tendermint Core documentation! Below you'll find an
-overview of the documentation.
+Welcome to the Tendermint Core documentation!
 
-Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state
-transition machine - written in any programming language - and securely
-replicates it on many machines. In other words, a blockchain.
+Tendermint Core is a blockchain application platform; it provides the equivalent
+of a web-server, database, and supporting libraries for blockchain applications
+written in any programming language. Like a web-server serving web applications,
+Tendermint serves blockchain applications.
 
-Tendermint requires an application running over the Application Blockchain
-Interface (ABCI) - and comes packaged with an example application to do so.
+More formally, Tendermint Core performs Byzantine Fault Tolerant (BFT)
+State Machine Replication (SMR) for arbitrary deterministic, finite state machines.
+For more background, see [What is
+Tendermint?](introduction/what-is-tendermint.md).
 
-## Getting Started
+To get started quickly with an example application, see the [quick start guide](introduction/quick-start.md).
 
-Here you'll find quick start guides and links to more advanced "get up and running"
-documentation.
+To learn about application development on Tendermint, see the [Application Blockchain Interface](spec/abci/).
 
-## Core
+For more details on using Tendermint, see the respective documentation for
+[Tendermint Core](tendermint-core/), [benchmarking and monitoring](tools/), and [network deployments](networks/).
 
-Details about the core functionality and configuration of Tendermint.
+## Contribute
 
-## Tools
-
-Benchmarking and monitoring tools.
-
-## Networks
-
-Setting up testnets manually or automated, local or in the cloud.
-
-## Apps
-
-Building appplications with the ABCI.
-
-## Specification
-
-Dive deep into the spec. There's one for each Tendermint and the ABCI
-
-## Edit the Documentation
-
-See [this file](./DOCS_README.md) for details of the build process and
+To contribute to the documentation, see [this file](./DOCS_README.md) for details of the build process and
 considerations when making changes.
+
+## Version
+
+This documentation is built from the following commit:

docs/app-dev/app-development.md

@@ -431,17 +431,30 @@ Note: these query formats are subject to change!
 In go:
 
 ```
-func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery types.ResponseQuery) {
-  if reqQuery.Prove {
-    value, proof, exists := app.state.Proof(reqQuery.Data)
-    resQuery.Index = -1 // TODO make Proof return index
-    resQuery.Key = reqQuery.Data
-    resQuery.Value = value
-    resQuery.Proof = proof
-    if exists {
-      resQuery.Log = "exists"
-    } else {
-      resQuery.Log = "does not exist"
+    func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery types.ResponseQuery) {
+      if reqQuery.Prove {
+        value, proof, exists := app.state.GetWithProof(reqQuery.Data)
+        resQuery.Index = -1 // TODO make Proof return index
+        resQuery.Key = reqQuery.Data
+        resQuery.Value = value
+        resQuery.Proof = proof
+        if exists {
+          resQuery.Log = "exists"
+        } else {
+          resQuery.Log = "does not exist"
+        }
+        return
+      } else {
+        index, value, exists := app.state.Get(reqQuery.Data)
+        resQuery.Index = int64(index)
+        resQuery.Value = value
+        if exists {
+          resQuery.Log = "exists"
+        } else {
+          resQuery.Log = "does not exist"
+        }
+        return
+      }
     }
     return
   } else {
@@ -461,22 +474,25 @@ func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery type
 In Java:
 
 ```
-ResponseQuery requestQuery(RequestQuery req) {
-    final boolean isProveQuery = req.getProve();
-    final ResponseQuery.Builder responseBuilder = ResponseQuery.newBuilder();
+    ResponseQuery requestQuery(RequestQuery req) {
+        final boolean isProveQuery = req.getProve();
+        final ResponseQuery.Builder responseBuilder = ResponseQuery.newBuilder();
+		byte[] queryData = req.getData().toByteArray();
 
-    if (isProveQuery) {
-        com.app.example.ProofResult proofResult = generateProof(req.getData().toByteArray());
-        final byte[] proofAsByteArray = proofResult.getAsByteArray();
-
-        responseBuilder.setProof(ByteString.copyFrom(proofAsByteArray));
-        responseBuilder.setKey(req.getData());
-        responseBuilder.setValue(ByteString.copyFrom(proofResult.getData()));
-        responseBuilder.setLog(result.getLogValue());
-    } else {
-        byte[] queryData = req.getData().toByteArray();
-
-        final com.app.example.QueryResult result = generateQueryResult(queryData);
+        if (isProveQuery) {
+            com.app.example.QueryResultWithProof result = generateQueryResultWithProof(queryData);
+            responseBuilder.setIndex(result.getLeftIndex());
+            responseBuilder.setKey(req.getData());
+            responseBuilder.setValue(result.getValueOrNull(0));
+            responseBuilder.setHeight(result.getHeight());
+            responseBuilder.setProof(result.getProof());
+            responseBuilder.setLog(result.getLogValue());
+        } else {
+            com.app.example.QueryResult result = generateQueryResult(queryData);
+            responseBuilder.setIndex(result.getIndex());
+            responseBuilder.setValue(result.getValue());
+            responseBuilder.setLog(result.getLogValue());
+        }
 
         responseBuilder.setIndex(result.getIndex());
         responseBuilder.setValue(ByteString.copyFrom(result.getValue()));

docs/app-dev/getting-started.md

@@ -7,8 +7,7 @@ application you want to run. So, to run a complete blockchain that does
 something useful, you must start two programs: one is Tendermint Core,
 the other is your application, which can be written in any programming
 language. Recall from [the intro to
-ABCI](../introduction/introduction.md#ABCI-Overview) that Tendermint Core handles all
-the p2p and consensus stuff, and just forwards transactions to the
+ABCI](../introduction/what-is-tendermint.md#abci-overview) that Tendermint Core handles all the p2p and consensus stuff, and just forwards transactions to the
 application when they need to be validated, or when they're ready to be
 committed to a block.
 

docs/app-dev/subscribing-to-events-via-websocket.md

@@ -20,7 +20,7 @@ method via Websocket.
 }
 ```
 
-Check out [API docs](https://tendermint.github.io/slate/#subscribe) for
+Check out [API docs](https://tendermint.com/rpc/) for
 more information on query syntax and other options.
 
 You can also use tags, given you had included them into DeliverTx
@@ -32,7 +32,7 @@ transactions](./indexing-transactions.md) for details.
 When validator set changes, ValidatorSetUpdates event is published. The
 event carries a list of pubkey/power pairs. The list is the same
 Tendermint receives from ABCI application (see [EndBlock
-section](https://tendermint.com/docs/app-dev/abci-spec.html#endblock) in
+section](../spec/abci/abci.md#endblock) in
 the ABCI spec).
 
 Response:

docs/architecture/adr-024-sign-bytes.md

@@ -0,0 +1,234 @@
+# ADR 024: SignBytes and validator types in privval
+
+## Context
+
+Currently, the messages exchanged between tendermint and a (potentially remote) signer/validator, 
+namely votes, proposals, and heartbeats, are encoded as a JSON string 
+(e.g., via `Vote.SignBytes(...)`) and then 
+signed . JSON encoding is sub-optimal for both, hardware wallets 
+and for usage in ethereum smart contracts. Both is laid down in detail in [issue#1622].  
+
+Also, there are currently no differences between sign-request and -replies. Also, there is no possibility 
+for a remote signer to include an error code or message in case something went wrong.
+The messages exchanged between tendermint and a remote signer currently live in 
+[privval/socket.go] and encapsulate the corresponding types in [types].
+
+
+[privval/socket.go]: https://github.com/tendermint/tendermint/blob/d419fffe18531317c28c29a292ad7d253f6cafdf/privval/socket.go#L496-L502
+[issue#1622]: https://github.com/tendermint/tendermint/issues/1622
+[types]: https://github.com/tendermint/tendermint/tree/master/types
+ 
+
+## Decision
+
+- restructure vote, proposal, and heartbeat such that their encoding is easily parseable by 
+hardware devices and smart contracts using a  binary encoding format ([amino] in this case)
+- split up the messages exchanged between tendermint and remote signers into requests and 
+responses (see details below)
+- include an error type in responses
+
+### Overview
+```
++--------------+                      +----------------+
+|              |     SignXRequest     |                |
+|Remote signer |<---------------------+  tendermint    |
+| (e.g. KMS)   |                      |                |
+|              +--------------------->|                |
++--------------+    SignedXReply      +----------------+
+
+
+SignXRequest {
+    x: X
+}
+
+SignedXReply {
+    x: X
+  sig: Signature // []byte
+  err: Error{ 
+    code: int
+    desc: string
+  }
+}
+```
+
+TODO: Alternatively, the type `X` might directly include the signature. A lot of places expect a vote with a 
+signature and do not necessarily deal with "Replies".
+Still exploring what would work best here. 
+This would look like (exemplified using X = Vote):
+```
+Vote {
+    // all fields besides signature
+}
+
+SignedVote {
+ Vote Vote
+ Signature []byte
+}
+
+SignVoteRequest {
+   Vote Vote
+}
+
+SignedVoteReply {
+    Vote SignedVote
+    Err  Error
+}
+```
+
+**Note:** There was a related discussion around including a fingerprint of, or, the whole public-key 
+into each sign-request to tell the signer which corresponding private-key to 
+use to sign the message. This is particularly relevant in the context of the KMS
+but is currently not considered in this ADR. 
+
+
+[amino]: https://github.com/tendermint/go-amino/
+
+### Vote
+
+As explained in [issue#1622] `Vote` will be changed to contain the following fields 
+(notation in protobuf-like syntax for easy readability):
+
+```proto
+// vanilla protobuf / amino encoded
+message Vote {
+    Version       fixed32                      
+    Height        sfixed64       
+    Round         sfixed32
+    VoteType      fixed32
+    Timestamp     Timestamp         // << using protobuf definition
+    BlockID       BlockID           // << as already defined 
+    ChainID       string            // at the end because length could vary a lot
+}
+
+// this is an amino registered type; like currently privval.SignVoteMsg: 
+// registered with "tendermint/socketpv/SignVoteRequest"
+message SignVoteRequest {
+   Vote vote
+}
+
+//  amino registered type
+// registered with "tendermint/socketpv/SignedVoteReply"
+message SignedVoteReply { 
+   Vote      Vote
+   Signature Signature 
+   Err       Error
+}
+
+// we will use this type everywhere below
+message Error {
+  Type        uint  // error code
+  Description string  // optional description
+}
+
+```
+
+The `ChainID` gets moved into the vote message directly. Previously, it was injected 
+using the [Signable] interface method `SignBytes(chainID string) []byte`. Also, the 
+signature won't be included directly, only in the corresponding `SignedVoteReply` message.
+
+[Signable]: https://github.com/tendermint/tendermint/blob/d419fffe18531317c28c29a292ad7d253f6cafdf/types/signable.go#L9-L11
+ 
+### Proposal
+
+```proto
+// vanilla protobuf / amino encoded
+message Proposal {                      
+    Height            sfixed64       
+    Round             sfixed32
+    Timestamp         Timestamp         // << using protobuf definition
+    BlockPartsHeader  PartSetHeader     // as already defined
+    POLRound          sfixed32
+    POLBlockID        BlockID           // << as already defined    
+}
+ 
+// amino registered with "tendermint/socketpv/SignProposalRequest"
+message SignProposalRequest {
+   Proposal proposal
+}
+
+// amino registered with "tendermint/socketpv/SignProposalReply"
+message SignProposalReply { 
+   Prop   Proposal
+   Sig    Signature 
+   Err    Error     // as defined above
+}
+```
+
+### Heartbeat
+
+**TODO**: clarify if heartbeat also needs a fixed offset and update the fields accordingly: 
+
+```proto
+message Heartbeat {
+	ValidatorAddress Address 
+	ValidatorIndex   int     
+	Height           int64   
+	Round            int     
+	Sequence         int     
+}
+// amino registered with "tendermint/socketpv/SignHeartbeatRequest"
+message SignHeartbeatRequest {
+   Hb Heartbeat
+}
+
+// amino registered with "tendermint/socketpv/SignHeartbeatReply"
+message SignHeartbeatReply { 
+   Hb     Heartbeat
+   Sig    Signature 
+   Err    Error     // as defined above
+}
+
+```
+
+## PubKey
+
+TBA -  this needs further thoughts: e.g. what todo like in the case of the KMS which holds
+several keys? How does it know with which key to reply?
+
+## SignBytes
+`SignBytes` will not require a `ChainID` parameter:
+
+```golang
+type Signable interface {
+	SignBytes() []byte
+}
+
+```
+And the implementation for vote, heartbeat, proposal will look like:
+```golang
+// type T is one of vote, sign, proposal
+func (tp *T) SignBytes() []byte {
+	bz, err := cdc.MarshalBinary(tp)
+	if err != nil {
+		panic(err)
+	}
+	return bz
+}
+```
+
+## Status
+
+DRAFT
+
+## Consequences
+
+
+
+### Positive
+
+The most relevant positive effect is that the signing bytes can easily be parsed by a 
+hardware module and a smart contract. Besides that:
+ 
+- clearer separation between requests and responses
+- added error messages enable better error handling 
+
+
+### Negative
+
+- relatively huge change / refactoring touching quite some code
+- lot's of places assume a `Vote` with a signature included -> they will need to 
+- need to modify some interfaces 
+
+### Neutral
+
+not even the swiss are neutral

docs/architecture/adr-025-commit.md

@@ -0,0 +1,75 @@
+# ADR 025 Commit
+
+## Context
+Currently the `Commit` structure contains a lot of potentially redundant or unnecessary data.
+In particular it contains an array of every precommit from the validators, which includes many copies of the same data. Such as `Height`, `Round`, `Type`, and `BlockID`. Also the `ValidatorIndex` could be derived from the vote's position in the array, and the `ValidatorAddress` could potentially be derived from runtime context. The only truely necessary data is the `Signature` and `Timestamp` associated with each `Vote`.
+
+```
+type Commit struct {
+    BlockID    BlockID `json:"block_id"`
+    Precommits []*Vote `json:"precommits"`
+}
+type Vote struct {
+    ValidatorAddress Address   `json:"validator_address"`
+    ValidatorIndex   int       `json:"validator_index"`
+    Height           int64     `json:"height"`
+    Round            int       `json:"round"`
+    Timestamp        time.Time `json:"timestamp"`
+    Type             byte      `json:"type"`
+    BlockID          BlockID   `json:"block_id"`
+    Signature        []byte    `json:"signature"`
+}
+```
+References:
+[#1648](https://github.com/tendermint/tendermint/issues/1648)
+[#2179](https://github.com/tendermint/tendermint/issues/2179)
+[#2226](https://github.com/tendermint/tendermint/issues/2226)
+
+## Proposed Solution
+We can improve efficiency by replacing the usage of the `Vote` struct with a subset of each vote, and by storing the constant values (`Height`, `Round`, `BlockID`) in the Commit itself.
+```
+type Commit struct {
+    Height  int64
+    Round   int
+    BlockID    BlockID      `json:"block_id"`
+    Precommits []*CommitSig `json:"precommits"`
+}
+type CommitSig struct {
+    ValidatorAddress Address
+    Signature []byte
+    Timestamp time.Time
+}
+```
+Continuing to store the `ValidatorAddress` in the `CommitSig` takes up extra space, but simplifies the process and allows for easier debugging.
+
+## Status
+Proposed
+
+## Consequences
+
+### Positive
+The size of a `Commit` transmitted over the network goes from:
+
+|BlockID| + n * (|Address| + |ValidatorIndex| + |Height| + |Round| + |Timestamp| + |Type| + |BlockID| + |Signature|)
+
+to:
+
+
+|BlockID|+|Height|+|Round| + n*(|Address| + |Signature| + |Timestamp|)
+
+This saves:
+
+n * (|BlockID| + |ValidatorIndex| + |Type|) + (n-1) * (Height + Round)
+
+In the current context, this would concretely be:
+(assuming all ints are int64, and hashes are 32 bytes)
+
+n *(72 + 8 + 1 + 8 + 8) - 16 = n * 97 - 16
+
+With 100 validators this is a savings of almost 10KB on every block.
+
+### Negative
+This would add some complexity to the processing and verification of blocks and commits, as votes would have to be reconstructed to be verified and gossiped. The reconstruction could be relatively straightforward, only requiring the copying of data from the `Commit` itself into the newly created `Vote`.
+
+### Neutral
+This design leaves the `ValidatorAddress` in the `CommitSig` and in the `Vote`. These could be removed at some point for additional savings, but that would introduce more complexity, and make printing of `Commit` and `VoteSet` objects less informative, which could harm debugging efficiency and UI/UX.

docs/architecture/adr-026-general-merkle-proof.md

@@ -0,0 +1,47 @@
+# ADR 026: General Merkle Proof
+
+## Context
+
+We are using raw `[]byte` for merkle proofs in `abci.ResponseQuery`. It makes hard to handle multilayer merkle proofs and general cases. Here, new interface `ProofOperator` is defined. The users can defines their own Merkle proof format and layer them easily. 
+
+Goals:
+- Layer Merkle proofs without decoding/reencoding
+- Provide general way to chain proofs
+- Make the proof format extensible, allowing thirdparty proof types
+
+## Decision
+
+### ProofOperator
+
+`type ProofOperator` is an interface for Merkle proofs. The definition is:
+
+```go
+type ProofOperator interface {
+    Run([][]byte) ([][]byte, error)
+    GetKey() []byte
+    ProofOp() ProofOp
+}
+```
+
+Since a proof can treat various data type, `Run()` takes `[][]byte` as the argument, not `[]byte`. For example, a range proof's `Run()` can take multiple key-values as its argument. It will then return the root of the tree for the further process, calculated with the input value.
+
+`ProofOperator` does not have to be a Merkle proof - it can be a function that transforms the argument for intermediate process e.g. prepending the length to the `[]byte`.
+
+### ProofOp
+
+`type ProofOp` is a protobuf message which is a triple of `Type string`, `Key []byte`, and `Data []byte`. `ProofOperator` and `ProofOp`are interconvertible, using `ProofOperator.ProofOp()` and `OpDecoder()`, where `OpDecoder` is a function that each proof type can register for their own encoding scheme. For example, we can add an byte for encoding scheme before the serialized proof, supporting JSON decoding.
+
+## Status
+
+## Consequences
+
+### Positive
+
+- Layering becomes easier (no encoding/decoding at each step)
+- Thirdparty proof format is available
+
+### Negative 
+
+- Larger size for abci.ResponseQuery
+- Unintuitive proof chaining(it is not clear what `Run()` is doing)
+- Additional codes for registering `OpDecoder`s

docs/architecture/adr-029-check-tx-consensus.md

@@ -0,0 +1,128 @@
+# ADR 029: Check block txs before prevote
+
+## Changelog
+
+04-10-2018: Update with link to issue
+[#2384](https://github.com/tendermint/tendermint/issues/2384) and reason for rejection
+19-09-2018: Initial Draft
+
+## Context
+
+We currently check a tx's validity through 2 ways.
+
+1. Through checkTx in mempool connection.
+2. Through deliverTx in consensus connection.
+
+The 1st is called when external tx comes in, so the node should be a proposer this time. The 2nd is called when external block comes in and reach the commit phase, the node doesn't need to be the proposer of the block, however it should check the txs in that block.
+
+In the 2nd situation, if there are many invalid txs in the block, it would be too late for all nodes to discover that most txs in the block are invalid, and we'd better not record invalid txs in the blockchain too.
+
+## Proposed solution
+
+Therefore, we should find a way to check the txs' validity before send out a prevote. Currently we have cs.isProposalComplete() to judge whether a block is complete. We can have
+
+```
+func (blockExec *BlockExecutor) CheckBlock(block *types.Block) error {
+   // check txs of block.
+   for _, tx := range block.Txs {
+      reqRes := blockExec.proxyApp.CheckTxAsync(tx)
+      reqRes.Wait()
+      if reqRes.Response == nil || reqRes.Response.GetCheckTx() == nil || reqRes.Response.GetCheckTx().Code != abci.CodeTypeOK {
+         return errors.Errorf("tx %v check failed. response: %v", tx, reqRes.Response)
+      }
+   }
+   return nil
+}
+```
+
+such a method in BlockExecutor to check all txs' validity in that block.
+
+However, this method should not be implemented like that, because checkTx will share the same state used in mempool in the app.  So we should define a new interface method checkBlock in Application to indicate it to use the same state as deliverTx.
+
+```
+type Application interface {
+   // Info/Query Connection
+   Info(RequestInfo) ResponseInfo                // Return application info
+   SetOption(RequestSetOption) ResponseSetOption // Set application option
+   Query(RequestQuery) ResponseQuery             // Query for state
+
+   // Mempool Connection
+   CheckTx(tx []byte) ResponseCheckTx // Validate a tx for the mempool
+
+   // Consensus Connection
+   InitChain(RequestInitChain) ResponseInitChain // Initialize blockchain with validators and other info from TendermintCore
+   CheckBlock(RequestCheckBlock) ResponseCheckBlock
+   BeginBlock(RequestBeginBlock) ResponseBeginBlock // Signals the beginning of a block
+   DeliverTx(tx []byte) ResponseDeliverTx           // Deliver a tx for full processing
+   EndBlock(RequestEndBlock) ResponseEndBlock       // Signals the end of a block, returns changes to the validator set
+   Commit() ResponseCommit                          // Commit the state and return the application Merkle root hash
+}
+```
+
+All app should implement that method. For example, counter:
+
+```
+func (app *CounterApplication) CheckBlock(block types.Request_CheckBlock) types.ResponseCheckBlock {
+   if app.serial {
+   	  app.originalTxCount = app.txCount   //backup the txCount state
+      for _, tx := range block.CheckBlock.Block.Txs {
+         if len(tx) > 8 {
+            return types.ResponseCheckBlock{
+               Code: code.CodeTypeEncodingError,
+               Log:  fmt.Sprintf("Max tx size is 8 bytes, got %d", len(tx))}
+         }
+         tx8 := make([]byte, 8)
+         copy(tx8[len(tx8)-len(tx):], tx)
+         txValue := binary.BigEndian.Uint64(tx8)
+         if txValue < uint64(app.txCount) {
+            return types.ResponseCheckBlock{
+               Code: code.CodeTypeBadNonce,
+               Log:  fmt.Sprintf("Invalid nonce. Expected >= %v, got %v", app.txCount, txValue)}
+         }
+         app.txCount++
+      }
+   }
+   return types.ResponseCheckBlock{Code: code.CodeTypeOK}
+}
+```
+
+In BeginBlock, the app should restore the state to the orignal state before checking the block:
+
+```
+func (app *CounterApplication) DeliverTx(tx []byte) types.ResponseDeliverTx {
+   if app.serial {
+      app.txCount = app.originalTxCount   //restore the txCount state
+   }
+   app.txCount++
+   return types.ResponseDeliverTx{Code: code.CodeTypeOK}
+}
+```
+
+The txCount is like the nonce in ethermint, it should be restored when entering the deliverTx phase. While some operation like checking the tx signature needs not to be done again. So the deliverTx can focus on how a tx can be applied, ignoring the checking of the tx, because all the checking has already been done in the checkBlock phase before.
+
+An optional optimization is alter the deliverTx to deliverBlock. For the block has already been checked by checkBlock, so all the txs in it are valid. So the app can cache the block, and in the deliverBlock phase, it just needs to apply the block in the cache. This optimization can save network current in deliverTx.
+
+
+
+## Status
+
+Rejected
+
+## Decision
+
+Performance impact is considered too great. See [#2384](https://github.com/tendermint/tendermint/issues/2384)
+
+## Consequences
+
+### Positive
+
+- more robust to defend the adversary to propose a block full of invalid txs.
+
+### Negative
+
+- add a new interface method. app logic needs to adjust to appeal to it.
+- sending all the tx data over the ABCI twice
+- potentially redundant validations (eg. signature checks in both CheckBlock and
+  DeliverTx)
+
+### Neutral

docs/architecture/adr-030-consensus-refactor.md

@@ -0,0 +1,152 @@
+# ADR 030: Consensus Refactor
+
+## Context
+
+One of the biggest challenges this project faces is to proof that the
+implementations of the specifications are correct, much like we strive to
+formaly verify our alogrithms and protocols we should work towards high
+confidence about the correctness of our program code. One of those is the core
+of Tendermint - Consensus - which currently resides in the `consensus` package.
+Over time there has been high friction making changes to the package due to the
+algorithm being scattered in a side-effectful container (the current
+`ConsensusState`). In order to test the algorithm a large object-graph needs to
+be set up and even than the non-deterministic parts of the container makes will
+prevent high certainty. Where ideally we have a 1-to-1 representation of the
+[spec](https://github.com/tendermint/spec), ready and easy to test for domain
+experts.
+
+Addresses:
+
+- [#1495](https://github.com/tendermint/tendermint/issues/1495)
+- [#1692](https://github.com/tendermint/tendermint/issues/1692)
+
+## Decision
+
+To remedy these issues we plan a gradual, non-invasive refactoring of the
+`consensus` package. Starting of by isolating the consensus alogrithm into
+a pure function and a finite state machine to address the most pressuring issue
+of lack of confidence. Doing so while leaving the rest of the package in tact
+and have follow-up optional changes to improve the sepration of concerns.
+
+### Implementation changes
+
+The core of Consensus can be modelled as a function with clear defined inputs:
+
+* `State` - data container for current round, height, etc.
+* `Event`- significant events in the network
+
+producing clear outputs;
+
+* `State` - updated input
+* `Message` - signal what actions to perform
+
+```go
+type Event int
+
+const (
+	EventUnknown Event = iota
+	EventProposal
+	Majority23PrevotesBlock
+	Majority23PrecommitBlock
+	Majority23PrevotesAny
+	Majority23PrecommitAny
+	TimeoutNewRound
+	TimeoutPropose
+	TimeoutPrevotes
+	TimeoutPrecommit
+)
+
+type Message int
+
+const (
+	MeesageUnknown Message = iota
+	MessageProposal
+	MessageVotes
+	MessageDecision
+)
+
+type State struct {
+	height      uint64
+	round       uint64
+	step        uint64
+	lockedValue interface{} // TODO: Define proper type.
+	lockedRound interface{} // TODO: Define proper type.
+	validValue  interface{} // TODO: Define proper type.
+	validRound  interface{} // TODO: Define proper type.
+	// From the original notes: valid(v)
+	valid       interface{} // TODO: Define proper type.
+	// From the original notes: proposer(h, r)
+	proposer    interface{} // TODO: Define proper type.
+}
+
+func Consensus(Event, State) (State, Message) {
+	// Consolidate implementation.
+}
+```
+
+Tracking of relevant information to feed `Event` into the function and act on
+the output is left to the `ConsensusExecutor` (formerly `ConsensusState`). 
+
+Benefits for testing surfacing nicely as testing for a sequence of events
+against algorithm could be as simple as the following example:
+
+``` go
+func TestConsensusXXX(t *testing.T) {
+	type expected struct {
+		message Message
+		state   State
+	}
+
+	// Setup order of events, initial state and expectation.
+	var (
+		events = []struct {
+			event Event
+			want  expected
+		}{
+		// ...
+		}
+		state = State{
+		// ...
+		}
+	)
+
+	for _, e := range events {
+		sate, msg = Consensus(e.event, state)
+
+		// Test message expectation.
+		if msg != e.want.message {
+			t.Fatalf("have %v, want %v", msg, e.want.message)
+		}
+
+		// Test state expectation.
+		if !reflect.DeepEqual(state, e.want.state) {
+			t.Fatalf("have %v, want %v", state, e.want.state)
+		}
+	}
+}
+```
+
+### Implementation roadmap
+
+* implement proposed implementation
+* replace currently scattered calls in `ConsensusState` with calls to the new
+  `Consensus` function
+* rename `ConsensusState` to `ConsensusExecutor` to avoid confusion
+* propose design for improved separation and clear information flow between
+  `ConsensusExecutor` and `ConsensusReactor`
+
+## Status
+
+Draft.
+
+## Consequences
+
+### Positive
+
+- isolated implementation of the algorithm
+- improved testability - simpler to proof correctness
+- clearer separation of concerns - easier to reason
+
+### Negative
+
+### Neutral

docs/introduction/README.md

@@ -0,0 +1,15 @@
+# Overview
+
+## Quick Start
+
+Get Tendermint up-and-running quickly with the [quick-start guide](./quick-start.md)!
+
+## Install
+
+Detailed [installation instructions](./install.md).
+
+## What is Tendermint?
+
+Dive into [what Tendermint is and why](./what-is-tendermint.md)!
+
+

docs/introduction/install.md

@@ -77,8 +77,13 @@ make install
 
 ## Compile with CLevelDB support
 
-Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7)
-with snappy. Example for Ubuntu:
+Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7).
+
+Build Tendermint with C libraries: `make build_c`.
+
+### Ubuntu
+
+Install LevelDB with snappy:
 
 ```
 sudo apt-get update
@@ -90,9 +95,9 @@ wget https://github.com/google/leveldb/archive/v1.20.tar.gz && \
   tar -zxvf v1.20.tar.gz && \
   cd leveldb-1.20/ && \
   make && \
-  sudo scp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \
+  cp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \
   cd include/ && \
-  sudo scp -r leveldb /usr/local/include/ && \
+  cp -r leveldb /usr/local/include/ && \
   sudo ldconfig && \
   rm -f v1.20.tar.gz
 ```

docs/introduction/introduction.md

@@ -1,5 +1,7 @@
 # What is Tendermint?
 
+DEPRECATED! See [What is Tendermint?](what-is-tendermint.md).
+
 Tendermint is software for securely and consistently replicating an
 application on many machines. By securely, we mean that Tendermint works
 even if up to 1/3 of machines fail in arbitrary ways. By consistently,

docs/introduction/quick-start.md

@@ -1,4 +1,4 @@
-# Tendermint
+# Quick Start
 
 ## Overview
 
@@ -9,45 +9,21 @@ works and want to get started right away, continue.
 
 ### Quick Install
 
-On a fresh Ubuntu 16.04 machine can be done with [this script](https://git.io/fFfOR), like so:
+To quickly get Tendermint installed on a fresh
+Ubuntu 16.04 machine, use [this script](https://git.io/fFfOR).
+
+WARNING: do not run this on your local machine.
 
 ```
 curl -L https://git.io/fFfOR | bash
 source ~/.profile
 ```
 
-WARNING: do not run the above on your local machine.
-
 The script is also used to facilitate cluster deployment below.
 
 ### Manual Install
 
-Requires:
-
-- `go` minimum version 1.10
-- `$GOPATH` environment variable must be set
-- `$GOPATH/bin` must be on your `$PATH` (see [here](https://github.com/tendermint/tendermint/wiki/Setting-GOPATH))
-
-To install Tendermint, run:
-
-```
-go get github.com/tendermint/tendermint
-cd $GOPATH/src/github.com/tendermint/tendermint
-make get_tools && make get_vendor_deps
-make install
-```
-
-Note that `go get` may return an error but it can be ignored.
-
-Confirm installation:
-
-```
-$ tendermint version
-0.23.0
-```
-
-Note: see the [releases page](https://github.com/tendermint/tendermint/releases) and the latest version
-should match what you see above.
+For manual installation, see the [install instructions](install.md)
 
 ## Initialization
 

docs/introduction/what-is-tendermint.md

@@ -0,0 +1,332 @@
+# What is Tendermint?
+
+Tendermint is software for securely and consistently replicating an
+application on many machines. By securely, we mean that Tendermint works
+even if up to 1/3 of machines fail in arbitrary ways. By consistently,
+we mean that every non-faulty machine sees the same transaction log and
+computes the same state. Secure and consistent replication is a
+fundamental problem in distributed systems; it plays a critical role in
+the fault tolerance of a broad range of applications, from currencies,
+to elections, to infrastructure orchestration, and beyond.
+
+The ability to tolerate machines failing in arbitrary ways, including
+becoming malicious, is known as Byzantine fault tolerance (BFT). The
+theory of BFT is decades old, but software implementations have only
+became popular recently, due largely to the success of "blockchain
+technology" like Bitcoin and Ethereum. Blockchain technology is just a
+reformalization of BFT in a more modern setting, with emphasis on
+peer-to-peer networking and cryptographic authentication. The name
+derives from the way transactions are batched in blocks, where each
+block contains a cryptographic hash of the previous one, forming a
+chain. In practice, the blockchain data structure actually optimizes BFT
+design.
+
+Tendermint consists of two chief technical components: a blockchain
+consensus engine and a generic application interface. The consensus
+engine, called Tendermint Core, ensures that the same transactions are
+recorded on every machine in the same order. The application interface,
+called the Application BlockChain Interface (ABCI), enables the
+transactions to be processed in any programming language. Unlike other
+blockchain and consensus solutions, which come pre-packaged with built
+in state machines (like a fancy key-value store, or a quirky scripting
+language), developers can use Tendermint for BFT state machine
+replication of applications written in whatever programming language and
+development environment is right for them.
+
+Tendermint is designed to be easy-to-use, simple-to-understand, highly
+performant, and useful for a wide variety of distributed applications.
+
+## Tendermint vs. X
+
+Tendermint is broadly similar to two classes of software. The first
+class consists of distributed key-value stores, like Zookeeper, etcd,
+and consul, which use non-BFT consensus. The second class is known as
+"blockchain technology", and consists of both cryptocurrencies like
+Bitcoin and Ethereum, and alternative distributed ledger designs like
+Hyperledger's Burrow.
+
+### Zookeeper, etcd, consul
+
+Zookeeper, etcd, and consul are all implementations of a key-value store
+atop a classical, non-BFT consensus algorithm. Zookeeper uses a version
+of Paxos called Zookeeper Atomic Broadcast, while etcd and consul use
+the Raft consensus algorithm, which is much younger and simpler. A
+typical cluster contains 3-5 machines, and can tolerate crash failures
+in up to 1/2 of the machines, but even a single Byzantine fault can
+destroy the system.
+
+Each offering provides a slightly different implementation of a
+featureful key-value store, but all are generally focused around
+providing basic services to distributed systems, such as dynamic
+configuration, service discovery, locking, leader-election, and so on.
+
+Tendermint is in essence similar software, but with two key differences:
+
+- It is Byzantine Fault Tolerant, meaning it can only tolerate up to a
+  1/3 of failures, but those failures can include arbitrary behaviour -
+  including hacking and malicious attacks. - It does not specify a
+  particular application, like a fancy key-value store. Instead, it
+  focuses on arbitrary state machine replication, so developers can build
+  the application logic that's right for them, from key-value store to
+  cryptocurrency to e-voting platform and beyond.
+
+The layout of this Tendermint website content is also ripped directly
+and without shame from [consul.io](https://www.consul.io/) and the other
+[Hashicorp sites](https://www.hashicorp.com/#tools).
+
+### Bitcoin, Ethereum, etc.
+
+Tendermint emerged in the tradition of cryptocurrencies like Bitcoin,
+Ethereum, etc. with the goal of providing a more efficient and secure
+consensus algorithm than Bitcoin's Proof of Work. In the early days,
+Tendermint had a simple currency built in, and to participate in
+consensus, users had to "bond" units of the currency into a security
+deposit which could be revoked if they misbehaved -this is what made
+Tendermint a Proof-of-Stake algorithm.
+
+Since then, Tendermint has evolved to be a general purpose blockchain
+consensus engine that can host arbitrary application states. That means
+it can be used as a plug-and-play replacement for the consensus engines
+of other blockchain software. So one can take the current Ethereum code
+base, whether in Rust, or Go, or Haskell, and run it as a ABCI
+application using Tendermint consensus. Indeed, [we did that with
+Ethereum](https://github.com/cosmos/ethermint). And we plan to do
+the same for Bitcoin, ZCash, and various other deterministic
+applications as well.
+
+Another example of a cryptocurrency application built on Tendermint is
+[the Cosmos network](http://cosmos.network).
+
+### Other Blockchain Projects
+
+[Fabric](https://github.com/hyperledger/fabric) takes a similar approach
+to Tendermint, but is more opinionated about how the state is managed,
+and requires that all application behaviour runs in potentially many
+docker containers, modules it calls "chaincode". It uses an
+implementation of [PBFT](http://pmg.csail.mit.edu/papers/osdi99.pdf).
+from a team at IBM that is [augmented to handle potentially
+non-deterministic
+chaincode](https://www.zurich.ibm.com/~cca/papers/sieve.pdf) It is
+possible to implement this docker-based behaviour as a ABCI app in
+Tendermint, though extending Tendermint to handle non-determinism
+remains for future work.
+
+[Burrow](https://github.com/hyperledger/burrow) is an implementation of
+the Ethereum Virtual Machine and Ethereum transaction mechanics, with
+additional features for a name-registry, permissions, and native
+contracts, and an alternative blockchain API. It uses Tendermint as its
+consensus engine, and provides a particular application state.
+
+## ABCI Overview
+
+The [Application BlockChain Interface
+(ABCI)](https://github.com/tendermint/tendermint/tree/develop/abci)
+allows for Byzantine Fault Tolerant replication of applications
+written in any programming language.
+
+### Motivation
+
+Thus far, all blockchains "stacks" (such as
+[Bitcoin](https://github.com/bitcoin/bitcoin)) have had a monolithic
+design. That is, each blockchain stack is a single program that handles
+all the concerns of a decentralized ledger; this includes P2P
+connectivity, the "mempool" broadcasting of transactions, consensus on
+the most recent block, account balances, Turing-complete contracts,
+user-level permissions, etc.
+
+Using a monolithic architecture is typically bad practice in computer
+science. It makes it difficult to reuse components of the code, and
+attempts to do so result in complex maintenance procedures for forks of
+the codebase. This is especially true when the codebase is not modular
+in design and suffers from "spaghetti code".
+
+Another problem with monolithic design is that it limits you to the
+language of the blockchain stack (or vice versa). In the case of
+Ethereum which supports a Turing-complete bytecode virtual-machine, it
+limits you to languages that compile down to that bytecode; today, those
+are Serpent and Solidity.
+
+In contrast, our approach is to decouple the consensus engine and P2P
+layers from the details of the application state of the particular
+blockchain application. We do this by abstracting away the details of
+the application to an interface, which is implemented as a socket
+protocol.
+
+Thus we have an interface, the Application BlockChain Interface (ABCI),
+and its primary implementation, the Tendermint Socket Protocol (TSP, or
+Teaspoon).
+
+### Intro to ABCI
+
+[Tendermint Core](https://github.com/tendermint/tendermint) (the
+"consensus engine") communicates with the application via a socket
+protocol that satisfies the ABCI.
+
+To draw an analogy, lets talk about a well-known cryptocurrency,
+Bitcoin. Bitcoin is a cryptocurrency blockchain where each node
+maintains a fully audited Unspent Transaction Output (UTXO) database. If
+one wanted to create a Bitcoin-like system on top of ABCI, Tendermint
+Core would be responsible for
+
+- Sharing blocks and transactions between nodes
+- Establishing a canonical/immutable order of transactions
+  (the blockchain)
+
+The application will be responsible for
+
+- Maintaining the UTXO database
+- Validating cryptographic signatures of transactions
+- Preventing transactions from spending non-existent transactions
+- Allowing clients to query the UTXO database.
+
+Tendermint is able to decompose the blockchain design by offering a very
+simple API (ie. the ABCI) between the application process and consensus
+process.
+
+The ABCI consists of 3 primary message types that get delivered from the
+core to the application. The application replies with corresponding
+response messages.
+
+The messages are specified here: [ABCI Message
+Types](https://github.com/tendermint/tendermint/blob/develop/abci/README.md#message-types).
+
+The **DeliverTx** message is the work horse of the application. Each
+transaction in the blockchain is delivered with this message. The
+application needs to validate each transaction received with the
+**DeliverTx** message against the current state, application protocol,
+and the cryptographic credentials of the transaction. A validated
+transaction then needs to update the application state — by binding a
+value into a key values store, or by updating the UTXO database, for
+instance.
+
+The **CheckTx** message is similar to **DeliverTx**, but it's only for
+validating transactions. Tendermint Core's mempool first checks the
+validity of a transaction with **CheckTx**, and only relays valid
+transactions to its peers. For instance, an application may check an
+incrementing sequence number in the transaction and return an error upon
+**CheckTx** if the sequence number is old. Alternatively, they might use
+a capabilities based system that requires capabilities to be renewed
+with every transaction.
+
+The **Commit** message is used to compute a cryptographic commitment to
+the current application state, to be placed into the next block header.
+This has some handy properties. Inconsistencies in updating that state
+will now appear as blockchain forks which catches a whole class of
+programming errors. This also simplifies the development of secure
+lightweight clients, as Merkle-hash proofs can be verified by checking
+against the block hash, and that the block hash is signed by a quorum.
+
+There can be multiple ABCI socket connections to an application.
+Tendermint Core creates three ABCI connections to the application; one
+for the validation of transactions when broadcasting in the mempool, one
+for the consensus engine to run block proposals, and one more for
+querying the application state.
+
+It's probably evident that applications designers need to very carefully
+design their message handlers to create a blockchain that does anything
+useful but this architecture provides a place to start. The diagram
+below illustrates the flow of messages via ABCI.
+
+![](../imgs/abci.png)
+
+## A Note on Determinism
+
+The logic for blockchain transaction processing must be deterministic.
+If the application logic weren't deterministic, consensus would not be
+reached among the Tendermint Core replica nodes.
+
+Solidity on Ethereum is a great language of choice for blockchain
+applications because, among other reasons, it is a completely
+deterministic programming language. However, it's also possible to
+create deterministic applications using existing popular languages like
+Java, C++, Python, or Go. Game programmers and blockchain developers are
+already familiar with creating deterministic programs by avoiding
+sources of non-determinism such as:
+
+- random number generators (without deterministic seeding)
+- race conditions on threads (or avoiding threads altogether)
+- the system clock
+- uninitialized memory (in unsafe programming languages like C
+  or C++)
+- [floating point
+  arithmetic](http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/)
+- language features that are random (e.g. map iteration in Go)
+
+While programmers can avoid non-determinism by being careful, it is also
+possible to create a special linter or static analyzer for each language
+to check for determinism. In the future we may work with partners to
+create such tools.
+
+## Consensus Overview
+
+Tendermint is an easy-to-understand, mostly asynchronous, BFT consensus
+protocol. The protocol follows a simple state machine that looks like
+this:
+
+![](../imgs/consensus_logic.png)
+
+Participants in the protocol are called **validators**; they take turns
+proposing blocks of transactions and voting on them. Blocks are
+committed in a chain, with one block at each **height**. A block may
+fail to be committed, in which case the protocol moves to the next
+**round**, and a new validator gets to propose a block for that height.
+Two stages of voting are required to successfully commit a block; we
+call them **pre-vote** and **pre-commit**. A block is committed when
+more than 2/3 of validators pre-commit for the same block in the same
+round.
+
+There is a picture of a couple doing the polka because validators are
+doing something like a polka dance. When more than two-thirds of the
+validators pre-vote for the same block, we call that a **polka**. Every
+pre-commit must be justified by a polka in the same round.
+
+Validators may fail to commit a block for a number of reasons; the
+current proposer may be offline, or the network may be slow. Tendermint
+allows them to establish that a validator should be skipped. Validators
+wait a small amount of time to receive a complete proposal block from
+the proposer before voting to move to the next round. This reliance on a
+timeout is what makes Tendermint a weakly synchronous protocol, rather
+than an asynchronous one. However, the rest of the protocol is
+asynchronous, and validators only make progress after hearing from more
+than two-thirds of the validator set. A simplifying element of
+Tendermint is that it uses the same mechanism to commit a block as it
+does to skip to the next round.
+
+Assuming less than one-third of the validators are Byzantine, Tendermint
+guarantees that safety will never be violated - that is, validators will
+never commit conflicting blocks at the same height. To do this it
+introduces a few **locking** rules which modulate which paths can be
+followed in the flow diagram. Once a validator precommits a block, it is
+locked on that block. Then,
+
+1.  it must prevote for the block it is locked on
+2.  it can only unlock, and precommit for a new block, if there is a
+    polka for that block in a later round
+
+## Stake
+
+In many systems, not all validators will have the same "weight" in the
+consensus protocol. Thus, we are not so much interested in one-third or
+two-thirds of the validators, but in those proportions of the total
+voting power, which may not be uniformly distributed across individual
+validators.
+
+Since Tendermint can replicate arbitrary applications, it is possible to
+define a currency, and denominate the voting power in that currency.
+When voting power is denominated in a native currency, the system is
+often referred to as Proof-of-Stake. Validators can be forced, by logic
+in the application, to "bond" their currency holdings in a security
+deposit that can be destroyed if they're found to misbehave in the
+consensus protocol. This adds an economic element to the security of the
+protocol, allowing one to quantify the cost of violating the assumption
+that less than one-third of voting power is Byzantine.
+
+The [Cosmos Network](https://cosmos.network) is designed to use this
+Proof-of-Stake mechanism across an array of cryptocurrencies implemented
+as ABCI applications.
+
+The following diagram is Tendermint in a (technical) nutshell. [See here
+for high resolution
+version](https://github.com/mobfoundry/hackatom/blob/master/tminfo.pdf).
+
+![](../imgs/tm-transaction-flow.png)

docs/networks/README.md

@@ -0,0 +1,9 @@
+# Overview
+
+Use [Docker Compose](./docker-compose.md) to spin up Tendermint testnets on your
+local machine.
+
+Use [Terraform and Ansible](./terraform-and-ansible.md) to deploy Tendermint
+testnets to the cloud.
+
+See the `tendermint testnet --help` command for more help initializing testnets.

docs/networks/deploy-testnets.md

@@ -1,8 +1,8 @@
 # Deploy a Testnet
 
-Now that we've seen how ABCI works, and even played with a few
-applications on a single validator node, it's time to deploy a test
-network to four validator nodes.
+DEPRECATED DOCS!
+
+See [Networks](../networks).
 
 ## Manual Deployments
 
@@ -21,17 +21,16 @@ Here are the steps to setting up a testnet manually:
 3.  Generate a private key and a node key for each validator using
     `tendermint init`
 4.  Compile a list of public keys for each validator into a
-    `genesis.json` file and replace the existing file with it.
-5.  Run
-    `tendermint node --proxy_app=kvstore --p2p.persistent_peers=< peer addresses >` on each node, where `< peer addresses >` is a comma separated
-    list of the ID@IP:PORT combination for each node. The default port for
-    Tendermint is `26656`. The ID of a node can be obtained by running
-    `tendermint show_node_id` command. Thus, if the IP addresses of your nodes
-    were `192.168.0.1, 192.168.0.2, 192.168.0.3, 192.168.0.4`, the command
-    would look like:
+    new `genesis.json` file and replace the existing file with it.
+5.  Get the node IDs of any peers you want other peers to connect to by
+    running `tendermint show_node_id` on the relevant machine
+6.  Set the `p2p.persistent_peers` in the config for all nodes to the comma
+    separated list of `ID@IP:PORT` for all nodes. Default port is 26656.
+
+Then start the node
 
 ```
-tendermint node --proxy_app=kvstore --p2p.persistent_peers=96663a3dd0d7b9d17d4c8211b191af259621c693@192.168.0.1:26656, 429fcf25974313b95673f58d77eacdd434402665@192.168.0.2:26656, 0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@192.168.0.3:26656, f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@192.168.0.4:26656
+tendermint node --proxy_app=kvstore
 ```
 
 After a few seconds, all the nodes should connect to each other and
@@ -67,7 +66,17 @@ make localnet-start
 ```
 
 from the root of the tendermint repository. This will spin up a 4-node
-local testnet. Review the target in the Makefile to debug any problems.
+local testnet. Note that this command expects a linux binary in the build directory. 
+If you built the binary using a non-linux OS, you may see 
+the error `Binary needs to be OS linux, ARCH amd64`, in which case you can
+run:
+
+```
+make build-linux
+make localnet-start
+```
+
+Review the target in the Makefile to debug any problems.
 
 ### Cloud
 

docs/networks/docker-compose.md

@@ -0,0 +1,85 @@
+# Docker Compose
+
+With Docker Compose, we can spin up local testnets in a single command:
+
+```
+make localnet-start
+```
+
+## Requirements
+
+- [Install tendermint](/docs/install.md)
+- [Install docker](https://docs.docker.com/engine/installation/)
+- [Install docker-compose](https://docs.docker.com/compose/install/)
+
+## Build
+
+Build the `tendermint` binary and the `tendermint/localnode` docker image.
+
+Note the binary will be mounted into the container so it can be updated without
+rebuilding the image.
+
+```
+cd $GOPATH/src/github.com/tendermint/tendermint
+
+# Build the linux binary in ./build
+make build-linux
+
+# Build tendermint/localnode image
+make build-docker-localnode
+```
+
+
+## Run a testnet
+
+To start a 4 node testnet run:
+
+```
+make localnet-start
+```
+
+The nodes bind their RPC servers to ports 26657, 26660, 26662, and 26664 on the host.
+This file creates a 4-node network using the localnode image.
+The nodes of the network expose their P2P and RPC endpoints to the host machine on ports 26656-26657, 26659-26660, 26661-26662, and 26663-26664 respectively.
+
+To update the binary, just rebuild it and restart the nodes:
+
+```
+make build-linux
+make localnet-stop
+make localnet-start
+```
+
+## Configuration
+
+The `make localnet-start` creates files for a 4-node testnet in `./build` by calling the `tendermint testnet` command.
+
+The `./build` directory is mounted to the `/tendermint` mount point to attach the binary and config files to the container.
+
+For instance, to create a single node testnet:
+
+```
+cd $GOPATH/src/github.com/tendermint/tendermint
+
+# Clear the build folder
+rm -rf ./build
+
+# Build binary
+make build-linux
+
+# Create configuration
+docker run -e LOG="stdout" -v `pwd`/build:/tendermint tendermint/localnode testnet --o . --v 1
+
+#Run the node
+docker run -v `pwd`/build:/tendermint tendermint/localnode
+
+```
+
+## Logging
+
+Log is saved under the attached volume, in the `tendermint.log` file. If the `LOG` environment variable is set to `stdout` at start, the log is not saved, but printed on the screen.
+
+## Special binaries
+
+If you have multiple binaries with different names, you can specify which one to run with the BINARY environment variable. The path of the binary is relative to the attached volume.
+

docs/networks/terraform-and-ansible.md

@@ -29,7 +29,7 @@ export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"
 
 These will be used by both `terraform` and `ansible`.
 
-### Terraform
+## Terraform
 
 This step will create four Digital Ocean droplets. First, go to the
 correct directory:
@@ -49,7 +49,7 @@ and you will get a list of IP addresses that belong to your droplets.
 
 With the droplets created and running, let's setup Ansible.
 
-### Ansible
+## Ansible
 
 The playbooks in [the ansible
 directory](https://github.com/tendermint/tendermint/tree/master/networks/remote/ansible)
@@ -144,7 +144,7 @@ Peek at the logs with the status role:
 ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml
 ```
 
-### Logging
+## Logging
 
 The crudest way is the status role described above. You can also ship
 logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana)
@@ -160,7 +160,7 @@ go get github.com/mheese/journalbeat
 ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345
 ```
 
-### Cleanup
+## Cleanup
 
 To remove your droplets, run:
 

docs/research/determinism.md

@@ -1,3 +1,3 @@
 # On Determinism
 
-Arguably, the most difficult part of blockchain programming is determinism - that is, ensuring that sources of indeterminism do not creep into the design of such systems.
+See [Determinism](../spec/abci/abci.md#determinism).

docs/research/transactional-semantics.md

@@ -1,25 +1,5 @@
 # Transactional Semantics
 
-In [Using Tendermint](../tendermint-core/using-tendermint.md#broadcast-api) we
-discussed different API endpoints for sending transactions and
-differences between them.
+See details of the [broadcast API](../tendermint-core/using-tendermint.md#broadcast-api)
+and the [mempool WAL](../tendermint-core/running-in-production.md#mempool-wal).
 
-What we have not yet covered is transactional semantics.
-
-When you send a transaction using one of the available methods, it first
-goes to the mempool. Currently, it does not provide strong guarantees
-like "if the transaction were accepted, it would be eventually included
-in a block (given CheckTx passes)."
-
-For instance a tx could enter the mempool, but before it can be sent to
-peers the node crashes.
-
-We are planning to provide such guarantees by using a WAL and replaying
-transactions (See
-[this issue](https://github.com/tendermint/tendermint/issues/248)), but
-it's non-trivial to do this all efficiently.
-
-The temporary solution is for clients to monitor the node and resubmit
-transaction(s) and/or send them to more nodes at once, so the
-probability of all of them crashing at the same time and losing the msg
-decreases substantially.

docs/spec/abci/README.md

@@ -1,7 +1,7 @@
-# ABCI
+# Overview
 
 ABCI is the interface between Tendermint (a state-machine replication engine)
-and an application (the actual state machine). It consists of a set of
+and your application (the actual state machine). It consists of a set of
 *methods*, where each method has a corresponding `Request` and `Response`
 message type. Tendermint calls the ABCI methods on the ABCI application by sending the `Request*`
 messages and receiving the `Response*` messages in return.
@@ -11,9 +11,9 @@ This allows Tendermint to run applications written in any programming language.
 
 This specification is split as follows:
 
-- [Methods and Types](abci.md) - complete details on all ABCI methods and
+- [Methods and Types](./abci.md) - complete details on all ABCI methods and
   message types
-- [Applications](apps.md) - how to manage ABCI application state and other
+- [Applications](./apps.md) - how to manage ABCI application state and other
   details about building ABCI applications
-- [Client and Server](client-server.md) - for those looking to implement their
+- [Client and Server](./client-server.md) - for those looking to implement their
   own ABCI application servers

docs/spec/abci/abci.md

@@ -7,9 +7,9 @@ file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.pro
 
 ABCI methods are split across 3 separate ABCI *connections*:
 
-- `Consensus Connection: InitChain, BeginBlock, DeliverTx, EndBlock, Commit`
-- `Mempool Connection: CheckTx`
-- `Info Connection: Info, SetOption, Query`
+- `Consensus Connection`: `InitChain, BeginBlock, DeliverTx, EndBlock, Commit`
+- `Mempool Connection`: `CheckTx`
+- `Info Connection`: `Info, SetOption, Query`
 
 The `Consensus Connection` is driven by a consensus protocol and is responsible
 for block execution.
@@ -29,10 +29,15 @@ Some methods (`Echo, Info, InitChain, BeginBlock, EndBlock, Commit`),
 don't return errors because an error would indicate a critical failure
 in the application and there's nothing Tendermint can do. The problem
 should be addressed and both Tendermint and the application restarted.
+
 All other methods (`SetOption, Query, CheckTx, DeliverTx`) return an
 application-specific response `Code uint32`, where only `0` is reserved
 for `OK`.
 
+Finally, `Query`, `CheckTx`, and `DeliverTx` include a `Codespace string`, whose
+intended use is to disambiguate `Code` values returned by different domains of the
+application. The `Codespace` is a namespace for the `Code`.
+
 ## Tags
 
 Some methods (`CheckTx, BeginBlock, DeliverTx, EndBlock`)
@@ -48,16 +53,50 @@ Keys and values in tags must be UTF-8 encoded strings (e.g.
 
 ## Determinism
 
-Some methods (`SetOption, Query, CheckTx, DeliverTx`) return
-non-deterministic data in the form of `Info` and `Log`. The `Log` is
-intended for the literal output from the application's logger, while the
-`Info` is any additional info that should be returned.
-
-All other fields in the `Response*` of all methods must be strictly deterministic.
+ABCI applications must implement deterministic finite-state machines to be
+securely replicated by the Tendermint consensus. This means block execution
+over the Consensus Connection must be strictly deterministic: given the same
+ordered set of requests, all nodes will compute identical responses, for all
+BeginBlock, DeliverTx, EndBlock, and Commit. This is critical, because the
+responses are included in the header of the next block, either via a Merkle root
+or directly, so all nodes must agree on exactly what they are.
 
 For this reason, it is recommended that applications not be exposed to any
 external user or process except via the ABCI connections to a consensus engine
-like Tendermint Core.
+like Tendermint Core. The application must only change its state based on input
+from block execution (BeginBlock, DeliverTx, EndBlock, Commit), and not through
+any other kind of request. This is the only way to ensure all nodes see the same
+transactions and compute the same results.
+
+If there is some non-determinism in the state machine, consensus will eventually
+fail as nodes disagree over the correct values for the block header. The
+non-determinism must be fixed and the nodes restarted.
+
+Sources of non-determinism in applications may include:
+
+- Hardware failures
+    - Cosmic rays, overheating, etc.
+- Node-dependent state
+    - Random numbers
+    - Time
+- Underspecification
+    - Library version changes
+    - Race conditions
+    - Floating point numbers
+    - JSON serialization
+    - Iterating through hash-tables/maps/dictionaries
+- External Sources
+    - Filesystem
+    - Network calls (eg. some external REST API service)
+
+See [#56](https://github.com/tendermint/abci/issues/56) for original discussion.
+
+Note that some methods (`SetOption, Query, CheckTx, DeliverTx`) return
+explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is
+intended for the literal output from the application's logger, while the
+`Info` is any additional info that should be returned. These are the only fields
+that are not included in block header computations, so we don't need agreement
+on them. All other fields in the `Response*` must be strictly deterministic.
 
 ## Block Execution
 
@@ -156,9 +195,9 @@ Commit are included in the header of the next block.
     of Path.
   - `Path (string)`: Path of request, like an HTTP GET path. Can be
     used with or in liue of Data.
-  - Apps MUST interpret '/store' as a query by key on the
+    - Apps MUST interpret '/store' as a query by key on the
     underlying store. The key SHOULD be specified in the Data field.
-  - Apps SHOULD allow queries over specific types like
+    - Apps SHOULD allow queries over specific types like
     '/accounts/...' or '/votes/...'
   - `Height (int64)`: The block height for which you want the query
     (default=0 returns data for the latest committed block). Note
@@ -175,15 +214,18 @@ Commit are included in the header of the next block.
   - `Index (int64)`: The index of the key in the tree.
   - `Key ([]byte)`: The key of the matching data.
   - `Value ([]byte)`: The value of the matching data.
-  - `Proof ([]byte)`: Serialized proof for the data, if requested, to be
+  - `Proof (Proof)`: Serialized proof for the value data, if requested, to be
     verified against the `AppHash` for the given Height.
   - `Height (int64)`: The block height from which data was derived.
     Note that this is the height of the block containing the
     application's Merkle root hash, which represents the state as it
     was after committing the block at Height-1
+  - `Codespace (string)`: Namespace for the `Code`.
 - **Usage**:
   - Query for data from the application at current or past height.
   - Optionally return Merkle proof.
+  - Merkle proof includes self-describing `type` field to support many types
+  of Merkle trees and encoding formats.
 
 ### BeginBlock
 
@@ -217,10 +259,11 @@ Commit are included in the header of the next block.
     be non-deterministic.
   - `Info (string)`: Additional information. May
     be non-deterministic.
-  - `GasWanted (int64)`: Amount of gas request for transaction.
+  - `GasWanted (int64)`: Amount of gas requested for transaction.
   - `GasUsed (int64)`: Amount of gas consumed by transaction.
   - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing
     transactions (eg. by account).
+  - `Codespace (string)`: Namespace for the `Code`.
 - **Usage**:
   - Technically optional - not involved in processing blocks.
   - Guardian of the mempool: every node runs CheckTx before letting a
@@ -248,6 +291,7 @@ Commit are included in the header of the next block.
   - `GasUsed (int64)`: Amount of gas consumed by transaction.
   - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing
     transactions (eg. by account).
+  - `Codespace (string)`: Namespace for the `Code`.
 - **Usage**:
   - The workhorse of the application - non-optional.
   - Execute the transaction in full.
@@ -379,3 +423,44 @@ Commit are included in the header of the next block.
   - `Round (int32)`: Commit round.
   - `Votes ([]VoteInfo)`: List of validators addresses in the last validator set
     with their voting power and whether or not they signed a vote.
+
+###  ConsensusParams
+
+- **Fields**:
+  - `BlockSize (BlockSize)`: Parameters limiting the size of a block.
+  - `EvidenceParams (EvidenceParams)`: Parameters limiting the validity of
+    evidence of byzantine behaviour.
+
+### BlockSize
+
+- **Fields**:
+  - `MaxBytes (int64)`: Max size of a block, in bytes.
+  - `MaxGas (int64)`: Max sum of `GasWanted` in a proposed block.
+    - NOTE: blocks that violate this may be committed if there are Byzantine proposers.
+        It's the application's responsibility to handle this when processing a
+        block!
+
+### EvidenceParams
+
+- **Fields**:
+  - `MaxAge (int64)`: Max age of evidence, in blocks. Evidence older than this
+    is considered stale and ignored.
+        - This should correspond with an app's "unbonding period" or other
+          similar mechanism for handling Nothing-At-Stake attacks.
+        - NOTE: this should change to time (instead of blocks)!
+
+### Proof
+
+- **Fields**:
+  - `Ops ([]ProofOp)`: List of chained Merkle proofs, of possibly different types
+    - The Merkle root of one op is the value being proven in the next op.
+    - The Merkle root of the final op should equal the ultimate root hash being
+      verified against.
+
+### ProofOp
+
+- **Fields**:
+  - `Type (string)`: Type of Merkle proof and how it's encoded.
+  - `Key ([]byte)`: Key in the Merkle tree that this proof is for.
+  - `Data ([]byte)`: Encoded Merkle proof for the key.
+

docs/spec/abci/apps.md

@@ -86,18 +86,51 @@ Otherwise it should never be modified.
 
 ## Transaction Results
 
-`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields, though they
-have slightly different effects.
+`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields.
 
-In both cases, `Info` and `Log` are non-deterministic values for debugging/convenience purposes
+The `Info` and `Log` fields are non-deterministic values for debugging/convenience purposes
 that are otherwise ignored.
 
-In both cases, `GasWanted` and `GasUsed` parameters are currently ignored,
-though see issues
-[#1861](https://github.com/tendermint/tendermint/issues/1861),
-[#2299](https://github.com/tendermint/tendermint/issues/2299) and
-[#2310](https://github.com/tendermint/tendermint/issues/2310) for how this may
-soon change.
+The `Data` field must be strictly deterministic, but can be arbitrary data.
+
+### Gas
+
+Ethereum introduced the notion of `gas` as an abstract representation of the
+cost of resources used by nodes when processing transactions. Every operation in the
+Ethereum Virtual Machine uses some amount of gas, and gas can be accepted at a market-variable price.
+Users propose a maximum amount of gas for their transaction; if the tx uses less, they get
+the difference credited back. Tendermint adopts a similar abstraction,
+though uses it only optionally and weakly, allowing applications to define
+their own sense of the cost of execution.
+
+In Tendermint, the `ConsensusParams.BlockSize.MaxGas` limits the amount of `gas` that can be used in a block.
+The default value is `-1`, meaning no limit, or that the concept of gas is
+meaningless.
+
+Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum
+amount of gas the sender of a tx is willing to use, and the later is how much it actually
+used. Applications should enforce that `GasUsed <= GasWanted` - ie. tx execution
+should halt before it can use more resources than it requested.
+
+When `MaxGas > -1`, Tendermint enforces the following rules:
+
+- `GasWanted <= MaxGas` for all txs in the mempool
+- `(sum of GasWanted in a block) <= MaxGas` when proposing a block
+
+If `MaxGas == -1`, no rules about gas are enforced.
+
+Note that Tendermint does not currently enforce anything about Gas in the consensus, only the mempool.
+This means it does not guarantee that committed blocks satisfy these rules!
+It is the application's responsibility to return non-zero response codes when gas limits are exceeded.
+
+The `GasUsed` field is ignored completely by Tendermint. That said, applications should enforce:
+
+- `GasUsed <= GasWanted` for any given transaction
+- `(sum of GasUsed in a block) <= MaxGas` for every block
+
+In the future, we intend to add a `Priority` field to the responses that can be
+used to explicitly prioritize txs in the mempool for inclusion in a block
+proposal. See [#1861](https://github.com/tendermint/tendermint/issues/1861).
 
 ### CheckTx
 
@@ -142,9 +175,6 @@ If the list is not empty, Tendermint will use it for the validator set.
 This way the application can determine the initial validator set for the
 blockchain.
 
-ResponseInitChain also includes ConsensusParams, but these are presently
-ignored.
-
 ### EndBlock
 
 Updates to the Tendermint validator set can be made by returning
@@ -179,14 +209,78 @@ following rules:
 
 Note the updates returned in block `H` will only take effect at block `H+2`.
 
+## Consensus Parameters
+
+ConsensusParams enforce certain limits in the blockchain, like the maximum size
+of blocks, amount of gas used in a block, and the maximum acceptable age of
+evidence. They can be set in InitChain and updated in EndBlock.
+
+### BlockSize.MaxBytes
+
+The maximum size of a complete Amino encoded block.
+This is enforced by Tendermint consensus.
+
+This implies a maximum tx size that is this MaxBytes, less the expected size of
+the header, the validator set, and any included evidence in the block.
+
+Must have `0 < MaxBytes < 100 MB`.
+
+### BlockSize.MaxGas
+
+The maximum of the sum of `GasWanted` in a proposed block.
+This is *not* enforced by Tendermint consensus.
+It is left to the app to enforce (ie. if txs are included past the
+limit, they should return non-zero codes). It is used by Tendermint to limit the
+txs included in a proposed block.
+
+Must have `MaxGas >= -1`.
+If `MaxGas == -1`, no limit is enforced.
+
+### EvidenceParams.MaxAge
+
+This is the maximum age of evidence.
+This is enforced by Tendermint consensus.
+If a block includes evidence older than this, the block will be rejected
+(validators won't vote for it).
+
+Must have `0 < MaxAge`.
+
+### Updates
+
+The application may set the ConsensusParams during InitChain, and update them during
+EndBlock. If the ConsensusParams is empty, it will be ignored. Each field
+that is not empty will be applied in full. For instance, if updating the
+BlockSize.MaxBytes, applications must also set the other BlockSize fields (like
+BlockSize.MaxGas), even if they are unchanged, as they will otherwise cause the
+value to be updated to 0.
+
+#### InitChain
+
+ResponseInitChain includes a ConsensusParams.
+If its nil, Tendermint will use the params loaded in the genesis
+file. If it's not nil, Tendermint will use it.
+This way the application can determine the initial consensus params for the
+blockchain.
+
+#### EndBlock
+
+ResponseEndBlock includes a ConsensusParams.
+If its nil, Tendermint will do nothing.
+If it's not nil, Tendermint will use it.
+This way the application can update the consensus params over time.
+
+Note the updates returned in block `H` will take effect right away for block
+`H+1`.
+
 ## Query
 
 Query is a generic method with lots of flexibility to enable diverse sets
 of queries on application state. Tendermint makes use of Query to filter new peers
 based on ID and IP, and exposes Query to the user over RPC.
+
 Note that calls to Query are not replicated across nodes, but rather query the
-local node's state - hence they may provide stale reads. For reads that require
-consensus, a transaction is required.
+local node's state - hence they may return stale reads. For reads that require
+consensus, use a transaction.
 
 The most important use of Query is to return Merkle proofs of the application state at some height
 that can be used for efficient application-specific lite-clients.
@@ -222,6 +316,30 @@ their state as follows:
 For instance, this allows an application's lite-client to verify proofs of
 absence in the application state, something which is much less efficient to do using the block hash.
 
+Some applications (eg. Ethereum, Cosmos-SDK) have multiple "levels" of Merkle trees,
+where the leaves of one tree are the root hashes of others. To support this, and
+the general variability in Merkle proofs, the `ResponseQuery.Proof` has some minimal structure:
+
+```
+message Proof {
+  repeated ProofOp ops
+}
+
+message ProofOp {
+  string type = 1;
+  bytes key = 2;
+  bytes data = 3;
+}
+```
+
+Each `ProofOp` contains a proof for a single key in a single Merkle tree, of the specified `type`.
+This allows ABCI to support many different kinds of Merkle trees, encoding
+formats, and proofs (eg. of presence and absence) just by varying the `type`.
+The `data` contains the actual encoded proof, encoded according to the `type`.
+When verifying the full proof, the root hash for one ProofOp is the value being
+verified for the next ProofOp in the list. The root hash of the final ProofOp in
+the list should match the `AppHash` being verified against.
+
 ### Peer Filtering
 
 When Tendermint connects to a peer, it sends two queries to the ABCI application
@@ -235,6 +353,15 @@ using the following paths, with no additional data:
 If either of these queries return a non-zero ABCI code, Tendermint will refuse
 to connect to the peer.
 
+### Paths
+
+Queries are directed at paths, and may optionally include additional data.
+
+The expectation is for there to be some number of high level paths
+differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently,
+Tendermint only uses `/p2p`, for filtering peers. For more advanced use, see the
+implementation of
+[Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/v0.23.1/baseapp/baseapp.go#L333).
 
 ## Crash Recovery
 

docs/spec/blockchain/blockchain.md

@@ -230,6 +230,15 @@ It must equal the weighted median of the timestamps of the valid votes in the bl
 Note: the timestamp of a vote must be greater by at least one millisecond than that of the
 block being voted on.
 
+The timestamp of the first block must be equal to the genesis time (since
+there's no votes to compute the median).
+
+```
+if block.Header.Height == 1 {
+    block.Header.Timestamp == genesisTime
+}
+```
+
 See the section on [BFT time](../consensus/bft-time.md) for more details.
 
 ### NumTxs
@@ -401,14 +410,23 @@ must be greater than 2/3 of the total voting power of the complete validator set
 
 A vote is a signed message broadcast in the consensus for a particular block at a particular height and round.
 When stored in the blockchain or propagated over the network, votes are encoded in Amino.
-For signing, votes are encoded in JSON, and the ChainID is included, in the form of the `CanonicalSignBytes`.
+For signing, votes are represented via `CanonicalVote` and also encoded using amino (protobuf compatible) via
+`Vote.SignBytes` which includes the `ChainID`, and uses a different ordering of
+the fields.
 
-We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the CanonicalSignBytes
+We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the `SignBytes`
 using the given ChainID:
 
 ```go
-func (v Vote) Verify(chainID string, pubKey PubKey) bool {
-    return pubKey.Verify(v.Signature, CanonicalSignBytes(chainID, v))
+func (vote *Vote) Verify(chainID string, pubKey crypto.PubKey) error {
+	if !bytes.Equal(pubKey.Address(), vote.ValidatorAddress) {
+		return ErrVoteInvalidValidatorAddress
+	}
+
+	if !pubKey.VerifyBytes(vote.SignBytes(chainID), vote.Signature) {
+		return ErrVoteInvalidSignature
+	}
+	return nil
 }
 ```
 

docs/spec/blockchain/encoding.md

@@ -298,14 +298,25 @@ Where the `"value"` is the base64 encoding of the raw pubkey bytes, and the
 
 ### Signed Messages
 
-Signed messages (eg. votes, proposals) in the consensus are encoded using Amino-JSON, rather than in the standard binary format
-(NOTE: this is subject to change: https://github.com/tendermint/tendermint/issues/1622)
+Signed messages (eg. votes, proposals) in the consensus are encoded using Amino.
 
-When signing, the elements of a message are sorted by key and prepended with
-a `@chain_id` and `@type` field.
-We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look
-like:
+When signing, the elements of a message are re-ordered so the fixed-length fields
+are first, making it easy to quickly check the version, height, round, and type.
+The `ChainID` is also appended to the end.
+We call this encoding the SignBytes. For instance, SignBytes for a vote is the Amino encoding of the following struct:
 
-```json
-{"@chain_id":"test_chain_id","@type":"vote","block_id":{"hash":"8B01023386C371778ECB6368573E539AFC3CC860","parts":{"hash":"72DB3D959635DFF1BB567BEDAA70573392C51596","total":"1000000"}},"height":"12345","round":"2","timestamp":"2017-12-25T03:00:01.234Z","type":2}
+```go
+type CanonicalVote struct {
+	Version   uint64           `binary:"fixed64"`
+	Height    int64            `binary:"fixed64"`
+	Round     int64            `binary:"fixed64"`
+	VoteType  byte
+	Timestamp time.Time
+	BlockID   CanonicalBlockID
+	ChainID   string
+}
 ```
+
+The field ordering and the fixed sized encoding for the first three fields is optimized to ease parsing of SignBytes
+in HSMs. It creates fixed offsets for relevant fields that need to be read in this context.
+See [#1622](https://github.com/tendermint/tendermint/issues/1622) for more details.

docs/spec/blockchain/state.md

@@ -45,7 +45,7 @@ processing transactions, like gas variables and tags - see
 ### Validator
 
 A validator is an active participant in the consensus with a public key and a voting power.
-Validator's also contain an address which is derived from the PubKey:
+Validator's also contain an address field, which is a hash digest of the PubKey.
 
 ```go
 type Validator struct {
@@ -55,6 +55,9 @@ type Validator struct {
 }
 ```
 
+When hashing the Validator struct, the pubkey is not hashed,
+because the address is already the hash of the pubkey.
+
 The `state.Validators`, `state.LastValidators`, and `state.NextValidators`, must always by sorted by validator address,
 so that there is a canonical order for computing the SimpleMerkleRoot.
 

docs/spec/consensus/consensus.md

@@ -17,7 +17,7 @@
   vote](https://godoc.org/github.com/tendermint/tendermint/types#FirstPrecommit)
   for something.
 - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R`
-  included in its [sign-bytes](../blockchain/blockchain.md).
+  included in its [sign-bytes](../blockchain/blockchain.md#vote).
 - _+2/3_ is short for "more than 2/3"
 - _1/3+_ is short for "1/3 or more"
 - A set of +2/3 of prevotes for a particular block or `<nil>` at
@@ -241,7 +241,7 @@ commit-set) are each justified in the JSet with no duplicitous vote
 signatures (by the committers).
 
 - **Lemma**: When a fork is detected by the existence of two
-  conflicting [commits](./validators.html#commiting-a-block), the
+  conflicting [commits](../blockchain/blockchain.md#commit), the
   union of the JSets for both commits (if they can be compiled) must
   include double-signing by at least 1/3+ of the validator set.
   **Proof**: The commit cannot be at the same round, because that

docs/spec/reactors/mempool/config.md

@@ -6,23 +6,21 @@ as command-line flags, but they can also be passed in as
 environmental variables or in the config.toml file. The
 following are all equivalent:
 
-Flag: `--mempool.recheck_empty=false`
+Flag: `--mempool.recheck=false`
 
-Environment: `TM_MEMPOOL_RECHECK_EMPTY=false`
+Environment: `TM_MEMPOOL_RECHECK=false`
 
 Config:
 
 ```
 [mempool]
-recheck_empty = false
+recheck = false
 ```
 
 ## Recheck
 
 `--mempool.recheck=false` (default: true)
 
-`--mempool.recheck_empty=false` (default: true)
-
 Recheck determines if the mempool rechecks all pending
 transactions after a block was committed. Once a block
 is committed, the mempool removes all valid transactions
@@ -31,9 +29,6 @@ that were successfully included in the block.
 If `recheck` is true, then it will rerun CheckTx on
 all remaining transactions with the new block state.
 
-If the block contained no transactions, it will skip the
-recheck unless `recheck_empty` is true.
-
 ## Broadcast
 
 `--mempool.broadcast=false` (default: true)

docs/tendermint-core/README.md

@@ -0,0 +1,4 @@
+# Overview
+
+See the side-bar for details on the various features of Tendermint Core.
+

docs/tendermint-core/configuration.md

@@ -115,15 +115,15 @@ addr_book_file = "addrbook.json"
 # Set false for private or local networks
 addr_book_strict = true
 
-# Time to wait before flushing messages out on the connection, in ms
-flush_throttle_timeout = 100
-
 # Maximum number of inbound peers
 max_num_inbound_peers = 40
 
 # Maximum number of outbound peers to connect to, excluding persistent peers
 max_num_outbound_peers = 10
 
+# Time to wait before flushing messages out on the connection
+flush_throttle_timeout = "100ms"
+
 # Maximum size of a message packet payload, in bytes
 max_packet_msg_payload_size = 1024
 
@@ -145,11 +145,17 @@ seed_mode = false
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = ""
 
+# Toggle to disable guard against peers connecting from the same ip.
+allow_duplicate_ip = true
+
+# Peer connection configuration.
+handshake_timeout = "20s"
+dial_timeout = "3s"
+
 ##### mempool configuration options #####
 [mempool]
 
 recheck = true
-recheck_empty = true
 broadcast = true
 wal_dir = "data/mempool.wal"
 
@@ -164,25 +170,24 @@ cache_size = 100000
 
 wal_file = "data/cs.wal/wal"
 
-# All timeouts are in milliseconds
-timeout_propose = 3000
-timeout_propose_delta = 500
-timeout_prevote = 1000
-timeout_prevote_delta = 500
-timeout_precommit = 1000
-timeout_precommit_delta = 500
-timeout_commit = 1000
+timeout_propose = "3000ms"
+timeout_propose_delta = "500ms"
+timeout_prevote = "1000ms"
+timeout_prevote_delta = "500ms"
+timeout_precommit = "1000ms"
+timeout_precommit_delta = "500ms"
+timeout_commit = "1000ms"
 
 # Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
 skip_timeout_commit = false
 
-# EmptyBlocks mode and possible interval between empty blocks in seconds
+# EmptyBlocks mode and possible interval between empty blocks
 create_empty_blocks = true
-create_empty_blocks_interval = 0
+create_empty_blocks_interval = "0s"
 
-# Reactor sleep duration parameters are in milliseconds
-peer_gossip_sleep_duration = 100
-peer_query_maj23_sleep_duration = 2000
+# Reactor sleep duration parameters
+peer_gossip_sleep_duration = "100ms"
+peer_query_maj23_sleep_duration = "2000ms"
 
 ##### transactions indexer configuration options #####
 [tx_index]
@@ -197,15 +202,15 @@ indexer = "kv"
 # Comma-separated list of tags to index (by default the only tag is "tx.hash")
 #
 # You can also index transactions by height by adding "tx.height" tag here.
-# 
+#
 # It's recommended to index only a subset of tags due to possible memory
 # bloat. This is, of course, depends on the indexer's DB and the volume of
 # transactions.
 index_tags = ""
 
 # When set to true, tells indexer to index all tags (predefined tags:
-# "tx.hash", "tx.height" and all tags from DeliverTx responses). 
-#	
+# "tx.hash", "tx.height" and all tags from DeliverTx responses).
+#
 # Note this may be not desirable (see the comment above). IndexTags has a
 # precedence over IndexAllTags (i.e. when given both, IndexTags will be
 # indexed).
@@ -227,4 +232,7 @@ prometheus_listen_addr = ":26660"
 # you increase your OS limits.
 # 0 - unlimited.
 max_open_connections = 3
+
+# Instrumentation namespace
+namespace = "tendermint"
 ```

docs/tendermint-core/metrics.md

@@ -8,35 +8,45 @@ This functionality is disabled by default.
 To enable the Prometheus metrics, set `instrumentation.prometheus=true` if your
 config file. Metrics will be served under `/metrics` on 26660 port by default.
 Listen address can be changed in the config file (see
-`instrumentation.prometheus_listen_addr`).
+`instrumentation.prometheus\_listen\_addr`).
 
 ## List of available metrics
 
 The following metrics are available:
 
-```
-| Name                                    | Type      | Since     | Description                                                                   |
-| --------------------------------------- | -------   | --------- | ----------------------------------------------------------------------------- |
-| consensus_height                        | Gauge     | 0.21.0    | Height of the chain                                                           |
-| consensus_validators                    | Gauge     | 0.21.0    | Number of validators                                                          |
-| consensus_validators_power              | Gauge     | 0.21.0    | Total voting power of all validators                                          |
-| consensus_missing_validators            | Gauge     | 0.21.0    | Number of validators who did not sign                                         |
-| consensus_missing_validators_power      | Gauge     | 0.21.0    | Total voting power of the missing validators                                  |
-| consensus_byzantine_validators          | Gauge     | 0.21.0    | Number of validators who tried to double sign                                 |
-| consensus_byzantine_validators_power    | Gauge     | 0.21.0    | Total voting power of the byzantine validators                                |
-| consensus_block_interval_seconds        | Histogram | 0.21.0    | Time between this and last block (Block.Header.Time) in seconds               |
-| consensus_rounds                        | Gauge     | 0.21.0    | Number of rounds                                                              |
-| consensus_num_txs                       | Gauge     | 0.21.0    | Number of transactions                                                        |
-| mempool_size                            | Gauge     | 0.21.0    | Number of uncommitted transactions                                            |
-| consensus_total_txs                     | Gauge     | 0.21.0    | Total number of transactions committed                                        |
-| consensus_block_size_bytes              | Gauge     | 0.21.0    | Block size in bytes                                                           |
-| p2p_peers                               | Gauge     | 0.21.0    | Number of peers node's connected to                                           |
-```
+| **Name**                                | **Type**  | **Since** | **Tags** | **Description**                                                 |
+|-----------------------------------------|-----------|-----------|----------|-----------------------------------------------------------------|
+| consensus\_height                       | Gauge     | 0.21.0    |          | Height of the chain                                             |
+| consensus\_validators                   | Gauge     | 0.21.0    |          | Number of validators                                            |
+| consensus\_validators\_power            | Gauge     | 0.21.0    |          | Total voting power of all validators                            |
+| consensus\_missing\_validators          | Gauge     | 0.21.0    |          | Number of validators who did not sign                           |
+| consensus\_missing\_validators\_power   | Gauge     | 0.21.0    |          | Total voting power of the missing validators                    |
+| consensus\_byzantine\_validators        | Gauge     | 0.21.0    |          | Number of validators who tried to double sign                   |
+| consensus\_byzantine\_validators\_power | Gauge     | 0.21.0    |          | Total voting power of the byzantine validators                  |
+| consensus\_block\_interval\_seconds     | Histogram | 0.21.0    |          | Time between this and last block (Block.Header.Time) in seconds |
+| consensus\_rounds                       | Gauge     | 0.21.0    |          | Number of rounds                                                |
+| consensus\_num\_txs                     | Gauge     | 0.21.0    |          | Number of transactions                                          |
+| consensus\_block\_parts                 | counter   | on dev    | peer\_id | number of blockparts transmitted by peer                        |
+| consensus\_latest\_block\_height        | gauge     | on dev    |          | /status sync\_info number                                       |
+| consensus\_fast\_syncing                | gauge     | on dev    |          | either 0 (not fast syncing) or 1 (syncing)                      |
+| consensus\_total\_txs                   | Gauge     | 0.21.0    |          | Total number of transactions committed                          |
+| consensus\_block\_size\_bytes           | Gauge     | 0.21.0    |          | Block size in bytes                                             |
+| p2p\_peers                              | Gauge     | 0.21.0    |          | Number of peers node's connected to                             |
+| p2p\_peer\_receive\_bytes\_total        | counter   | on dev    | peer\_id | number of bytes received from a given peer                      |
+| p2p\_peer\_send\_bytes\_total           | counter   | on dev    | peer\_id | number of bytes sent to a given peer                            |
+| p2p\_peer\_pending\_send\_bytes         | gauge     | on dev    | peer\_id | number of pending bytes to be sent to a given peer              |
+| p2p\_num\_txs                           | gauge     | on dev    | peer\_id | number of transactions submitted by each peer\_id               |
+| p2p\_pending\_send\_bytes               | gauge     | on dev    | peer\_id | amount of data pending to be sent to peer                       |
+| mempool\_size                           | Gauge     | 0.21.0    |          | Number of uncommitted transactions                              |
+| mempool\_tx\_size\_bytes                | histogram | on dev    |          | transaction sizes in bytes                                      |
+| mempool\_failed\_txs                    | counter   | on dev    |          | number of failed transactions                                   |
+| mempool\_recheck\_times                 | counter   | on dev    |          | number of transactions rechecked in the mempool                 |
+| state\_block\_processing\_time          | histogram | on dev    |          | time between BeginBlock and EndBlock in ms                      |
 
 ## Useful queries
 
 Percentage of missing + byzantine validators:
 
 ```
-((consensus_byzantine_validators_power + consensus_missing_validators_power) / consensus_validators_power) * 100
+((consensus\_byzantine\_validators\_power + consensus\_missing\_validators\_power) / consensus\_validators\_power) * 100
 ```