Merge pull request #2473 from tendermint/release/v0.25.0

Release/v0.25.0

CHANGELOG.md

@@ -1,5 +1,60 @@
 # Changelog
 
+## v0.25.0
+
+*September 22, 2018*
+
+Special thanks to external contributors on this release:
+@scriptionist, @bradyjoestar, @WALL-E
+
+This release is mostly about the ConsensusParams - removing fields and enforcing MaxGas.
+It also addresses some issues found via security audit, removes various unused
+functions from `libs/common`, and implements
+[ADR-012](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-012-peer-transport.md).
+
+Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint).
+
+BREAKING CHANGES:
+
+* CLI/RPC/Config
+  * [rpc] [\#2391](https://github.com/tendermint/tendermint/issues/2391) /status `result.node_info.other` became a map
+  * [types] [\#2364](https://github.com/tendermint/tendermint/issues/2364) Remove `TxSize` and `BlockGossip` from `ConsensusParams`
+    * Maximum tx size is now set implicitly via the `BlockSize.MaxBytes`
+    * The size of block parts in the consensus is now fixed to 64kB
+
+* Apps
+  * [mempool] [\#2360](https://github.com/tendermint/tendermint/issues/2360) Mempool tracks the `ResponseCheckTx.GasWanted` and
+    `ConsensusParams.BlockSize.MaxGas` and enforces:
+    - `GasWanted <= MaxGas` for every tx
+    - `(sum of GasWanted in block) <= MaxGas` for block proposal
+
+* Go API
+  * [libs/common] [\#2431](https://github.com/tendermint/tendermint/issues/2431) Remove Word256 due to lack of use
+  * [libs/common] [\#2452](https://github.com/tendermint/tendermint/issues/2452) Remove the following functions due to lack of use:
+    * byteslice.go: cmn.IsZeros, cmn.RightPadBytes, cmn.LeftPadBytes, cmn.PrefixEndBytes
+    * strings.go: cmn.IsHex, cmn.StripHex
+    * int.go: Uint64Slice, all put/get int64 methods
+
+FEATURES:
+- [rpc] [\#2415](https://github.com/tendermint/tendermint/issues/2415) New `/consensus_params?height=X` endpoint to query the consensus
+  params at any height (@scriptonist)
+- [types] [\#1714](https://github.com/tendermint/tendermint/issues/1714) Add Address to GenesisValidator
+- [metrics] [\#2337](https://github.com/tendermint/tendermint/issues/2337) `consensus.block_interval_metrics` is now gauge, not histogram (you will be able to see spikes, if any)
+- [libs] [\#2286](https://github.com/tendermint/tendermint/issues/2286) Panic if `autofile` or `db/fsdb` permissions change from 0600.
+
+IMPROVEMENTS:
+- [libs/db] [\#2371](https://github.com/tendermint/tendermint/issues/2371) Output error instead of panic when the given `db_backend` is not initialised (@bradyjoestar)
+- [mempool] [\#2399](https://github.com/tendermint/tendermint/issues/2399) Make mempool cache a proper LRU (@bradyjoestar)
+- [p2p] [\#2126](https://github.com/tendermint/tendermint/issues/2126) Introduce PeerTransport interface to improve isolation of concerns
+- [libs/common] [\#2326](https://github.com/tendermint/tendermint/issues/2326) Service returns ErrNotStarted
+
+BUG FIXES:
+- [node] [\#2294](https://github.com/tendermint/tendermint/issues/2294) Delay starting node until Genesis time
+- [consensus] [\#2048](https://github.com/tendermint/tendermint/issues/2048) Correct peer statistics for marking peer as good
+- [rpc] [\#2460](https://github.com/tendermint/tendermint/issues/2460) StartHTTPAndTLSServer() now passes StartTLS() errors back to the caller rather than hanging forever.
+- [p2p] [\#2047](https://github.com/tendermint/tendermint/issues/2047) Accept new connections asynchronously
+- [tm-bench] [\#2410](https://github.com/tendermint/tendermint/issues/2410) Enforce minimum transaction size (@WALL-E)
+
 ## 0.24.0
 
 *September 6th, 2018*

CHANGELOG_PENDING.md

@@ -1,35 +1,17 @@
 # Pending
 
-Special thanks to external contributors with PRs included in this release:
+Special thanks to external contributors on this release:
 
 BREAKING CHANGES:
 
 * CLI/RPC/Config
-  * [rpc] [\#2391](https://github.com/tendermint/tendermint/issues/2391) /status `result.node_info.other` became a map
 
 * Apps
-  * [mempool] \#2310 Mempool tracks the `ResponseCheckTx.GasWanted` and enforces `ConsensusParams.BlockSize.MaxGas` on proposals.
 
 * Go API
-  * [libs/common] \#2431 Remove Word256 due to lack of use
-  * [libs/common] \#2452 Remove the following functions due to lack of use:
-    * byteslice.go: cmn.IsZeros, cmn.RightPadBytes, cmn.LeftPadBytes, cmn.PrefixEndBytes
-    * strings.go: cmn.IsHex, cmn.StripHex
-    * int.go: Uint64Slice, all put/get int64 methods
-
-* Blockchain Protocol
-
-* P2P Protocol
-
 
 FEATURES:
 
 IMPROVEMENTS:
-- [libs/db] \#2371 Output error instead of panic when the given db_backend is not initialised (@bradyjoestar)
-- [mempool] [\#2399](https://github.com/tendermint/tendermint/issues/2399) Make mempool cache a proper LRU (@bradyjoestar)
-- [types] [\#1714](https://github.com/tendermint/tendermint/issues/1714) Add Address to GenesisValidator
-- [metrics] `consensus.block_interval_metrics` is now gauge, not histogram (you will be able to see spikes, if any)
-- [p2p] \#2126 Introduce PeerTransport interface to improve isolation of concerns
 
 BUG FIXES:
-- [node] \#2294 Delay starting node until Genesis time

Makefile

@@ -23,11 +23,14 @@ check: check_tools get_vendor_deps
 build:
 	CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint/
 
+build_c:
+	CGO_ENABLED=1 go build $(BUILD_FLAGS) -tags "$(BUILD_TAGS) gcc" -o build/tendermint ./cmd/tendermint/
+
 build_race:
 	CGO_ENABLED=0 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint
 
 install:
-	CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
+	CGO_ENABLED=0 go install  $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
 
 ########################################
 ### Protobuf

UPGRADING.md

@@ -3,6 +3,12 @@
 This guide provides steps to be followed when you upgrade your applications to
 a newer version of Tendermint Core.
 
+## v0.25.0
+
+This release has minimal impact.
+
+If you use GasWanted in ABCI and want to enforce it, set the MaxGas in the genesis file (default is no max).
+
 ## v0.24.0
 
 New 0.24.0 release contains a lot of changes to the state and types. It's not

cmd/tendermint/commands/init.go

@@ -59,8 +59,8 @@ func initFilesWithConfig(config *cfg.Config) error {
 		}
 		genDoc.Validators = []types.GenesisValidator{{
 			Address: pv.GetPubKey().Address(),
-			PubKey: pv.GetPubKey(),
-			Power:  10,
+			PubKey:  pv.GetPubKey(),
+			Power:   10,
 		}}
 
 		if err := genDoc.SaveAs(genFile); err != nil {

cmd/tendermint/commands/testnet.go

@@ -92,9 +92,9 @@ func testnetFiles(cmd *cobra.Command, args []string) error {
 		pv := privval.LoadFilePV(pvFile)
 		genVals[i] = types.GenesisValidator{
 			Address: pv.GetPubKey().Address(),
-			PubKey: pv.GetPubKey(),
-			Power:  1,
-			Name:   nodeDirName,
+			PubKey:  pv.GetPubKey(),
+			Power:   1,
+			Name:    nodeDirName,
 		}
 	}
 

consensus/metrics.go

@@ -85,7 +85,6 @@ func PrometheusMetrics() *Metrics {
 			Help:      "Total power of the byzantine validators.",
 		}, []string{}),
 
-
 		BlockIntervalSeconds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
 			Subsystem: "consensus",
 			Name:      "block_interval_seconds",

consensus/reactor.go

@@ -29,6 +29,7 @@ const (
 	maxMsgSize = 1048576 // 1MB; NOTE/TODO: keep in sync with types.PartSet sizes.
 
 	blocksToContributeToBecomeGoodPeer = 10000
+	votesToContributeToBecomeGoodPeer  = 10000
 )
 
 //-----------------------------------------------------------------------------
@@ -60,6 +61,9 @@ func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *Consens
 func (conR *ConsensusReactor) OnStart() error {
 	conR.Logger.Info("ConsensusReactor ", "fastSync", conR.FastSync())
 
+	// start routine that computes peer statistics for evaluating peer quality
+	go conR.peerStatsRoutine()
+
 	conR.subscribeToBroadcastEvents()
 
 	if !conR.FastSync() {
@@ -258,9 +262,7 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
 			ps.ApplyProposalPOLMessage(msg)
 		case *BlockPartMessage:
 			ps.SetHasProposalBlockPart(msg.Height, msg.Round, msg.Part.Index)
-			if numBlocks := ps.RecordBlockPart(msg); numBlocks%blocksToContributeToBecomeGoodPeer == 0 {
-				conR.Switch.MarkPeerAsGood(src)
-			}
+
 			conR.conS.peerMsgQueue <- msgInfo{msg, src.ID()}
 		default:
 			conR.Logger.Error(fmt.Sprintf("Unknown message type %v", reflect.TypeOf(msg)))
@@ -280,9 +282,6 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte)
 			ps.EnsureVoteBitArrays(height, valSize)
 			ps.EnsureVoteBitArrays(height-1, lastCommitSize)
 			ps.SetHasVote(msg.Vote)
-			if blocks := ps.RecordVote(msg.Vote); blocks%blocksToContributeToBecomeGoodPeer == 0 {
-				conR.Switch.MarkPeerAsGood(src)
-			}
 
 			cs.peerMsgQueue <- msgInfo{msg, src.ID()}
 
@@ -794,6 +793,43 @@ OUTER_LOOP:
 	}
 }
 
+func (conR *ConsensusReactor) peerStatsRoutine() {
+	for {
+		if !conR.IsRunning() {
+			conR.Logger.Info("Stopping peerStatsRoutine")
+			return
+		}
+
+		select {
+		case msg := <-conR.conS.statsMsgQueue:
+			// Get peer
+			peer := conR.Switch.Peers().Get(msg.PeerID)
+			if peer == nil {
+				conR.Logger.Debug("Attempt to update stats for non-existent peer",
+					"peer", msg.PeerID)
+				continue
+			}
+			// Get peer state
+			ps := peer.Get(types.PeerStateKey).(*PeerState)
+			switch msg.Msg.(type) {
+			case *VoteMessage:
+				if numVotes := ps.RecordVote(); numVotes%votesToContributeToBecomeGoodPeer == 0 {
+					conR.Switch.MarkPeerAsGood(peer)
+				}
+			case *BlockPartMessage:
+				if numParts := ps.RecordBlockPart(); numParts%blocksToContributeToBecomeGoodPeer == 0 {
+					conR.Switch.MarkPeerAsGood(peer)
+				}
+			}
+		case <-conR.conS.Quit():
+			return
+
+		case <-conR.Quit():
+			return
+		}
+	}
+}
+
 // String returns a string representation of the ConsensusReactor.
 // NOTE: For now, it is just a hard-coded string to avoid accessing unprotected shared variables.
 // TODO: improve!
@@ -836,15 +872,13 @@ type PeerState struct {
 
 // peerStateStats holds internal statistics for a peer.
 type peerStateStats struct {
-	LastVoteHeight      int64 `json:"last_vote_height"`
-	Votes               int   `json:"votes"`
-	LastBlockPartHeight int64 `json:"last_block_part_height"`
-	BlockParts          int   `json:"block_parts"`
+	Votes      int `json:"votes"`
+	BlockParts int `json:"block_parts"`
 }
 
 func (pss peerStateStats) String() string {
-	return fmt.Sprintf("peerStateStats{lvh: %d, votes: %d, lbph: %d, blockParts: %d}",
-		pss.LastVoteHeight, pss.Votes, pss.LastBlockPartHeight, pss.BlockParts)
+	return fmt.Sprintf("peerStateStats{votes: %d, blockParts: %d}",
+		pss.Votes, pss.BlockParts)
 }
 
 // NewPeerState returns a new PeerState for the given Peer
@@ -1080,18 +1114,14 @@ func (ps *PeerState) ensureVoteBitArrays(height int64, numValidators int) {
 	}
 }
 
-// RecordVote updates internal statistics for this peer by recording the vote.
-// It returns the total number of votes (1 per block). This essentially means
-// the number of blocks for which peer has been sending us votes.
-func (ps *PeerState) RecordVote(vote *types.Vote) int {
+// RecordVote increments internal votes related statistics for this peer.
+// It returns the total number of added votes.
+func (ps *PeerState) RecordVote() int {
 	ps.mtx.Lock()
 	defer ps.mtx.Unlock()
 
-	if ps.Stats.LastVoteHeight >= vote.Height {
-		return ps.Stats.Votes
-	}
-	ps.Stats.LastVoteHeight = vote.Height
 	ps.Stats.Votes++
+
 	return ps.Stats.Votes
 }
 
@@ -1104,25 +1134,17 @@ func (ps *PeerState) VotesSent() int {
 	return ps.Stats.Votes
 }
 
-// RecordBlockPart updates internal statistics for this peer by recording the
-// block part. It returns the total number of block parts (1 per block). This
-// essentially means the number of blocks for which peer has been sending us
-// block parts.
-func (ps *PeerState) RecordBlockPart(bp *BlockPartMessage) int {
+// RecordBlockPart increments internal block part related statistics for this peer.
+// It returns the total number of added block parts.
+func (ps *PeerState) RecordBlockPart() int {
 	ps.mtx.Lock()
 	defer ps.mtx.Unlock()
 
-	if ps.Stats.LastBlockPartHeight >= bp.Height {
-		return ps.Stats.BlockParts
-	}
-
-	ps.Stats.LastBlockPartHeight = bp.Height
 	ps.Stats.BlockParts++
 	return ps.Stats.BlockParts
 }
 
-// BlockPartsSent returns the number of blocks for which peer has been sending
-// us block parts.
+// BlockPartsSent returns the number of useful block parts the peer has sent us.
 func (ps *PeerState) BlockPartsSent() int {
 	ps.mtx.Lock()
 	defer ps.mtx.Unlock()

consensus/reactor_test.go

@@ -11,20 +11,16 @@ import (
 	"testing"
 	"time"
 
-	abcicli "github.com/tendermint/tendermint/abci/client"
+	"github.com/tendermint/tendermint/abci/client"
 	"github.com/tendermint/tendermint/abci/example/kvstore"
 	abci "github.com/tendermint/tendermint/abci/types"
 	bc "github.com/tendermint/tendermint/blockchain"
-	cmn "github.com/tendermint/tendermint/libs/common"
+	cfg "github.com/tendermint/tendermint/config"
 	dbm "github.com/tendermint/tendermint/libs/db"
 	"github.com/tendermint/tendermint/libs/log"
 	mempl "github.com/tendermint/tendermint/mempool"
-	sm "github.com/tendermint/tendermint/state"
-	tmtime "github.com/tendermint/tendermint/types/time"
-
-	cfg "github.com/tendermint/tendermint/config"
 	"github.com/tendermint/tendermint/p2p"
-	p2pdummy "github.com/tendermint/tendermint/p2p/dummy"
+	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
 
 	"github.com/stretchr/testify/assert"
@@ -246,110 +242,25 @@ func TestReactorProposalHeartbeats(t *testing.T) {
 	}, css)
 }
 
-// Test we record block parts from other peers
-func TestReactorRecordsBlockParts(t *testing.T) {
-	// create dummy peer
-	peer := p2pdummy.NewPeer()
-	ps := NewPeerState(peer).SetLogger(log.TestingLogger())
-	peer.Set(types.PeerStateKey, ps)
+// Test we record stats about votes and block parts from other peers.
+func TestReactorRecordsVotesAndBlockParts(t *testing.T) {
+	N := 4
+	css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
+	reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
+	defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
 
-	// create reactor
-	css := randConsensusNet(1, "consensus_reactor_records_block_parts_test", newMockTickerFunc(true), newPersistentKVStore)
-	reactor := NewConsensusReactor(css[0], false) // so we dont start the consensus states
-	reactor.SetEventBus(css[0].eventBus)
-	reactor.SetLogger(log.TestingLogger())
-	sw := p2p.MakeSwitch(cfg.DefaultP2PConfig(), 1, "testing", "123.123.123", func(i int, sw *p2p.Switch) *p2p.Switch { return sw })
-	reactor.SetSwitch(sw)
-	err := reactor.Start()
-	require.NoError(t, err)
-	defer reactor.Stop()
+	// wait till everyone makes the first new block
+	timeoutWaitGroup(t, N, func(j int) {
+		<-eventChans[j]
+	}, css)
 
-	// 1) new block part
-	parts := types.NewPartSetFromData(cmn.RandBytes(100), 10)
-	msg := &BlockPartMessage{
-		Height: 2,
-		Round:  0,
-		Part:   parts.GetPart(0),
-	}
-	bz, err := cdc.MarshalBinaryBare(msg)
-	require.NoError(t, err)
+	// Get peer
+	peer := reactors[1].Switch.Peers().List()[0]
+	// Get peer state
+	ps := peer.Get(types.PeerStateKey).(*PeerState)
 
-	reactor.Receive(DataChannel, peer, bz)
-	require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should have increased by 1")
-
-	// 2) block part with the same height, but different round
-	msg.Round = 1
-
-	bz, err = cdc.MarshalBinaryBare(msg)
-	require.NoError(t, err)
-
-	reactor.Receive(DataChannel, peer, bz)
-	require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should stay the same")
-
-	// 3) block part from earlier height
-	msg.Height = 1
-	msg.Round = 0
-
-	bz, err = cdc.MarshalBinaryBare(msg)
-	require.NoError(t, err)
-
-	reactor.Receive(DataChannel, peer, bz)
-	require.Equal(t, 1, ps.BlockPartsSent(), "number of block parts sent should stay the same")
-}
-
-// Test we record votes from other peers.
-func TestReactorRecordsVotes(t *testing.T) {
-	// Create dummy peer.
-	peer := p2pdummy.NewPeer()
-	ps := NewPeerState(peer).SetLogger(log.TestingLogger())
-	peer.Set(types.PeerStateKey, ps)
-
-	// Create reactor.
-	css := randConsensusNet(1, "consensus_reactor_records_votes_test", newMockTickerFunc(true), newPersistentKVStore)
-	reactor := NewConsensusReactor(css[0], false) // so we dont start the consensus states
-	reactor.SetEventBus(css[0].eventBus)
-	reactor.SetLogger(log.TestingLogger())
-	sw := p2p.MakeSwitch(cfg.DefaultP2PConfig(), 1, "testing", "123.123.123", func(i int, sw *p2p.Switch) *p2p.Switch { return sw })
-	reactor.SetSwitch(sw)
-	err := reactor.Start()
-	require.NoError(t, err)
-	defer reactor.Stop()
-	_, val := css[0].state.Validators.GetByIndex(0)
-
-	// 1) new vote
-	vote := &types.Vote{
-		ValidatorIndex:   0,
-		ValidatorAddress: val.Address,
-		Height:           2,
-		Round:            0,
-		Timestamp:        tmtime.Now(),
-		Type:             types.VoteTypePrevote,
-		BlockID:          types.BlockID{},
-	}
-	bz, err := cdc.MarshalBinaryBare(&VoteMessage{vote})
-	require.NoError(t, err)
-
-	reactor.Receive(VoteChannel, peer, bz)
-	assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should have increased by 1")
-
-	// 2) vote with the same height, but different round
-	vote.Round = 1
-
-	bz, err = cdc.MarshalBinaryBare(&VoteMessage{vote})
-	require.NoError(t, err)
-
-	reactor.Receive(VoteChannel, peer, bz)
-	assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should stay the same")
-
-	// 3) vote from earlier height
-	vote.Height = 1
-	vote.Round = 0
-
-	bz, err = cdc.MarshalBinaryBare(&VoteMessage{vote})
-	require.NoError(t, err)
-
-	reactor.Receive(VoteChannel, peer, bz)
-	assert.Equal(t, 1, ps.VotesSent(), "number of votes sent should stay the same")
+	assert.Equal(t, true, ps.VotesSent() > 0, "number of votes sent should have increased")
+	assert.Equal(t, true, ps.BlockPartsSent() > 0, "number of votes sent should have increased")
 }
 
 //-------------------------------------------------------------

consensus/state.go

@@ -91,6 +91,10 @@ type ConsensusState struct {
 	internalMsgQueue chan msgInfo
 	timeoutTicker    TimeoutTicker
 
+	// information about about added votes and block parts are written on this channel
+	// so statistics can be computed by reactor
+	statsMsgQueue chan msgInfo
+
 	// we use eventBus to trigger msg broadcasts in the reactor,
 	// and to notify external subscribers, eg. through a websocket
 	eventBus *types.EventBus
@@ -141,6 +145,7 @@ func NewConsensusState(
 		peerMsgQueue:     make(chan msgInfo, msgQueueSize),
 		internalMsgQueue: make(chan msgInfo, msgQueueSize),
 		timeoutTicker:    NewTimeoutTicker(),
+		statsMsgQueue:    make(chan msgInfo, msgQueueSize),
 		done:             make(chan struct{}),
 		doWALCatchup:     true,
 		wal:              nilWAL{},
@@ -639,7 +644,11 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
 		err = cs.setProposal(msg.Proposal)
 	case *BlockPartMessage:
 		// if the proposal is complete, we'll enterPrevote or tryFinalizeCommit
-		_, err = cs.addProposalBlockPart(msg, peerID)
+		added, err := cs.addProposalBlockPart(msg, peerID)
+		if added {
+			cs.statsMsgQueue <- mi
+		}
+
 		if err != nil && msg.Round != cs.Round {
 			cs.Logger.Debug("Received block part from wrong round", "height", cs.Height, "csRound", cs.Round, "blockRound", msg.Round)
 			err = nil
@@ -647,7 +656,11 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
 	case *VoteMessage:
 		// attempt to add the vote and dupeout the validator if its a duplicate signature
 		// if the vote gives us a 2/3-any or 2/3-one, we transition
-		err := cs.tryAddVote(msg.Vote, peerID)
+		added, err := cs.tryAddVote(msg.Vote, peerID)
+		if added {
+			cs.statsMsgQueue <- mi
+		}
+
 		if err == ErrAddingVote {
 			// TODO: punish peer
 			// We probably don't want to stop the peer here. The vote does not
@@ -1454,7 +1467,7 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
 			int64(cs.state.ConsensusParams.BlockSize.MaxBytes),
 		)
 		if err != nil {
-			return true, err
+			return added, err
 		}
 		// NOTE: it's possible to receive complete proposal blocks for future rounds without having the proposal
 		cs.Logger.Info("Received complete proposal block", "height", cs.ProposalBlock.Height, "hash", cs.ProposalBlock.Hash())
@@ -1484,35 +1497,35 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
 			// If we're waiting on the proposal block...
 			cs.tryFinalizeCommit(height)
 		}
-		return true, nil
+		return added, nil
 	}
 	return added, nil
 }
 
 // Attempt to add the vote. if its a duplicate signature, dupeout the validator
-func (cs *ConsensusState) tryAddVote(vote *types.Vote, peerID p2p.ID) error {
-	_, err := cs.addVote(vote, peerID)
+func (cs *ConsensusState) tryAddVote(vote *types.Vote, peerID p2p.ID) (bool, error) {
+	added, err := cs.addVote(vote, peerID)
 	if err != nil {
 		// If the vote height is off, we'll just ignore it,
 		// But if it's a conflicting sig, add it to the cs.evpool.
 		// If it's otherwise invalid, punish peer.
 		if err == ErrVoteHeightMismatch {
-			return err
+			return added, err
 		} else if voteErr, ok := err.(*types.ErrVoteConflictingVotes); ok {
 			if bytes.Equal(vote.ValidatorAddress, cs.privValidator.GetAddress()) {
 				cs.Logger.Error("Found conflicting vote from ourselves. Did you unsafe_reset a validator?", "height", vote.Height, "round", vote.Round, "type", vote.Type)
-				return err
+				return added, err
 			}
 			cs.evpool.AddEvidence(voteErr.DuplicateVoteEvidence)
-			return err
+			return added, err
 		} else {
 			// Probably an invalid signature / Bad peer.
 			// Seems this can also err sometimes with "Unexpected step" - perhaps not from a bad peer ?
 			cs.Logger.Error("Error attempting to add vote", "err", err)
-			return ErrAddingVote
+			return added, ErrAddingVote
 		}
 	}
-	return nil
+	return added, nil
 }
 
 //-----------------------------------------------------------------------------

consensus/state_test.go

@@ -7,9 +7,13 @@ import (
 	"testing"
 	"time"
 
+	"github.com/stretchr/testify/require"
+
 	cstypes "github.com/tendermint/tendermint/consensus/types"
+	cmn "github.com/tendermint/tendermint/libs/common"
 	"github.com/tendermint/tendermint/libs/log"
 	tmpubsub "github.com/tendermint/tendermint/libs/pubsub"
+	p2pdummy "github.com/tendermint/tendermint/p2p/dummy"
 	"github.com/tendermint/tendermint/types"
 )
 
@@ -1081,6 +1085,80 @@ func TestStateHalt1(t *testing.T) {
 	}
 }
 
+func TestStateOutputsBlockPartsStats(t *testing.T) {
+	// create dummy peer
+	cs, _ := randConsensusState(1)
+	peer := p2pdummy.NewPeer()
+
+	// 1) new block part
+	parts := types.NewPartSetFromData(cmn.RandBytes(100), 10)
+	msg := &BlockPartMessage{
+		Height: 1,
+		Round:  0,
+		Part:   parts.GetPart(0),
+	}
+
+	cs.ProposalBlockParts = types.NewPartSetFromHeader(parts.Header())
+	cs.handleMsg(msgInfo{msg, peer.ID()})
+
+	statsMessage := <-cs.statsMsgQueue
+	require.Equal(t, msg, statsMessage.Msg, "")
+	require.Equal(t, peer.ID(), statsMessage.PeerID, "")
+
+	// sending the same part from different peer
+	cs.handleMsg(msgInfo{msg, "peer2"})
+
+	// sending the part with the same height, but different round
+	msg.Round = 1
+	cs.handleMsg(msgInfo{msg, peer.ID()})
+
+	// sending the part from the smaller height
+	msg.Height = 0
+	cs.handleMsg(msgInfo{msg, peer.ID()})
+
+	// sending the part from the bigger height
+	msg.Height = 3
+	cs.handleMsg(msgInfo{msg, peer.ID()})
+
+	select {
+	case <-cs.statsMsgQueue:
+		t.Errorf("Should not output stats message after receiving the known block part!")
+	case <-time.After(50 * time.Millisecond):
+	}
+
+}
+
+func TestStateOutputVoteStats(t *testing.T) {
+	cs, vss := randConsensusState(2)
+	// create dummy peer
+	peer := p2pdummy.NewPeer()
+
+	vote := signVote(vss[1], types.VoteTypePrecommit, []byte("test"), types.PartSetHeader{})
+
+	voteMessage := &VoteMessage{vote}
+	cs.handleMsg(msgInfo{voteMessage, peer.ID()})
+
+	statsMessage := <-cs.statsMsgQueue
+	require.Equal(t, voteMessage, statsMessage.Msg, "")
+	require.Equal(t, peer.ID(), statsMessage.PeerID, "")
+
+	// sending the same part from different peer
+	cs.handleMsg(msgInfo{&VoteMessage{vote}, "peer2"})
+
+	// sending the vote for the bigger height
+	incrementHeight(vss[1])
+	vote = signVote(vss[1], types.VoteTypePrecommit, []byte("test"), types.PartSetHeader{})
+
+	cs.handleMsg(msgInfo{&VoteMessage{vote}, peer.ID()})
+
+	select {
+	case <-cs.statsMsgQueue:
+		t.Errorf("Should not output stats message after receiving the known vote or vote from bigger height")
+	case <-time.After(50 * time.Millisecond):
+	}
+
+}
+
 // subscribe subscribes test client to the given query and returns a channel with cap = 1.
 func subscribe(eventBus *types.EventBus, q tmpubsub.Query) <-chan interface{} {
 	out := make(chan interface{}, 1)

docs/.vuepress/config.js

@@ -106,14 +106,6 @@ module.exports = {
           "/spec/abci/apps",
           "/spec/abci/client-server"
         ]
-      },
-      {
-        title: "Research",
-        collapsable: false,
-        children: [
-	  "/research/determinism",
-	  "/research/transactional-semantics"
-	]
       }
     ]
   }

docs/app-dev/getting-started.md

@@ -7,7 +7,7 @@ application you want to run. So, to run a complete blockchain that does
 something useful, you must start two programs: one is Tendermint Core,
 the other is your application, which can be written in any programming
 language. Recall from [the intro to
-ABCI](../introduction/introduction.md#ABCI-Overview) that Tendermint Core handles all
+ABCI](../introduction/introduction.html#abci-overview) that Tendermint Core handles all
 the p2p and consensus stuff, and just forwards transactions to the
 application when they need to be validated, or when they're ready to be
 committed to a block.

docs/introduction/install.md

@@ -77,8 +77,13 @@ make install
 
 ## Compile with CLevelDB support
 
-Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7)
-with snappy. Example for Ubuntu:
+Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7).
+
+Build Tendermint with C libraries: `make build_c`.
+
+### Ubuntu
+
+Install LevelDB with snappy:
 
 ```
 sudo apt-get update
@@ -90,9 +95,9 @@ wget https://github.com/google/leveldb/archive/v1.20.tar.gz && \
   tar -zxvf v1.20.tar.gz && \
   cd leveldb-1.20/ && \
   make && \
-  sudo scp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \
+  cp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \
   cd include/ && \
-  sudo scp -r leveldb /usr/local/include/ && \
+  cp -r leveldb /usr/local/include/ && \
   sudo ldconfig && \
   rm -f v1.20.tar.gz
 ```

docs/research/determinism.md

@@ -1,3 +1,3 @@
 # On Determinism
 
-Arguably, the most difficult part of blockchain programming is determinism - that is, ensuring that sources of indeterminism do not creep into the design of such systems.
+See [Determinism](../spec/abci/abci.md#determinism).

docs/research/transactional-semantics.md

@@ -1,25 +1,5 @@
 # Transactional Semantics
 
-In [Using Tendermint](../tendermint-core/using-tendermint.md#broadcast-api) we
-discussed different API endpoints for sending transactions and
-differences between them.
+See details of the [broadcast API](../tendermint-core/using-tendermint.md#broadcast-api)
+and the [mempool WAL](../tendermint-core/running-in-production.md#mempool-wal).
 
-What we have not yet covered is transactional semantics.
-
-When you send a transaction using one of the available methods, it first
-goes to the mempool. Currently, it does not provide strong guarantees
-like "if the transaction were accepted, it would be eventually included
-in a block (given CheckTx passes)."
-
-For instance a tx could enter the mempool, but before it can be sent to
-peers the node crashes.
-
-We are planning to provide such guarantees by using a WAL and replaying
-transactions (See
-[this issue](https://github.com/tendermint/tendermint/issues/248)), but
-it's non-trivial to do this all efficiently.
-
-The temporary solution is for clients to monitor the node and resubmit
-transaction(s) and/or send them to more nodes at once, so the
-probability of all of them crashing at the same time and losing the msg
-decreases substantially.

docs/spec/abci/abci.md

@@ -48,16 +48,50 @@ Keys and values in tags must be UTF-8 encoded strings (e.g.
 
 ## Determinism
 
-Some methods (`SetOption, Query, CheckTx, DeliverTx`) return
-non-deterministic data in the form of `Info` and `Log`. The `Log` is
-intended for the literal output from the application's logger, while the
-`Info` is any additional info that should be returned.
-
-All other fields in the `Response*` of all methods must be strictly deterministic.
+ABCI applications must implement deterministic finite-state machines to be
+securely replicated by the Tendermint consensus. This means block execution
+over the Consensus Connection must be strictly deterministic: given the same
+ordered set of requests, all nodes will compute identical responses, for all
+BeginBlock, DeliverTx, EndBlock, and Commit. This is critical, because the
+responses are included in the header of the next block, either via a Merkle root
+or directly, so all nodes must agree on exactly what they are.
 
 For this reason, it is recommended that applications not be exposed to any
 external user or process except via the ABCI connections to a consensus engine
-like Tendermint Core.
+like Tendermint Core. The application must only change its state based on input
+from block execution (BeginBlock, DeliverTx, EndBlock, Commit), and not through
+any other kind of request. This is the only way to ensure all nodes see the same
+transactions and compute the same results.
+
+If there is some non-determinism in the state machine, consensus will eventually
+fail as nodes disagree over the correct values for the block header. The
+non-determinism must be fixed and the nodes restarted.
+
+Sources of non-determinism in applications may include:
+
+- Hardware failures
+    - Cosmic rays, overheating, etc.
+- Node-dependent state
+    - Random numbers
+    - Time
+- Underspecification
+    - Library version changes
+    - Race conditions
+    - Floating point numbers
+    - JSON serialization
+    - Iterating through hash-tables/maps/dictionaries
+- External Sources
+    - Filesystem
+    - Network calls (eg. some external REST API service)
+
+See [#56](https://github.com/tendermint/abci/issues/56) for original discussion.
+
+Note that some methods (`SetOption, Query, CheckTx, DeliverTx`) return
+explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is
+intended for the literal output from the application's logger, while the
+`Info` is any additional info that should be returned. These are the only fields
+that are not included in block header computations, so we don't need agreement
+on them. All other fields in the `Response*` must be strictly deterministic.
 
 ## Block Execution
 
@@ -217,7 +251,7 @@ Commit are included in the header of the next block.
     be non-deterministic.
   - `Info (string)`: Additional information. May
     be non-deterministic.
-  - `GasWanted (int64)`: Amount of gas request for transaction.
+  - `GasWanted (int64)`: Amount of gas requested for transaction.
   - `GasUsed (int64)`: Amount of gas consumed by transaction.
   - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing
     transactions (eg. by account).

docs/spec/abci/apps.md

@@ -86,18 +86,51 @@ Otherwise it should never be modified.
 
 ## Transaction Results
 
-`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields, though they
-have slightly different effects.
+`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields.
 
-In both cases, `Info` and `Log` are non-deterministic values for debugging/convenience purposes
+The `Info` and `Log` fields are non-deterministic values for debugging/convenience purposes
 that are otherwise ignored.
 
-In both cases, `GasWanted` and `GasUsed` parameters are currently ignored,
-though see issues
-[#1861](https://github.com/tendermint/tendermint/issues/1861),
-[#2299](https://github.com/tendermint/tendermint/issues/2299) and
-[#2310](https://github.com/tendermint/tendermint/issues/2310) for how this may
-soon change.
+The `Data` field must be strictly deterministic, but can be arbitrary data.
+
+### Gas
+
+Ethereum introduced the notion of `gas` as an abstract representation of the
+cost of resources used by nodes when processing transactions. Every operation in the
+Ethereum Virtual Machine uses some amount of gas, and gas can be accepted at a market-variable price.
+Users propose a maximum amount of gas for their transaction; if the tx uses less, they get
+the difference credited back. Tendermint adopts a similar abstraction,
+though uses it only optionally and weakly, allowing applications to define
+their own sense of the cost of execution.
+
+In Tendermint, the `ConsensusParams.BlockSize.MaxGas` limits the amount of `gas` that can be used in a block.
+The default value is `-1`, meaning no limit, or that the concept of gas is
+meaningless.
+
+Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum
+amount of gas the sender of a tx is willing to use, and the later is how much it actually
+used. Applications should enforce that `GasUsed <= GasWanted` - ie. tx execution
+should halt before it can use more resources than it requested.
+
+When `MaxGas > -1`, Tendermint enforces the following rules:
+
+- `GasWanted <= MaxGas` for all txs in the mempool
+- `(sum of GasWanted in a block) <= MaxGas` when proposing a block
+
+If `MaxGas == -1`, no rules about gas are enforced.
+
+Note that Tendermint does not currently enforce anything about Gas in the consensus, only the mempool.
+This means it does not guarantee that committed blocks satisfy these rules!
+It is the application's responsibility to return non-zero response codes when gas limits are exceeded.
+
+The `GasUsed` field is ignored completely by Tendermint. That said, applications should enforce:
+
+- `GasUsed <= GasWanted` for any given transaction
+- `(sum of GasUsed in a block) <= MaxGas` for every block
+
+In the future, we intend to add a `Priority` field to the responses that can be
+used to explicitly prioritize txs in the mempool for inclusion in a block
+proposal. See [#1861](https://github.com/tendermint/tendermint/issues/1861).
 
 ### CheckTx
 
@@ -142,9 +175,6 @@ If the list is not empty, Tendermint will use it for the validator set.
 This way the application can determine the initial validator set for the
 blockchain.
 
-ResponseInitChain also includes ConsensusParams, but these are presently
-ignored.
-
 ### EndBlock
 
 Updates to the Tendermint validator set can be made by returning
@@ -179,14 +209,74 @@ following rules:
 
 Note the updates returned in block `H` will only take effect at block `H+2`.
 
+## Consensus Parameters
+
+ConsensusParams enforce certain limits in the blockchain, like the maximum size
+of blocks, amount of gas used in a block, and the maximum acceptable age of
+evidence. They can be set in InitChain and updated in EndBlock.
+
+### BlockSize.MaxBytes
+
+The maximum size of a complete Amino encoded block.
+This is enforced by Tendermint consensus.
+
+This implies a maximum tx size that is this MaxBytes, less the expected size of
+the header, the validator set, and any included evidence in the block.
+
+Must have `0 < MaxBytes < 100 MB`.
+
+### BlockSize.MaxGas
+
+The maximum of the sum of `GasWanted` in a proposed block.
+This is *not* enforced by Tendermint consensus.
+It is left to the app to enforce (ie. if txs are included past the
+limit, they should return non-zero codes). It is used by Tendermint to limit the
+txs included in a proposed block.
+
+Must have `MaxGas >= -1`.
+If `MaxGas == -1`, no limit is enforced.
+
+### EvidenceParams.MaxAge
+
+This is the maximum age of evidence.
+This is enforced by Tendermint consensus.
+If a block includes evidence older than this, the block will be rejected
+(validators won't vote for it).
+
+Must have `0 < MaxAge`.
+
+### Updates
+
+The application may set the consensus params during InitChain, and update them during
+EndBlock.
+
+#### InitChain
+
+ResponseInitChain includes a ConsensusParams.
+If its nil, Tendermint will use the params loaded in the genesis
+file. If it's not nil, Tendermint will use it.
+This way the application can determine the initial consensus params for the
+blockchain.
+
+#### EndBlock
+
+ResponseEndBlock includes a ConsensusParams.
+If its nil, Tendermint will do nothing.
+If it's not nil, Tendermint will use it.
+This way the application can update the consensus params over time.
+
+Note the updates returned in block `H` will take effect right away for block
+`H+1`.
+
 ## Query
 
 Query is a generic method with lots of flexibility to enable diverse sets
 of queries on application state. Tendermint makes use of Query to filter new peers
 based on ID and IP, and exposes Query to the user over RPC.
+
 Note that calls to Query are not replicated across nodes, but rather query the
-local node's state - hence they may provide stale reads. For reads that require
-consensus, a transaction is required.
+local node's state - hence they may return stale reads. For reads that require
+consensus, use a transaction.
 
 The most important use of Query is to return Merkle proofs of the application state at some height
 that can be used for efficient application-specific lite-clients.
@@ -235,6 +325,15 @@ using the following paths, with no additional data:
 If either of these queries return a non-zero ABCI code, Tendermint will refuse
 to connect to the peer.
 
+### Paths
+
+Queries are directed at paths, and may optionally include additional data.
+
+The expectation is for there to be some number of high level paths
+differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently,
+Tendermint only uses `/p2p`, for filtering peers. For more advanced use, see the
+implementation of
+[Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/v0.23.1/baseapp/baseapp.go#L333).
 
 ## Crash Recovery
 

docs/spec/consensus/consensus.md

@@ -17,7 +17,7 @@
   vote](https://godoc.org/github.com/tendermint/tendermint/types#FirstPrecommit)
   for something.
 - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R`
-  included in its [sign-bytes](../blockchain/blockchain.md).
+  included in its [sign-bytes](../blockchain/blockchain.md#vote).
 - _+2/3_ is short for "more than 2/3"
 - _1/3+_ is short for "1/3 or more"
 - A set of +2/3 of prevotes for a particular block or `<nil>` at

docs/tendermint-core/running-in-production.md

@@ -1,5 +1,41 @@
 # Running in production
 
+## Database
+
+By default, Tendermint uses the `syndtr/goleveldb` package for it's in-process
+key-value database. Unfortunately, this implementation of LevelDB seems to suffer under heavy load (see
+[#226](https://github.com/syndtr/goleveldb/issues/226)). It may be best to
+install the real C-implementation of LevelDB and compile Tendermint to use
+that using `make build_c`. See the [install instructions](../introduction/install.md) for details.
+
+Tendermint keeps multiple distinct LevelDB databases in the `$TMROOT/data`:
+
+- `blockstore.db`: Keeps the entire blockchain - stores blocks,
+  block commits, and block meta data, each indexed by height. Used to sync new
+  peers.
+- `evidence.db`: Stores all verified evidence of misbehaviour.
+- `state.db`: Stores the current blockchain state (ie. height, validators,
+  consensus params). Only grows if consensus params or validators change. Also
+  used to temporarily store intermediate results during block processing.
+- `tx_index.db`: Indexes txs (and their results) by tx hash and by DeliverTx result tags.
+
+By default, Tendermint will only index txs by their hash, not by their DeliverTx
+result tags. See [indexing transactions](../app-dev/indexing-transactions.md) for
+details.
+
+There is no current strategy for pruning the databases. Consider reducing
+block production by [controlling empty blocks](../tendermint-core/using-tendermint.md#no-empty-blocks)
+or by increasing the `consensus.timeout_commit` param. Note both of these are
+local settings and not enforced by the consensus.
+
+We're working on [state
+syncing](https://github.com/tendermint/tendermint/issues/828),
+which will enable history to be thrown away
+and recent application state to be directly synced. We'll need to develop solutions
+for archival nodes that allow queries on historical transactions and states.
+The Cosmos project has had much success just dumping the latest state of a
+blockchain to disk and starting a new chain from that state.
+
 ## Logging
 
 Default logging level (`main:info,state:info,*:`) should suffice for
@@ -11,6 +47,33 @@ you're trying to debug Tendermint or asked to provide logs with debug
 logging level, you can do so by running tendermint with
 `--log_level="*:debug"`.
 
+## Write Ahead Logs (WAL)
+
+Tendermint uses write ahead logs for the consensus (`cs.wal`) and the mempool
+(`mempool.wal`). Both WALs have a max size of 1GB and are automatically rotated.
+
+### Consensus WAL
+
+The `consensus.wal` is used to ensure we can recover from a crash at any point
+in the consensus state machine.
+It writes all consensus messages (timeouts, proposals, block part, or vote)
+to a single file, flushing to disk before processing messages from its own
+validator. Since Tendermint validators are expected to never sign a conflicting vote, the
+WAL ensures we can always recover deterministically to the latest state of the consensus without
+using the network or re-signing any consensus messages.
+
+If your `consensus.wal` is corrupted, see [below](#wal-corruption).
+
+### Mempool WAL
+
+The `mempool.wal` logs all incoming txs before running CheckTx, but is
+otherwise not used in any programmatic way. It's just a kind of manual
+safe guard. Note the mempool provides no durability guarantees - a tx sent to one or many nodes
+may never make it into the blockchain if those nodes crash before being able to
+propose it. Clients must monitor their txs by subscribing over websockets,
+polling for them, or using `/broadcast_tx_commit`. In the worst case, txs can be
+resent from the mempool WAL manually.
+
 ## DOS Exposure and Mitigation
 
 Validators are supposed to setup [Sentry Node

docs/tendermint-core/using-tendermint.md

@@ -305,6 +305,12 @@ can take on the order of a second. For a quick result, use
 `broadcast_tx_sync`, but the transaction will not be committed until
 later, and by that point its effect on the state may change.
 
+Note the mempool does not provide strong guarantees - just because a tx passed
+CheckTx (ie. was accepted into the mempool), doesn't mean it will be committed,
+as nodes with the tx in their mempool may crash before they get to propose.
+For more information, see the [mempool
+write-ahead-log](../tendermint-core/running-in-production.md#mempool-wal)
+
 ## Tendermint Networks
 
 When `tendermint init` is run, both a `genesis.json` and

mempool/mempool.go

@@ -22,6 +22,16 @@ import (
 	"github.com/tendermint/tendermint/types"
 )
 
+// PreCheckFunc is an optional filter executed before CheckTx and rejects
+// transaction if false is returned. An example would be to ensure that a
+// transaction doesn't exceeded the block size.
+type PreCheckFunc func(types.Tx) bool
+
+// PostCheckFunc is an optional filter executed after CheckTx and rejects
+// transaction if false is returned. An example would be to ensure a
+// transaction doesn't require more gas than available for the block.
+type PostCheckFunc func(types.Tx, *abci.ResponseCheckTx) bool
+
 /*
 
 The mempool pushes new txs onto the proxyAppConn.
@@ -58,6 +68,27 @@ var (
 	ErrMempoolIsFull = errors.New("Mempool is full")
 )
 
+// PreCheckAminoMaxBytes checks that the size of the transaction plus the amino
+// overhead is smaller or equal to the expected maxBytes.
+func PreCheckAminoMaxBytes(maxBytes int64) PreCheckFunc {
+	return func(tx types.Tx) bool {
+		// We have to account for the amino overhead in the tx size as well
+		aminoOverhead := amino.UvarintSize(uint64(len(tx)))
+		return int64(len(tx)+aminoOverhead) <= maxBytes
+	}
+}
+
+// PostCheckMaxGas checks that the wanted gas is smaller or equal to the passed
+// maxGas. Returns true if maxGas is -1.
+func PostCheckMaxGas(maxGas int64) PostCheckFunc {
+	return func(tx types.Tx, res *abci.ResponseCheckTx) bool {
+		if maxGas == -1 {
+			return true
+		}
+		return res.GasWanted <= maxGas
+	}
+}
+
 // TxID is the hex encoded hash of the bytes as a types.Tx.
 func TxID(tx []byte) string {
 	return fmt.Sprintf("%X", types.Tx(tx).Hash())
@@ -80,8 +111,8 @@ type Mempool struct {
 	recheckEnd           *clist.CElement // re-checking stops here
 	notifiedTxsAvailable bool
 	txsAvailable         chan struct{} // fires once for each height, when the mempool is not empty
-	// Filter mempool to only accept txs for which filter(tx) returns true.
-	filter func(types.Tx) bool
+	preCheck             PreCheckFunc
+	postCheck            PostCheckFunc
 
 	// Keep a cache of already-seen txs.
 	// This reduces the pressure on the proxyApp.
@@ -141,10 +172,16 @@ func (mem *Mempool) SetLogger(l log.Logger) {
 	mem.logger = l
 }
 
-// WithFilter sets a filter for mempool to only accept txs for which f(tx)
-// returns true.
-func WithFilter(f func(types.Tx) bool) MempoolOption {
-	return func(mem *Mempool) { mem.filter = f }
+// WithPreCheck sets a filter for the mempool to reject a tx if f(tx) returns
+// false. This is ran before CheckTx.
+func WithPreCheck(f PreCheckFunc) MempoolOption {
+	return func(mem *Mempool) { mem.preCheck = f }
+}
+
+// WithPostCheck sets a filter for the mempool to reject a tx if f(tx) returns
+// false. This is ran after CheckTx.
+func WithPostCheck(f PostCheckFunc) MempoolOption {
+	return func(mem *Mempool) { mem.postCheck = f }
 }
 
 // WithMetrics sets the metrics.
@@ -248,7 +285,7 @@ func (mem *Mempool) CheckTx(tx types.Tx, cb func(*abci.Response)) (err error) {
 		return ErrMempoolIsFull
 	}
 
-	if mem.filter != nil && !mem.filter(tx) {
+	if mem.preCheck != nil && !mem.preCheck(tx) {
 		return
 	}
 
@@ -298,7 +335,8 @@ func (mem *Mempool) resCbNormal(req *abci.Request, res *abci.Response) {
 	switch r := res.Value.(type) {
 	case *abci.Response_CheckTx:
 		tx := req.GetCheckTx().Tx
-		if r.CheckTx.Code == abci.CodeTypeOK {
+		if (r.CheckTx.Code == abci.CodeTypeOK) &&
+			mem.isPostCheckPass(tx, r.CheckTx) {
 			mem.counter++
 			memTx := &mempoolTx{
 				counter:   mem.counter,
@@ -326,10 +364,15 @@ func (mem *Mempool) resCbRecheck(req *abci.Request, res *abci.Response) {
 	case *abci.Response_CheckTx:
 		memTx := mem.recheckCursor.Value.(*mempoolTx)
 		if !bytes.Equal(req.GetCheckTx().Tx, memTx.tx) {
-			cmn.PanicSanity(fmt.Sprintf("Unexpected tx response from proxy during recheck\n"+
-				"Expected %X, got %X", r.CheckTx.Data, memTx.tx))
+			cmn.PanicSanity(
+				fmt.Sprintf(
+					"Unexpected tx response from proxy during recheck\nExpected %X, got %X",
+					r.CheckTx.Data,
+					memTx.tx,
+				),
+			)
 		}
-		if r.CheckTx.Code == abci.CodeTypeOK {
+		if (r.CheckTx.Code == abci.CodeTypeOK) && mem.isPostCheckPass(memTx.tx, r.CheckTx) {
 			// Good, nothing to do.
 		} else {
 			// Tx became invalidated due to newly committed block.
@@ -444,7 +487,12 @@ func (mem *Mempool) ReapMaxTxs(max int) types.Txs {
 // Update informs the mempool that the given txs were committed and can be discarded.
 // NOTE: this should be called *after* block is committed by consensus.
 // NOTE: unsafe; Lock/Unlock must be managed by caller
-func (mem *Mempool) Update(height int64, txs types.Txs, filter func(types.Tx) bool) error {
+func (mem *Mempool) Update(
+	height int64,
+	txs types.Txs,
+	preCheck PreCheckFunc,
+	postCheck PostCheckFunc,
+) error {
 	// First, create a lookup map of txns in new txs.
 	txsMap := make(map[string]struct{}, len(txs))
 	for _, tx := range txs {
@@ -455,8 +503,11 @@ func (mem *Mempool) Update(height int64, txs types.Txs, filter func(types.Tx) bo
 	mem.height = height
 	mem.notifiedTxsAvailable = false
 
-	if filter != nil {
-		mem.filter = filter
+	if preCheck != nil {
+		mem.preCheck = preCheck
+	}
+	if postCheck != nil {
+		mem.postCheck = postCheck
 	}
 
 	// Remove transactions that are already in txs.
@@ -514,6 +565,10 @@ func (mem *Mempool) recheckTxs(goodTxs []types.Tx) {
 	mem.proxyAppConn.FlushAsync()
 }
 
+func (mem *Mempool) isPostCheckPass(tx types.Tx, r *abci.ResponseCheckTx) bool {
+	return mem.postCheck == nil || mem.postCheck(tx, r)
+}
+
 //--------------------------------------------------------------------------------
 
 // mempoolTx is a transaction that successfully ran

mempool/mempool_test.go

@@ -14,6 +14,7 @@ import (
 	"github.com/stretchr/testify/assert"
 
 	"github.com/stretchr/testify/require"
+	amino "github.com/tendermint/go-amino"
 	"github.com/tendermint/tendermint/abci/example/counter"
 	"github.com/tendermint/tendermint/abci/example/kvstore"
 	abci "github.com/tendermint/tendermint/abci/types"
@@ -119,6 +120,62 @@ func TestReapMaxBytesMaxGas(t *testing.T) {
 	}
 }
 
+func TestMempoolFilters(t *testing.T) {
+	app := kvstore.NewKVStoreApplication()
+	cc := proxy.NewLocalClientCreator(app)
+	mempool := newMempoolWithApp(cc)
+	emptyTxArr := []types.Tx{[]byte{}}
+
+	nopPreFilter := func(tx types.Tx) bool { return true }
+	nopPostFilter := func(tx types.Tx, res *abci.ResponseCheckTx) bool { return true }
+
+	// This is the same filter we expect to be used within node/node.go and state/execution.go
+	nBytePreFilter := func(n int) func(tx types.Tx) bool {
+		return func(tx types.Tx) bool {
+			// We have to account for the amino overhead in the tx size as well
+			aminoOverhead := amino.UvarintSize(uint64(len(tx)))
+			return (len(tx) + aminoOverhead) <= n
+		}
+	}
+
+	nGasPostFilter := func(n int64) func(tx types.Tx, res *abci.ResponseCheckTx) bool {
+		return func(tx types.Tx, res *abci.ResponseCheckTx) bool {
+			if n == -1 {
+				return true
+			}
+			return res.GasWanted <= n
+		}
+	}
+
+	// each table driven test creates numTxsToCreate txs with checkTx, and at the end clears all remaining txs.
+	// each tx has 20 bytes + amino overhead = 21 bytes, 1 gas
+	tests := []struct {
+		numTxsToCreate int
+		preFilter      func(tx types.Tx) bool
+		postFilter     func(tx types.Tx, res *abci.ResponseCheckTx) bool
+		expectedNumTxs int
+	}{
+		{10, nopPreFilter, nopPostFilter, 10},
+		{10, nBytePreFilter(10), nopPostFilter, 0},
+		{10, nBytePreFilter(20), nopPostFilter, 0},
+		{10, nBytePreFilter(21), nopPostFilter, 10},
+		{10, nopPreFilter, nGasPostFilter(-1), 10},
+		{10, nopPreFilter, nGasPostFilter(0), 0},
+		{10, nopPreFilter, nGasPostFilter(1), 10},
+		{10, nopPreFilter, nGasPostFilter(3000), 10},
+		{10, nBytePreFilter(10), nGasPostFilter(20), 0},
+		{10, nBytePreFilter(30), nGasPostFilter(20), 10},
+		{10, nBytePreFilter(21), nGasPostFilter(1), 10},
+		{10, nBytePreFilter(21), nGasPostFilter(0), 0},
+	}
+	for tcIndex, tt := range tests {
+		mempool.Update(1, emptyTxArr, tt.preFilter, tt.postFilter)
+		checkTxs(t, mempool, tt.numTxsToCreate)
+		require.Equal(t, tt.expectedNumTxs, mempool.Size(), "mempool had the incorrect size, on test case %d", tcIndex)
+		mempool.Flush()
+	}
+}
+
 func TestTxsAvailable(t *testing.T) {
 	app := kvstore.NewKVStoreApplication()
 	cc := proxy.NewLocalClientCreator(app)
@@ -139,7 +196,7 @@ func TestTxsAvailable(t *testing.T) {
 	// it should fire once now for the new height
 	// since there are still txs left
 	committedTxs, txs := txs[:50], txs[50:]
-	if err := mempool.Update(1, committedTxs, nil); err != nil {
+	if err := mempool.Update(1, committedTxs, nil, nil); err != nil {
 		t.Error(err)
 	}
 	ensureFire(t, mempool.TxsAvailable(), timeoutMS)
@@ -151,7 +208,7 @@ func TestTxsAvailable(t *testing.T) {
 
 	// now call update with all the txs. it should not fire as there are no txs left
 	committedTxs = append(txs, moreTxs...)
-	if err := mempool.Update(2, committedTxs, nil); err != nil {
+	if err := mempool.Update(2, committedTxs, nil, nil); err != nil {
 		t.Error(err)
 	}
 	ensureNoFire(t, mempool.TxsAvailable(), timeoutMS)
@@ -208,7 +265,7 @@ func TestSerialReap(t *testing.T) {
 			binary.BigEndian.PutUint64(txBytes, uint64(i))
 			txs = append(txs, txBytes)
 		}
-		if err := mempool.Update(0, txs, nil); err != nil {
+		if err := mempool.Update(0, txs, nil, nil); err != nil {
 			t.Error(err)
 		}
 	}

node/node.go

@@ -7,22 +7,23 @@ import (
 	"fmt"
 	"net"
 	"net/http"
+	_ "net/http/pprof"
+	"strings"
 	"time"
 
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promhttp"
-
 	amino "github.com/tendermint/go-amino"
-	abci "github.com/tendermint/tendermint/abci/types"
-	"github.com/tendermint/tendermint/crypto/ed25519"
-	cmn "github.com/tendermint/tendermint/libs/common"
-	dbm "github.com/tendermint/tendermint/libs/db"
-	"github.com/tendermint/tendermint/libs/log"
 
+	abci "github.com/tendermint/tendermint/abci/types"
 	bc "github.com/tendermint/tendermint/blockchain"
 	cfg "github.com/tendermint/tendermint/config"
 	cs "github.com/tendermint/tendermint/consensus"
+	"github.com/tendermint/tendermint/crypto/ed25519"
 	"github.com/tendermint/tendermint/evidence"
+	cmn "github.com/tendermint/tendermint/libs/common"
+	dbm "github.com/tendermint/tendermint/libs/db"
+	"github.com/tendermint/tendermint/libs/log"
 	mempl "github.com/tendermint/tendermint/mempool"
 	"github.com/tendermint/tendermint/p2p"
 	"github.com/tendermint/tendermint/p2p/pex"
@@ -40,9 +41,6 @@ import (
 	"github.com/tendermint/tendermint/types"
 	tmtime "github.com/tendermint/tendermint/types/time"
 	"github.com/tendermint/tendermint/version"
-
-	_ "net/http/pprof"
-	"strings"
 )
 
 //------------------------------------------------------------------------------
@@ -255,7 +253,17 @@ func NewNode(config *cfg.Config,
 		proxyApp.Mempool(),
 		state.LastBlockHeight,
 		mempl.WithMetrics(memplMetrics),
-		mempl.WithFilter(sm.TxFilter(state)),
+		mempl.WithPreCheck(
+			mempl.PreCheckAminoMaxBytes(
+				types.MaxDataBytesUnknownEvidence(
+					state.ConsensusParams.BlockSize.MaxBytes,
+					state.Validators.Size(),
+				),
+			),
+		),
+		mempl.WithPostCheck(
+			mempl.PostCheckMaxGas(state.ConsensusParams.BlockSize.MaxGas),
+		),
 	)
 	mempoolLogger := logger.With("module", "mempool")
 	mempool.SetLogger(mempoolLogger)

rpc/lib/server/http_server.go

@@ -102,15 +102,16 @@ func StartHTTPAndTLSServer(
 		listener = netutil.LimitListener(listener, config.MaxOpenConnections)
 	}
 
-	go func() {
-		err := http.ServeTLS(
-			listener,
-			RecoverAndLogHandler(maxBytesHandler{h: handler, n: maxBodyBytes}, logger),
-			certFile,
-			keyFile,
-		)
+	err = http.ServeTLS(
+		listener,
+		RecoverAndLogHandler(maxBytesHandler{h: handler, n: maxBodyBytes}, logger),
+		certFile,
+		keyFile,
+	)
+	if err != nil {
 		logger.Error("RPC HTTPS server stopped", "err", err)
-	}()
+		return nil, err
+	}
 	return listener, nil
 }
 

rpc/lib/server/http_server_test.go

@@ -5,11 +5,14 @@ import (
 	"io"
 	"io/ioutil"
 	"net/http"
+	"os"
 	"sync"
 	"sync/atomic"
 	"testing"
 	"time"
 
+	"github.com/stretchr/testify/require"
+
 	"github.com/tendermint/tendermint/libs/log"
 )
 
@@ -60,3 +63,15 @@ func TestMaxOpenConnections(t *testing.T) {
 		t.Errorf("%d requests failed within %d attempts", failed, attempts)
 	}
 }
+
+func TestStartHTTPAndTLSServer(t *testing.T) {
+	// set up fixtures
+	listenerAddr := "tcp://0.0.0.0:0"
+	mux := http.NewServeMux()
+	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {})
+
+	// test failure
+	gotListener, err := StartHTTPAndTLSServer(listenerAddr, mux, "", "", log.TestingLogger(), Config{MaxOpenConnections: 1})
+	require.Nil(t, gotListener)
+	require.IsType(t, (*os.PathError)(nil), err)
+}

state/execution.go

@@ -7,6 +7,7 @@ import (
 	abci "github.com/tendermint/tendermint/abci/types"
 	dbm "github.com/tendermint/tendermint/libs/db"
 	"github.com/tendermint/tendermint/libs/log"
+	"github.com/tendermint/tendermint/mempool"
 	"github.com/tendermint/tendermint/proxy"
 	"github.com/tendermint/tendermint/types"
 )
@@ -115,11 +116,16 @@ func (blockExec *BlockExecutor) ApplyBlock(state State, blockID types.BlockID, b
 	return state, nil
 }
 
-// Commit locks the mempool, runs the ABCI Commit message, and updates the mempool.
+// Commit locks the mempool, runs the ABCI Commit message, and updates the
+// mempool.
 // It returns the result of calling abci.Commit (the AppHash), and an error.
-// The Mempool must be locked during commit and update because state is typically reset on Commit and old txs must be replayed
-// against committed state before new txs are run in the mempool, lest they be invalid.
-func (blockExec *BlockExecutor) Commit(state State, block *types.Block) ([]byte, error) {
+// The Mempool must be locked during commit and update because state is
+// typically reset on Commit and old txs must be replayed against committed
+// state before new txs are run in the mempool, lest they be invalid.
+func (blockExec *BlockExecutor) Commit(
+	state State,
+	block *types.Block,
+) ([]byte, error) {
 	blockExec.mempool.Lock()
 	defer blockExec.mempool.Unlock()
 
@@ -134,22 +140,35 @@ func (blockExec *BlockExecutor) Commit(state State, block *types.Block) ([]byte,
 	// Commit block, get hash back
 	res, err := blockExec.proxyApp.CommitSync()
 	if err != nil {
-		blockExec.logger.Error("Client error during proxyAppConn.CommitSync", "err", err)
+		blockExec.logger.Error(
+			"Client error during proxyAppConn.CommitSync",
+			"err", err,
+		)
 		return nil, err
 	}
 	// ResponseCommit has no error code - just data
 
-	blockExec.logger.Info("Committed state",
+	blockExec.logger.Info(
+		"Committed state",
 		"height", block.Height,
 		"txs", block.NumTxs,
-		"appHash", fmt.Sprintf("%X", res.Data))
+		"appHash", fmt.Sprintf("%X", res.Data),
+	)
 
 	// Update mempool.
-	if err := blockExec.mempool.Update(block.Height, block.Txs, TxFilter(state)); err != nil {
-		return nil, err
-	}
+	err = blockExec.mempool.Update(
+		block.Height,
+		block.Txs,
+		mempool.PreCheckAminoMaxBytes(
+			types.MaxDataBytesUnknownEvidence(
+				state.ConsensusParams.BlockSize.MaxBytes,
+				state.Validators.Size(),
+			),
+		),
+		mempool.PostCheckMaxGas(state.ConsensusParams.MaxGas),
+	)
 
-	return res.Data, nil
+	return res.Data, err
 }
 
 //---------------------------------------------------------

state/services.go

@@ -2,6 +2,7 @@ package state
 
 import (
 	abci "github.com/tendermint/tendermint/abci/types"
+	"github.com/tendermint/tendermint/mempool"
 	"github.com/tendermint/tendermint/types"
 )
 
@@ -23,7 +24,7 @@ type Mempool interface {
 	Size() int
 	CheckTx(types.Tx, func(*abci.Response)) error
 	ReapMaxBytesMaxGas(maxBytes, maxGas int64) types.Txs
-	Update(height int64, txs types.Txs, filter func(types.Tx) bool) error
+	Update(int64, types.Txs, mempool.PreCheckFunc, mempool.PostCheckFunc) error
 	Flush()
 	FlushAppConn() error
 
@@ -36,16 +37,23 @@ type MockMempool struct{}
 
 var _ Mempool = MockMempool{}
 
-func (MockMempool) Lock()                                                                {}
-func (MockMempool) Unlock()                                                              {}
-func (MockMempool) Size() int                                                            { return 0 }
-func (MockMempool) CheckTx(tx types.Tx, cb func(*abci.Response)) error                   { return nil }
-func (MockMempool) ReapMaxBytesMaxGas(maxBytes, maxGas int64) types.Txs              { return types.Txs{} }
-func (MockMempool) Update(height int64, txs types.Txs, filter func(types.Tx) bool) error { return nil }
-func (MockMempool) Flush()                                                               {}
-func (MockMempool) FlushAppConn() error                                                  { return nil }
-func (MockMempool) TxsAvailable() <-chan struct{}                                        { return make(chan struct{}) }
-func (MockMempool) EnableTxsAvailable()                                                  {}
+func (MockMempool) Lock()                                            {}
+func (MockMempool) Unlock()                                          {}
+func (MockMempool) Size() int                                        { return 0 }
+func (MockMempool) CheckTx(_ types.Tx, _ func(*abci.Response)) error { return nil }
+func (MockMempool) ReapMaxBytesMaxGas(_, _ int64) types.Txs          { return types.Txs{} }
+func (MockMempool) Update(
+	_ int64,
+	_ types.Txs,
+	_ mempool.PreCheckFunc,
+	_ mempool.PostCheckFunc,
+) error {
+	return nil
+}
+func (MockMempool) Flush()                        {}
+func (MockMempool) FlushAppConn() error           { return nil }
+func (MockMempool) TxsAvailable() <-chan struct{} { return make(chan struct{}) }
+func (MockMempool) EnableTxsAvailable()           {}
 
 //------------------------------------------------------
 // blockstore
@@ -82,5 +90,5 @@ type EvidencePool interface {
 type MockEvidencePool struct{}
 
 func (m MockEvidencePool) PendingEvidence(int64) []types.Evidence { return nil }
-func (m MockEvidencePool) AddEvidence(types.Evidence) error     { return nil }
-func (m MockEvidencePool) Update(*types.Block, State)           {}
+func (m MockEvidencePool) AddEvidence(types.Evidence) error       { return nil }
+func (m MockEvidencePool) Update(*types.Block, State)             {}

version/version.go

@@ -3,14 +3,14 @@ package version
 // Version components
 const (
 	Maj = "0"
-	Min = "24"
+	Min = "25"
 	Fix = "0"
 )
 
 var (
 	// Version is the current version of Tendermint
 	// Must be a string because scripts like dist.sh read this file.
-	Version = "0.24.0"
+	Version = "0.25.0"
 
 	// GitCommit is the current HEAD set using ldflags.
 	GitCommit string