Commit Graph

228 Commits

Author SHA1 Message Date
8d50bb9dad conesnsu: follow up to removing some consensus params (#2427)
* follow up to removing some consensus params Refs #2382
* change args type to int64 in state#makeParams
* make valsCount and evidenceCount ints again
* MaxEvidenceBytesPerBlock: include magic number in godoc
* [spec] creating a proposal
* test state#TxFilter
* panic if MaxDataBytes is less than 0
* fixes after review
* use amino#UvarintSize to calculate overhead
0c74291f3b/encoder.go (L85-L90)
* avoid cyclic imports
* you can do better Go, come on
* remove testdouble package
2018-09-21 11:00:36 +02:00
26aa978456 Make mempool cache a proper LRU (#2407)
Closes #2399
2018-09-19 21:40:22 +04:00
1ea64fc27f Make mempool aware of MaxGas requirement (#2360)
* Make mempool aware of MaxGas requirement

* update spec

* Add tests

* Switch GasWanted from kv store to persistent kv store

* Fix typo in test name

* switch back to using kvstore, not persistent kv store
2018-09-12 16:41:19 -04:00
166fd82b70 max-bytes PR follow-up (#2318)
* ReapMaxTxs: return all txs if max is negative

this mirrors ReapMaxBytes behavior

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214439950

* increase MaxAminoOverheadForBlock

tested with:

```
func TestMaxAminoOverheadForBlock(t *testing.T) {

        maxChainID := ""
        for i := 0; i < MaxChainIDLen; i++ {
                maxChainID += "𠜎"
        }

        h := Header{
                ChainID:            maxChainID,
                Height:             10,
                Time:               time.Now().UTC(),
                NumTxs:             100,
                TotalTxs:           200,
                LastBlockID:        makeBlockID(make([]byte, 20), 300, make([]byte, 20)),
                LastCommitHash:     tmhash.Sum([]byte("last_commit_hash")),
                DataHash:           tmhash.Sum([]byte("data_hash")),
                ValidatorsHash:     tmhash.Sum([]byte("validators_hash")),
                NextValidatorsHash: tmhash.Sum([]byte("next_validators_hash")),
                ConsensusHash:      tmhash.Sum([]byte("consensus_hash")),
                AppHash:            tmhash.Sum([]byte("app_hash")),
                LastResultsHash:    tmhash.Sum([]byte("last_results_hash")),
                EvidenceHash:       tmhash.Sum([]byte("evidence_hash")),
                ProposerAddress:    tmhash.Sum([]byte("proposer_address")),
        }
        b := Block{
                Header:     h,
                Data:       Data{Txs: makeTxs(10000, 100)},
                Evidence:   EvidenceData{},
                LastCommit: &Commit{},
        }

        bz, err := cdc.MarshalBinary(b)
        require.NoError(t, err)

        assert.Equal(t, MaxHeaderBytes+MaxAminoOverheadForBlock-2, len(bz)-1000000-20000-1)
}
```

* fix MaxYYY constants calculation

by using math.MaxInt64

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214444244

* pass mempool filter as an option

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214445869

* fixes after Dev's comments
2018-09-04 11:46:34 +04:00
e873fed815 calculate amino overhead on the fly 2018-08-31 16:01:22 +04:00
d73c5cbdb1 reap max bytes from the mempool & check transaction size
See ADR 020: Limiting txs size inside a block docs/architecture/adr-020-block-size.md

Refs #2035
2018-08-31 16:01:21 +04:00
bd531401a0 mempool: Store txs by hash inside of cache (#2234)
* mempool: Store txs by hash inside of cache

This allows for large cachesizes, without fear of the memory
consumption growing rapidly.

* (squash this) rename hashedTx -> txHash
2018-08-29 08:10:06 +04:00
8a1a79257e mempool: Keep cache hashmap and linked list in sync (#2188)
* mempool: Keep cache hashmap and linked list in sync

This removes bugs with the linked list being full, but hashmap empty

* address PR comments

* switch clist back to list
2018-08-13 14:42:33 +04:00
2756be5a59 libs: Remove usage of custom Fmt, in favor of fmt.Sprintf (#2199)
* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!
2018-08-10 09:25:57 +04:00
e1b9bf7c81 set capacity of txsmap (#2166) 2018-08-07 20:50:10 +04:00
96ae535fb8 proto3 timestamp (#2064)
This PR changes ABCI time format from int64 (Unix seconds) to WKT (WellKnownType) google.protobuf.Timestamp.

Refs #1857

Reasons:

better precision
standard DT for proto

* update Gopkg.lock
* [makefile] remove extra grep
    - go list excludes vendor by default now
* proto3 timestamp
* [docs/abci-spec] note about serialisation format
* make time non-nullable
2018-07-27 04:23:19 +02:00
15b112e669 mempool: chan bool -> chan struct{} 2018-07-23 21:06:47 -04:00
e36ce6f893 fix race condition on proposal height for published txs (#2021)
* #1920 try to fix race condition on proposal height for published txs

- related to create_empty_blocks=false
- published height for accepted tx can be wrong (too low)
- use the actual mempool height + 1 for the proposal
- expose Height() on mempool

* #1920 add initial test for mempool.Height()

- not sure how to test the lock
- can the mutex reference be of type Locker?
-- this way, we can use a "mock" of the mutex to test triggering

* #1920 use the ConsensusState height in favor of mempool

- gets rid of indirections
- doesn't need any "+1" magic

* #1920 cosmetic

- if we use cs.Height, it's enough to evaluate right before propose

* #1920 cleanup TODO and non-needed code

* #1920 add changelog entry
2018-07-23 15:34:45 +04:00
9120fd5d14 unexport DecodeMessage functions
Refs #646
2018-07-09 13:01:23 +04:00
d55243f0e6 fix import paths 2018-07-01 22:36:49 -04:00
ab04201c3d fix empty packet size calculation
Fixes #1762#issuecomment-398940107
2018-06-29 12:40:32 +04:00
aa20c45ae9 log total when adding new txs to the mempool 2018-06-27 18:14:45 +04:00
fa3bd05d44 mempool: log hashes, not whole tx 2018-06-27 18:14:45 +04:00
a605b66c5a Move abci imports 2018-06-22 07:28:07 +02:00
9cd9f3338b Merge branch 'master' into develop 2018-06-21 12:57:09 -07:00
205d8b8062 fixes after @xla review
- move prometheus metrics into internal packages
- *Option structs
- misc. format changes
2018-06-20 12:40:25 +04:00
e4bb3566a0 move metrics constructors to a separate package 2018-06-20 12:40:25 +04:00
84812145cb friendly apis for constructors 2018-06-20 12:40:25 +04:00
b10b0da3fd bundle imports 2018-06-20 12:40:11 +04:00
7efb73aa18 mempool size metric 2018-06-20 12:38:45 +04:00
4f5492c831 add nopTxCache (Nil Object Pattern)
to better handle zero cache size
2018-06-19 11:59:07 +04:00
70d973016e output msg only once during start 2018-06-19 11:40:40 +04:00
4b2348f697 mempool: fix cache_size==0. closes #1761 2018-06-18 18:21:19 -07:00
a519825bf8 consensus: fixes #1754
* updateToState exits early if the state isn't new, which happens after
* fast syncing. This results in not sending a NewRoundStep message. The mempool
* reactor depends on PeerState, which is updated by NewRoundStep
* messages. If the peer never sends a NewRoundStep, the mempool reactor
* will think they're behind, and never forward transactions. Note this
* only happens when `create_empty_blocks = false`, because otherwise
* peers will move through the consensus state and send a NewRoundStep
* for a new step soon anyways. Simple fix is just to send the
* NewRoundStep message during updateToState even if exit early
2018-06-18 17:08:09 -07:00
3fa734ef5a recheck only if there are txs left in the mempool (#1645) 2018-06-04 14:28:47 +04:00
98b0c51b5f fix possible mempool deadlock 2018-06-03 16:11:52 -04:00
202a43a5af remove TODO
no longer relevant, I guess. since ABCI only defines 0 (success) code.
2018-05-18 10:48:13 +04:00
c9001d5a11 bound the mempool
Refs #345
2018-05-18 10:45:51 +04:00
ae572b9038 remove extra call to Exists
Refs #345

https://github.com/tendermint/tendermint/issues/1539#issuecomment-387240606
2018-05-18 10:17:09 +04:00
bbe1355957 log only hash, not tx itself 2018-05-17 10:00:59 +04:00
7c14fa820d do not log txs at info level
BEFORE:

```
./tm-bench -c 5 -r 1000 127.0.0.1:46657
Stats          Avg       StdDev     Max
Txs/sec        1826      843        2744
Blocks/sec     1.100     0.300      2
```

AFTER:

```
./tm-bench -T 30 -c 5 -r 1000 127.0.0.1:46657
Stats          Avg       StdDev     Max
Txs/sec        6120      1970       9776
Blocks/sec     1.000     0.000      1
```
2018-05-17 10:00:59 +04:00
1a1e4e767b check max msg size in DecodeMessage 2018-04-09 15:18:47 +03:00
35a1d747b0 Fix mempool 2018-03-31 11:51:32 +02:00
4242352852 stop peer on decoding error 2018-03-15 11:58:19 +04:00
2cc63069c6 rename dummy to kvstore (#1223)
* remove accidental binary

* docs: s/Dummy&dummy/KVStore&kvstore/g

* glide update to abci

* update abci import paths

* dummy begone, hello kvstore

* RequestInitChain needs genesisBytes

* glide update
2018-02-27 18:01:10 +04:00
0ae0155cba restore mempool memory leak tests 2018-02-19 15:34:33 -05:00
2a24ae90c1 fixes from Jae's review
1. remove pointer
2. add Quit() method to Service interface
2018-02-12 14:32:09 +04:00
84a0a1987c comment out tests for now
https://github.com/tendermint/tendermint/pull/1173#issuecomment-363173047
2018-02-05 22:26:14 +04:00
11b68f1934 rewrite broadcastTxRoutine to use channels
https://play.golang.org/p/gN21yO9IRs3

```
func waitWithCancel(f func() *clist.CElement, ctx context.Context) *clist.CElement {
	el := make(chan *clist.CElement, 1)
	select {
	case el <- f():
```
will just run f() blockingly, so this doesn't change much in terms of behavior.
2018-02-05 16:36:26 +04:00
202d9a2c0c fix memory leak in mempool reactor
Leaking goroutine:
```
114 @ 0x42f2bc 0x42f3ae 0x440794 0x4403b9 0x468002 0x9fe32d 0x9ff78f 0xa025ed 0x45e571
```

Explanation:
it blocks on an empty clist forever. so unless theres txs coming in,
this go routine will just sit there, holding onto the peer too.
if we're constantly reconnecting to some peer, old instances are not
garbage collected, leading to memory leak.

Fixes https://github.com/cosmos/gaia/issues/108
Previous attempt https://github.com/tendermint/tendermint/pull/1156
2018-02-05 13:52:18 +04:00
cd0fd06b0d update for sdk2 libs. need to fix kv test
NOTE we only updating for tmlibs and abci
2018-02-03 03:35:02 -05:00
57cc8ab977 Merge pull request #1143 from tendermint/1091-race-condition
call FlushSync before calling CommitSync
2018-01-24 14:22:43 -05:00
260affd037 docs consolidation 2018-01-23 23:46:28 -05:00
5f3048bd09 call FlushSync before calling CommitSync
if we call it after, we might receive a "fresh" transaction from
  `broadcast_tx_sync` before old transactions (which were not
  committed).

Refs #1091

```
Commit is called with a lock on the mempool, meaning no calls to CheckTx
can start. However, since CheckTx is called async in the mempool
connection, some CheckTx might have already "sailed", when the lock is
released in the mempool and Commit proceeds.

Then, that spurious CheckTx has not yet "begun" in the ABCI app (stuck
in transport?). Instead, ABCI app manages to start to process the
Commit. Next, the spurious, "sailed" CheckTx happens in the wrong place.
```
2018-01-23 16:56:14 +04:00
ee674f919f StopPeerForError in blockchain and consensus 2018-01-21 13:32:04 -05:00