358 Commits

Author SHA1 Message Date
Ismail Khoffi
a6011c007d Close and retry a RemoteSigner on err (#2923)
* Close and recreate a RemoteSigner on err

* Update changelog

* Address Anton's comments / suggestions:

 - update changelog
 - restart TCPVal
 - shut down on `ErrUnexpectedResponse`

* re-init remote signer client with fresh connection if Ping fails

- add/update TODOs in secret connection
- rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose

* account for `conn returned by waitConnection can be `nil`

- also add TODO about RemoteSigner conn field

* Tests for retrying: IPC / TCP

 - shorter info log on success
 - set conn and use it in tests to close conn

* Tests for retrying: IPC / TCP

 - shorter info log on success
 - set conn and use it in tests to close conn
 - add rwmutex for conn field in IPC

* comments and doc.go

* fix ipc tests. fixes #2677

* use constants for tests

* cleanup some error statements

* fixes #2784, race in tests

* remove print statement

* minor fixes from review

* update comment on sts spec

* cosmetics

* p2p/conn: add failing tests

* p2p/conn: make SecretConnection thread safe

* changelog

* IPCVal signer refactor

- use a .reset() method
- don't use embedded RemoteSignerClient
- guard RemoteSignerClient with mutex
- drop the .conn
- expose Close() on RemoteSignerClient

* apply IPCVal refactor to TCPVal

* remove mtx from RemoteSignerClient

* consolidate IPCVal and TCPVal, fixes #3104

- done in tcp_client.go
- now called SocketVal
- takes a listener in the constructor
- make tcpListener and unixListener contain all the differences

* delete ipc files

* introduce unix and tcp dialer for RemoteSigner

* rename files

- drop tcp_ prefix
- rename priv_validator.go to file.go

* bring back listener options

* fix node

* fix priv_val_server

* fix node test

* minor cleanup and comments
2019-01-13 14:31:31 -05:00
Ismail Khoffi
6a80412a01 Remove privval.GetAddress(), memoize pubkey (#2948)
privval: remove GetAddress(), memoize pubkey
2018-12-22 00:36:45 -05:00
yutianwu
41e2eeee9c R4R: Split immutable and mutable parts of priv_validator.json (#2870)
* split immutable and mutable parts of priv_validator.json

* fix bugs

* minor changes

* retrig test

* delete scripts/wire2amino.go

* fix test

* fixes from review

* privval: remove mtx

* rearrange priv_validator.go

* upgrade path

* write tests for the upgrade

* fix for unsafe_reset_all

* add test

* add reset test
2018-12-21 16:58:27 -05:00
Ethan Buchman
b3141d7d02
makeNodeInfo returns error (#3029)
* makeNodeInfo returns error

* version and changelog
2018-12-16 14:05:58 -05:00
Jae Kwon
9a6dd96cba Revert to using defers in addrbook. (#3025)
* Revert to using defers in addrbook.  ValidateBasic->Validate since it requires DNS

* Update CHANGELOG_PENDING
2018-12-16 12:27:16 -05:00
Ethan Buchman
222b8978c8
Minor log changes (#2959)
* node: allow state and code to have diff block versions

* node: pex is a log module
2018-12-04 08:30:29 -05:00
Anton Kaliaev
b487feba42
node: refactor privValidator ext client code & tests (#2895)
* update ConsensusState#OnStop comment

* consensus: set logger for WAL in tests

* refactor privValidator client code and tests

follow-up on https://github.com/tendermint/tendermint/pull/2866
2018-11-21 21:24:13 +04:00
Joe Bowman
72f86b5192 [pv] add ability to use ipc validator (#2866)
Ref #2827

(I have since seen #2847 which is a fix for the same issue; this PR has tests and docs too ;) )
2018-11-21 10:45:20 +04:00
Ethan Buchman
6168b404a7
p2p: NewMultiplexTransport takes an MConnConfig (#2869)
* p2p: NewMultiplexTransport takes an MConnConfig

* changelog

* move test func to test file
2018-11-17 03:16:49 -05:00
Alessio Treglia
b646437ec7 Decouple StartHTTP{,AndTLS}Server from Listen() (#2791)
* Decouple StartHTTP{,AndTLS}Server from Listen()

This should help solve cosmos/cosmos-sdk#2715

* Fix small mistake

* Update StartGRPCServer

* s/rpc/rpcserver/

* Start grpccore.StartGRPCServer in a goroutine

* Reinstate l.Close()

* Fix rpc/lib/test/main.go

* Update code comment

* update changelog and comments

* fix tm-monitor. more comments
2018-11-15 15:33:04 -05:00
Hleb Albau
6353862ac0 2582 Enable CORS on RPC API (#2800) 2018-11-14 16:47:41 +04:00
Ismail Khoffi
3ff820bdf4 fix amino overhead computation for Tx (#2792)
* fix amino overhead computation for Tx:

- also count the fieldnum / typ3
- add method to compute overhead per Tx
- slightly clarify comment on MaxAminoOverheadForBlock
- add tests

* fix TestReapMaxBytesMaxGas according to amino overhead

* fix TestMempoolFilters according to amino overhead

* address review comments:

 - add a note about fieldNum = 1
 - add forgotten godoc comment

* fix and use sm.TxPreCheck

* fix test

* remove print statement
2018-11-11 10:09:33 -05:00
Anton Kaliaev
b8a9b0bf78 Mempool WAL is still created by default in home directory, leads to permission errors (#2758)
* only invoke InitWAL/CloseWAL if WalPath is not empty

Closes #2717

* panic if WAL is not initialized when calling CloseWAL

* add a changelog entry
2018-11-05 22:39:05 -08:00
Ethan Buchman
fe1d59ab7b
Set protocol versions in NodeInfo from state (#2686)
* use types.NewValidator

* node: set p2p.ProtocolVersion from state, not globals
2018-10-22 17:55:49 -04:00
Ethan Buchman
ed4ce5ff6c
ADR-016: Update ABCI Info method for versions (#2662)
* abci: update RequestInfo for versions

* abci: update ResponseInfo for versions

* compile fix

* fix test

* software_version -> version

* comment fix

* update spec

* add test

* comments and fix test
2018-10-18 16:51:17 -04:00
Ethan Buchman
14c1baeb24
ADR-016: Add protocol Version to NodeInfo (#2654)
* p2p: add protocol Version to NodeInfo

* update node pkg. remove extraneous version files

* update changelog and docs

* fix test

* p2p: Version -> ProtocolVersion; more ValidateBasic and tests
2018-10-18 10:29:59 -04:00
Ethan Buchman
455d34134c
ADR-016: Add versions to Block and State (#2644)
* types: add Version to Header

* abci: add Version to Header

* state: add Version to State

* node: check software and state protocol versions match

* update changelog

* docs/spec: update for versions

* state: more tests

* remove TODOs

* remove empty test
2018-10-17 15:30:53 -04:00
Hendrik Hofstadt
f60713bca8 privval: Add IPCPV and fix SocketPV (#2568)
Ref: #2563

I added IPC as an unencrypted alternative to SocketPV.

Besides I fixed the following aspects of SocketPV:

Added locking since we are operating on a single socket
The connection deadline is extended every time a successful packet exchange happens; otherwise the connection would always die permanently x seconds after the connection was established.
Added a ping/heartbeat mechanism to keep the connection alive; native TCP keepalives do not work in this use-case


* Extend the SecureConn socket to extend its deadline
* Add locking & ping/heartbeat packets to SocketPV
* Implement IPC PV and abstract socket signing
* Refactored IPC and SocketPV
* Implement @melekes comments
* Fixes to rebase
2018-10-17 10:26:14 +02:00
Ethan Buchman
0baa7588c2
p2p: NodeInfo is an interface; General cleanup (#2556)
* p2p: NodeInfo is an interface

* (squash) fixes from review

* (squash) more fixes from review

* p2p: remove peerConn.HandshakeTimeout

* p2p: NodeInfo is two interfaces. Remove String()

* fixes from review

* remove test code from peer.RemoteIP()

* p2p: remove peer.OriginalAddr(). See #2618

* use a mockPeer in peer_set_test.go

* p2p: fix testNodeInfo naming

* p2p: remove unused var

* remove testRandNodeInfo

* fix linter

* fix retry dialing self

* fix rpc
2018-10-12 19:25:33 -04:00
Zarko Milosevic
2363d88979 consensus: Wait for proposal or timeout before prevote (#2540)
* Fix termination issues and improve tests

* Improve formatting and tests based on reviewer feedback
2018-10-12 16:13:01 -04:00
Matthew Slipper
92343ef484 Add additional metrics (#2500)
* Add additional metrics
Continues addressing https://github.com/cosmos/cosmos-sdk/issues/2169.
* Add nop metrics to fix NPE
* Tweak buckets, code review
* Update buckets
* Update docs with new metrics
* Code review updates
2018-10-10 18:27:43 +02:00
Ethan Buchman
52e21cebcf
remove some xxx comments and the config.mempool.recheck_empty (#2505)
* remove some XXX

* config: remove Mempool.RecheckEmpty

* docs: remove recheck_empty
2018-09-30 13:28:34 -04:00
Matthew Slipper
587116dae1 metrics: Add additional metrics to p2p and consensus (#2425)
* Add additional metrics to p2p and consensus
Partially addresses https://github.com/cosmos/cosmos-sdk/issues/2169.
* WIP
* Updates from code review
* Updates from code review
* Add instrumentation namespace to configuration
* Fix test failure
* Updates from code review
* Add quotes
* Add atomic load
* Use storeint64
* Use addInt64 in writePacketMsgTo
2018-09-25 13:14:38 +02:00
Anton Kaliaev
d12e55c494 node: Respond always to OS interrupts (#2479)
* stop node upon receiving SIGTERM or CTRL-Ceven during genesis sleep by setting up interrupt before starting a node

Closes #2434

* call Start, not OnStart when starting a component to avoid:

```
E[09-24|10:13:15.805] Not stopping PubSub -- have not been started yet module=pubsub impl=PubSub
```

being printed on exit
2018-09-25 12:24:18 +02:00
Dev Ojha
111e627037 mempool: Filter new txs if they have insufficient gas (#2385)
This also refactors the prior mempool to filter to be known as
"precheck filter" and this new filter is called "postcheck filter"

This PR also fixes a bug where the precheck filter previously didn't
account for the amino overhead, which could a maliciously sized tx to
halt blocks from getting any txs in them.

* Move maxGas outside of function definition to avoid race condition
* Type filter funcs and make public
* Use helper method for post check
* Remove superfluous Filter suffix
* Move default pre/post checks into package
* Fix broken references
* Fix typos
* Expand on examples for checks
2018-09-22 02:50:06 +02:00
Anton Kaliaev
8d50bb9dad conesnsu: follow up to removing some consensus params (#2427)
* follow up to removing some consensus params Refs #2382
* change args type to int64 in state#makeParams
* make valsCount and evidenceCount ints again
* MaxEvidenceBytesPerBlock: include magic number in godoc
* [spec] creating a proposal
* test state#TxFilter
* panic if MaxDataBytes is less than 0
* fixes after review
* use amino#UvarintSize to calculate overhead
0c74291f3b/encoder.go (L85-L90)
* avoid cyclic imports
* you can do better Go, come on
* remove testdouble package
2018-09-21 11:00:36 +02:00
Anton Kaliaev
aa5495f24e p2p: add RPCAddress to NodeInfoOther.String() (#2442)
- remove second AminoVersion from
https://github.com/tendermint/tendermint/pull/2426/files#diff-f2fefc7a06ea0d1e0a910196901e50aaR86

Refs #2433
2018-09-19 15:54:14 +02:00
Ethan Buchman
91a8767083 proxy: remove Handshaker from proxy pkg (#2437)
Handshaker was removed from proxy package so it can be called
independently of starting the abci app connections and can return a
result to the caller.
2018-09-19 15:35:09 +02:00
Alexander Simmerl
bdd01310a0 p2p: Integrate new Transport
We are swapping the exisiting listener implementation with the newly
introduced Transport and its default implementation MultiplexTransport,
removing a large chunk of old connection setup and handling scattered
over the Peer and Switch code. The Switch requires a Transport now and
handles externally passed Peer filters.
2018-09-18 22:26:43 +02:00
Zarko Milosevic
2fbf810cd8 Delay starting node until Genesis time (#2389) 2018-09-18 13:16:50 +04:00
Anton Kaliaev
fc7f9bcaf6 rpc: Transform /status result.node_info.other into map (#2417)
* [rpc] transform /status result.node_info.other into map
* amino does not support maps, duh

Refs #2391
2018-09-17 18:39:52 +02:00
Anton Kaliaev
0e1cd88863 Remove ConsensusParams.TxSize and ConsensusParams.BlockGossip (#2364)
* remove ConsensusParams.TxSize and ConsensusParams.BlockGossip

Refs #2347

* block part size is now fixed

Refs #2347

* use max data size, not max bytes for tx limit

Refs #2347
2018-09-12 15:44:43 -04:00
JamesRay
d0bb1ab2b0 Filter out empty addresses in persistent_peers/seeds lists (#2323)
Fixes #2320
2018-09-05 10:13:25 +04:00
Anton Kaliaev
166fd82b70
max-bytes PR follow-up (#2318)
* ReapMaxTxs: return all txs if max is negative

this mirrors ReapMaxBytes behavior

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214439950

* increase MaxAminoOverheadForBlock

tested with:

```
func TestMaxAminoOverheadForBlock(t *testing.T) {

        maxChainID := ""
        for i := 0; i < MaxChainIDLen; i++ {
                maxChainID += "𠜎"
        }

        h := Header{
                ChainID:            maxChainID,
                Height:             10,
                Time:               time.Now().UTC(),
                NumTxs:             100,
                TotalTxs:           200,
                LastBlockID:        makeBlockID(make([]byte, 20), 300, make([]byte, 20)),
                LastCommitHash:     tmhash.Sum([]byte("last_commit_hash")),
                DataHash:           tmhash.Sum([]byte("data_hash")),
                ValidatorsHash:     tmhash.Sum([]byte("validators_hash")),
                NextValidatorsHash: tmhash.Sum([]byte("next_validators_hash")),
                ConsensusHash:      tmhash.Sum([]byte("consensus_hash")),
                AppHash:            tmhash.Sum([]byte("app_hash")),
                LastResultsHash:    tmhash.Sum([]byte("last_results_hash")),
                EvidenceHash:       tmhash.Sum([]byte("evidence_hash")),
                ProposerAddress:    tmhash.Sum([]byte("proposer_address")),
        }
        b := Block{
                Header:     h,
                Data:       Data{Txs: makeTxs(10000, 100)},
                Evidence:   EvidenceData{},
                LastCommit: &Commit{},
        }

        bz, err := cdc.MarshalBinary(b)
        require.NoError(t, err)

        assert.Equal(t, MaxHeaderBytes+MaxAminoOverheadForBlock-2, len(bz)-1000000-20000-1)
}
```

* fix MaxYYY constants calculation

by using math.MaxInt64

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214444244

* pass mempool filter as an option

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214445869

* fixes after Dev's comments
2018-09-04 11:46:34 +04:00
Anton Kaliaev
d73c5cbdb1
reap max bytes from the mempool & check transaction size
See ADR 020: Limiting txs size inside a block docs/architecture/adr-020-block-size.md

Refs #2035
2018-08-31 16:01:21 +04:00
bradyjoestar
5446452b01 pass in NodeKey to NewNode (#2212)
Fixes #1544
2018-08-15 12:29:45 +04:00
Dev Ojha
2756be5a59 libs: Remove usage of custom Fmt, in favor of fmt.Sprintf (#2199)
* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!
2018-08-10 09:25:57 +04:00
Anton Kaliaev
75a26ebd6d
do not overwrite metrics provider in node#NewNode
also, make running Prometheus server optional.

Closes #2019
2018-07-24 16:01:11 +04:00
ValarDragon
c798702764 crypto: Remove Ed25519 and Secp256k1 suffix on GenPrivKey 2018-07-20 10:44:21 -07:00
ValarDragon
571e602f07 Merge remote-tracking branch 'origin/develop' into dev/refactor_crypto 2018-07-18 08:54:51 -07:00
ValarDragon
99e582d79a crypto: Refactor to move files out of the top level directory
Currently the top level directory contains basically all of the code
for the crypto package. This PR moves the crypto code into submodules
in a similar manner to what `golang/x/crypto` does. This improves code
organization.

Ref discussion: https://github.com/tendermint/tendermint/pull/1966

Closes #1956
2018-07-18 08:38:44 -07:00
Jeremiah Andrews
6c4ca140ed Add private peer ID tracking to AddrBook (#1989)
* Add private peer ID tracking to AddrBook

* Remove private peer tracking/blocking from pex

* debug level msg when we fail to add private address
2018-07-18 13:22:09 +04:00
Ethan Buchman
d903057011 fix stopping pubsub 2018-07-14 14:50:56 +01:00
Anton Kaliaev
4de9d42e4c
limit the number of requests in flights for Prometheus server (#1927)
* limit the number of requests in flights for Prometheus server

Closes #1804

Default to 1 because usually there's just one collector.

* config: Up default for prom connections
2018-07-10 15:49:48 +04:00
Ethan Buchman
737c5c065d fixes from review 2018-07-02 12:46:02 -04:00
Ethan Buchman
1c018d3fd2 p2p: external address
* new config option for external address to advertise
* if blank, defaults to best guess from listener
* if laddr ip address is also blank, default to IPv4
2018-07-02 12:44:48 -04:00
Ethan Buchman
d55243f0e6 fix import paths 2018-07-01 22:36:49 -04:00
Anton Kaliaev
ada5ef0669
[p2p] add godoc comments to Listener (#1828)
* update changelog

* document p2p/listener

* do not expose underlying net.Listener

* add a TODO

* replace todo with a comment
2018-06-29 11:48:52 +04:00
kph7
e556e3336e Adding IPv6 support to peering (#1818) 2018-06-28 11:29:27 +04:00
Dev Ojha
b1d6deaf0b config: rename skip_upnp to upnp (#1827)
* config: rename skip_upnp to upnp

Change default option to enable upnp.

Closes #1806

* doc updates

- fix comment and set UPNP to false in TestP2PConfig
- add UPNP to config template
- update changelog
2018-06-28 11:09:39 +04:00