Compare commits

...

164 Commits

Author SHA1 Message Date
Ethan Buchman
0d985ede28 Merge pull request #3417 from tendermint/release/v0.31.0
Release/v0.31.0
2019-03-19 19:53:37 -04:00
Ismail Khoffi
1e3469789d Ensure WriteTimeout > TimeoutBroadcastTxCommit (#3443)
* Make sure config.TimeoutBroadcastTxCommit < rpcserver.WriteTimeout()

* remove redundant comment

* libs/rpc/http_server: move Read/WriteTimeout into Config

* increase defaults for read/write timeouts

Based on this article
https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration

* WriteTimeout should be larger than TimeoutBroadcastTxCommit

* set a deadline for subscribing to txs

* extract duration into const

* add two changelog entries

* Update CHANGELOG_PENDING.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* Update CHANGELOG_PENDING.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* 12 -> 10

* changelog

* changelog
2019-03-19 19:45:51 -04:00
Ethan Buchman
5f68fbae37 Merge pull request #3449 from tendermint/ismail/merge_develop_into_release/0.31.0
Merge develop into release/0.31.0
2019-03-19 19:25:26 -04:00
Ismail Khoffi
e276f35f86 remove 3421 from changelog 2019-03-19 14:36:42 +01:00
Ismail Khoffi
8e62a3d62a Add #3421 to changelog and reorder alphabetically 2019-03-19 12:19:02 +01:00
Ismail Khoffi
48aaccab8f Merge in develop and update CHANGELOG.md 2019-03-19 12:09:26 +01:00
Anton Kaliaev
4162ebe8b5 types: refactor PB2TM.ConsensusParams to take BlockTimeIota as an arg (#3442)
See https://github.com/tendermint/tendermint/pull/3403/files#r266208947

In #3403 we unexposed BlockTimeIota from the ABCI, but it's still part
of the ConsensusParams struct, so we have to remember to add it back
after calling PB2TM.ConsensusParams. Instead, PB2TM.ConsensusParams
should take it as an argument

Fixes #3432
2019-03-19 11:38:32 +04:00
Ethan Buchman
551b6322f5 Update v0.31.0 release notes (#3434)
* changelog: fix formatting

* update release notes

* update changelog

* linkify

* update UPGRADING
2019-03-16 19:24:12 -04:00
Ismail Khoffi
52c4e15eb2 changelog: more review fixes/release/v0.31.0 (#3427)
* Update release summary

* Add pubsub config changes

* Add link to issue for pubsub changes
2019-03-14 19:07:06 +04:00
Ismail Khoffi
5483ac6b0a minor changes / fixes to release 0.31.0 (#3422)
* bump ABCIVersion due to renaming BlockSizeParams -> BlockParams
(https://github.com/tendermint/tendermint/pull/3417#discussion_r264974791)

* Move changelog on consensus params entry to breaking

* Add @melekes' suggestion for breaking change in pubsub into upgrading.md

* Add changelog entry for #3351

* Add changelog entry for #3358 & #3359

* Add changelog entry for #3397

* remove changelog entry for #3397 (was already released in 0.30.2)

* move 3351 to improvements

* Update changelog comment
2019-03-14 15:17:49 +04:00
Anton Kaliaev
7457133307 grpcdb: close Iterator/ReverseIterator after use (#3424)
Fixes #3402
2019-03-14 15:00:58 +04:00
Anton Kaliaev
a59930a327 localnet: fix $LOG variable (#3423)
Fixes #3421

Before: it was creating a file named ${LOG:-tendermint.log} in .build/nodeX
After: it creates a file named tendermint.log
2019-03-13 16:09:05 +04:00
Ismail Khoffi
85c023db88 Prep release v0.31.0:
- update changelog, reset pending
 - bump versions
 - add external contributors (partly manually)
2019-03-12 20:07:26 +01:00
Anton Kaliaev
4cbd36f341 Merge pull request #3415 from tendermint/master
Merge master back to develop (do not squash)
2019-03-12 21:22:03 +04:00
Anton Kaliaev
e42f833fd4 Merge master back to develop (#3412)
* libs/db: close batch (#3397)

ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842

* update changelog and bump version to 0.30.2
2019-03-12 16:20:59 +04:00
Anton Kaliaev
ad3e990c6a fix GO_VERSION in installation scripts (#3411)
there is no such file https://storage.googleapis.com/golang/go1.12.0.linux-amd64.tar.gz

Fixes #3405
2019-03-11 23:59:00 +04:00
srmo
676212fa8f cmd: make sure to have 'testnet' create the data directory for nonvals (#3409)
Fixes #3408
2019-03-11 23:06:03 +04:00
Anton Kaliaev
3035572034 cs: comment out log.Error to avoid TestReactorValidatorSetChanges timing out (#3401) 2019-03-11 22:52:09 +04:00
Anton Kaliaev
d741c7b478 limit number of /subscribe clients and queries per client (#3269)
* limit number of /subscribe clients and queries per client

Add the following config variables (under [rpc] section):
  * max_subscription_clients
  * max_subscriptions_per_client
  * timeout_broadcast_tx_commit

Fixes #2826

new HTTPClient interface for subscriptions

finalize HTTPClient events interface

remove EventSubscriber

fix data race

```
WARNING: DATA RACE
Read at 0x00c000a36060 by goroutine 129:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0

Previous write at 0x00c000a36060 by goroutine 132:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 129 (running) created at:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 132 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
```

lite client works (tested manually)

godoc comments

httpclient: do not close the out channel

use TimeoutBroadcastTxCommit

no timeout for unsubscribe

but 1s Local (5s HTTP) timeout for resubscribe

format code

change Subscribe#out cap to 1

and replace config vars with RPCConfig

TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout

rpc: Context as first parameter to all functions

reformat code

fixes after my own review

fixes after Ethan's review

add test stubs

fix config.toml

* fixes after manual testing

- rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes
Tendermint resources (pubsub)
- rpc: better error in Subscribe and BroadcastTxCommit
- HTTPClient: do not resubscribe if err = ErrAlreadySubscribed

* fixes after Ismail's review

* Update rpc/grpc/grpc_test.go

Co-Authored-By: melekes <anton.kalyaev@gmail.com>
2019-03-11 22:45:58 +04:00
Anton Kaliaev
15f621141d remove TimeIotaMs from ABCI consensus params (#3403)
Also

- init substructures to avoid panic in pb2tm.ConsensusParams
Before: if csp.Block is nil and we later try to access/write to it,
we'll panic.
After: if csp.Block is nil and we later try to access/write to it,
there'll be no panic.
2019-03-11 22:21:17 +04:00
Anca Zamfir
dc359bd3a5 types: remove check for priority order of existing validators (#3407)
When scaling and averaging is invoked, it is possible to have validators
with close priorities ending up with same priority. With the current code,
this  makes it impossible to verify the priority orders before and after updates.

Fixes #3383
2019-03-11 18:17:25 +04:00
Ethan Buchman
976819537d Merge pull request #3399 from tendermint/release/v0.30.2
Release/v0.30.2
2019-03-11 08:17:14 -04:00
Anton Kaliaev
100ff08de9 p2p: do not panic when filter times out (#3384)
Fixes #3369
2019-03-11 15:31:53 +04:00
Anton Kaliaev
f996b10f47 update changelog and bump version to 0.30.2 2019-03-10 13:06:34 +04:00
Yumin Xia
36d7180ca2 libs/db: close batch (#3397)
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842
2019-03-10 12:56:04 +04:00
Yumin Xia
b021f1e505 libs/db: close batch (#3397)
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842
2019-03-10 12:46:32 +04:00
mircea-c
90794260bc circleci: removed complexity from docs deployment job (#3396) 2019-03-09 19:13:36 +04:00
Anton Kaliaev
b6a510a3e7 make ineffassign linter pass (#3386)
Refs #3262

This fixes two small bugs:

1) lite/dbprovider: return `ok` instead of true in parse* functions. It's weird that we're ignoring `ok` value before.
2) consensus/state: previously because of the shadowing we almost never output "Error with msg". Now we declare both `added` and `err` in the beginning of the function, so there's no shadowing.
2019-03-08 09:46:09 +04:00
Ismail Khoffi
e415c326f9 update golang.org/x/crypto (#3392)
Update Gopkg.lock via dep ensure --update golang.org/x/crypto

see #3391 (comment) (nothing to review here really).
2019-03-08 09:40:59 +04:00
Ismail Khoffi
28e9e9e714 update levigo to 1.0.0 (#3389)
Although the version we were pinning to is from Nov. 2016 there were no substantial changes:

    jmhodges/levigo@2b8c778 added go-modules support (no code changes)
    jmhodges/levigo@853d788 added a badge to the readme

closes #3381
2019-03-07 20:03:57 +04:00
Anton Kaliaev
3ebfa99f2c do not pin repos without releases to exact revisions (#3382)
We're pinning repos without releases because it's very easy to upgrade all the dependencies by executing dep ensure --upgrade. Instead, we should just never run this command directly, only dep ensure --upgrade <some repo>. And we can defend that in PRs.

Refs #3374

The problem with pinning to exact revisions: people who import Tendermint as a library (e.g. abci/types) are stuck with these revisions even though the code they import may not even use them.
2019-03-07 19:35:04 +04:00
YOSHIDA Masanori
91b488f9a5 docs: fix the reverse of meaning in spec (#3387)
https://tools.ietf.org/html/rfc6962#section-2.1
"The largest power of two less than the number of items" is actually correct!

    For n > 1, let k be the largest power of two smaller than n (i.e.,
    k < n <= 2k).
2019-03-07 17:02:13 +04:00
Anton Kaliaev
f25d727035 make dupl linter pass (#3385)
Refs #3262
2019-03-07 09:10:34 +04:00
Anca Zamfir
411bc5e49f types: followup after validator set changes (#3301)
* fix failure in TestProposerFrequency

* Add test to check priority order after updates

* Changed applyRemovals() and removed Remove()

Changed applyRemovals() similar to applyUpdates()
Removed function Remove()
Updated comments

* review comments

* simplify applyRemovals and add more comments

* small correction in comment

* Fix check in test

* Fix priority check for centering, address review comments

* fix assert for priority centering

* review comments

* review comments

* cleanup and review comments

added upper limit check for validator voting power
moved check for empty validator set earlier
moved panic on potential negative set length in verifyRemovals
added more tests

* review comments
2019-03-06 12:54:49 +04:00
Silas Davis
858875fbb8 Copy secp256k1 code from go-ethereum to avoid GPL vendoring issues in (#3371)
downstream

Signed-off-by: Silas Davis <silas@monax.io>
2019-03-06 12:22:35 +04:00
Ismail Khoffi
1eaa42cd25 golang 1.12.0 (#3376)
- update docker image on circleci
 - remove GOCACHE=off from Makefile (see:
 https://tip.golang.org/doc/go1.12#gocache)
 - update badge in readme
 - update in scripts/install
 - update Vagrantfile
 - update in networks/remote/integration.sh
 - tools/build/Makefile
2019-03-05 11:08:52 +04:00
Jack Zampolin
8c9df30e28 libs/db: Add cleveldb.Stats() (#3379)
Fixes: #3378

* Add stats to cleveldb implementation

* update changelog

* remote TODO

also
- sort keys
- preallocate memory

* fix const initializer []string literal is not a constant

* add test
2019-03-05 10:56:46 +04:00
Anton Kaliaev
52771e1287 make BlockTimeIota a consensus parameter, not a locally configurable … (#3048)
* make BlockTimeIota a consensus parameter, not a locally configurable option

Refs #2920

* make TimeIota int64 ms

Refs #2920

* update Gopkg.toml

* fixes after Ethan's review

* fix TestRemoteSignerProposalSigningFailed

* update changelog
2019-03-04 13:24:44 +04:00
Anton Kaliaev
f39138aa2e remove RoundState from EventDataRoundState (#3354)
Before we're using it to get a round state in tests. Now it can be done
by calling csX.GetRoundState. We will need to rewrite
TestStateSlashingPrevotes and TestStateSlashingPrecommits, which are
commented right now, to not rely on EventDataRoundState#RoundState
field.

Refs #1527
2019-03-04 12:18:32 +04:00
Anton Kaliaev
8a962ffc46 deps: update gogo/protobuf from 1.1.1 to 1.2.1 and golang/protobuf from 1.1.0 to 1.3.0 (#3357)
* deps: update gogo/proto from 1.1.1 to 1.2.1

- verified changes manually
  git diff 636bf030~ ba06b47c --stat -- ':!*.pb.go' ':!test'

* deps: update golang/protobuf from 1.1.0 to 1.3.0

- verified changes manually
  git diff b4deda0~ c823c79 -- ':!*.pb.go' ':!test'
2019-03-04 12:17:38 +04:00
zjubfd
3421e4dcd7 make lightd availbe (#3364)
1."abci_query": rpcserver.NewRPCFunc(c.ABCIQuery, "path,data,prove")
"validators": rpcserver.NewRPCFunc(c.Validators, "height"),
the parameters and function do not match, cause index out of range error.
2. the prove of query is forced to be true, while default option is false.
3. fix the wrong key of merkle
2019-03-02 16:36:52 -05:00
zjubfd
976b1c2ef7 fix pool timer leak bug, resolve#3353 (#3358)
When remove peer, block pool simple remove bpPeer,
but do not stop timer, that cause stopError for recorrected
peers. Stop timer when remove from pool.
2019-03-02 15:21:21 -05:00
zjubfd
d95894152b fix dirty data in peerset,resolve #3304 (#3359)
* fix dirty data in peerset

startInitPeer before PeerSet add the peer,
once mconnection start and Receive of one
Reactor faild, will try to remove it from PeerSet
while PeerSet still not contain the peer. Fix
this by change the order.

* fix test FilterDuplicate

* fix start/stop race

* fix err
2019-03-02 15:17:37 -05:00
Sven-Hendrik Haase
37a548414b docs: fix typo (#3373) 2019-03-02 10:06:57 +04:00
Juan Leni
853dd34d31 privval: improve Remote Signer implementation (#3351)
This issue is related to #3107
This is a first renaming/refactoring step before reworking and removing heartbeats.
As discussed with @Liamsi , we preferred to go for a couple of independent and separate PRs to simplify review work.

The changes:

    Help to clarify the relation between the validator and remote signer endpoints
    Differentiate between timeouts and deadlines
    Prepare to encapsulate networking related code behind RemoteSigner in the next PR

My intention is to separate and encapsulate the "network related" code from the actual signer.

SignerRemote ---(uses/contains)--> SignerValidatorEndpoint <--(connects to)--> SignerServiceEndpoint ---> SignerService (future.. not here yet but would like to decouple too)

All reconnection/heartbeat/whatever code goes in the endpoints. Signer[Remote/Service] do not need to know about that.

I agree Endpoint may not be the perfect name. I tried to find something "Go-ish" enough. It is a common name in go-kit, kubernetes, etc.

Right now:
SignerValidatorEndpoint:

    handles the listener
    contains SignerRemote
    Implements the PrivValidator interface
    connects and sets a connection object in a contained SignerRemote
    delegates PrivValidator some calls to SignerRemote which in turn uses the conn object that was set externally

SignerRemote:

    Implements the PrivValidator interface
    read/writes from a connection object directly
    handles heartbeats

SignerServiceEndpoint:

    Does most things in a single place
    delegates to a PrivValidator IIRC.

* cleanup

* Refactoring step 1

* Refactoring step 2

* move messages to another file

* mark for future work / next steps

* mark deprecated classes in docs

* Fix linter problems

* additional linter fixes
2019-02-28 11:48:20 +04:00
Anton Kaliaev
d6e2fb453d update docs (#3349)
* docs: explain create_empty_blocks configurations

Closes #3307

* Vagrantfile: install nodejs for docs

* update docs instructions

npm install does not make sense since there's no packages.json file

* explain broadcast_tx_* tx format

Closes #536

* docs: explain how transaction ordering works

Closes #2904

* bring in consensus parameters explained

* example for create_empty_blocks_interval

* bring in explanation from https://github.com/tendermint/tendermint/issues/2487#issuecomment-424899799

* link to formatting instead of duplicating info
2019-02-28 11:31:59 +04:00
Anton Kaliaev
ec9bff5234 rename WAL#Flush to WAL#FlushAndSync (#3345)
* rename WAL#Flush to WAL#FlushAndSync

- rename auto#Flush to auto#FlushAndSync
- cleanup WAL interface to not leak implementation details!
  * remove Group()
  * add WALReader interface and return it in SearchForEndHeight()
- add interface assertions

Refs #3337

* replace WALReader with io.ReadCloser
2019-02-25 09:11:07 +04:00
Ismail Khoffi
6797d85851 p2p: fix comment in secret connection (#3348)
Just a minor followup on the review if #3347: Fixes a comment. [#3347 (comment)](https://github.com/tendermint/tendermint/pull/3347#discussion_r259582330)
2019-02-25 09:06:21 +04:00
Anton Kaliaev
cdf3a74f48 Unclean shutdown on SIGINT / SIGTERM (#3308)
* libs/common: TrapSignal accepts logger as a first parameter

 and does not block anymore
* previously it was dumping "captured ..." msg to os.Stdout
* TrapSignal should not be responsible for blocking thread of execution

Refs #3238

* exit with zero (0) code upon receiving SIGTERM/SIGINT

Refs #3238

* fix formatting in docs/app-dev/abci-cli.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* fix formatting in docs/app-dev/abci-cli.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>
2019-02-23 10:48:28 -05:00
Anton Kaliaev
41f91318e9 bound mempool memory usage (#3248)
* bound mempool memory usage

Closes #3079

* rename SizeBytes to TxsTotalBytes

and other small fixes after Zarko's review

* rename MaxBytes to MaxTxsTotalBytes

* make ErrMempoolIsFull more informative

* expose mempool's txs_total_bytes via RPC

* test full response

* fixes after Ethan's review

* config: rename mempool.size to mempool.max_txs

https://github.com/tendermint/tendermint/pull/3248#discussion_r254034004

* test more cases

https://github.com/tendermint/tendermint/pull/3248#discussion_r254036532

* simplify test

* Revert "config: rename mempool.size to mempool.max_txs"

This reverts commit 39bfa3696177aa46195000b90655419a975d6ff7.

* rename count back to n_txs

to make a change non-breaking

* rename max_txs_total_bytes to max_txs_bytes

* format code

* fix TestWALPeriodicSync

The test was sometimes failing due to processFlushTicks being called too
early. The solution is to call wal#Start later in the test.

* Apply suggestions from code review
2019-02-23 10:32:31 -05:00
Ismail Khoffi
e0adc5e807 secret connection check all zeroes (#3347)
* reject the shared secret if is all zeros in case the blacklist was not
sufficient

* Add test that verifies lower order pub-keys are rejected at the DH step

* Update changelog

* fix typo in test-comment
2019-02-23 10:25:57 -05:00
Anton Kaliaev
2137ecc130 pubsub 2.0 (#3227)
* green pubsub tests :OK:

* get rid of clientToQueryMap

* Subscribe and SubscribeUnbuffered

* start adapting other pkgs to new pubsub

* nope

* rename MsgAndTags to Message

* remove TagMap

it does not bring any additional benefits

* bring back EventSubscriber

* fix test

* fix data race in TestStartNextHeightCorrectly

```
Write at 0x00c0001c7418 by goroutine 796:
  github.com/tendermint/tendermint/consensus.TestStartNextHeightCorrectly()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1296 +0xad
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Previous read at 0x00c0001c7418 by goroutine 858:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1631 +0x1366
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1476 +0x8f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:667 +0xa1e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:628 +0x794

Goroutine 796 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1117 +0x4ee
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:214 +0x332

Goroutine 858 (running) created at:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:334 +0x221
  github.com/tendermint/tendermint/consensus.startTestRound()
      /go/src/github.com/tendermint/tendermint/consensus/common_test.go:122 +0x63
  github.com/tendermint/tendermint/consensus.TestStateFullRound1()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:255 +0x397
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
```

* fixes after my own review

* fix formatting

* wait 100ms before kicking a subscriber out

+ a test for indexer_service

* fixes after my second review

* no timeout

* add changelog entries

* fix merge conflicts

* fix typos after Thane's review

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* reformat code

* rewrite indexer service in the attempt to fix failing test

https://github.com/tendermint/tendermint/pull/3227/#issuecomment-462316527

* Revert "rewrite indexer service in the attempt to fix failing test"

This reverts commit 0d9107a098.

* another attempt to fix indexer

* fixes after Ethan's review

* use unbuffered channel when indexing transactions

Refs https://github.com/tendermint/tendermint/pull/3227#discussion_r258786716

* add a comment for EventBus#SubscribeUnbuffered

* format code
2019-02-22 23:11:27 -05:00
Anton Kaliaev
67fd428354 fix TestWALPeriodicSync (#3342)
The test was sometimes failing due to processFlushTicks being called too
early. The solution is to call wal#Start later in the test.
2019-02-22 11:49:08 +04:00
Anton Kaliaev
f22ada442a refactor decideProposal in common_test (#3343) 2019-02-22 11:48:40 +04:00
Anton Kaliaev
ed1de13548 cs: update wal comments (#3334)
* cs: update wal comments

Follow-up to https://github.com/tendermint/tendermint/pull/3300

* Update consensus/wal.go

Co-Authored-By: melekes <anton.kalyaev@gmail.com>
2019-02-21 18:28:02 +04:00
Ethan Buchman
4f83eec782 Merge pull request #3339 from tendermint/master
Merge master back to develop
2019-02-20 10:11:33 -05:00
Ethan Buchman
e0f8936455 Merge pull request #3326 from tendermint/release/v0.30.1
Release/v0.30.1
2019-02-20 10:10:49 -05:00
Ethan Buchman
f2351dc758 update changelog 2019-02-20 10:05:57 -05:00
Daniil Lashin
db5d7602fe docs: fix rpc Tx() method docs (#3331) 2019-02-20 15:34:52 +04:00
Thane Thomson
dff3deb2a9 cs: sync WAL more frequently (#3300)
As per #3043, this adds a ticker to sync the WAL every 2s while the WAL is running.

* Flush WAL every 2s

This adds a ticker that flushes the WAL every 2s while the WAL is
running. This is related to #3043.

* Fix spelling

* Increase timeout to 2mins for slower build environments

* Make WAL sync interval configurable

* Add TODO to replace testChan with more comprehensive testBus

* Remove extraneous debug statement

* Remove testChan in favour of using system time

As per
https://github.com/tendermint/tendermint/pull/3300#discussion_r255886586,
this removes the `testChan` WAL member and replaces the approach with a
system time-oriented one. In this new approach, we keep track of the
system time at which each flush and periodic flush successfully
occurred.

The naming of the various functions is also updated here to be more
consistent with "flushing" as opposed to "sync'ing".

* Update naming convention and ensure lock for timestamp update

* Add Flush method as part of WAL interface

Adds a `Flush` method as part of the WAL interface to enforce the idea
that we can manually trigger a WAL flush from outside of the WAL. This
is employed in the consensus state management to flush the WAL prior to
signing votes/proposals, as per https://github.com/tendermint/tendermint/issues/3043#issuecomment-453853630

* Update CHANGELOG_PENDING

* Remove mutex approach and replace with DI

The dependency injection approach to dealing with testing concerns could
allow similar effects to some kind of "testing bus"-based approach. This
commit introduces an example of this, where instead of relying on
(potentially fragile) timing of things between the code and the test, we
inject code into the function under test that can signal the test
through a channel.

This allows us to avoid the `time.Sleep()`-based approach previously
employed.

* Update comment on WAL flushing during vote signing

Co-Authored-By: thanethomson <connect@thanethomson.com>

* Simplify flush interval definition

Co-Authored-By: thanethomson <connect@thanethomson.com>

* Expand commentary on WAL disk flushing

Co-Authored-By: thanethomson <connect@thanethomson.com>

* Add broken test to illustrate WAL sync test problem

Removes test-related state (dependency injection code) from the WAL data
structure and adds test code to illustrate the problem with using
`WALGenerateNBlocks` and `wal.SearchForEndHeight` to test periodic
sync'ing.

* Fix test error messages

* Use WAL group buffer size to check for flush

A function is added to `libs/autofile/group.go#Group` in order to return
the size of the buffered data (i.e. data that has not yet been flushed
to disk). The test now checks that, prior to a `time.Sleep`, the group
buffer has data in it. After the `time.Sleep` (during which time the
periodic flush should have been called), the buffer should be empty.

* Remove config root dir removal from #3291

* Add godoc for NewWAL mentioning periodic sync
2019-02-20 09:45:18 +04:00
Anton Kaliaev
9d4f59b836 update changelog and bump version 2019-02-18 15:27:26 +04:00
Ismail Khoffi
d2c7f8dbcf p2p: check secret conn id matches dialed id (#3321)
ref: [#3010 (comment)](https://github.com/tendermint/tendermint/issues/3010#issuecomment-464287627)

> I tried searching for code where we authenticate a peer against its NetAddress.ID and couldn't find it. I don't see a reason to switch to Noise, but a need to ensure that the node's ID is authenticated e.g. after dialing from the address book.

* p2p: check secret conn id matches dialed id

* Fix all p2p tests & make code compile

* add simple test for dialing with wrong ID

* update changelog

* address review comments

* yet another place where to use IDAddressString and fix
testSetupMultiplexTransport
2019-02-18 14:08:22 +04:00
Anton Kaliaev
8283ca7ddb 3291 follow-up (#3323)
* changelog: use issue number instead of PR number

* follow up to #3291

- rpc/test/helpers.go add StopTendermint(node) func
- remove ensureDir(filepath.Dir(walFile), 0700)
- mempool/mempool_test.go add type cleanupFunc func()

* cmd/show_validator: wrap err to make it more clear
2019-02-18 13:23:40 +04:00
Alessio Treglia
59cc6d36c9 improve ResetTestRootWithChainID() concurrency safety (#3291)
* improve ResetTestRootWithChainID() concurrency safety

Rely on ioutil.TempDir() to create test root directories and ensure
multiple same-chain id test cases can run in parallel.

* Update config/toml.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* clean up test directories after completion

Closes: #1034

* Remove redundant EnsureDir call

* s/PanicSafety()/panic()/s

* Put create dir functionality back in ResetTestRootWithChainID

* Place test directories in OS's tempdir

In modern UNIX and UNIX-like systems /tmp is very often
mounted as tmpfs. This might speed test execution a bit.

* Set 0700 to a const

* rootsDirs -> configRootDirs

* Don't double remove directories

* Avoid global variables

* Fix consensus tests

* Reduce defer stack

* Address review comments

* Try to fix tests

* Update CHANGELOG_PENDING.md

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* Update consensus/common_test.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>

* Update consensus/common_test.go

Co-Authored-By: alessio <quadrispro@ubuntu.com>
2019-02-18 11:45:27 +04:00
Zarko Milosevic
af8793c01a cs: reset triggered timeout precommit (#3310)
* Reset TriggeredTimeoutPrecommit as part of updateToState

* Add failing test and fix

* fix DATA RACE in TestResetTimeoutPrecommitUponNewHeight

```
WARNING: DATA RACE
Read at 0x00c001691d28 by goroutine 691:
  github.com/tendermint/tendermint/consensus.decideProposal()
      /go/src/github.com/tendermint/tendermint/consensus/common_test.go:133 +0x121
  github.com/tendermint/tendermint/consensus.TestResetTimeoutPrecommitUponNewHeight()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1389 +0x958
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Previous write at 0x00c001691d28 by goroutine 931:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).updateToState()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:562 +0x5b2
  github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1340 +0x141e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1255 +0x66e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1201 +0x135
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1232 +0x94b
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1657 +0x132e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1503 +0x8f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:694 +0xa1e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd

Goroutine 691 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:216 +0x332
```

* fix another DATA RACE by locking consensus

```
WARNING: DATA RACE
Read at 0x00c009b835a8 by goroutine 871:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).createProposalBlock()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:955 +0x7c
  github.com/tendermint/tendermint/consensus.decideProposal()
      /go/src/github.com/tendermint/tendermint/consensus/common_test.go:127 +0x53
  github.com/tendermint/tendermint/consensus.TestResetTimeoutPrecommitUponNewHeight()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1389 +0x958
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Previous write at 0x00c009b835a8 by goroutine 931:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).updateHeight()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:446 +0xb7
  github.com/tendermint/tendermint/consensus.(*ConsensusState).updateToState()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:542 +0x22f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1340 +0x141e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1255 +0x66e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1201 +0x135
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1232 +0x94b
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1657 +0x132e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1503 +0x8f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:694 +0xa1e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:642 +0x948
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:655 +0x7dd
```

* Fix failing test

* Delete profile.out

* fix data races
2019-02-18 11:29:41 +04:00
Anton Kaliaev
0b0a8b3128 cs/wal: refuse to encode msg that is bigger than maxMsgSizeBytes (#3303)
Earlier this week somebody posted this in GoS Riot chat:

```
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 878916964 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 825701731 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 1631073634 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.596] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[length 912418148 exceeded maximum possible value of 1048576 bytes]"
E[2019-02-12|10:38:37.600] Corrupted entry. Skipping... module=consensus wal=/home/gaia/.gaiad/data/cs.wal/wal err="DataCorruptionError[failed to read data: EOF]"
E[2019-02-12|10:38:37.600] Error on catchup replay. Proceeding to start ConsensusState anyway module=consensus err="Cannot replay height 7242. WAL does not contain #ENDHEIGHT for 7241"
E[2019-02-12|10:38:37.861] Error dialing peer module=p2p err="dial tcp 35.183.126.181:26656: i/o timeout
```

Note the length error messages. What has happened is the length field got corrupted probably. I've looked at the code and noticed that we don't check the msg size during encoding. This PR fixes that. It also improves a few error messages in WALDecoder.
2019-02-18 11:23:06 +04:00
Anton Kaliaev
7ced9e416b rpc/net_info: change RemoteIP type from net.IP to String (#3309)
* rpc/net_info: change RemoteIP type from net.IP to String

Before:
"AAAAAAAAAAAAAP//rB8ktw=="
which is amino-encoded net.IP byte slice

After:
"192.0.2.1"

Fixes #3251

* rpc/net_info: non-empty response in docs
2019-02-16 09:46:02 +04:00
Ismail Khoffi
af3ba5145a fix test-vectors to actually match the sign bytes: (#3312)
- sign bytes are length prefixed
 - change test to use SignBytes methods instead of calling amino methods
 directly

Resolves #3311
ref: #2455 #2508
2019-02-16 09:12:00 +04:00
Alexander Bezobchuk
cf737ec85c return an error on show_validator (#3316)
* return a command error prior to init

* add a pending log entry

* Update CHANGELOG_PENDING.md

Co-Authored-By: alexanderbez <alexanderbez@users.noreply.github.com>

closes: #3314
2019-02-16 09:07:18 +04:00
Zach
d32f7d2416 update codeowners (#3305)
limit the scope of @zramsay because he's annoyed by notifications
2019-02-13 08:26:01 +04:00
Ethan Buchman
dc6567c677 consensus: flush wal on stop (#3297)
Should fix #3295
Also partial fix of #3043
2019-02-12 09:32:40 +04:00
Ethan Buchman
08dabab024 types: validator set update tests (#3284)
* types: validator set update tests

* docs: abci val updates must not include duplicates
2019-02-12 09:04:23 +04:00
Anca Zamfir
8a9eecce7f test blockExec does not panic if all vals removed (#3241)
Fix for #3224
Also address #2084
2019-02-12 09:02:44 +04:00
Ismail Khoffi
b089587b42 make gosec linter pass (#3294)
* not related to linter: remove obsolete constants:
 - `Insecure` and `Secure` and type `Security` are not used anywhere

* not related to linter: update example

 - NewInsecure was deleted; change example to NewRemoteDB

* address: Binds to all network interfaces (gosec):

 - bind to localhost instead of 0.0.0.0
 - regenerate test key and cert for this purpose (was valid for ::) and
 otherwise we would see:
 transport: authentication handshake failed: x509: certificate is
 valid for ::, not 127.0.0.1\"

(used https://github.com/google/keytransparency/blob/master/scripts/gen_server_keys.sh
to regenerate certs)

* use sha256 in tests instead of md5; time difference is negligible

* nolint usage of math/rand in test and add comment on its import

 - crypto/rand is slower and we do not need sth more secure in tests

* enable linter in circle-ci

* another nolint math/rand in test

* replace another occurrence of md5

* consistent comment about importing math/rand
2019-02-12 08:54:12 +04:00
Anton Kaliaev
7fd51e6ade make govet linter pass (#3292)
* make govet linter pass

Refs #3262

* close PipeReader and check for err
2019-02-11 16:31:34 +04:00
Anca Zamfir
966b5bdf6e fix failure in TestProposerFrequency (#3293)
```
--- FAIL: TestProposerFrequency (2.50s)

panic: empty validator set [recovered]

        panic: empty validator set



goroutine 91 [running]:

testing.tRunner.func1(0xc000a98c00)

        /usr/local/go/src/testing/testing.go:792 +0x6a7

panic(0xeae7e0, 0x11fbb30)

        /usr/local/go/src/runtime/panic.go:513 +0x1b9

github.com/tendermint/tendermint/types.(*ValidatorSet).RescalePriorities(0xc0000e7380, 0x0)

        /go/src/github.com/tendermint/tendermint/types/validator_set.go:106 +0x1ac

github.com/tendermint/tendermint/state.TestProposerFrequency(0xc000a98c00)

        /go/src/github.com/tendermint/tendermint/state/state_test.go:335 +0xb44

testing.tRunner(0xc000a98c00, 0x111a4d8)

        /usr/local/go/src/testing/testing.go:827 +0x163

created by testing.(*T).Run

        /usr/local/go/src/testing/testing.go:878 +0x65a

FAIL    github.com/tendermint/tendermint/state  3.139s
```
2019-02-11 16:21:51 +04:00
Ethan Buchman
021b5cc7f6 Merge pull request #3290 from tendermint/master
Merge pull request #3288 from tendermint/release/v0.30.0
2019-02-09 10:07:49 -05:00
Ethan Buchman
28d75ec801 Merge pull request #3288 from tendermint/release/v0.30.0
Release/v0.30.0
2019-02-09 10:07:22 -05:00
Ethan Buchman
792b12573e Prepare v0.30.0 (#3287)
* changelog, upgrading, version

* update for evidence fixes

* linkify

* fix an entry
2019-02-08 18:50:02 -05:00
Ethan Buchman
4f2ef36701 types.NewCommit (#3275)
* types.NewCommit

* use types.NewCommit everywhere

* fix log in unsafe_reset

* memoize height and round in constructor

* notes about deprecating toVote

* bring back memoizeHeightRound
2019-02-08 18:40:41 -05:00
Ethan Buchman
6b1b595951 Merge pull request #3286 from tendermint/bucky/fix-duplicate-evidence
Fixes for duplicate evidence
2019-02-08 18:33:45 -05:00
Ismail Khoffi
87bdc42bf8 Reject blocks with committed evidence (#37)
* evidence: NewEvidencePool takes evidenceDB

* evidence: failing TestStoreCommitDuplicate

tendermint/security#35

* GetEvidence -> GetEvidenceInfo

* fix TestStoreCommitDuplicate

* comment in VerifyEvidence

* add check if evidence was already seen
 - modify EventPool interface (EventStore is not known in ApplyBlock):
   - add IsCommitted method to iface
 - add test

* update changelog

* fix TestStoreMark:
 - priority in evidence info gets reset to zero after evidence gets committed

* review comments: simplify EvidencePool.IsCommitted
 - delete obsolete EvidenceStore.IsCommitted

* add simple test for IsCommitted

* update changelog: this is actually breaking (PR number still missing)

* fix TestStoreMark:
 - priority in evidence info gets reset to zero after evidence gets
 committed

* review suggestion: simplify return
2019-02-08 18:30:45 -05:00
Ethan Buchman
90ba63948a Sec/bucky/35 commit duplicate evidence (#36)
Don't add committed evidence to evpool
2019-02-08 18:25:48 -05:00
Anca Zamfir
cce4d21ccb treat validator updates as set (#3222)
* Initial commit for 3181..still early

* unit test updates

* unit test updates

* fix check of dups accross updates and deletes

* simplify the processChange() func

* added overflow check utest

* Added checks for empty valset, new utest

* deepcopy changes in processUpdate()

* moved to new API, fixed tests

* test cleanup

* address review comments

* make sure votePower > 0

* gofmt fixes

* handle duplicates and invalid values

* more work on tests, review comments

* Renamed and explained K

* make TestVal private

* split verifyUpdatesAndComputeNewPriorities.., added check for deletes

* return error if validator set is empty after processing changes

* address review comments

* lint err

* Fixed the total voting power and added comments

* fix lint

* fix lint
2019-02-08 13:05:09 -05:00
Ismail Khoffi
c1f7399a86 review comment: cleaner constant for N/2, delete secp256k1N and use (#3279)
`secp256k1.S256().N` directly instead
2019-02-08 09:48:09 -05:00
Ethan Buchman
44a89a3537 Merge pull request #3282 from tendermint/master
Merge master back to develop
2019-02-08 09:43:36 -05:00
Ethan Buchman
a8dbc64319 Merge pull request #3276 from tendermint/release/v0.29.2
Release/v0.29.2
2019-02-08 09:42:52 -05:00
Ethan Buchman
af6e6cd350 remove MixEntropy (#3278)
* remove MixEntropy

* changelog
2019-02-07 20:12:57 -05:00
Ethan Buchman
ad4bd92fec secp256k1: change build tags (#3277) 2019-02-07 19:57:30 -05:00
Ethan Buchman
f571ee8876 prepare v0.29.2 (#3272)
* update changelog

* linkify

* bump version

* update main changelog

* final fixes

* entry for wal fix

* changelog preamble

* remove a line
2019-02-07 19:34:01 -05:00
Anton Kaliaev
11e36d0bfb addrbook_test: preallocate memory for bookSizes (#3268)
Fixes https://circleci.com/gh/tendermint/tendermint/44901
2019-02-07 17:16:31 +04:00
Ethan Buchman
354a08c25a p2p: fix infinite loop in addrbook (#3232)
* failing test

* fix infinite loop in addrbook

There are cases where we only have a small number of addresses marked
good ("old"), but the selection mechanism keeps trying to select more of these
addresses, and hence ends up in an infinite loop. Here we fix this to
only try and select such "old" addresses if we have enough of them. Note this
means, if we don't have enough of them, we may return more "new"
addresses than otherwise expected by the newSelectionBias.

This whole GetSelectionWithBias method probably needs to be rewritten,
but this is a quick fix for the issue.

* changelog

* fix infinite loop if not enough new addrs

* fix another potential infinite loop

if a.nNew == 0 -> pickFromOldBucket=true, but we don't have enough items
  (a.nOld > len(oldBucketToAddrsMap) false)

* Revert "fix another potential infinite loop"

This reverts commit 146540c1125597162bd89820d611f6531f5e5e4b.

* check num addresses instead of buckets, new test

* fixed the int division

* add slack to bias % in test, lint fixes

* Added checks for selection content in test

* test cleanup

* Apply suggestions from code review

Co-Authored-By: ebuchman <ethan@coinculture.info>

* address review comments

* change after  Anton's review comments

* use the same docker image we use for testing

when building a binary for localnet

* switch back to circleci classic

* more review comments

* more review comments

* refactor addrbook_test

* build linux binary inside docker

in attempt to fix

```
--> Running dep
+ make build-linux
GOOS=linux GOARCH=amd64 make build
make[1]: Entering directory `/home/circleci/.go_workspace/src/github.com/tendermint/tendermint'
CGO_ENABLED=0 go build -ldflags "-X github.com/tendermint/tendermint/version.GitCommit=`git rev-parse --short=8 HEAD`" -tags 'tendermint' -o build/tendermint ./cmd/tendermint/
p2p/pex/addrbook.go:373:13: undefined: math.Round
```

* change dir from /usr to /go

* use concrete Go version for localnet binary

* check for nil addresses just to be sure
2019-02-07 15:58:23 +04:00
Thane Thomson
e70f27c8e4 Add remote signer test harness (KMS) (#3149)
* WIP: Starts adding remote signer test harness

This commit adds a new command to Tendermint to allow for us to build a
standalone binary to test remote signers such as KMS
(https://github.com/tendermint/kms).

Right now, all it does is test that the local public key matches the
public key reported by the client, and fails at the point where it
attempts to get the client to sign a proposal.

* Fixes typo

* Fixes proposal validation test

This commit fixes the proposal validation test as per #3149. It also
moves the test harness into its own internal package to isolate its
exports from the `privval` package.

* Adds vote signing validation

* Applying recommendations from #3149

* Adds function descriptions for test harness

* Adds ability to ask remote signer to shut down

Prior to this commit, the remote signer needs to manually be shut down,
which is not ideal for automated testing. This commit allows us to send
a poison pill message to the KMS to let it shut down gracefully once
testing is done (whether the tests pass or fail).

* Adds tests for remote signer test harness

This commit makes some minor modifications to a few files to allow for
testing of the remote signer test harness. Two tests are added here:
checking for a fully successful (the ideal) case, and for the case where
the maximum number of retries has been reached when attempting to accept
incoming connections from the remote signer.

* Condenses serialization of proposals and votes using existing Tendermint functions

* Removes now-unnecessary amino import and codec

* Adds error message for vote signing failure

* Adds key extraction command for integration test

Took the code from here:
https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to
create a simple utility command to extract a key from a local Tendermint
validator for use in KMS integration testing.

* Makes path expansion success non-compulsory

* Fixes segfault on SIGTERM

We need an additional variable to keep track of whether we're
successfully connected, otherwise hitting Ctrl+Break during execution
causes a segmentation fault. This now allows for a clean shutdown.

* Consolidates shutdown checks

* Adds comments indicating codes for easy lookup

* Adds Docker build for remote signer harness

Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to
override the image name and Dockerfile using environment variables.
Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow
for building the `remote_val_harness` Docker image.

* Adds build_remote_val_harness_docker_image to .PHONY

* Removes remote signer poison pill messaging functionality

* Reduces fluff code in command line parsing

As per
https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788,
this reduces the amount of fluff code in the PR down to the bare
minimum.

* Fixes ordering of error check and info log

* Moves remove_val_harness cmd into tools folder

It seems to make sense to rather keep the remote signer test harness in
its own tool folder (now rather named `tm-signer-harness` to keep with
the tool naming convention). It is actually a separate tool, not meant
to be one of the core binaries, but supplementary and supportive.

* Updates documentation for tm-signer-harness

* Refactors flag parsing to be more compact and less redundant

* Adds version sub-command help

* Removes extraneous flags parsing

* Adds CHANGELOG_PENDING entry for tm-signer-harness

* Improves test coverage

Adds a few extra parameters to the `MockPV` type to fake broken vote and
proposal signing. Also adds some more tests for the test harness so as
to increase coverage for failed cases.

* Fixes formatting for CHANGELOG_PENDING.md

* Fix formatting for documentation config

* Point users towards official Tendermint docs for tools documentation

* Point users towards official Tendermint docs for tm-signer-harness

* Remove extraneous constant

* Rename TestHarness.sc to TestHarness.spv for naming consistency

* Refactor to remove redundant goroutine

* Refactor conditional to cleaner switch statement and better error handling for listener protocol

* Remove extraneous goroutine

* Add note about installing tmkms via Cargo

* Fix typo in naming of output signing key

* Add note about where to find chain ID

* Replace /home/user with ~/ for brevity

* Fixes "signer.key" typo

* Minor edits for clarification for tm-signer-harness bulid/setup process
2019-02-07 10:32:32 +04:00
Anton Kaliaev
fcebdf6720 Merge pull request #3261 from tendermint/anton/circle
Revert "quick fix for CircleCI (#2279)"
2019-02-07 10:28:22 +04:00
Ethan Buchman
9e9026452c p2p/conn: don't hold stopMtx while waiting (#3254)
* p2p/conn: fix deadlock in FlushStop/OnStop

* makefile: set_with_deadlock

* close doneSendRoutine at end of sendRoutine

* conn: initialize channs in OnStart
2019-02-06 10:29:51 -05:00
Ethan Buchman
45b70ae031 fix non deterministic test failures and race in privval socket (#3258)
* node: decrease retry conn timeout in test

Should fix #3256

The retry timeout was set to the default, which is the same as the
accept timeout, so it's no wonder this would fail. Here we decrease the
retry timeout so we can try many times before the accept timeout.

* p2p: increase handshake timeout in test

This fails sometimes, presumably because the handshake timeout is so low
(only 50ms). So increase it to 1s. Should fix #3187

* privval: fix race with ping. closes #3237

Pings happen in a go-routine and can happen concurrently with other
messages. Since we use a request/response protocol, we expect to send a
request and get back the corresponding response. But with pings
happening concurrently, this assumption could be violated. We were using
a mutex, but only a RWMutex, where the RLock was being held for sending
messages - this was to allow the underlying connection to be replaced if
it fails. Turns out we actually need to use a full lock (not just a read
lock) to prevent multiple requests from happening concurrently.

* node: fix test name. DelayedStop -> DelayedStart

* autofile: Wait() method

In the TestWALTruncate in consensus/wal_test.go we remove the WAL
directory at the end of the test. However the wal.Stop() does not
properly wait for the autofile group to finish shutting down. Hence it
was possible that the group's go-routine is still running when the
cleanup happens, which causes a panic since the directory disappeared.
Here we add a Wait() method to properly wait until the go-routine exits
so we can safely clean up. This fixes #2852.
2019-02-06 10:24:43 -05:00
Ethan Buchman
4429826229 cmn: GetFreePort (#3255) 2019-02-06 10:14:03 -05:00
Anton Kaliaev
1386707ceb rename TestGCRandom to _TestGCRandom 2019-02-06 18:36:54 +04:00
Anton Kaliaev
1a35895ac8 remove gotype linter
WARN [runner/nolint] Found unknown linters in //nolint directives: gotype
2019-02-06 18:26:56 +04:00
Anton Kaliaev
6941d1bb35 use nolint label instead of commenting 2019-02-06 18:21:32 +04:00
Anton Kaliaev
23314daee4 comment out clist tests w/ depend on runtime.SetFinalizer 2019-02-06 15:38:02 +04:00
Anton Kaliaev
3c8156a55a preallocating memory when we can 2019-02-06 15:24:54 +04:00
Anton Kaliaev
ffd3bf8448 remove or comment out unused code 2019-02-06 15:16:38 +04:00
Anton Kaliaev
da33dd04cc switch to golangci-lint from gometalinter 🚤 2019-02-06 15:00:55 +04:00
Anton Kaliaev
d8f0bc3e60 Revert "quick fix for CircleCI (#2279)"
This reverts commit 1cf6712a36.
2019-02-06 14:11:35 +04:00
Ethan Buchman
1809efa350 Introduce CommitSig alias for Vote in Commit (#3245)
* types: memoize height/round in commit instead of first vote

* types: commit.ValidateBasic in VerifyCommit

* types: new CommitSig alias for Vote

In preparation for reducing the redundancy in Commits, we introduce the
CommitSig as an alias for Vote. This is non-breaking on the protocol,
and minor breaking on the Go API, as Commit now contains a list of
CommitSig instead of Vote.

* remove dependence on ToVote

* update some comments

* fix tests

* fix tests

* fixes from review
2019-02-04 13:01:59 -05:00
Ethan Buchman
39eba4e154 WAL: better errors and new fail point (#3246)
* privval: more info in errors

* wal: change Debug logs to Info

* wal: log and return error on corrupted wal instead of panicing

* fail: Exit right away instead of sending interupt

* consensus: FAIL before handling our own vote

allows to replicate #3089:
- run using `FAIL_TEST_INDEX=0`
- delete some bytes from the end of the WAL
- start normally

Results in logs like:

```
I[2019-02-03|18:12:58.225] Searching for height module=consensus wal=/Users/ethanbuchman/.tendermint/data/cs.wal/wal height=1 min=0 max=0
E[2019-02-03|18:12:58.225] Error on catchup replay. Proceeding to start ConsensusState anyway module=consensus err="failed to read data: EOF"
I[2019-02-03|18:12:58.225] Started node module=main nodeInfo="{ProtocolVersion:{P2P:6 Block:9 App:1} ID_:35e87e93f2e31f305b65a5517fd2102331b56002 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-J8JvJH Version:0.29.1 Channels:4020212223303800 Moniker:Ethans-MacBook-Pro.local Other:{TxIndex:on RPCAddress:tcp://0.0.0.0:26657}}"
E[2019-02-03|18:12:58.226] Couldn't connect to any seeds module=p2p
I[2019-02-03|18:12:59.229] Timed out module=consensus dur=998.568ms height=1 round=0 step=RoundStepNewHeight
I[2019-02-03|18:12:59.230] enterNewRound(1/0). Current: 1/0/RoundStepNewHeight module=consensus height=1 round=0
I[2019-02-03|18:12:59.230] enterPropose(1/0). Current: 1/0/RoundStepNewRound module=consensus height=1 round=0
I[2019-02-03|18:12:59.230] enterPropose: Our turn to propose module=consensus height=1 round=0 proposer=AD278B7767B05D7FBEB76207024C650988FA77D5 privValidator="PrivValidator{AD278B7767B05D7FBEB76207024C650988FA77D5 LH:1, LR:0, LS:2}"
E[2019-02-03|18:12:59.230] enterPropose: Error signing proposal module=consensus height=1 round=0 err="Error signing proposal: Step regression at height 1 round 0. Got 1, last step 2"
I[2019-02-03|18:13:02.233] Timed out module=consensus dur=3s height=1 round=0 step=RoundStepPropose
I[2019-02-03|18:13:02.233] enterPrevote(1/0). Current: 1/0/RoundStepPropose module=consensus
I[2019-02-03|18:13:02.233] enterPrevote: ProposalBlock is nil module=consensus height=1 round=0
E[2019-02-03|18:13:02.234] Error signing vote module=consensus height=1 round=0 vote="Vote{0:AD278B7767B0 1/00/1(Prevote) 000000000000 000000000000 @ 2019-02-04T02:13:02.233897Z}" err="Error signing vote: Conflicting data"
```

Notice the EOF, the step regression, and the conflicting data.

* wal: change errors to be DataCorruptionError

* exit on corrupt WAL

* fix log

* fix new line
2019-02-04 13:00:06 -05:00
Ethan Buchman
eb4e23b91e fix FlushStop (#3247)
* p2p/pex: failing test

* p2p/conn: add stopMtx for FlushStop and OnStop

* changelog
2019-02-04 07:30:24 -08:00
Ismail Khoffi
6485e68beb Use ethereum's secp256k1 lib (#3234)
* switch from fork (tendermint/btcd) to orig package (btcsuite/btcd); also

 - remove obsolete check in test `size != -1` is always true
 - WIP as the serialization still needs to be wrapped

* WIP: wrap signature & privkey, pubkey needs to be wrapped as well

* wrap pubkey too

* use "github.com/ethereum/go-ethereum/crypto/secp256k1" if cgo is
available, else use "github.com/btcsuite/btcd/btcec" and take care of
lower-S when verifying

Annoyingly, had to disable pruning when importing
github.com/ethereum/go-ethereum/ :-/

* update comment

* update comment

* emulate signature_nocgo.go for additional benchmarks:
592bf6a59c/crypto/signature_nocgo.go (L60-L76)

* use our format (r || s) in lower-s form when in the non-cgo case

* remove comment about using the C library directly

* vendor github.com/btcsuite/btcd too

* Add test for the !cgo case

* update changelog pending

Closes #3162 #3163
Refs #1958, #2091, tendermint/btcd#1
2019-02-04 12:24:54 +04:00
Anton Kaliaev
d470945503 update gometalinter to 3.0.0 (#3233)
in the attempt to fix https://circleci.com/gh/tendermint/tendermint/43165

also

    code is simplified by running gofmt -s .
    remove unused vars
    enable linters we're currently passing
    remove deprecated linters
2019-01-30 12:24:26 +04:00
Anton Kaliaev
8985a1fa63 pubsub: fixes after Ethan's review (#3212)
in https://github.com/tendermint/tendermint/pull/3209
2019-01-29 13:16:43 +04:00
Ismail Khoffi
6dd817cbbc secret connection: check for low order points (#3040)
> Implement a check for the blacklisted low order points, ala the X25519 has_small_order() function in libsodium

(#3010 (comment))
resolves first half of #3010
2019-01-29 12:44:59 +04:00
rickyyangz
0b3a87a323 mempool: correct args order in the log msg (#3221)
Before: Unexpected tx response from proxy during recheck\n Expected: {r.CheckTx.Data}, got: {memTx.tx}
After: Unexpected tx response from proxy during recheck\n Expected: {memTx.tx}, got: {tx}

Closes #3214
2019-01-29 10:12:07 +04:00
Zach
e1edd2aa6a hardcode rpc link (#3223) 2019-01-28 23:41:37 +04:00
Thane Thomson
9a0bfafef6 docs: fix links (#3220)
Because there's nothing worse than having to copy/paste a link from a
web page to navigate to it 😁
2019-01-28 17:41:39 +04:00
Thane Thomson
a335caaedb alias amino imports (#3219)
As per conversation here: https://github.com/tendermint/tendermint/pull/3218#discussion_r251364041

This is the result of running the following code on the repo:

```bash
find . -name '*.go' | grep -v 'vendor/' | xargs -n 1 goimports -w
```
2019-01-28 16:13:17 +04:00
Anton Kaliaev
ff3c4bfc76 add go-deadlock tool to help detect deadlocks (#3218)
* add go-deadlock tool to help detect deadlocks

Run it with `make test_with_deadlock`. After it's done, use Git to
cleanup `git checkout .`

Link: https://github.com/sasha-s/go-deadlock/
Replaces https://github.com/tendermint/tendermint/pull/3148

* add a target to cleanup changes
2019-01-28 14:57:47 +04:00
Anton Kaliaev
8d2dd7e554 refactor TestListenerConnectDeadlines to avoid data races (#3201)
Fixes #3179
2019-01-28 12:38:11 +04:00
cong
71e5939441 start eventBus & indexerService before replay and use them while replaying blocks (#3194)
so if we did not index the last block (because of panic or smth else), we index it during replay

Closes #3186
2019-01-28 11:36:35 +04:00
Ethan Buchman
d91ea9b59d adr-033 update 2019-01-26 14:30:29 +04:00
Ethan Buchman
9b6b792ce7 pubsub: comments 2019-01-26 14:30:29 +04:00
Ethan Buchman
57af99d901 types: comments on user vs internal events
Distinguish between user events and internal consensus events
2019-01-26 14:30:29 +04:00
Ethan Buchman
a58d5897e4 note about TmCoreSemVer 2019-01-26 14:30:29 +04:00
Jae Kwon
ddbdffb4e5 add design philosophy doc (#3034) 2019-01-25 23:00:55 +04:00
Ismail Khoffi
d6dd43cdaa adr: style fixes (#3206)
- links to issues
 - fix a few markdown glitches
 - inline code
 - etc
2019-01-25 17:38:26 +04:00
Joon
75cbe4a1c1 R4R: Config TestRoot modification for LCD test (#3177)
* add ResetTestRootWithChainID

* modify chainid
2019-01-25 08:10:36 -05:00
Ethan Buchman
27c1563bf0 Merge pull request #3205 from tendermint/master
Merge master back to develop
2019-01-24 11:35:01 -05:00
Ethan Buchman
4d7b29cd8f Merge pull request #3203 from tendermint/release/v0.29.1
Release/v0.29.1
2019-01-24 11:34:20 -05:00
Ethan Buchman
90970d0ddc fix changelog 2019-01-24 11:19:52 -05:00
Anton Kaliaev
bb0a9b3d6d bump version 2019-01-24 19:18:32 +04:00
Anton Kaliaev
8992596192 update changelog 2019-01-24 19:17:53 +04:00
Zarko Milosevic
c20fbed2f7 [WIP] fix halting issue (#3197)
fix halting issue
2019-01-24 09:33:47 -05:00
Anton Kaliaev
c4157549ab only log "Reached max attempts to dial" once (#3144)
Closes #3037
2019-01-24 13:53:02 +04:00
Infinytum
fbd1e79465 docs: fix lite client formatting (#3198)
Closes #3180
2019-01-24 12:10:34 +04:00
Anton Kaliaev
1efacaa8d3 docs: update pubsub ADR (#3131)
* docs: update pubsub ADR

* third version
2019-01-24 11:33:58 +04:00
Gautham Santhosh
98b42e9eb2 docs: explain how someone can run his/her own ABCI app on localnet (#3195)
Closes #3192.
2019-01-24 10:45:43 +04:00
Anton Kaliaev
2449bf7300 p2p: file descriptor leaks (#3150)
* close peer's connection to avoid fd leak

Fixes #2967

* rename peer#Addr to RemoteAddr

* fix test

* fixes after Ethan's review

* bring back the check

* changelog entry

* write a test for switch#acceptRoutine

* increase timeouts? :(

* remove extra assertNPeersWithTimeout

* simplify test

* assert number of peers (just to be safe)

* Cleanup in OnStop

* run tests with verbose flag on CircleCI

* spawn a reading routine to prevent connection from closing

* get port from the listener

random port is faster, but often results in

```
panic: listen tcp 127.0.0.1:44068: bind: address already in use [recovered]
        panic: listen tcp 127.0.0.1:44068: bind: address already in use

goroutine 79 [running]:
testing.tRunner.func1(0xc0001bd600)
        /usr/local/go/src/testing/testing.go:792 +0x387
panic(0x974d20, 0xc0001b0500)
        /usr/local/go/src/runtime/panic.go:513 +0x1b9
github.com/tendermint/tendermint/p2p.MakeSwitch(0xc0000f42a0, 0x0, 0x9fb9cc, 0x9, 0x9fc346, 0xb, 0xb42128, 0x0, 0x0, 0x0, ...)
        /home/vagrant/go/src/github.com/tendermint/tendermint/p2p/test_util.go:182 +0xa28
github.com/tendermint/tendermint/p2p.MakeConnectedSwitches(0xc0000f42a0, 0x2, 0xb42128, 0xb41eb8, 0x4f1205, 0xc0001bed80, 0x4f16ed)
        /home/vagrant/go/src/github.com/tendermint/tendermint/p2p/test_util.go:75 +0xf9
github.com/tendermint/tendermint/p2p.MakeSwitchPair(0xbb8d20, 0xc0001bd600, 0xb42128, 0x2f7, 0x4f16c0)
        /home/vagrant/go/src/github.com/tendermint/tendermint/p2p/switch_test.go:94 +0x4c
github.com/tendermint/tendermint/p2p.TestSwitches(0xc0001bd600)
        /home/vagrant/go/src/github.com/tendermint/tendermint/p2p/switch_test.go:117 +0x58
testing.tRunner(0xc0001bd600, 0xb42038)
        /usr/local/go/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:878 +0x353
exit status 2
FAIL    github.com/tendermint/tendermint/p2p    0.350s
```
2019-01-22 13:23:18 -05:00
Ethan Buchman
3362da0a69 Merge pull request #3191 from tendermint/master
Merge master to develop
2019-01-22 13:19:58 -05:00
Ethan Buchman
4514842a63 Merge pull request #3185 from tendermint/release/v0.29.0
Release/v0.29.0
2019-01-22 13:19:28 -05:00
Ethan Buchman
a97d6995c9 fix changelog indent (#3190) 2019-01-22 12:24:26 -05:00
Ethan Buchman
d9d4f3e629 Prepare v0.29.0 (#3184)
* update changelog and upgrading

* add note about max voting power in abci spec

* update version

* changelog
2019-01-21 19:32:10 -05:00
Ethan Buchman
7a8aeff4b0 update spec for Merkle RFC 6962 (#3175)
* spec: specify when MerkleRoot is on hashes

* remove unnecessary hash methods

* update changelog

* fix test
2019-01-21 10:02:57 -05:00
Ethan Buchman
de5a6010f0 fix DynamicVerifier for large validator set changes (#3171)
* base verifier: bc->bv and check chainid

* improve some comments

* comments in dynamic verifier

* fix comment in doc about BaseVerifier

It requires the validator set to perfectly match.

* failing test for #2862

* move errTooMuchChange to types. fixes #2862

* changelog, comments

* ic -> dv

* update comment, link to issue
2019-01-21 09:21:04 -05:00
Ethan Buchman
da95f4aa6d mempool: enforce maxMsgSize limit in CheckTx (#3168)
- fixes #3008
- reactor requires encoded messages are less than maxMsgSize
- requires size of tx + amino-overhead to not exceed maxMsgSize
2019-01-20 17:27:49 -05:00
Ethan Buchman
4f8769175e [types] hash of ConsensusParams includes only a subset of fields (#3165)
* types: dont hash entire ConsensusParams

* update encoding spec

* update blockchain spec

* spec: consensus params hash

* changelog
2019-01-19 16:08:57 -05:00
Ismail Khoffi
40c887baf7 Normalize priorities to not exceed total voting power (#3049)
* more proposer priority tests

 - test that we don't reset to zero when updating / adding
 - test that same power validators alternate

* add another test to track / simulate similar behaviour as in #2960

* address some of Chris' review comments

* address some more of Chris' review comments

* temporarily pushing branch with the following changes:
The total power might change if:
   - a validator is added
   - a validator is removed
   - a validator is updated

Decrement the accums (of all validators) directly after any of these events
(by the inverse of the change)

* Fix 2960 by re-normalizing / scaling priorities to be in bounds of total
power, additionally:

 - remove heap where it doesn't make sense
 - avg. only at the end of IncrementProposerPriority instead of each
   iteration
 - update (and slightly improve)
   TestAveragingInIncrementProposerPriorityWithVotingPower to reflect
   above changes

* Fix 2960 by re-normalizing / scaling priorities to be in bounds of total
power, additionally:

 - remove heap where it doesn't make sense
 - avg. only at the end of IncrementProposerPriority instead of each
   iteration
 - update (and slightly improve)
   TestAveragingInIncrementProposerPriorityWithVotingPower to reflect
   above changes

* fix tests

* add comment

* update changelog pending & some minor changes

* comment about division will floor the result & fix typo

* Update TestLargeGenesisValidator:
 - remove TODO and increase large genesis validator's voting power
accordingly

* move changelog entry to P2P Protocol

* Ceil instead of flooring when dividing & update test

* quickly fix failing TestProposerPriorityDoesNotGetResetToZero:

 - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower)

* fix typo: rename getValWitMostPriority -> getValWithMostPriority

* test proposer frequencies

* return absolute value for diff. keep testing

* use for loop for div

* cleanup, more tests

* spellcheck

* get rid of using floats: manually ceil where necessary

* Remove float, simplify, fix tests to match chris's proof (#3157)
2019-01-19 15:55:08 -05:00
Ethan Buchman
d3e8889411 update btcd fork for v0.1.1 (#3164)
* update btcd fork for v0.1.1

* changelog
2019-01-19 14:08:41 -05:00
Ethan Buchman
d17969e378 Merge pull request #3154 from tendermint/master
Merge master back to develop
2019-01-18 12:39:21 -05:00
bradyjoestar
f5f1416a14 json2wal: increase reader's buffer size (#3147)
```
panic: failed to unmarshal json: unexpected end of JSON input

goroutine 1 [running]:
main.main()
	/root/gelgo/src/github.com/tendermint/tendermint/scripts/json2wal/main.go:66 +0x73f
```

Closes #3146
2019-01-18 12:09:12 +04:00
Henry Harder
4d36647eea Add ParadigmCore to ecosystem.json (#3145)
Adding the OrderStream network client (alpha), ParadigmCore, to the `ecosystem.json` file.
2019-01-18 11:46:34 +04:00
Anton Kaliaev
87991059aa docs: fix RPC links (#3141) 2019-01-17 08:42:57 -05:00
Thane Thomson
c69dbb25ce Consolidates deadline tests for privval Unix/TCP (#3143)
* Consolidates deadline tests for privval Unix/TCP

Following on from #3115 and #3132, this converts fundamental timeout
errors from the client's `acceptConnection()` method so that these can
be detected by the test for the TCP connection.

Timeout deadlines are now tested for both TCP and Unix domain socket
connections.

There is also no need for the additional secret connection code: the
connection will time out at the `acceptConnection()` phase for TCP
connections, and will time out when attempting to obtain the
`RemoteSigner`'s public key for Unix domain socket connections.

* Removes extraneous logging

* Adds IsConnTimeout helper function

This commit adds a helper function to detect whether an error is either
a fundamental networking timeout error, or an `ErrConnTimeout` error
specific to the `RemoteSigner` class.

* Adds a test for the IsConnTimeout() helper function

* Separates tests logically for IsConnTimeout
2019-01-17 08:10:56 -05:00
Gautham Santhosh
bc8874020f docs: fix broken link (#3142) 2019-01-17 15:30:51 +04:00
Dev Ojha
55d7238708 Add comment to simple_merkle get_split_point (#3136)
* Add comment to simple_merkle get_split_point

* fix grammar error
2019-01-16 16:03:19 -05:00
Ethan Buchman
4a037f9fe6 Merge pull request #3138 from tendermint/master
Merge master back to develop
2019-01-16 16:02:51 -05:00
Anton Kaliaev
dcb8f88525 add "chain_id" label for all metrics (#3123)
* add "chain_id" label for all metrics

Refs #3082

* fix labels extraction
2019-01-15 12:16:33 -05:00
Kunal Dhariwal
73ea5effe5 docs: update link for rpc docs (#3129) 2019-01-15 17:12:35 +04:00
Anton Kaliaev
4daca1a634 return maxPerPage (not defaultPerPage) if per_page is greater than max (#3124)
it's more user-friendly.

Refs #3065
2019-01-14 17:35:31 -05:00
Anton Kaliaev
bc00a032c1 makefile: fix build-docker-localnode target (#3122)
cd does not work because it's executed in a subprocess
so it has to be either chained by && or ;
See https://stackoverflow.com/q/1789594/820520
for more details.

Fixes #3058
2019-01-14 17:33:33 -05:00
Anton Kaliaev
5f4d8e031e [log] fix year format (#3125)
Refs #3060
2019-01-14 14:10:13 -05:00
Ethan Buchman
7b2c4bb493 update ADR-020 (#3116) 2019-01-14 11:53:43 -05:00
Dev Ojha
ec53ce359b Simple merkle rfc compatibility (#2713)
* Begin simple merkle compatibility PR

* Fix query_test

* Use trillian test vectors

* Change the split point per RFC 6962

* update spec

* refactor innerhash to match spec

* Update changelog

* Address @liamsi's comments

* Write the comment requested by @liamsi
2019-01-13 18:02:38 -05:00
Ismail Khoffi
1f68318875 fix order of BlockID and Timestamp in Vote and Proposal (#3078)
* Consistent order fields of Timestamp/BlockID fields in CanonicalVote and
CanonicalProposal

* update spec too

* Introduce and use IsZero & IsComplete:
 - update IsZero method according to spec and introduce IsComplete
 - use methods in validate basic to validate: proposals come with a
 "complete" blockId and votes are either complete or empty
 - update spec: BlockID.IsNil() -> BlockID.IsZero() and fix typo

* BlockID comes first

* fix tests
2019-01-13 17:56:36 -05:00
Ismail Khoffi
1ccc0918f5 More ProposerPriority tests (#2966)
* more proposer priority tests

 - test that we don't reset to zero when updating / adding
 - test that same power validators alternate

* add another test to track / simulate similar behaviour as in #2960

* address some of Chris' review comments

* address some more of Chris' review comments
2019-01-13 17:40:50 -05:00
419 changed files with 32881 additions and 4783 deletions

View File

@@ -3,14 +3,14 @@ version: 2
defaults: &defaults
working_directory: /go/src/github.com/tendermint/tendermint
docker:
- image: circleci/golang:1.11.4
- image: circleci/golang:1.12.0
environment:
GOBIN: /tmp/workspace/bin
docs_update_config: &docs_update_config
working_directory: ~/repo
docker:
- image: tendermint/docs_deployment
- image: tendermintdev/jq_curl
environment:
AWS_REGION: us-east-1
@@ -48,10 +48,10 @@ jobs:
key: v3-pkg-cache
paths:
- /go/pkg
# - save_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
# paths:
# - /go/src/github.com/tendermint/tendermint
- save_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
paths:
- /go/src/github.com/tendermint/tendermint
build_slate:
<<: *defaults
@@ -60,23 +60,8 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# https://discuss.circleci.com/t/saving-cache-stopped-working-warning-skipping-this-step-disabled-in-configuration/24423/2
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: slate docs
command: |
@@ -91,29 +76,14 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
make get_dev_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: metalinter
command: |
set -ex
export PATH="$GOBIN:$PATH"
make metalinter
make lint
- run:
name: check_dep
command: |
@@ -128,22 +98,8 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: Run abci apps tests
command: |
@@ -159,22 +115,8 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: Run abci-cli tests
command: |
@@ -188,22 +130,8 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run: sudo apt-get update && sudo apt-get install -y --no-install-recommends bsdmainutils
- run:
name: Run tests
@@ -217,22 +145,8 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run: mkdir -p /tmp/logs
- run:
name: Run tests
@@ -240,7 +154,7 @@ jobs:
for pkg in $(go list github.com/tendermint/tendermint/... | circleci tests split --split-by=timings); do
id=$(basename "$pkg")
GOCACHE=off go test -timeout 5m -race -coverprofile=/tmp/workspace/profiles/$id.out -covermode=atomic "$pkg" | tee "/tmp/logs/$id-$RANDOM.log"
go test -v -timeout 5m -race -coverprofile=/tmp/workspace/profiles/$id.out -covermode=atomic "$pkg" | tee "/tmp/logs/$id-$RANDOM.log"
done
- persist_to_workspace:
root: /tmp/workspace
@@ -256,22 +170,8 @@ jobs:
at: /tmp/workspace
- restore_cache:
key: v3-pkg-cache
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: Run tests
command: bash test/persist/test_failure_indices.sh
@@ -292,9 +192,7 @@ jobs:
name: run localnet and exit on failure
command: |
set -x
make get_tools
make get_vendor_deps
make build-linux
docker run --rm -v "$PWD":/go/src/github.com/tendermint/tendermint -w /go/src/github.com/tendermint/tendermint golang:1.11.4 make build-linux
make localnet-start &
./scripts/localnet-blocks-test.sh 40 5 10 localhost
@@ -317,22 +215,10 @@ jobs:
steps:
- attach_workspace:
at: /tmp/workspace
# - restore_cache:
# key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- checkout
- run:
name: tools
command: |
export PATH="$GOBIN:$PATH"
make get_tools
- run:
name: dependencies
command: |
export PATH="$GOBIN:$PATH"
make get_vendor_deps
- run: mkdir -p $GOPATH/src/github.com/tendermint
- run: ln -sf /home/circleci/project $GOPATH/src/github.com/tendermint/tendermint
- restore_cache:
key: v3-pkg-cache
- restore_cache:
key: v3-tree-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: gather
command: |
@@ -353,7 +239,22 @@ jobs:
- run:
name: Trigger website build
command: |
chamber exec tendermint -- start_website_build
curl --silent \
--show-error \
-X POST \
--header "Content-Type: application/json" \
-d "{\"branch\": \"$CIRCLE_BRANCH\"}" \
"https://circleci.com/api/v1.1/project/github/$CIRCLE_PROJECT_USERNAME/$WEBSITE_REPO_NAME/build?circle-token=$TENDERBOT_API_TOKEN" > response.json
RESULT=`jq -r '.status' response.json`
MESSAGE=`jq -r '.message' response.json`
if [[ ${RESULT} == "null" ]] || [[ ${RESULT} -ne "200" ]]; then
echo "CircleCI API call failed: $MESSAGE"
exit 1
else
echo "Website build started"
fi
workflows:
version: 2

4
.github/CODEOWNERS vendored
View File

@@ -4,4 +4,6 @@
* @ebuchman @melekes @xla
# Precious documentation
/docs/ @zramsay
/docs/README.md @zramsay
/docs/DOCS_README.md @zramsay
/docs/.vuepress/ @zramsay

57
.golangci.yml Normal file
View File

@@ -0,0 +1,57 @@
run:
deadline: 1m
linters:
enable-all: true
disable:
- gocyclo
- golint
- maligned
- errcheck
- staticcheck
- interfacer
- unconvert
- goconst
- unparam
- nakedret
- lll
- gochecknoglobals
- gocritic
- gochecknoinits
- scopelint
- stylecheck
# linters-settings:
# govet:
# check-shadowing: true
# golint:
# min-confidence: 0
# gocyclo:
# min-complexity: 10
# maligned:
# suggest-new: true
# dupl:
# threshold: 100
# goconst:
# min-len: 2
# min-occurrences: 2
# depguard:
# list-type: blacklist
# packages:
# # logging is allowed only by logutils.Log, logrus
# # is allowed to use only in logutils package
# - github.com/sirupsen/logrus
# misspell:
# locale: US
# lll:
# line-length: 140
# goimports:
# local-prefixes: github.com/golangci/golangci-lint
# gocritic:
# enabled-tags:
# - performance
# - style
# - experimental
# disabled-checks:
# - wrapperFunc
# - commentFormatting # https://github.com/go-critic/go-critic/issues/755

View File

@@ -1,5 +1,329 @@
# Changelog
## v0.31.0
*March 16th, 2019*
Special thanks to external contributors on this release:
@danil-lashin, @guagualvcha, @siburu, @silasdavis, @srmo, @Stumble, @svenstaro
This release is primarily about the new pubsub implementation, dubbed `pubsub 2.0`, and related changes,
like configurable limits on the number of active RPC subscriptions at a time (`max_subscription_clients`).
Pubsub 2.0 is an improved version of the older pubsub that is non-blocking and has a nicer API.
Note the improved pubsub API also resulted in some improvements to the HTTPClient interface and the API for WebSocket subscriptions.
This release also adds a configurable limit to the mempool size (`max_txs_bytes`, default 1GB)
and a configurable timeout for the `/broadcast_tx_commit` endpoint.
See the [v0.31.0
Milestone](https://github.com/tendermint/tendermint/milestone/19?closed=1) for
more details.
Friendly reminder, we have a [bug bounty
program](https://hackerone.com/tendermint).
### BREAKING CHANGES:
* CLI/RPC/Config
- [config] [\#2920](https://github.com/tendermint/tendermint/issues/2920) Remove `consensus.blocktime_iota` parameter
- [rpc] [\#3227](https://github.com/tendermint/tendermint/issues/3227) New PubSub design does not block on clients when publishing
messages. Slow clients may miss messages and receive an error, terminating
the subscription.
- [rpc] [\#3269](https://github.com/tendermint/tendermint/issues/2826) Limit number of unique clientIDs with open subscriptions. Configurable via `rpc.max_subscription_clients`
- [rpc] [\#3269](https://github.com/tendermint/tendermint/issues/2826) Limit number of unique queries a given client can subscribe to at once. Configurable via `rpc.max_subscriptions_per_client`.
- [rpc] [\#3435](https://github.com/tendermint/tendermint/issues/3435) Default ReadTimeout and WriteTimeout changed to 10s. WriteTimeout can increased by setting `rpc.timeout_broadcast_tx_commit` in the config.
- [rpc/client] [\#3269](https://github.com/tendermint/tendermint/issues/3269) Update `EventsClient` interface to reflect new pubsub/eventBus API [ADR-33](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-033-pubsub.md). This includes `Subscribe`, `Unsubscribe`, and `UnsubscribeAll` methods.
* Apps
- [abci] [\#3403](https://github.com/tendermint/tendermint/issues/3403) Remove `time_iota_ms` from BlockParams. This is a
ConsensusParam but need not be exposed to the app for now.
- [abci] [\#2920](https://github.com/tendermint/tendermint/issues/2920) Rename `consensus_params.block_size` to `consensus_params.block` in ABCI ConsensusParams
* Go API
- [libs/common] TrapSignal accepts logger as a first parameter and does not block anymore
* previously it was dumping "captured ..." msg to os.Stdout
* TrapSignal should not be responsible for blocking thread of execution
- [libs/db] [\#3397](https://github.com/tendermint/tendermint/pull/3397) Add possibility to `Close()` `Batch` to prevent memory leak when using ClevelDB. (@Stumble)
- [types] [\#3354](https://github.com/tendermint/tendermint/issues/3354) Remove RoundState from EventDataRoundState
- [rpc] [\#3435](https://github.com/tendermint/tendermint/issues/3435) `StartHTTPServer` / `StartHTTPAndTLSServer` now require a Config (use `rpcserver.DefaultConfig`)
* Blockchain Protocol
* P2P Protocol
### FEATURES:
- [config] [\#3269](https://github.com/tendermint/tendermint/issues/2826) New configuration values for controlling RPC subscriptions:
- `rpc.max_subscription_clients` sets the maximum number of unique clients
with open subscriptions
- `rpc.max_subscriptions_per_client`sets the maximum number of unique
subscriptions from a given client
- `rpc.timeout_broadcast_tx_commit` sets the time to wait for a tx to be committed during `/broadcast_tx_commit`
- [types] [\#2920](https://github.com/tendermint/tendermint/issues/2920) Add `time_iota_ms` to block's consensus parameters (not exposed to the application)
- [lite] [\#3269](https://github.com/tendermint/tendermint/issues/3269) Add `/unsubscribe_all` endpoint to unsubscribe from all events
- [mempool] [\#3079](https://github.com/tendermint/tendermint/issues/3079) Bound mempool memory usage via the `mempool.max_txs_bytes` configuration value. Set to 1GB by default. The mempool's current `txs_total_bytes` is exposed via `total_bytes` field in
`/num_unconfirmed_txs` and `/unconfirmed_txs` RPC endpoints.
### IMPROVEMENTS:
- [all] [\#3385](https://github.com/tendermint/tendermint/issues/3385), [\#3386](https://github.com/tendermint/tendermint/issues/3386) Various linting improvements
- [crypto] [\#3371](https://github.com/tendermint/tendermint/issues/3371) Copy in secp256k1 package from go-ethereum instead of importing
go-ethereum (@silasdavis)
- [deps] [\#3382](https://github.com/tendermint/tendermint/issues/3382) Don't pin repos without releases
- [deps] [\#3357](https://github.com/tendermint/tendermint/issues/3357), [\#3389](https://github.com/tendermint/tendermint/issues/3389), [\#3392](https://github.com/tendermint/tendermint/issues/3392) Update gogo/protobuf, golang/protobuf, levigo, golang.org/x/crypto
- [libs/common] [\#3238](https://github.com/tendermint/tendermint/issues/3238) exit with zero (0) code upon receiving SIGTERM/SIGINT
- [libs/db] [\#3378](https://github.com/tendermint/tendermint/issues/3378) CLevelDB#Stats now returns the following properties:
- leveldb.num-files-at-level{n}
- leveldb.stats
- leveldb.sstables
- leveldb.blockpool
- leveldb.cachedblock
- leveldb.openedtables
- leveldb.alivesnaps
- leveldb.aliveiters
- [privval] [\#3351](https://github.com/tendermint/tendermint/pull/3351) First part of larger refactoring that clarifies and separates concerns in the privval package.
### BUG FIXES:
- [blockchain] [\#3358](https://github.com/tendermint/tendermint/pull/3358) Fix timer leak in `BlockPool` (@guagualvcha)
- [cmd] [\#3408](https://github.com/tendermint/tendermint/issues/3408) Fix `testnet` command's panic when creating non-validator configs (using `--n` flag) (@srmo)
- [libs/db/remotedb/grpcdb] [\#3402](https://github.com/tendermint/tendermint/issues/3402) Close Iterator/ReverseIterator after use
- [libs/pubsub] [\#951](https://github.com/tendermint/tendermint/issues/951), [\#1880](https://github.com/tendermint/tendermint/issues/1880) Use non-blocking send when dispatching messages [ADR-33](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-033-pubsub.md)
- [lite] [\#3364](https://github.com/tendermint/tendermint/issues/3364) Fix `/validators` and `/abci_query` proxy endpoints
(@guagualvcha)
- [p2p/conn] [\#3347](https://github.com/tendermint/tendermint/issues/3347) Reject all-zero shared secrets in the Diffie-Hellman step of secret-connection
- [p2p] [\#3369](https://github.com/tendermint/tendermint/issues/3369) Do not panic when filter times out
- [p2p] [\#3359](https://github.com/tendermint/tendermint/pull/3359) Fix reconnecting report duplicate ID error due to race condition between adding peer to peerSet and starting it (@guagualvcha)
## v0.30.2
*March 10th, 2019*
This release fixes a CLevelDB memory leak. It was happening because we were not
closing the WriteBatch object after use. See [levigo's
godoc](https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close) for the
Close method. Special thanks goes to @Stumble who both reported an issue in
[cosmos-sdk](https://github.com/cosmos/cosmos-sdk/issues/3842) and provided a
fix here.
### BREAKING CHANGES:
* Go API
- [libs/db] [\#3842](https://github.com/cosmos/cosmos-sdk/issues/3842) Add Close() method to Batch interface (@Stumble)
### BUG FIXES:
- [libs/db] [\#3842](https://github.com/cosmos/cosmos-sdk/issues/3842) Fix CLevelDB memory leak (@Stumble)
## v0.30.1
*February 20th, 2019*
This release fixes a consensus halt and a DataCorruptionError after restart
discovered in `game_of_stakes_6`. It also fixes a security issue in the p2p
handshake by authenticating the NetAddress.ID of the peer we're dialing.
### IMPROVEMENTS:
* [config] [\#3291](https://github.com/tendermint/tendermint/issues/3291) Make
config.ResetTestRootWithChainID() create concurrency-safe test directories.
### BUG FIXES:
* [consensus] [\#3295](https://github.com/tendermint/tendermint/issues/3295)
Flush WAL on stop to prevent data corruption during graceful shutdown.
* [consensus] [\#3302](https://github.com/tendermint/tendermint/issues/3302)
Fix possible halt by resetting TriggeredTimeoutPrecommit before starting next height.
* [rpc] [\#3251](https://github.com/tendermint/tendermint/issues/3251) Fix
`/net_info#peers#remote_ip` format. New format spec:
* dotted decimal ("192.0.2.1"), if ip is an IPv4 or IP4-mapped IPv6 address
* IPv6 ("2001:db8::1"), if ip is a valid IPv6 address
* [cmd] [\#3314](https://github.com/tendermint/tendermint/issues/3314) Return
an error on `show_validator` when the private validator file does not exist.
* [p2p] [\#3010](https://github.com/tendermint/tendermint/issues/3010#issuecomment-464287627)
Authenticate a peer against its NetAddress.ID when dialing.
## v0.30.0
*February 8th, 2019*
This release fixes yet another issue with the proposer selection algorithm.
We hope it's the last one, but we won't be surprised if it's not.
We plan to one day expose the selection algorithm more directly to
the application ([\#3285](https://github.com/tendermint/tendermint/issues/3285)), and even to support randomness ([\#763](https://github.com/tendermint/tendermint/issues/763)).
For more, see issues marked
[proposer-selection](https://github.com/tendermint/tendermint/labels/proposer-selection).
This release also includes a fix to prevent Tendermint from including the same
piece of evidence in more than one block. This issue was reported by @chengwenxi in our
[bug bounty program](https://hackerone.com/tendermint).
### BREAKING CHANGES:
* Apps
- [state] [\#3222](https://github.com/tendermint/tendermint/issues/3222)
Duplicate updates for the same validator are forbidden. Apps must ensure
that a given `ResponseEndBlock.ValidatorUpdates` contains only one entry per pubkey.
* Go API
- [types] [\#3222](https://github.com/tendermint/tendermint/issues/3222)
Remove `Add` and `Update` methods from `ValidatorSet` in favor of new
`UpdateWithChangeSet`. This allows updates to be applied as a set, instead of
one at a time.
* Block Protocol
- [state] [\#3286](https://github.com/tendermint/tendermint/issues/3286) Blocks that include already committed evidence are invalid.
* P2P Protocol
- [consensus] [\#3222](https://github.com/tendermint/tendermint/issues/3222)
Validator updates are applied as a set, instead of one at a time, thus
impacting the proposer priority calculation. This ensures that the proposer
selection algorithm does not depend on the order of updates in
`ResponseEndBlock.ValidatorUpdates`.
### IMPROVEMENTS:
- [crypto] [\#3279](https://github.com/tendermint/tendermint/issues/3279) Use `btcec.S256().N` directly instead of hard coding a copy.
### BUG FIXES:
- [state] [\#3222](https://github.com/tendermint/tendermint/issues/3222) Fix validator set updates so they are applied as a set, rather
than one at a time. This makes the proposer selection algorithm independent of
the order of updates in `ResponseEndBlock.ValidatorUpdates`.
- [evidence] [\#3286](https://github.com/tendermint/tendermint/issues/3286) Don't add committed evidence to evidence pool.
## v0.29.2
*February 7th, 2019*
Special thanks to external contributors on this release:
@ackratos, @rickyyangz
**Note**: This release contains security sensitive patches in the `p2p` and
`crypto` packages:
- p2p:
- Partial fix for MITM attacks on the p2p connection. MITM conditions may
still exist. See [\#3010](https://github.com/tendermint/tendermint/issues/3010).
- crypto:
- Eliminate our fork of `btcd` and use the `btcd/btcec` library directly for
native secp256k1 signing. Note we still modify the signature encoding to
prevent malleability.
- Support the libsecp256k1 library via CGo through the `go-ethereum/crypto/secp256k1` package.
- Eliminate MixEntropy functions
### BREAKING CHANGES:
* Go API
- [crypto] [\#3278](https://github.com/tendermint/tendermint/issues/3278) Remove
MixEntropy functions
- [types] [\#3245](https://github.com/tendermint/tendermint/issues/3245) Commit uses `type CommitSig Vote` instead of `Vote` directly.
In preparation for removing redundant fields from the commit [\#1648](https://github.com/tendermint/tendermint/issues/1648)
### IMPROVEMENTS:
- [consensus] [\#3246](https://github.com/tendermint/tendermint/issues/3246) Better logging and notes on recovery for corrupted WAL file
- [crypto] [\#3163](https://github.com/tendermint/tendermint/issues/3163) Use ethereum's libsecp256k1 go-wrapper for signatures when cgo is available
- [crypto] [\#3162](https://github.com/tendermint/tendermint/issues/3162) Wrap btcd instead of forking it to keep up with fixes (used if cgo is not available)
- [makefile] [\#3233](https://github.com/tendermint/tendermint/issues/3233) Use golangci-lint instead of go-metalinter
- [tools] [\#3218](https://github.com/tendermint/tendermint/issues/3218) Add go-deadlock tool to help detect deadlocks
- [tools] [\#3106](https://github.com/tendermint/tendermint/issues/3106) Add tm-signer-harness test harness for remote signers
- [tests] [\#3258](https://github.com/tendermint/tendermint/issues/3258) Fixed a bunch of non-deterministic test failures
### BUG FIXES:
- [node] [\#3186](https://github.com/tendermint/tendermint/issues/3186) EventBus and indexerService should be started before first block (for replay last block on handshake) execution (@ackratos)
- [p2p] [\#3232](https://github.com/tendermint/tendermint/issues/3232) Fix infinite loop leading to addrbook deadlock for seed nodes
- [p2p] [\#3247](https://github.com/tendermint/tendermint/issues/3247) Fix panic in SeedMode when calling FlushStop and OnStop
concurrently
- [p2p] [\#3040](https://github.com/tendermint/tendermint/issues/3040) Fix MITM on secret connection by checking low-order points
- [privval] [\#3258](https://github.com/tendermint/tendermint/issues/3258) Fix race between sign requests and ping requests in socket that was causing messages to be corrupted
## v0.29.1
*January 24, 2019*
Special thanks to external contributors on this release:
@infinytum, @gauthamzz
This release contains two important fixes: one for p2p layer where we sometimes
were not closing connections and one for consensus layer where consensus with
no empty blocks (`create_empty_blocks = false`) could halt.
Friendly reminder, we have a [bug bounty
program](https://hackerone.com/tendermint).
### IMPROVEMENTS:
- [pex] [\#3037](https://github.com/tendermint/tendermint/issues/3037) Only log "Reached max attempts to dial" once
- [rpc] [\#3159](https://github.com/tendermint/tendermint/issues/3159) Expose
`triggered_timeout_commit` in the `/dump_consensus_state`
### BUG FIXES:
- [consensus] [\#3199](https://github.com/tendermint/tendermint/issues/3199) Fix consensus halt with no empty blocks from not resetting triggeredTimeoutCommit
- [p2p] [\#2967](https://github.com/tendermint/tendermint/issues/2967) Fix file descriptor leak
## v0.29.0
*January 21, 2019*
Special thanks to external contributors on this release:
@bradyjoestar, @kunaldhariwal, @gauthamzz, @hrharder
This release is primarily about making some breaking changes to
the Block protocol version before Cosmos launch, and to fixing more issues
in the proposer selection algorithm discovered on Cosmos testnets.
The Block protocol changes include using a standard Merkle tree format (RFC 6962),
fixing some inconsistencies between field orders in Vote and Proposal structs,
and constraining the hash of the ConsensusParams to include only a few fields.
The proposer selection algorithm saw significant progress,
including a [formal proof by @cwgoes for the base-case in Idris](https://github.com/cwgoes/tm-proposer-idris)
and a [much more detailed specification (still in progress) by
@ancazamfir](https://github.com/tendermint/tendermint/pull/3140).
Fixes to the proposer selection algorithm include normalizing the proposer
priorities to mitigate the effects of large changes to the validator set.
That said, we just discovered [another bug](https://github.com/tendermint/tendermint/issues/3181),
which will be fixed in the next breaking release.
While we are trying to stabilize the Block protocol to preserve compatibility
with old chains, there may be some final changes yet to come before Cosmos
launch as we continue to audit and test the software.
Friendly reminder, we have a [bug bounty
program](https://hackerone.com/tendermint).
### BREAKING CHANGES:
* CLI/RPC/Config
* Apps
- [state] [\#3049](https://github.com/tendermint/tendermint/issues/3049) Total voting power of the validator set is upper bounded by
`MaxInt64 / 8`. Apps must ensure they do not return changes to the validator
set that cause this maximum to be exceeded.
* Go API
- [node] [\#3082](https://github.com/tendermint/tendermint/issues/3082) MetricsProvider now requires you to pass a chain ID
- [types] [\#2713](https://github.com/tendermint/tendermint/issues/2713) Rename `TxProof.LeafHash` to `TxProof.Leaf`
- [crypto/merkle] [\#2713](https://github.com/tendermint/tendermint/issues/2713) `SimpleProof.Verify` takes a `leaf` instead of a
`leafHash` and performs the hashing itself
* Blockchain Protocol
* [crypto/merkle] [\#2713](https://github.com/tendermint/tendermint/issues/2713) Merkle trees now match the RFC 6962 specification
* [types] [\#3078](https://github.com/tendermint/tendermint/issues/3078) Re-order Timestamp and BlockID in CanonicalVote so it's
consistent with CanonicalProposal (BlockID comes
first)
* [types] [\#3165](https://github.com/tendermint/tendermint/issues/3165) Hash of ConsensusParams only includes BlockSize.MaxBytes and
BlockSize.MaxGas
* P2P Protocol
- [consensus] [\#3049](https://github.com/tendermint/tendermint/issues/3049) Normalize priorities to not exceed `2*TotalVotingPower` to mitigate unfair proposer selection
heavily preferring earlier joined validators in the case of an early bonded large validator unbonding
### FEATURES:
### IMPROVEMENTS:
- [rpc] [\#3065](https://github.com/tendermint/tendermint/issues/3065) Return maxPerPage (100), not defaultPerPage (30) if `per_page` is greater than the max 100.
- [instrumentation] [\#3082](https://github.com/tendermint/tendermint/issues/3082) Add `chain_id` label for all metrics
### BUG FIXES:
- [crypto] [\#3164](https://github.com/tendermint/tendermint/issues/3164) Update `btcd` fork for rare signRFC6979 bug
- [lite] [\#3171](https://github.com/tendermint/tendermint/issues/3171) Fix verifying large validator set changes
- [log] [\#3125](https://github.com/tendermint/tendermint/issues/3125) Fix year format
- [mempool] [\#3168](https://github.com/tendermint/tendermint/issues/3168) Limit tx size to fit in the max reactor msg size
- [scripts] [\#3147](https://github.com/tendermint/tendermint/issues/3147) Fix json2wal for large block parts (@bradyjoestar)
## v0.28.1
*January 18th, 2019*

View File

@@ -1,8 +1,6 @@
## v0.29.0
## v0.32.0
*TBD*
Special thanks to external contributors on this release:
**
### BREAKING CHANGES:

36
Gopkg.lock generated
View File

@@ -10,12 +10,11 @@
revision = "3a771d992973f24aa725d07868b467d1ddfceafb"
[[projects]]
branch = "master"
digest = "1:c0decf632843204d2b8781de7b26e7038584e2dcccc7e2f401e88ae85b1df2b7"
digest = "1:093bf93a65962e8191e3e8cd8fc6c363f83d43caca9739c906531ba7210a9904"
name = "github.com/btcsuite/btcd"
packages = ["btcec"]
pruneopts = "UT"
revision = "67e573d211ace594f1366b4ce9d39726c4b19bd0"
revision = "ed77733ec07dfc8a513741138419b8d9d3de9d2d"
[[projects]]
digest = "1:1d8e1cb71c33a9470bbbae09bfec09db43c6bf358dfcae13cd8807c4e2a9a2bf"
@@ -84,7 +83,7 @@
version = "v1.8.0"
[[projects]]
digest = "1:35621fe20f140f05a0c4ef662c26c0ab4ee50bca78aa30fe87d33120bd28165e"
digest = "1:95e1006e41c641abd2f365dfa0f1213c04da294e7cd5f0bf983af234b775db64"
name = "github.com/gogo/protobuf"
packages = [
"gogoproto",
@@ -95,11 +94,11 @@
"types",
]
pruneopts = "UT"
revision = "636bf0302bc95575d69441b25a2603156ffdddf1"
version = "v1.1.1"
revision = "ba06b47c162d49f2af050fb4c75bcbc86a159d5c"
version = "v1.2.1"
[[projects]]
digest = "1:17fe264ee908afc795734e8c4e63db2accabaf57326dbf21763a7d6b86096260"
digest = "1:239c4c7fd2159585454003d9be7207167970194216193a8a210b8d29576f19c9"
name = "github.com/golang/protobuf"
packages = [
"proto",
@@ -109,8 +108,8 @@
"ptypes/timestamp",
]
pruneopts = "UT"
revision = "b4deda0973fb4c70b50d226b1af49f3da59f5265"
version = "v1.1.0"
revision = "c823c79ea1570fb5ff454033735a8e68575d1d0f"
version = "v1.3.0"
[[projects]]
branch = "master"
@@ -155,11 +154,12 @@
version = "v1.0"
[[projects]]
digest = "1:39b27d1381a30421f9813967a5866fba35dc1d4df43a6eefe3b7a5444cb07214"
digest = "1:a74b5a8e34ee5843cd6e65f698f3e75614f812ff170c2243425d75bc091e9af2"
name = "github.com/jmhodges/levigo"
packages = ["."]
pruneopts = "UT"
revision = "c42d9e0ca023e2198120196f842701bb4c55d7b9"
revision = "853d788c5c416eaaee5b044570784a96c7a26975"
version = "v1.0.0"
[[projects]]
branch = "master"
@@ -360,13 +360,6 @@
pruneopts = "UT"
revision = "6b91fda63f2e36186f1c9d0e48578defb69c5d43"
[[projects]]
digest = "1:605b6546f3f43745695298ec2d342d3e952b6d91cdf9f349bea9315f677d759f"
name = "github.com/tendermint/btcd"
packages = ["btcec"]
pruneopts = "UT"
revision = "e5840949ff4fff0c56f9b6a541e22b63581ea9df"
[[projects]]
digest = "1:ad9c4c1a4e7875330b1f62906f2830f043a23edb5db997e3a5ac5d3e6eadf80a"
name = "github.com/tendermint/go-amino"
@@ -376,7 +369,8 @@
version = "v0.14.1"
[[projects]]
digest = "1:00d2b3e64cdc3fa69aa250dfbe4cc38c4837d4f37e62279be2ae52107ffbbb44"
branch = "master"
digest = "1:f4edb30d5ff238e2abba10457010f74cd55ae20bbda8c54db1a07155fa020490"
name = "golang.org/x/crypto"
packages = [
"bcrypt",
@@ -397,7 +391,7 @@
"salsa20/salsa",
]
pruneopts = "UT"
revision = "505ab145d0a99da450461ae2c1a9f6cd10d1f447"
revision = "8dd112bcdc25174059e45e07517d9fc663123347"
[[projects]]
digest = "1:d36f55a999540d29b6ea3c2ea29d71c76b1d9853fdcd3e5c5cb4836f2ba118f1"
@@ -503,6 +497,7 @@
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"github.com/btcsuite/btcd/btcec",
"github.com/btcsuite/btcutil/base58",
"github.com/btcsuite/btcutil/bech32",
"github.com/fortytw2/leaktest",
@@ -534,7 +529,6 @@
"github.com/syndtr/goleveldb/leveldb/errors",
"github.com/syndtr/goleveldb/leveldb/iterator",
"github.com/syndtr/goleveldb/leveldb/opt",
"github.com/tendermint/btcd/btcec",
"github.com/tendermint/go-amino",
"golang.org/x/crypto/bcrypt",
"golang.org/x/crypto/chacha20poly1305",

View File

@@ -28,11 +28,11 @@
[[constraint]]
name = "github.com/gogo/protobuf"
version = "~1.1.1"
version = "~1.2.1"
[[constraint]]
name = "github.com/golang/protobuf"
version = "~1.1.0"
version = "~1.3.0"
# Allow only minor releases for other libraries
[[constraint]]
@@ -75,34 +75,18 @@
name = "github.com/prometheus/client_golang"
version = "^0.9.1"
###################################
## Some repos dont have releases.
## Pin to revision
[[constraint]]
name = "golang.org/x/crypto"
revision = "505ab145d0a99da450461ae2c1a9f6cd10d1f447"
[[override]]
name = "github.com/jmhodges/levigo"
revision = "c42d9e0ca023e2198120196f842701bb4c55d7b9"
version = "^1.0.0"
# last revision used by go-crypto
[[constraint]]
name = "github.com/btcsuite/btcutil"
revision = "d4cc87b860166d00d6b5b9e0d3b3d71d6088d4d4"
###################################
## Repos which don't have releases.
[[constraint]]
name = "github.com/tendermint/btcd"
revision = "e5840949ff4fff0c56f9b6a541e22b63581ea9df"
[[constraint]]
name = "github.com/rcrowley/go-metrics"
revision = "e2704e165165ec55d062f5919b4b29494e9fa790"
[[constraint]]
name = "golang.org/x/net"
revision = "292b43bbf7cb8d35ddf40f8d5100ef3837cced3f"
## - github.com/btcsuite/btcd
## - golang.org/x/crypto
## - github.com/btcsuite/btcutil
## - github.com/rcrowley/go-metrics
## - golang.org/x/net
[prune]
go-tests = true

View File

@@ -1,7 +1,7 @@
GOTOOLS = \
github.com/mitchellh/gox \
github.com/golang/dep/cmd/dep \
github.com/alecthomas/gometalinter \
github.com/golangci/golangci-lint/cmd/golangci-lint \
github.com/gogo/protobuf/protoc-gen-gogo \
github.com/square/certstrap
GOBIN?=${GOPATH}/bin
@@ -11,8 +11,6 @@ INCLUDE = -I=. -I=${GOPATH}/src -I=${GOPATH}/src/github.com/gogo/protobuf/protob
BUILD_TAGS?='tendermint'
BUILD_FLAGS = -ldflags "-X github.com/tendermint/tendermint/version.GitCommit=`git rev-parse --short=8 HEAD`"
LINT_FLAGS = --exclude '.*\.pb\.go' --exclude 'vendor/*' --vendor --deadline=600s
all: check build test install
check: check_tools get_vendor_deps
@@ -82,10 +80,6 @@ get_tools:
@echo "--> Installing tools"
./scripts/get_tools.sh
get_dev_tools:
@echo "--> Downloading linters (this may take awhile)"
$(GOPATH)/src/github.com/alecthomas/gometalinter/scripts/install.sh -b $(GOBIN)
update_tools:
@echo "--> Updating tools"
./scripts/get_tools.sh
@@ -137,7 +131,7 @@ clean_certs:
rm -f db/remotedb/::.crt db/remotedb/::.key
test_libs: gen_certs
GOCACHE=off go test -tags gcc $(PACKAGES)
go test -tags gcc $(PACKAGES)
make clean_certs
grpc_dbserver:
@@ -220,12 +214,28 @@ vagrant_test:
### go tests
test:
@echo "--> Running go test"
@GOCACHE=off go test -p 1 $(PACKAGES)
@go test -p 1 $(PACKAGES)
test_race:
@echo "--> Running go test --race"
@GOCACHE=off go test -p 1 -v -race $(PACKAGES)
@go test -p 1 -v -race $(PACKAGES)
# uses https://github.com/sasha-s/go-deadlock/ to detect potential deadlocks
test_with_deadlock:
make set_with_deadlock
make test
make cleanup_after_test_with_deadlock
set_with_deadlock:
find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/sync.RWMutex/deadlock.RWMutex/'
find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/sync.Mutex/deadlock.Mutex/'
find . -name "*.go" | grep -v "vendor/" | xargs -n 1 goimports -w
# cleanes up after you ran test_with_deadlock
cleanup_after_test_with_deadlock:
find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/deadlock.RWMutex/sync.RWMutex/'
find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/deadlock.Mutex/sync.Mutex/'
find . -name "*.go" | grep -v "vendor/" | xargs -n 1 goimports -w
########################################
### Formatting, linting, and vetting
@@ -233,38 +243,9 @@ test_race:
fmt:
@go fmt ./...
metalinter:
lint:
@echo "--> Running linter"
@gometalinter $(LINT_FLAGS) --disable-all \
--enable=deadcode \
--enable=gosimple \
--enable=misspell \
--enable=safesql \
./...
#--enable=gas \
#--enable=maligned \
#--enable=dupl \
#--enable=errcheck \
#--enable=goconst \
#--enable=gocyclo \
#--enable=goimports \
#--enable=golint \ <== comments on anything exported
#--enable=gotype \
#--enable=ineffassign \
#--enable=interfacer \
#--enable=megacheck \
#--enable=staticcheck \
#--enable=structcheck \
#--enable=unconvert \
#--enable=unparam \
#--enable=unused \
#--enable=varcheck \
#--enable=vet \
#--enable=vetshadow \
metalinter_all:
@echo "--> Running linter (all)"
gometalinter $(LINT_FLAGS) --enable-all --disable=lll ./...
@golangci-lint run
DESTINATION = ./index.html.md
@@ -288,13 +269,11 @@ build-docker:
### Local testnet using docker
# Build linux binary on other platforms
build-linux:
build-linux: get_tools get_vendor_deps
GOOS=linux GOARCH=amd64 $(MAKE) build
build-docker-localnode:
cd networks/local
make
cd -
@cd networks/local && make
# Run a 4-node testnet locally
localnet-start: localnet-stop
@@ -332,4 +311,4 @@ build-slate:
# To avoid unintended conflicts with file names, always add to .PHONY
# unless there is a reason not to.
# https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html
.PHONY: check build build_race build_abci dist install install_abci check_dep check_tools get_tools get_dev_tools update_tools get_vendor_deps draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all build_c install_c
.PHONY: check build build_race build_abci dist install install_abci check_dep check_tools get_tools update_tools get_vendor_deps draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all build_c install_c test_with_deadlock cleanup_after_test_with_deadlock lint

158
PHILOSOPHY.md Normal file
View File

@@ -0,0 +1,158 @@
## Design goals
The design goals for Tendermint (and the SDK and related libraries) are:
* Simplicity and Legibility
* Parallel performance, namely ability to utilize multicore architecture
* Ability to evolve the codebase bug-free
* Debuggability
* Complete correctness that considers all edge cases, esp in concurrency
* Future-proof modular architecture, message protocol, APIs, and encapsulation
### Justification
Legibility is key to maintaining bug-free software as it evolves toward more
optimizations, more ease of debugging, and additional features.
It is too easy to introduce bugs over time by replacing lines of code with
those that may panic, which means ideally locks are unlocked by defer
statements.
For example,
```go
func (obj *MyObj) something() {
mtx.Lock()
obj.something = other
mtx.Unlock()
}
```
It is too easy to refactor the codebase in the future to replace `other` with
`other.String()` for example, and this may introduce a bug that causes a
deadlock. So as much as reasonably possible, we need to be using defer
statements, even though it introduces additional overhead.
If it is necessary to optimize the unlocking of mutex locks, the solution is
more modularity via smaller functions, so that defer'd unlocks are scoped
within a smaller function.
Similarly, idiomatic for-loops should always be preferred over those that use
custom counters, because it is too easy to evolve the body of a for-loop to
become more complicated over time, and it becomes more and more difficult to
assess the correctness of such a for-loop by visual inspection.
### On performance
It doesn't matter whether there are alternative implementations that are 2x or
3x more performant, when the software doesn't work, deadlocks, or if bugs
cannot be debugged. By taking advantage of multicore concurrency, the
Tendermint implementation will at least be an order of magnitude within the
range of what is theoretically possible. The design philosophy of Tendermint,
and the choice of Go as implementation language, is designed to make Tendermint
implementation the standard specification for concurrent BFT software.
By focusing on the message protocols (e.g. ABCI, p2p messages), and
encapsulation e.g. IAVL module, (relatively) independent reactors, we are both
implementing a standard implementation to be used as the specification for
future implementations in more optimizable languages like Rust, Java, and C++;
as well as creating sufficiently performant software. Tendermint Core will
never be as fast as future implementations of the Tendermint Spec, because Go
isn't designed to be as fast as possible. The advantage of using Go is that we
can develop the whole stack of modular components **faster** than in other
languages.
Furthermore, the real bottleneck is in the application layer, and it isn't
necessary to support more than a sufficiently decentralized set of validators
(e.g. 100 ~ 300 validators is sufficient, with delegated bonded PoS).
Instead of optimizing Tendermint performance down to the metal, lets focus on
optimizing on other matters, namely ability to push feature complete software
that works well enough, can be debugged and maintained, and can serve as a spec
for future implementations.
### On encapsulation
In order to create maintainable, forward-optimizable software, it is critical
to develop well-encapsulated objects that have well understood properties, and
to re-use these easy-to-use-correctly components as building blocks for further
encapsulated meta-objects.
For example, mutexes are cheap enough for Tendermint's design goals when there
isn't goroutine contention, so it is encouraged to create concurrency safe
structures with struct-level mutexes. If they are used in the context of
non-concurrent logic, then the performance is good enough. If they are used in
the context of concurrent logic, then it will still perform correctly.
Examples of this design principle can be seen in the types.ValidatorSet struct,
and the cmn.Rand struct. It's one single struct declaration that can be used
in both concurrent and non-concurrent logic, and due to its well encapsulation,
it's easy to get the usage of the mutex right.
#### example: cmn.Rand:
`The default Source is safe for concurrent use by multiple goroutines, but
Sources created by NewSource are not`. The reason why the default
package-level source is safe for concurrent use is because it is protected (see
`lockedSource` in https://golang.org/src/math/rand/rand.go).
But we shouldn't rely on the global source, we should be creating our own
Rand/Source instances and using them, especially for determinism in testing.
So it is reasonable to have cmn.Rand be protected by a mutex. Whether we want
our own implementation of Rand is another question, but the answer there is
also in the affirmative. Sometimes you want to know where Rand is being used
in your code, so it becomes a simple matter of dropping in a log statement to
inject inspectability into Rand usage. Also, it is nice to be able to extend
the functionality of Rand with custom methods. For these reasons, and for the
reasons which is outlined in this design philosophy document, we should
continue to use the cmn.Rand object, with mutex protection.
Another key aspect of good encapsulation is the choice of exposed vs unexposed
methods. It should be clear to the reader of the code, which methods are
intended to be used in what context, and what safe usage is. Part of this is
solved by hiding methods via unexported methods. Another part of this is
naming conventions on the methods (e.g. underscores) with good documentation,
and code organization. If there are too many exposed methods and it isn't
clear what methods have what side effects, then there is something wrong about
the design of abstractions that should be revisited.
### On concurrency
In order for Tendermint to remain relevant in the years to come, it is vital
for Tendermint to take advantage of multicore architectures. Due to the nature
of the problem, namely consensus across a concurrent p2p gossip network, and to
handle RPC requests for a large number of consuming subscribers, it is
unavoidable for Tendermint development to require expertise in concurrency
design, especially when it comes to the reactor design, and also for RPC
request handling.
## Guidelines
Here are some guidelines for designing for (sufficient) performance and concurrency:
* Mutex locks are cheap enough when there isn't contention.
* Do not optimize code without analytical or observed proof that it is in a hot path.
* Don't over-use channels when mutex locks w/ encapsulation are sufficient.
* The need to drain channels are often a hint of unconsidered edge cases.
* The creation of O(N) one-off goroutines is generally technical debt that
needs to get addressed sooner than later. Avoid creating too many
goroutines as a patch around incomplete concurrency design, or at least be
aware of the debt and do not invest in the debt. On the other hand, Tendermint
is designed to have a limited number of peers (e.g. 10 or 20), so the creation
of O(C) goroutines per O(P) peers is still O(C\*P=constant).
* Use defer statements to unlock as much as possible. If you want to unlock sooner,
try to create more modular functions that do make use of defer statements.
## Matras
* Premature optimization kills
* Readability is paramount
* Beautiful is better than fast.
* In the face of ambiguity, refuse the temptation to guess.
* In the face of bugs, refuse the temptation to cover the bug.
* There should be one-- and preferably only one --obvious way to do it.

View File

@@ -8,7 +8,7 @@ Or [Blockchain](https://en.wikipedia.org/wiki/Blockchain_(database)), for short.
[![API Reference](
https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667
)](https://godoc.org/github.com/tendermint/tendermint)
[![Go version](https://img.shields.io/badge/go-1.10.4-blue.svg)](https://github.com/moovweb/gvm)
[![Go version](https://img.shields.io/badge/go-1.12.0-blue.svg)](https://github.com/moovweb/gvm)
[![riot.im](https://img.shields.io/badge/riot.im-JOIN%20CHAT-green.svg)](https://riot.im/app/#/room/#tendermint:matrix.org)
[![license](https://img.shields.io/github/license/tendermint/tendermint.svg)](https://github.com/tendermint/tendermint/blob/master/LICENSE)
[![](https://tokei.rs/b1/github/tendermint/tendermint?category=lines)](https://github.com/tendermint/tendermint)
@@ -96,6 +96,7 @@ include the in-process Go APIs.
That said, breaking changes in the following packages will be documented in the
CHANGELOG even if they don't lead to MINOR version bumps:
- crypto
- types
- rpc/client
- config

View File

@@ -3,6 +3,92 @@
This guide provides steps to be followed when you upgrade your applications to
a newer version of Tendermint Core.
## v0.31.0
This release contains a breaking change to the behaviour of the pubsub system.
It also contains some minor breaking changes in the Go API and ABCI.
There are no changes to the block or p2p protocols, so v0.31.0 should work fine
with blockchains created from the v0.30 series.
### RPC
The pubsub no longer blocks on publishing. This may cause some WebSocket (WS) clients to stop working as expected.
If your WS client is not consuming events fast enough, Tendermint can terminate the subscription.
In this case, the WS client will receive an error with description:
```json
{
"jsonrpc": "2.0",
"id": "{ID}#event",
"error": {
"code": -32000,
"msg": "Server error",
"data": "subscription was cancelled (reason: client is not pulling messages fast enough)" // or "subscription was cancelled (reason: Tendermint exited)"
}
}
Additionally, there are now limits on the number of subscribers and
subscriptions that can be active at once. See the new
`rpc.max_subscription_clients` and `rpc.max_subscriptions_per_client` values to
configure this.
```
### Applications
Simple rename of `ConsensusParams.BlockSize` to `ConsensusParams.Block`.
The `ConsensusParams.Block.TimeIotaMS` field was also removed. It's configured
in the ConsensusParsm in genesis.
### Go API
See the [CHANGELOG](CHANGELOG.md). These are relatively straight forward.
## v0.30.0
This release contains a breaking change to both the block and p2p protocols,
however it may be compatible with blockchains created with v0.29.0 depending on
the chain history. If your blockchain has not included any pieces of evidence,
or no piece of evidence has been included in more than one block,
and if your application has never returned multiple updates
for the same validator in a single block, then v0.30.0 will work fine with
blockchains created with v0.29.0.
The p2p protocol change is to fix the proposer selection algorithm again.
Note that proposer selection is purely a p2p concern right
now since the algorithm is only relevant during real time consensus.
This change is thus compatible with v0.29.0, but
all nodes must be upgraded to avoid disagreements on the proposer.
### Applications
Applications must ensure they do not return duplicates in
`ResponseEndBlock.ValidatorUpdates`. A pubkey must only appear once per set of
updates. Duplicates will cause irrecoverable failure. If you have a very good
reason why we shouldn't do this, please open an issue.
## v0.29.0
This release contains some breaking changes to the block and p2p protocols,
and will not be compatible with any previous versions of the software, primarily
due to changes in how various data structures are hashed.
Any implementations of Tendermint blockchain verification, including lite clients,
will need to be updated. For specific details:
- [Merkle tree](./docs/spec/blockchain/encoding.md#merkle-trees)
- [ConsensusParams](./docs/spec/blockchain/state.md#consensusparams)
There was also a small change to field ordering in the vote struct. Any
implementations of an out-of-process validator (like a Key-Management Server)
will need to be updated. For specific details:
- [Vote](https://github.com/tendermint/tendermint/blob/develop/docs/spec/consensus/signing.md#votes)
Finally, the proposer selection algorithm continues to evolve. See the
[work-in-progress
specification](https://github.com/tendermint/tendermint/pull/3140).
For everything else, please see the [CHANGELOG](./CHANGELOG.md#v0.29.0).
## v0.28.0
This release breaks the format for the `priv_validator.json` file

12
Vagrantfile vendored
View File

@@ -29,10 +29,14 @@ Vagrant.configure("2") do |config|
usermod -a -G docker vagrant
# install go
wget -q https://dl.google.com/go/go1.11.linux-amd64.tar.gz
tar -xvf go1.11.linux-amd64.tar.gz
wget -q https://dl.google.com/go/go1.12.linux-amd64.tar.gz
tar -xvf go1.12.linux-amd64.tar.gz
mv go /usr/local
rm -f go1.11.linux-amd64.tar.gz
rm -f go1.12.linux-amd64.tar.gz
# install nodejs (for docs)
curl -sL https://deb.nodesource.com/setup_11.x | bash -
apt-get install -y nodejs
# cleanup
apt-get autoremove -y
@@ -53,6 +57,6 @@ Vagrant.configure("2") do |config|
# get all deps and tools, ready to install/test
su - vagrant -c 'source /home/vagrant/.bash_profile'
su - vagrant -c 'cd /home/vagrant/go/src/github.com/tendermint/tendermint && make get_tools && make get_dev_tools && make get_vendor_deps'
su - vagrant -c 'cd /home/vagrant/go/src/github.com/tendermint/tendermint && make get_tools && make get_vendor_deps'
SHELL
end

View File

@@ -129,7 +129,7 @@ func (cli *grpcClient) EchoAsync(msg string) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Echo{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Echo{Echo: res}})
}
func (cli *grpcClient) FlushAsync() *ReqRes {
@@ -138,7 +138,7 @@ func (cli *grpcClient) FlushAsync() *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Flush{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Flush{Flush: res}})
}
func (cli *grpcClient) InfoAsync(params types.RequestInfo) *ReqRes {
@@ -147,7 +147,7 @@ func (cli *grpcClient) InfoAsync(params types.RequestInfo) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Info{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Info{Info: res}})
}
func (cli *grpcClient) SetOptionAsync(params types.RequestSetOption) *ReqRes {
@@ -156,7 +156,7 @@ func (cli *grpcClient) SetOptionAsync(params types.RequestSetOption) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_SetOption{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_SetOption{SetOption: res}})
}
func (cli *grpcClient) DeliverTxAsync(tx []byte) *ReqRes {
@@ -165,7 +165,7 @@ func (cli *grpcClient) DeliverTxAsync(tx []byte) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_DeliverTx{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_DeliverTx{DeliverTx: res}})
}
func (cli *grpcClient) CheckTxAsync(tx []byte) *ReqRes {
@@ -174,7 +174,7 @@ func (cli *grpcClient) CheckTxAsync(tx []byte) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_CheckTx{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_CheckTx{CheckTx: res}})
}
func (cli *grpcClient) QueryAsync(params types.RequestQuery) *ReqRes {
@@ -183,7 +183,7 @@ func (cli *grpcClient) QueryAsync(params types.RequestQuery) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Query{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Query{Query: res}})
}
func (cli *grpcClient) CommitAsync() *ReqRes {
@@ -192,7 +192,7 @@ func (cli *grpcClient) CommitAsync() *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Commit{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_Commit{Commit: res}})
}
func (cli *grpcClient) InitChainAsync(params types.RequestInitChain) *ReqRes {
@@ -201,7 +201,7 @@ func (cli *grpcClient) InitChainAsync(params types.RequestInitChain) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_InitChain{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_InitChain{InitChain: res}})
}
func (cli *grpcClient) BeginBlockAsync(params types.RequestBeginBlock) *ReqRes {
@@ -210,7 +210,7 @@ func (cli *grpcClient) BeginBlockAsync(params types.RequestBeginBlock) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_BeginBlock{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_BeginBlock{BeginBlock: res}})
}
func (cli *grpcClient) EndBlockAsync(params types.RequestEndBlock) *ReqRes {
@@ -219,7 +219,7 @@ func (cli *grpcClient) EndBlockAsync(params types.RequestEndBlock) *ReqRes {
if err != nil {
cli.StopForError(err)
}
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_EndBlock{res}})
return cli.finishAsyncCall(req, &types.Response{Value: &types.Response_EndBlock{EndBlock: res}})
}
func (cli *grpcClient) finishAsyncCall(req *types.Request, res *types.Response) *ReqRes {

View File

@@ -394,7 +394,6 @@ func cmdConsole(cmd *cobra.Command, args []string) error {
return err
}
}
return nil
}
func muxOnCommands(cmd *cobra.Command, pArgs []string) error {
@@ -637,9 +636,7 @@ func cmdQuery(cmd *cobra.Command, args []string) error {
}
func cmdCounter(cmd *cobra.Command, args []string) error {
app := counter.NewCounterApplication(flagSerial)
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
// Start the listener
@@ -652,12 +649,14 @@ func cmdCounter(cmd *cobra.Command, args []string) error {
return err
}
// Wait forever
cmn.TrapSignal(func() {
// Stop upon receiving SIGTERM or CTRL-C.
cmn.TrapSignal(logger, func() {
// Cleanup
srv.Stop()
})
return nil
// Run forever.
select {}
}
func cmdKVStore(cmd *cobra.Command, args []string) error {
@@ -682,12 +681,14 @@ func cmdKVStore(cmd *cobra.Command, args []string) error {
return err
}
// Wait forever
cmn.TrapSignal(func() {
// Stop upon receiving SIGTERM or CTRL-C.
cmn.TrapSignal(logger, func() {
// Cleanup
srv.Stop()
})
return nil
// Run forever.
select {}
}
//--------------------------------------------------------------------------------

View File

@@ -83,7 +83,7 @@ func TestWriteReadMessage2(t *testing.T) {
Log: phrase,
GasWanted: 10,
Tags: []cmn.KVPair{
cmn.KVPair{Key: []byte("abc"), Value: []byte("def")},
{Key: []byte("abc"), Value: []byte("def")},
},
},
// TODO: add the rest

File diff suppressed because it is too large Load Diff

View File

@@ -4,9 +4,9 @@ package types;
// For more information on gogo.proto, see:
// https://github.com/gogo/protobuf/blob/master/extensions.md
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
import "google/protobuf/timestamp.proto";
import "github.com/tendermint/tendermint/libs/common/types.proto";
import "github.com/tendermint/tendermint/crypto/merkle/merkle.proto";
import "github.com/tendermint/tendermint/libs/common/types.proto";
import "google/protobuf/timestamp.proto";
// This file is copied from http://github.com/tendermint/abci
// NOTE: When using custom types, mind the warnings.
@@ -207,13 +207,13 @@ message ResponseCommit {
// ConsensusParams contains all consensus-relevant parameters
// that can be adjusted by the abci app
message ConsensusParams {
BlockSizeParams block_size = 1;
BlockParams block = 1;
EvidenceParams evidence = 2;
ValidatorParams validator = 3;
}
// BlockSize contains limits on the block size.
message BlockSizeParams {
// BlockParams contains limits on the block size and timestamp.
message BlockParams {
// Note: must be greater than 0
int64 max_bytes = 1;
// Note: must be greater or equal to -1

View File

@@ -1479,15 +1479,15 @@ func TestConsensusParamsMarshalTo(t *testing.T) {
}
}
func TestBlockSizeParamsProto(t *testing.T) {
func TestBlockParamsProto(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSizeParams(popr, false)
p := NewPopulatedBlockParams(popr, false)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockSizeParams{}
msg := &BlockParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -1510,10 +1510,10 @@ func TestBlockSizeParamsProto(t *testing.T) {
}
}
func TestBlockSizeParamsMarshalTo(t *testing.T) {
func TestBlockParamsMarshalTo(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSizeParams(popr, false)
p := NewPopulatedBlockParams(popr, false)
size := p.Size()
dAtA := make([]byte, size)
for i := range dAtA {
@@ -1523,7 +1523,7 @@ func TestBlockSizeParamsMarshalTo(t *testing.T) {
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockSizeParams{}
msg := &BlockParams{}
if err := github_com_gogo_protobuf_proto.Unmarshal(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -2675,16 +2675,16 @@ func TestConsensusParamsJSON(t *testing.T) {
t.Fatalf("seed = %d, %#v !Json Equal %#v", seed, msg, p)
}
}
func TestBlockSizeParamsJSON(t *testing.T) {
func TestBlockParamsJSON(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSizeParams(popr, true)
p := NewPopulatedBlockParams(popr, true)
marshaler := github_com_gogo_protobuf_jsonpb.Marshaler{}
jsondata, err := marshaler.MarshalToString(p)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
msg := &BlockSizeParams{}
msg := &BlockParams{}
err = github_com_gogo_protobuf_jsonpb.UnmarshalString(jsondata, msg)
if err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
@@ -3637,12 +3637,12 @@ func TestConsensusParamsProtoCompactText(t *testing.T) {
}
}
func TestBlockSizeParamsProtoText(t *testing.T) {
func TestBlockParamsProtoText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSizeParams(popr, true)
p := NewPopulatedBlockParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.MarshalTextString(p)
msg := &BlockSizeParams{}
msg := &BlockParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -3651,12 +3651,12 @@ func TestBlockSizeParamsProtoText(t *testing.T) {
}
}
func TestBlockSizeParamsProtoCompactText(t *testing.T) {
func TestBlockParamsProtoCompactText(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSizeParams(popr, true)
p := NewPopulatedBlockParams(popr, true)
dAtA := github_com_gogo_protobuf_proto.CompactTextString(p)
msg := &BlockSizeParams{}
msg := &BlockParams{}
if err := github_com_gogo_protobuf_proto.UnmarshalText(dAtA, msg); err != nil {
t.Fatalf("seed = %d, err = %v", seed, err)
}
@@ -4573,10 +4573,10 @@ func TestConsensusParamsSize(t *testing.T) {
}
}
func TestBlockSizeParamsSize(t *testing.T) {
func TestBlockParamsSize(t *testing.T) {
seed := time.Now().UnixNano()
popr := math_rand.New(math_rand.NewSource(seed))
p := NewPopulatedBlockSizeParams(popr, true)
p := NewPopulatedBlockParams(popr, true)
size2 := github_com_gogo_protobuf_proto.Size(p)
dAtA, err := github_com_gogo_protobuf_proto.Marshal(p)
if err != nil {

View File

@@ -4,7 +4,7 @@ import (
"testing"
"time"
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
proto "github.com/tendermint/tendermint/benchmarks/proto"
"github.com/tendermint/tendermint/crypto/ed25519"

View File

@@ -299,6 +299,9 @@ func (pool *BlockPool) removePeer(peerID p2p.ID) {
requester.redo(peerID)
}
}
if p, exist := pool.peers[peerID]; exist && p.timeout != nil {
p.timeout.Stop()
}
delete(pool.peers, peerID)
}
@@ -363,7 +366,8 @@ func (pool *BlockPool) sendError(err error, peerID p2p.ID) {
pool.errorsCh <- peerError{err, peerID}
}
// unused by tendermint; left for debugging purposes
// for debugging purposes
//nolint:unused
func (pool *BlockPool) debug() string {
pool.mtx.Lock()
defer pool.mtx.Unlock()

View File

@@ -8,7 +8,6 @@ import (
amino "github.com/tendermint/go-amino"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/libs/log"
"github.com/tendermint/tendermint/p2p"
sm "github.com/tendermint/tendermint/state"
@@ -302,7 +301,7 @@ FOR_LOOP:
firstParts := first.MakePartSet(types.BlockPartSizeBytes)
firstPartsHeader := firstParts.Header()
firstID := types.BlockID{first.Hash(), firstPartsHeader}
firstID := types.BlockID{Hash: first.Hash(), PartsHeader: firstPartsHeader}
// Finally, verify the first block using the second's commit
// NOTE: we can probably make this more efficient, but note that calling
// first.Hash() doesn't verify the tx contents, so MakePartSet() is
@@ -338,8 +337,7 @@ FOR_LOOP:
state, err = bcR.blockExec.ApplyBlock(state, firstID, first)
if err != nil {
// TODO This is bad, are we zombie?
cmn.PanicQ(fmt.Sprintf("Failed to process committed block (%d:%X): %v",
first.Height, first.Hash(), err))
panic(fmt.Sprintf("Failed to process committed block (%d:%X): %v", first.Height, first.Hash(), err))
}
blocksSynced++

View File

@@ -1,6 +1,7 @@
package blockchain
import (
"os"
"sort"
"testing"
"time"
@@ -95,13 +96,13 @@ func newBlockchainReactor(logger log.Logger, genDoc *types.GenesisDoc, privVals
// let's add some blocks in
for blockHeight := int64(1); blockHeight <= maxBlockHeight; blockHeight++ {
lastCommit := &types.Commit{}
lastCommit := types.NewCommit(types.BlockID{}, nil)
if blockHeight > 1 {
lastBlockMeta := blockStore.LoadBlockMeta(blockHeight - 1)
lastBlock := blockStore.LoadBlock(blockHeight - 1)
vote := makeVote(&lastBlock.Header, lastBlockMeta.BlockID, state.Validators, privVals[0])
lastCommit = &types.Commit{Precommits: []*types.Vote{vote}, BlockID: lastBlockMeta.BlockID}
vote := makeVote(&lastBlock.Header, lastBlockMeta.BlockID, state.Validators, privVals[0]).CommitSig()
lastCommit = types.NewCommit(lastBlockMeta.BlockID, []*types.CommitSig{vote})
}
thisBlock := makeBlock(blockHeight, state, lastCommit)
@@ -125,6 +126,7 @@ func newBlockchainReactor(logger log.Logger, genDoc *types.GenesisDoc, privVals
func TestNoBlockResponse(t *testing.T) {
config = cfg.ResetTestRoot("blockchain_reactor_test")
defer os.RemoveAll(config.RootDir)
genDoc, privVals := randGenesisDoc(1, false, 30)
maxBlockHeight := int64(65)
@@ -184,6 +186,7 @@ func TestNoBlockResponse(t *testing.T) {
// that seems extreme.
func TestBadBlockStopsPeer(t *testing.T) {
config = cfg.ResetTestRoot("blockchain_reactor_test")
defer os.RemoveAll(config.RootDir)
genDoc, privVals := randGenesisDoc(1, false, 30)
maxBlockHeight := int64(148)

View File

@@ -3,9 +3,11 @@ package blockchain
import (
"bytes"
"fmt"
"os"
"runtime/debug"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -20,7 +22,17 @@ import (
tmtime "github.com/tendermint/tendermint/types/time"
)
func makeStateAndBlockStore(logger log.Logger) (sm.State, *BlockStore) {
// A cleanupFunc cleans up any config / test files created for a particular
// test.
type cleanupFunc func()
// make a Commit with a single vote containing just the height and a timestamp
func makeTestCommit(height int64, timestamp time.Time) *types.Commit {
commitSigs := []*types.CommitSig{{Height: height, Timestamp: timestamp}}
return types.NewCommit(types.BlockID{}, commitSigs)
}
func makeStateAndBlockStore(logger log.Logger) (sm.State, *BlockStore, cleanupFunc) {
config := cfg.ResetTestRoot("blockchain_reactor_test")
// blockDB := dbm.NewDebugDB("blockDB", dbm.NewMemDB())
// stateDB := dbm.NewDebugDB("stateDB", dbm.NewMemDB())
@@ -30,7 +42,7 @@ func makeStateAndBlockStore(logger log.Logger) (sm.State, *BlockStore) {
if err != nil {
panic(cmn.ErrorWrap(err, "error constructing state from genesis file"))
}
return state, NewBlockStore(blockDB)
return state, NewBlockStore(blockDB), func() { os.RemoveAll(config.RootDir) }
}
func TestLoadBlockStoreStateJSON(t *testing.T) {
@@ -80,20 +92,32 @@ func freshBlockStore() (*BlockStore, db.DB) {
}
var (
state, _ = makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
block = makeBlock(1, state, new(types.Commit))
partSet = block.MakePartSet(2)
part1 = partSet.GetPart(0)
part2 = partSet.GetPart(1)
seenCommit1 = &types.Commit{Precommits: []*types.Vote{{Height: 10,
Timestamp: tmtime.Now()}}}
state sm.State
block *types.Block
partSet *types.PartSet
part1 *types.Part
part2 *types.Part
seenCommit1 *types.Commit
)
func TestMain(m *testing.M) {
var cleanup cleanupFunc
state, _, cleanup = makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
block = makeBlock(1, state, new(types.Commit))
partSet = block.MakePartSet(2)
part1 = partSet.GetPart(0)
part2 = partSet.GetPart(1)
seenCommit1 = makeTestCommit(10, tmtime.Now())
code := m.Run()
cleanup()
os.Exit(code)
}
// TODO: This test should be simplified ...
func TestBlockStoreSaveLoadBlock(t *testing.T) {
state, bs := makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
state, bs, cleanup := makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
defer cleanup()
require.Equal(t, bs.Height(), int64(0), "initially the height should be zero")
// check there are no blocks at various heights
@@ -107,8 +131,7 @@ func TestBlockStoreSaveLoadBlock(t *testing.T) {
// save a block
block := makeBlock(bs.Height()+1, state, new(types.Commit))
validPartSet := block.MakePartSet(2)
seenCommit := &types.Commit{Precommits: []*types.Vote{{Height: 10,
Timestamp: tmtime.Now()}}}
seenCommit := makeTestCommit(10, tmtime.Now())
bs.SaveBlock(block, partSet, seenCommit)
require.Equal(t, bs.Height(), block.Header.Height, "expecting the new height to be changed")
@@ -127,8 +150,7 @@ func TestBlockStoreSaveLoadBlock(t *testing.T) {
// End of setup, test data
commitAtH10 := &types.Commit{Precommits: []*types.Vote{{Height: 10,
Timestamp: tmtime.Now()}}}
commitAtH10 := makeTestCommit(10, tmtime.Now())
tuples := []struct {
block *types.Block
parts *types.PartSet
@@ -311,7 +333,7 @@ func TestLoadBlockPart(t *testing.T) {
gotPart, _, panicErr := doFn(loadPart)
require.Nil(t, panicErr, "an existent and proper block should not panic")
require.Nil(t, res, "a properly saved block should return a proper block")
require.Equal(t, gotPart.(*types.Part).Hash(), part1.Hash(),
require.Equal(t, gotPart.(*types.Part), part1,
"expecting successful retrieval of previously saved block")
}
@@ -346,14 +368,13 @@ func TestLoadBlockMeta(t *testing.T) {
}
func TestBlockFetchAtHeight(t *testing.T) {
state, bs := makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
state, bs, cleanup := makeStateAndBlockStore(log.NewTMLogger(new(bytes.Buffer)))
defer cleanup()
require.Equal(t, bs.Height(), int64(0), "initially the height should be zero")
block := makeBlock(bs.Height()+1, state, new(types.Commit))
partSet := block.MakePartSet(2)
seenCommit := &types.Commit{Precommits: []*types.Vote{{Height: 10,
Timestamp: tmtime.Now()}}}
seenCommit := makeTestCommit(10, tmtime.Now())
bs.SaveBlock(block, partSet, seenCommit)
require.Equal(t, bs.Height(), block.Header.Height, "expecting the new height to be changed")

View File

@@ -1,7 +1,7 @@
package blockchain
import (
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/tendermint/types"
)

View File

@@ -35,7 +35,7 @@ func main() {
pv := privval.LoadFilePV(*privValKeyPath, *privValStatePath)
var dialer privval.Dialer
var dialer privval.SocketDialer
protocol, address := cmn.ProtocolAndAddress(*addr)
switch protocol {
case "unix":
@@ -48,16 +48,20 @@ func main() {
os.Exit(1)
}
rs := privval.NewRemoteSigner(logger, *chainID, pv, dialer)
rs := privval.NewSignerServiceEndpoint(logger, *chainID, pv, dialer)
err := rs.Start()
if err != nil {
panic(err)
}
cmn.TrapSignal(func() {
// Stop upon receiving SIGTERM or CTRL-C.
cmn.TrapSignal(logger, func() {
err := rs.Stop()
if err != nil {
panic(err)
}
})
// Run forever.
select {}
}

View File

@@ -43,7 +43,7 @@ func init() {
LiteCmd.Flags().IntVar(&cacheSize, "cache-size", 10, "Specify the memory trust store cache size")
}
func ensureAddrHasSchemeOrDefaultToTCP(addr string) (string, error) {
func EnsureAddrHasSchemeOrDefaultToTCP(addr string) (string, error) {
u, err := url.Parse(addr)
if err != nil {
return "", err
@@ -59,11 +59,16 @@ func ensureAddrHasSchemeOrDefaultToTCP(addr string) (string, error) {
}
func runProxy(cmd *cobra.Command, args []string) error {
nodeAddr, err := ensureAddrHasSchemeOrDefaultToTCP(nodeAddr)
// Stop upon receiving SIGTERM or CTRL-C.
cmn.TrapSignal(logger, func() {
// TODO: close up shop
})
nodeAddr, err := EnsureAddrHasSchemeOrDefaultToTCP(nodeAddr)
if err != nil {
return err
}
listenAddr, err := ensureAddrHasSchemeOrDefaultToTCP(listenAddr)
listenAddr, err := EnsureAddrHasSchemeOrDefaultToTCP(listenAddr)
if err != nil {
return err
}
@@ -86,9 +91,6 @@ func runProxy(cmd *cobra.Command, args []string) error {
return cmn.ErrorWrap(err, "starting proxy")
}
cmn.TrapSignal(func() {
// TODO: close up shop
})
return nil
// Run forever
select {}
}

View File

@@ -61,7 +61,7 @@ func resetFilePV(privValKeyFile, privValStateFile string, logger log.Logger) {
} else {
pv := privval.GenFilePV(privValKeyFile, privValStateFile)
pv.Save()
logger.Info("Generated private validator file", "file", "keyFile", privValKeyFile,
logger.Info("Generated private validator file", "keyFile", privValKeyFile,
"stateFile", privValStateFile)
}
}

View File

@@ -22,10 +22,6 @@ var (
defaultRoot = os.ExpandEnv("$HOME/.some/test/dir")
)
const (
rootName = "root"
)
// clearConfig clears env vars, the given root dir, and resets viper.
func clearConfig(dir string) {
if err := os.Unsetenv("TMHOME"); err != nil {

View File

@@ -2,12 +2,10 @@ package commands
import (
"fmt"
"os"
"os/signal"
"syscall"
"github.com/spf13/cobra"
cmn "github.com/tendermint/tendermint/libs/common"
nm "github.com/tendermint/tendermint/node"
)
@@ -57,28 +55,20 @@ func NewRunNodeCmd(nodeProvider nm.NodeProvider) *cobra.Command {
return fmt.Errorf("Failed to create node: %v", err)
}
// Stop upon receiving SIGTERM or CTRL-C
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
go func() {
for sig := range c {
logger.Error(fmt.Sprintf("captured %v, exiting...", sig))
if n.IsRunning() {
n.Stop()
}
os.Exit(1)
// Stop upon receiving SIGTERM or CTRL-C.
cmn.TrapSignal(logger, func() {
if n.IsRunning() {
n.Stop()
}
}()
})
if err := n.Start(); err != nil {
return fmt.Errorf("Failed to start node: %v", err)
}
logger.Info("Started node", "nodeInfo", n.Switch().NodeInfo())
// Run forever
// Run forever.
select {}
return nil
},
}

View File

@@ -16,12 +16,11 @@ var ShowNodeIDCmd = &cobra.Command{
}
func showNodeID(cmd *cobra.Command, args []string) error {
nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile())
if err != nil {
return err
}
fmt.Println(nodeKey.ID())
fmt.Println(nodeKey.ID())
return nil
}

View File

@@ -3,8 +3,10 @@ package commands
import (
"fmt"
"github.com/pkg/errors"
"github.com/spf13/cobra"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/privval"
)
@@ -12,11 +14,21 @@ import (
var ShowValidatorCmd = &cobra.Command{
Use: "show_validator",
Short: "Show this node's validator info",
Run: showValidator,
RunE: showValidator,
}
func showValidator(cmd *cobra.Command, args []string) {
privValidator := privval.LoadOrGenFilePV(config.PrivValidatorKeyFile(), config.PrivValidatorStateFile())
pubKeyJSONBytes, _ := cdc.MarshalJSON(privValidator.GetPubKey())
fmt.Println(string(pubKeyJSONBytes))
func showValidator(cmd *cobra.Command, args []string) error {
keyFilePath := config.PrivValidatorKeyFile()
if !cmn.FileExists(keyFilePath) {
return fmt.Errorf("private validator file %s does not exist", keyFilePath)
}
pv := privval.LoadFilePV(keyFilePath, config.PrivValidatorStateFile())
bz, err := cdc.MarshalJSON(pv.GetPubKey())
if err != nil {
return errors.Wrap(err, "failed to marshal private validator pubkey")
}
fmt.Println(string(bz))
return nil
}

View File

@@ -115,6 +115,12 @@ func testnetFiles(cmd *cobra.Command, args []string) error {
return err
}
err = os.MkdirAll(filepath.Join(nodeDir, "data"), nodeDirPerm)
if err != nil {
_ = os.RemoveAll(outputDir)
return err
}
initFilesWithConfig(config)
}

View File

@@ -1,7 +1,7 @@
package commands
import (
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
cryptoAmino "github.com/tendermint/tendermint/crypto/encoding/amino"
)

View File

@@ -323,6 +323,22 @@ type RPCConfig struct {
// Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
// 1024 - 40 - 10 - 50 = 924 = ~900
MaxOpenConnections int `mapstructure:"max_open_connections"`
// Maximum number of unique clientIDs that can /subscribe
// If you're using /broadcast_tx_commit, set to the estimated maximum number
// of broadcast_tx_commit calls per block.
MaxSubscriptionClients int `mapstructure:"max_subscription_clients"`
// Maximum number of unique queries a given client can /subscribe to
// If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set
// to the estimated maximum number of broadcast_tx_commit calls per block.
MaxSubscriptionsPerClient int `mapstructure:"max_subscriptions_per_client"`
// How long to wait for a tx to be committed during /broadcast_tx_commit
// WARNING: Using a value larger than 10s will result in increasing the
// global HTTP write timeout, which applies to all connections and endpoints.
// See https://github.com/tendermint/tendermint/issues/3435
TimeoutBroadcastTxCommit time.Duration `mapstructure:"timeout_broadcast_tx_commit"`
}
// DefaultRPCConfig returns a default configuration for the RPC server
@@ -337,6 +353,10 @@ func DefaultRPCConfig() *RPCConfig {
Unsafe: false,
MaxOpenConnections: 900,
MaxSubscriptionClients: 100,
MaxSubscriptionsPerClient: 5,
TimeoutBroadcastTxCommit: 10 * time.Second,
}
}
@@ -358,6 +378,15 @@ func (cfg *RPCConfig) ValidateBasic() error {
if cfg.MaxOpenConnections < 0 {
return errors.New("max_open_connections can't be negative")
}
if cfg.MaxSubscriptionClients < 0 {
return errors.New("max_subscription_clients can't be negative")
}
if cfg.MaxSubscriptionsPerClient < 0 {
return errors.New("max_subscriptions_per_client can't be negative")
}
if cfg.TimeoutBroadcastTxCommit < 0 {
return errors.New("timeout_broadcast_tx_commit can't be negative")
}
return nil
}
@@ -530,12 +559,13 @@ func DefaultFuzzConnConfig() *FuzzConnConfig {
// MempoolConfig defines the configuration options for the Tendermint mempool
type MempoolConfig struct {
RootDir string `mapstructure:"home"`
Recheck bool `mapstructure:"recheck"`
Broadcast bool `mapstructure:"broadcast"`
WalPath string `mapstructure:"wal_dir"`
Size int `mapstructure:"size"`
CacheSize int `mapstructure:"cache_size"`
RootDir string `mapstructure:"home"`
Recheck bool `mapstructure:"recheck"`
Broadcast bool `mapstructure:"broadcast"`
WalPath string `mapstructure:"wal_dir"`
Size int `mapstructure:"size"`
MaxTxsBytes int64 `mapstructure:"max_txs_bytes"`
CacheSize int `mapstructure:"cache_size"`
}
// DefaultMempoolConfig returns a default configuration for the Tendermint mempool
@@ -544,10 +574,11 @@ func DefaultMempoolConfig() *MempoolConfig {
Recheck: true,
Broadcast: true,
WalPath: "",
// Each signature verification takes .5ms, size reduced until we implement
// Each signature verification takes .5ms, Size reduced until we implement
// ABCI Recheck
Size: 5000,
CacheSize: 10000,
Size: 5000,
MaxTxsBytes: 1024 * 1024 * 1024, // 1GB
CacheSize: 10000,
}
}
@@ -574,6 +605,9 @@ func (cfg *MempoolConfig) ValidateBasic() error {
if cfg.Size < 0 {
return errors.New("size can't be negative")
}
if cfg.MaxTxsBytes < 0 {
return errors.New("max_txs_bytes can't be negative")
}
if cfg.CacheSize < 0 {
return errors.New("cache_size can't be negative")
}
@@ -608,9 +642,6 @@ type ConsensusConfig struct {
// Reactor sleep duration parameters
PeerGossipSleepDuration time.Duration `mapstructure:"peer_gossip_sleep_duration"`
PeerQueryMaj23SleepDuration time.Duration `mapstructure:"peer_query_maj23_sleep_duration"`
// Block time parameters. Corresponds to the minimum time increment between consecutive blocks.
BlockTimeIota time.Duration `mapstructure:"blocktime_iota"`
}
// DefaultConsensusConfig returns a default configuration for the consensus service
@@ -629,7 +660,6 @@ func DefaultConsensusConfig() *ConsensusConfig {
CreateEmptyBlocksInterval: 0 * time.Second,
PeerGossipSleepDuration: 100 * time.Millisecond,
PeerQueryMaj23SleepDuration: 2000 * time.Millisecond,
BlockTimeIota: 1000 * time.Millisecond,
}
}
@@ -646,16 +676,9 @@ func TestConsensusConfig() *ConsensusConfig {
cfg.SkipTimeoutCommit = true
cfg.PeerGossipSleepDuration = 5 * time.Millisecond
cfg.PeerQueryMaj23SleepDuration = 250 * time.Millisecond
cfg.BlockTimeIota = 10 * time.Millisecond
return cfg
}
// MinValidVoteTime returns the minimum acceptable block time.
// See the [BFT time spec](https://godoc.org/github.com/tendermint/tendermint/docs/spec/consensus/bft-time.md).
func (cfg *ConsensusConfig) MinValidVoteTime(lastBlockTime time.Time) time.Time {
return lastBlockTime.Add(cfg.BlockTimeIota)
}
// WaitForTxs returns true if the consensus should wait for transactions before entering the propose step
func (cfg *ConsensusConfig) WaitForTxs() bool {
return !cfg.CreateEmptyBlocks || cfg.CreateEmptyBlocksInterval > 0
@@ -733,9 +756,6 @@ func (cfg *ConsensusConfig) ValidateBasic() error {
if cfg.PeerQueryMaj23SleepDuration < 0 {
return errors.New("peer_query_maj23_sleep_duration can't be negative")
}
if cfg.BlockTimeIota < 0 {
return errors.New("blocktime_iota can't be negative")
}
return nil
}

View File

@@ -2,13 +2,17 @@ package config
import (
"bytes"
"os"
"fmt"
"io/ioutil"
"path/filepath"
"text/template"
cmn "github.com/tendermint/tendermint/libs/common"
)
// DefaultDirPerm is the default permissions used when creating directories.
const DefaultDirPerm = 0700
var configTemplate *template.Template
func init() {
@@ -23,13 +27,13 @@ func init() {
// EnsureRoot creates the root, config, and data directories if they don't exist,
// and panics if it fails.
func EnsureRoot(rootDir string) {
if err := cmn.EnsureDir(rootDir, 0700); err != nil {
if err := cmn.EnsureDir(rootDir, DefaultDirPerm); err != nil {
cmn.PanicSanity(err.Error())
}
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultConfigDir), 0700); err != nil {
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultConfigDir), DefaultDirPerm); err != nil {
cmn.PanicSanity(err.Error())
}
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultDataDir), 0700); err != nil {
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultDataDir), DefaultDirPerm); err != nil {
cmn.PanicSanity(err.Error())
}
@@ -161,6 +165,22 @@ unsafe = {{ .RPC.Unsafe }}
# 1024 - 40 - 10 - 50 = 924 = ~900
max_open_connections = {{ .RPC.MaxOpenConnections }}
# Maximum number of unique clientIDs that can /subscribe
# If you're using /broadcast_tx_commit, set to the estimated maximum number
# of broadcast_tx_commit calls per block.
max_subscription_clients = {{ .RPC.MaxSubscriptionClients }}
# Maximum number of unique queries a given client can /subscribe to
# If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set to
# the estimated # maximum number of broadcast_tx_commit calls per block.
max_subscriptions_per_client = {{ .RPC.MaxSubscriptionsPerClient }}
# How long to wait for a tx to be committed during /broadcast_tx_commit.
# WARNING: Using a value larger than 10s will result in increasing the
# global HTTP write timeout, which applies to all connections and endpoints.
# See https://github.com/tendermint/tendermint/issues/3435
timeout_broadcast_tx_commit = "{{ .RPC.TimeoutBroadcastTxCommit }}"
##### peer to peer configuration options #####
[p2p]
@@ -233,10 +253,15 @@ recheck = {{ .Mempool.Recheck }}
broadcast = {{ .Mempool.Broadcast }}
wal_dir = "{{ js .Mempool.WalPath }}"
# size of the mempool
# Maximum number of transactions in the mempool
size = {{ .Mempool.Size }}
# size of the cache (used to filter transactions we saw earlier)
# Limit the total size of all txs in the mempool.
# This only accounts for raw transactions (e.g. given 1MB transactions and
# max_txs_bytes=5MB, mempool will only accept 5 transactions).
max_txs_bytes = {{ .Mempool.MaxTxsBytes }}
# Size of the cache (used to filter transactions we saw earlier) in transactions
cache_size = {{ .Mempool.CacheSize }}
##### consensus configuration options #####
@@ -263,9 +288,6 @@ create_empty_blocks_interval = "{{ .Consensus.CreateEmptyBlocksInterval }}"
peer_gossip_sleep_duration = "{{ .Consensus.PeerGossipSleepDuration }}"
peer_query_maj23_sleep_duration = "{{ .Consensus.PeerQueryMaj23SleepDuration }}"
# Block time parameters. Corresponds to the minimum time increment between consecutive blocks.
blocktime_iota = "{{ .Consensus.BlockTimeIota }}"
##### transactions indexer configuration options #####
[tx_index]
@@ -317,29 +339,21 @@ namespace = "{{ .Instrumentation.Namespace }}"
/****** these are for test settings ***********/
func ResetTestRoot(testName string) *Config {
rootDir := os.ExpandEnv("$HOME/.tendermint_test")
rootDir = filepath.Join(rootDir, testName)
// Remove ~/.tendermint_test_bak
if cmn.FileExists(rootDir + "_bak") {
if err := os.RemoveAll(rootDir + "_bak"); err != nil {
cmn.PanicSanity(err.Error())
}
return ResetTestRootWithChainID(testName, "")
}
func ResetTestRootWithChainID(testName string, chainID string) *Config {
// create a unique, concurrency-safe test directory under os.TempDir()
rootDir, err := ioutil.TempDir("", fmt.Sprintf("%s-%s_", chainID, testName))
if err != nil {
panic(err)
}
// Move ~/.tendermint_test to ~/.tendermint_test_bak
if cmn.FileExists(rootDir) {
if err := os.Rename(rootDir, rootDir+"_bak"); err != nil {
cmn.PanicSanity(err.Error())
}
// ensure config and data subdirs are created
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultConfigDir), DefaultDirPerm); err != nil {
panic(err)
}
// Create new dir
if err := cmn.EnsureDir(rootDir, 0700); err != nil {
cmn.PanicSanity(err.Error())
}
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultConfigDir), 0700); err != nil {
cmn.PanicSanity(err.Error())
}
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultDataDir), 0700); err != nil {
cmn.PanicSanity(err.Error())
if err := cmn.EnsureDir(filepath.Join(rootDir, defaultDataDir), DefaultDirPerm); err != nil {
panic(err)
}
baseConfig := DefaultBaseConfig()
@@ -353,6 +367,10 @@ func ResetTestRoot(testName string) *Config {
writeDefaultConfigFile(configFilePath)
}
if !cmn.FileExists(genesisFilePath) {
if chainID == "" {
chainID = "tendermint_test"
}
testGenesis := fmt.Sprintf(testGenesisFmt, chainID)
cmn.MustWriteFile(genesisFilePath, []byte(testGenesis), 0644)
}
// we always overwrite the priv val
@@ -363,9 +381,9 @@ func ResetTestRoot(testName string) *Config {
return config
}
var testGenesis = `{
var testGenesisFmt = `{
"genesis_time": "2018-10-10T08:20:13.695936996Z",
"chain_id": "tendermint_test",
"chain_id": "%s",
"validators": [
{
"pub_key": {

View File

@@ -48,6 +48,7 @@ func TestEnsureTestRoot(t *testing.T) {
// create root dir
cfg := ResetTestRoot(testName)
defer os.RemoveAll(cfg.RootDir)
rootDir := cfg.RootDir
// make sure config is set properly

View File

@@ -13,10 +13,6 @@ import (
"github.com/tendermint/tendermint/types"
)
func init() {
config = ResetConfig("consensus_byzantine_test")
}
//----------------------------------------------
// byzantine failures
@@ -29,7 +25,8 @@ func init() {
func TestByzantine(t *testing.T) {
N := 4
logger := consensusLogger().With("test", "byzantine")
css := randConsensusNet(N, "consensus_byzantine_test", newMockTickerFunc(false), newCounter)
css, cleanup := randConsensusNet(N, "consensus_byzantine_test", newMockTickerFunc(false), newCounter)
defer cleanup()
// give the byzantine validator a normal ticker
ticker := NewTimeoutTicker()
@@ -49,7 +46,7 @@ func TestByzantine(t *testing.T) {
switches[i].SetLogger(p2pLogger.With("validator", i))
}
eventChans := make([]chan interface{}, N)
blocksSubs := make([]types.Subscription, N)
reactors := make([]p2p.Reactor, N)
for i := 0; i < N; i++ {
// make first val byzantine
@@ -68,16 +65,15 @@ func TestByzantine(t *testing.T) {
eventBus := css[i].eventBus
eventBus.SetLogger(logger.With("module", "events", "validator", i))
eventChans[i] = make(chan interface{}, 1)
err := eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock, eventChans[i])
var err error
blocksSubs[i], err = eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock)
require.NoError(t, err)
conR := NewConsensusReactor(css[i], true) // so we dont start the consensus states
conR.SetLogger(logger.With("validator", i))
conR.SetEventBus(eventBus)
var conRI p2p.Reactor // nolint: gotype, gosimple
conRI = conR
var conRI p2p.Reactor = conR
// make first val byzantine
if i == 0 {
@@ -135,7 +131,7 @@ func TestByzantine(t *testing.T) {
p2p.Connect2Switches(switches, ind1, ind2)
// wait for someone in the big partition (B) to make a block
<-eventChans[ind2]
<-blocksSubs[ind2].Out()
t.Log("A block has been committed. Healing partition")
p2p.Connect2Switches(switches, ind0, ind1)
@@ -147,7 +143,7 @@ func TestByzantine(t *testing.T) {
wg.Add(2)
for i := 1; i < N-1; i++ {
go func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
wg.Done()
}(i)
}

View File

@@ -7,7 +7,6 @@ import (
"io/ioutil"
"os"
"path/filepath"
"reflect"
"sort"
"sync"
"testing"
@@ -25,6 +24,7 @@ import (
cmn "github.com/tendermint/tendermint/libs/common"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/libs/log"
tmpubsub "github.com/tendermint/tendermint/libs/pubsub"
mempl "github.com/tendermint/tendermint/mempool"
"github.com/tendermint/tendermint/p2p"
"github.com/tendermint/tendermint/privval"
@@ -37,8 +37,13 @@ const (
testSubscriber = "test-client"
)
// A cleanupFunc cleans up any config / test files created for a particular
// test.
type cleanupFunc func()
// genesis, chain_id, priv_val
var config *cfg.Config // NOTE: must be reset for each _test.go file
var consensusReplayConfig *cfg.Config
var ensureTimeout = time.Millisecond * 100
func ensureDir(dir string, mode os.FileMode) {
@@ -122,17 +127,21 @@ func startTestRound(cs *ConsensusState, height int64, round int) {
cs.startRoutines(0)
}
// Create proposal block from cs1 but sign it with vs
// Create proposal block from cs1 but sign it with vs.
func decideProposal(cs1 *ConsensusState, vs *validatorStub, height int64, round int) (proposal *types.Proposal, block *types.Block) {
cs1.mtx.Lock()
block, blockParts := cs1.createProposalBlock()
if block == nil { // on error
panic("error creating proposal block")
validRound := cs1.ValidRound
chainID := cs1.state.ChainID
cs1.mtx.Unlock()
if block == nil {
panic("Failed to createProposalBlock. Did you forget to add commit for previous block?")
}
// Make proposal
polRound, propBlockID := cs1.ValidRound, types.BlockID{block.Hash(), blockParts.Header()}
polRound, propBlockID := validRound, types.BlockID{block.Hash(), blockParts.Header()}
proposal = types.NewProposal(height, round, polRound, propBlockID)
if err := vs.SignProposal(cs1.state.ChainID, proposal); err != nil {
if err := vs.SignProposal(chainID, proposal); err != nil {
panic(err)
}
return
@@ -218,30 +227,29 @@ func validatePrevoteAndPrecommit(t *testing.T, cs *ConsensusState, thisRound, lo
cs.mtx.Unlock()
}
// genesis
func subscribeToVoter(cs *ConsensusState, addr []byte) chan interface{} {
voteCh0 := make(chan interface{})
err := cs.eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryVote, voteCh0)
func subscribeToVoter(cs *ConsensusState, addr []byte) <-chan tmpubsub.Message {
votesSub, err := cs.eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryVote)
if err != nil {
panic(fmt.Sprintf("failed to subscribe %s to %v", testSubscriber, types.EventQueryVote))
}
voteCh := make(chan interface{})
ch := make(chan tmpubsub.Message)
go func() {
for v := range voteCh0 {
vote := v.(types.EventDataVote)
for msg := range votesSub.Out() {
vote := msg.Data().(types.EventDataVote)
// we only fire for our own votes
if bytes.Equal(addr, vote.Vote.ValidatorAddress) {
voteCh <- v
ch <- msg
}
}
}()
return voteCh
return ch
}
//-------------------------------------------------------------------------------
// consensus states
func newConsensusState(state sm.State, pv types.PrivValidator, app abci.Application) *ConsensusState {
config := cfg.ResetTestRoot("consensus_state_test")
return newConsensusStateWithConfig(config, state, pv, app)
}
@@ -311,7 +319,7 @@ func randConsensusState(nValidators int) (*ConsensusState, []*validatorStub) {
//-------------------------------------------------------------------------------
func ensureNoNewEvent(ch <-chan interface{}, timeout time.Duration,
func ensureNoNewEvent(ch <-chan tmpubsub.Message, timeout time.Duration,
errorMessage string) {
select {
case <-time.After(timeout):
@@ -321,28 +329,28 @@ func ensureNoNewEvent(ch <-chan interface{}, timeout time.Duration,
}
}
func ensureNoNewEventOnChannel(ch <-chan interface{}) {
func ensureNoNewEventOnChannel(ch <-chan tmpubsub.Message) {
ensureNoNewEvent(
ch,
ensureTimeout,
"We should be stuck waiting, not receiving new event on the channel")
}
func ensureNoNewRoundStep(stepCh <-chan interface{}) {
func ensureNoNewRoundStep(stepCh <-chan tmpubsub.Message) {
ensureNoNewEvent(
stepCh,
ensureTimeout,
"We should be stuck waiting, not receiving NewRoundStep event")
}
func ensureNoNewUnlock(unlockCh <-chan interface{}) {
func ensureNoNewUnlock(unlockCh <-chan tmpubsub.Message) {
ensureNoNewEvent(
unlockCh,
ensureTimeout,
"We should be stuck waiting, not receiving Unlock event")
}
func ensureNoNewTimeout(stepCh <-chan interface{}, timeout int64) {
func ensureNoNewTimeout(stepCh <-chan tmpubsub.Message, timeout int64) {
timeoutDuration := time.Duration(timeout*5) * time.Nanosecond
ensureNoNewEvent(
stepCh,
@@ -350,172 +358,157 @@ func ensureNoNewTimeout(stepCh <-chan interface{}, timeout int64) {
"We should be stuck waiting, not receiving NewTimeout event")
}
func ensureNewEvent(
ch <-chan interface{},
height int64,
round int,
timeout time.Duration,
errorMessage string) {
func ensureNewEvent(ch <-chan tmpubsub.Message, height int64, round int, timeout time.Duration, errorMessage string) {
select {
case <-time.After(timeout):
panic(errorMessage)
case ev := <-ch:
rs, ok := ev.(types.EventDataRoundState)
case msg := <-ch:
roundStateEvent, ok := msg.Data().(types.EventDataRoundState)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataRoundState, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
panic(fmt.Sprintf("expected a EventDataRoundState, got %T. Wrong subscription channel?",
msg.Data()))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
if roundStateEvent.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, roundStateEvent.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
if roundStateEvent.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, roundStateEvent.Round))
}
// TODO: We could check also for a step at this point!
}
}
func ensureNewRoundStep(stepCh <-chan interface{}, height int64, round int) {
ensureNewEvent(
stepCh,
height,
round,
ensureTimeout,
"Timeout expired while waiting for NewStep event")
}
func ensureNewVote(voteCh <-chan interface{}, height int64, round int) {
select {
case <-time.After(ensureTimeout):
break
case v := <-voteCh:
edv, ok := v.(types.EventDataVote)
if !ok {
panic(fmt.Sprintf("expected a *types.Vote, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(v)))
}
vote := edv.Vote
if vote.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, vote.Height))
}
if vote.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, vote.Round))
}
}
}
func ensureNewRound(roundCh <-chan interface{}, height int64, round int) {
func ensureNewRound(roundCh <-chan tmpubsub.Message, height int64, round int) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewRound event")
case ev := <-roundCh:
rs, ok := ev.(types.EventDataNewRound)
case msg := <-roundCh:
newRoundEvent, ok := msg.Data().(types.EventDataNewRound)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataNewRound, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
panic(fmt.Sprintf("expected a EventDataNewRound, got %T. Wrong subscription channel?",
msg.Data()))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
if newRoundEvent.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, newRoundEvent.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
if newRoundEvent.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, newRoundEvent.Round))
}
}
}
func ensureNewTimeout(timeoutCh <-chan interface{}, height int64, round int, timeout int64) {
timeoutDuration := time.Duration(timeout*3) * time.Nanosecond
func ensureNewTimeout(timeoutCh <-chan tmpubsub.Message, height int64, round int, timeout int64) {
timeoutDuration := time.Duration(timeout*5) * time.Nanosecond
ensureNewEvent(timeoutCh, height, round, timeoutDuration,
"Timeout expired while waiting for NewTimeout event")
}
func ensureNewProposal(proposalCh <-chan interface{}, height int64, round int) {
func ensureNewProposal(proposalCh <-chan tmpubsub.Message, height int64, round int) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewProposal event")
case ev := <-proposalCh:
rs, ok := ev.(types.EventDataCompleteProposal)
case msg := <-proposalCh:
proposalEvent, ok := msg.Data().(types.EventDataCompleteProposal)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataCompleteProposal, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
panic(fmt.Sprintf("expected a EventDataCompleteProposal, got %T. Wrong subscription channel?",
msg.Data()))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
if proposalEvent.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, proposalEvent.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
if proposalEvent.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, proposalEvent.Round))
}
}
}
func ensureNewValidBlock(validBlockCh <-chan interface{}, height int64, round int) {
func ensureNewValidBlock(validBlockCh <-chan tmpubsub.Message, height int64, round int) {
ensureNewEvent(validBlockCh, height, round, ensureTimeout,
"Timeout expired while waiting for NewValidBlock event")
}
func ensureNewBlock(blockCh <-chan interface{}, height int64) {
func ensureNewBlock(blockCh <-chan tmpubsub.Message, height int64) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewBlock event")
case ev := <-blockCh:
block, ok := ev.(types.EventDataNewBlock)
case msg := <-blockCh:
blockEvent, ok := msg.Data().(types.EventDataNewBlock)
if !ok {
panic(fmt.Sprintf("expected a *types.EventDataNewBlock, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(block)))
panic(fmt.Sprintf("expected a EventDataNewBlock, got %T. Wrong subscription channel?",
msg.Data()))
}
if block.Block.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, block.Block.Height))
if blockEvent.Block.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, blockEvent.Block.Height))
}
}
}
func ensureNewBlockHeader(blockCh <-chan interface{}, height int64, blockHash cmn.HexBytes) {
func ensureNewBlockHeader(blockCh <-chan tmpubsub.Message, height int64, blockHash cmn.HexBytes) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewBlockHeader event")
case ev := <-blockCh:
blockHeader, ok := ev.(types.EventDataNewBlockHeader)
case msg := <-blockCh:
blockHeaderEvent, ok := msg.Data().(types.EventDataNewBlockHeader)
if !ok {
panic(fmt.Sprintf("expected a *types.EventDataNewBlockHeader, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(blockHeader)))
panic(fmt.Sprintf("expected a EventDataNewBlockHeader, got %T. Wrong subscription channel?",
msg.Data()))
}
if blockHeader.Header.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, blockHeader.Header.Height))
if blockHeaderEvent.Header.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, blockHeaderEvent.Header.Height))
}
if !bytes.Equal(blockHeader.Header.Hash(), blockHash) {
panic(fmt.Sprintf("expected header %X, got %X", blockHash, blockHeader.Header.Hash()))
if !bytes.Equal(blockHeaderEvent.Header.Hash(), blockHash) {
panic(fmt.Sprintf("expected header %X, got %X", blockHash, blockHeaderEvent.Header.Hash()))
}
}
}
func ensureNewUnlock(unlockCh <-chan interface{}, height int64, round int) {
func ensureNewUnlock(unlockCh <-chan tmpubsub.Message, height int64, round int) {
ensureNewEvent(unlockCh, height, round, ensureTimeout,
"Timeout expired while waiting for NewUnlock event")
}
func ensureVote(voteCh <-chan interface{}, height int64, round int,
func ensureProposal(proposalCh <-chan tmpubsub.Message, height int64, round int, propID types.BlockID) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewProposal event")
case msg := <-proposalCh:
proposalEvent, ok := msg.Data().(types.EventDataCompleteProposal)
if !ok {
panic(fmt.Sprintf("expected a EventDataCompleteProposal, got %T. Wrong subscription channel?",
msg.Data()))
}
if proposalEvent.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, proposalEvent.Height))
}
if proposalEvent.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, proposalEvent.Round))
}
if !proposalEvent.BlockID.Equals(propID) {
panic("Proposed block does not match expected block")
}
}
}
func ensurePrecommit(voteCh <-chan tmpubsub.Message, height int64, round int) {
ensureVote(voteCh, height, round, types.PrecommitType)
}
func ensurePrevote(voteCh <-chan tmpubsub.Message, height int64, round int) {
ensureVote(voteCh, height, round, types.PrevoteType)
}
func ensureVote(voteCh <-chan tmpubsub.Message, height int64, round int,
voteType types.SignedMsgType) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewVote event")
case v := <-voteCh:
edv, ok := v.(types.EventDataVote)
case msg := <-voteCh:
voteEvent, ok := msg.Data().(types.EventDataVote)
if !ok {
panic(fmt.Sprintf("expected a *types.Vote, "+
"got %v. wrong subscription channel?",
reflect.TypeOf(v)))
panic(fmt.Sprintf("expected a EventDataVote, got %T. Wrong subscription channel?",
msg.Data()))
}
vote := edv.Vote
vote := voteEvent.Vote
if vote.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, vote.Height))
}
@@ -528,39 +521,7 @@ func ensureVote(voteCh <-chan interface{}, height int64, round int,
}
}
func ensureProposal(proposalCh <-chan interface{}, height int64, round int, propId types.BlockID) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for NewProposal event")
case ev := <-proposalCh:
rs, ok := ev.(types.EventDataCompleteProposal)
if !ok {
panic(
fmt.Sprintf(
"expected a EventDataCompleteProposal, got %v.Wrong subscription channel?",
reflect.TypeOf(rs)))
}
if rs.Height != height {
panic(fmt.Sprintf("expected height %v, got %v", height, rs.Height))
}
if rs.Round != round {
panic(fmt.Sprintf("expected round %v, got %v", round, rs.Round))
}
if !rs.BlockID.Equals(propId) {
panic("Proposed block does not match expected block")
}
}
}
func ensurePrecommit(voteCh <-chan interface{}, height int64, round int) {
ensureVote(voteCh, height, round, types.PrecommitType)
}
func ensurePrevote(voteCh <-chan interface{}, height int64, round int) {
ensureVote(voteCh, height, round, types.PrevoteType)
}
func ensureNewEventOnChannel(ch <-chan interface{}) {
func ensureNewEventOnChannel(ch <-chan tmpubsub.Message) {
select {
case <-time.After(ensureTimeout):
panic("Timeout expired while waiting for new activity on the channel")
@@ -584,14 +545,17 @@ func consensusLogger() log.Logger {
}).With("module", "consensus")
}
func randConsensusNet(nValidators int, testName string, tickerFunc func() TimeoutTicker, appFunc func() abci.Application, configOpts ...func(*cfg.Config)) []*ConsensusState {
func randConsensusNet(nValidators int, testName string, tickerFunc func() TimeoutTicker,
appFunc func() abci.Application, configOpts ...func(*cfg.Config)) ([]*ConsensusState, cleanupFunc) {
genDoc, privVals := randGenesisDoc(nValidators, false, 30)
css := make([]*ConsensusState, nValidators)
logger := consensusLogger()
configRootDirs := make([]string, 0, nValidators)
for i := 0; i < nValidators; i++ {
stateDB := dbm.NewMemDB() // each state needs its own db
state, _ := sm.LoadStateFromDBOrGenesisDoc(stateDB, genDoc)
thisConfig := ResetConfig(fmt.Sprintf("%s_%d", testName, i))
configRootDirs = append(configRootDirs, thisConfig.RootDir)
for _, opt := range configOpts {
opt(thisConfig)
}
@@ -604,18 +568,26 @@ func randConsensusNet(nValidators int, testName string, tickerFunc func() Timeou
css[i].SetTimeoutTicker(tickerFunc())
css[i].SetLogger(logger.With("validator", i, "module", "consensus"))
}
return css
return css, func() {
for _, dir := range configRootDirs {
os.RemoveAll(dir)
}
}
}
// nPeers = nValidators + nNotValidator
func randConsensusNetWithPeers(nValidators, nPeers int, testName string, tickerFunc func() TimeoutTicker, appFunc func() abci.Application) []*ConsensusState {
func randConsensusNetWithPeers(nValidators, nPeers int, testName string, tickerFunc func() TimeoutTicker,
appFunc func() abci.Application) ([]*ConsensusState, cleanupFunc) {
genDoc, privVals := randGenesisDoc(nValidators, false, testMinPower)
css := make([]*ConsensusState, nPeers)
logger := consensusLogger()
configRootDirs := make([]string, 0, nPeers)
for i := 0; i < nPeers; i++ {
stateDB := dbm.NewMemDB() // each state needs its own db
state, _ := sm.LoadStateFromDBOrGenesisDoc(stateDB, genDoc)
thisConfig := ResetConfig(fmt.Sprintf("%s_%d", testName, i))
configRootDirs = append(configRootDirs, thisConfig.RootDir)
ensureDir(filepath.Dir(thisConfig.Consensus.WalFile()), 0700) // dir for wal
var privVal types.PrivValidator
if i < nValidators {
@@ -641,7 +613,11 @@ func randConsensusNetWithPeers(nValidators, nPeers int, testName string, tickerF
css[i].SetTimeoutTicker(tickerFunc())
css[i].SetLogger(logger.With("validator", i, "module", "consensus"))
}
return css
return css, func() {
for _, dir := range configRootDirs {
os.RemoveAll(dir)
}
}
}
func getSwitchIndex(switches []*p2p.Switch, peer p2p.Peer) int {
@@ -651,7 +627,6 @@ func getSwitchIndex(switches []*p2p.Switch, peer p2p.Peer) int {
}
}
panic("didnt find peer in switches")
return -1
}
//-------------------------------------------------------------------------------
@@ -729,8 +704,7 @@ func (m *mockTicker) Chan() <-chan timeoutInfo {
return m.c
}
func (mockTicker) SetLogger(log.Logger) {
}
func (*mockTicker) SetLogger(log.Logger) {}
//------------------------------------
@@ -739,6 +713,9 @@ func newCounter() abci.Application {
}
func newPersistentKVStore() abci.Application {
dir, _ := ioutil.TempDir("/tmp", "persistent-kvstore")
dir, err := ioutil.TempDir("", "persistent-kvstore")
if err != nil {
panic(err)
}
return kvstore.NewPersistentKVStoreApplication(dir)
}

View File

@@ -3,6 +3,7 @@ package consensus
import (
"encoding/binary"
"fmt"
"os"
"testing"
"time"
@@ -14,10 +15,6 @@ import (
"github.com/tendermint/tendermint/types"
)
func init() {
config = ResetConfig("consensus_mempool_test")
}
// for testing
func assertMempool(txn txNotifier) sm.Mempool {
return txn.(sm.Mempool)
@@ -25,6 +22,7 @@ func assertMempool(txn txNotifier) sm.Mempool {
func TestMempoolNoProgressUntilTxsAvailable(t *testing.T) {
config := ResetConfig("consensus_mempool_txs_available_test")
defer os.RemoveAll(config.RootDir)
config.Consensus.CreateEmptyBlocks = false
state, privVals := randGenesisState(1, false, 10)
cs := newConsensusStateWithConfig(config, state, privVals[0], NewCounterApplication())
@@ -43,6 +41,7 @@ func TestMempoolNoProgressUntilTxsAvailable(t *testing.T) {
func TestMempoolProgressAfterCreateEmptyBlocksInterval(t *testing.T) {
config := ResetConfig("consensus_mempool_txs_available_test")
defer os.RemoveAll(config.RootDir)
config.Consensus.CreateEmptyBlocksInterval = ensureTimeout
state, privVals := randGenesisState(1, false, 10)
cs := newConsensusStateWithConfig(config, state, privVals[0], NewCounterApplication())
@@ -58,6 +57,7 @@ func TestMempoolProgressAfterCreateEmptyBlocksInterval(t *testing.T) {
func TestMempoolProgressInHigherRound(t *testing.T) {
config := ResetConfig("consensus_mempool_txs_available_test")
defer os.RemoveAll(config.RootDir)
config.Consensus.CreateEmptyBlocks = false
state, privVals := randGenesisState(1, false, 10)
cs := newConsensusStateWithConfig(config, state, privVals[0], NewCounterApplication())
@@ -117,9 +117,9 @@ func TestMempoolTxConcurrentWithCommit(t *testing.T) {
for nTxs := 0; nTxs < NTxs; {
ticker := time.NewTicker(time.Second * 30)
select {
case b := <-newBlockCh:
evt := b.(types.EventDataNewBlock)
nTxs += int(evt.Block.Header.NumTxs)
case msg := <-newBlockCh:
blockEvent := msg.Data().(types.EventDataNewBlock)
nTxs += int(blockEvent.Block.Header.NumTxs)
case <-ticker.C:
panic("Timed out waiting to commit blocks with transactions")
}

View File

@@ -8,7 +8,11 @@ import (
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const MetricsSubsystem = "consensus"
const (
// MetricsSubsystem is a subsystem shared by all metrics exposed by this
// package.
MetricsSubsystem = "consensus"
)
// Metrics contains metrics exposed by this package.
type Metrics struct {
@@ -50,101 +54,107 @@ type Metrics struct {
}
// PrometheusMetrics returns Metrics build using Prometheus client library.
func PrometheusMetrics(namespace string) *Metrics {
// Optionally, labels can be provided along with their values ("foo",
// "fooValue").
func PrometheusMetrics(namespace string, labelsAndValues ...string) *Metrics {
labels := []string{}
for i := 0; i < len(labelsAndValues); i += 2 {
labels = append(labels, labelsAndValues[i])
}
return &Metrics{
Height: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "height",
Help: "Height of the chain.",
}, []string{}),
}, labels).With(labelsAndValues...),
Rounds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "rounds",
Help: "Number of rounds.",
}, []string{}),
}, labels).With(labelsAndValues...),
Validators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators",
Help: "Number of validators.",
}, []string{}),
}, labels).With(labelsAndValues...),
ValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "validators_power",
Help: "Total power of all validators.",
}, []string{}),
}, labels).With(labelsAndValues...),
MissingValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators",
Help: "Number of validators who did not sign.",
}, []string{}),
}, labels).With(labelsAndValues...),
MissingValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "missing_validators_power",
Help: "Total power of the missing validators.",
}, []string{}),
}, labels).With(labelsAndValues...),
ByzantineValidators: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators",
Help: "Number of validators who tried to double sign.",
}, []string{}),
}, labels).With(labelsAndValues...),
ByzantineValidatorsPower: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "byzantine_validators_power",
Help: "Total power of the byzantine validators.",
}, []string{}),
}, labels).With(labelsAndValues...),
BlockIntervalSeconds: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_interval_seconds",
Help: "Time between this and the last block.",
}, []string{}),
}, labels).With(labelsAndValues...),
NumTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "num_txs",
Help: "Number of transactions.",
}, []string{}),
}, labels).With(labelsAndValues...),
BlockSizeBytes: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_size_bytes",
Help: "Size of the block.",
}, []string{}),
}, labels).With(labelsAndValues...),
TotalTxs: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "total_txs",
Help: "Total number of transactions.",
}, []string{}),
}, labels).With(labelsAndValues...),
CommittedHeight: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "latest_block_height",
Help: "The latest block height.",
}, []string{}),
}, labels).With(labelsAndValues...),
FastSyncing: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "fast_syncing",
Help: "Whether or not a node is fast syncing. 1 if yes, 0 if no.",
}, []string{}),
}, labels).With(labelsAndValues...),
BlockParts: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: namespace,
Subsystem: MetricsSubsystem,
Name: "block_parts",
Help: "Number of blockparts transmitted by peer.",
}, []string{"peer_id"}),
}, append(labels, "peer_id")).With(labelsAndValues...),
}
}

View File

@@ -8,7 +8,7 @@ import (
"github.com/pkg/errors"
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
cstypes "github.com/tendermint/tendermint/consensus/types"
cmn "github.com/tendermint/tendermint/libs/common"
tmevents "github.com/tendermint/tendermint/libs/events"
@@ -438,9 +438,9 @@ func (conR *ConsensusReactor) broadcastHasVoteMessage(vote *types.Vote) {
func makeRoundStepMessage(rs *cstypes.RoundState) (nrsMsg *NewRoundStepMessage) {
nrsMsg = &NewRoundStepMessage{
Height: rs.Height,
Round: rs.Round,
Step: rs.Step,
Height: rs.Height,
Round: rs.Round,
Step: rs.Step,
SecondsSinceStartTime: int(time.Since(rs.StartTime).Seconds()),
LastCommitRound: rs.LastCommit.Round(),
}
@@ -896,7 +896,7 @@ type PeerState struct {
peer p2p.Peer
logger log.Logger
mtx sync.Mutex `json:"-"` // NOTE: Modify below using setters, never directly.
mtx sync.Mutex // NOTE: Modify below using setters, never directly.
PRS cstypes.PeerRoundState `json:"round_state"` // Exposed.
Stats *peerStateStats `json:"stats"` // Exposed.
}

View File

@@ -14,7 +14,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/tendermint/tendermint/abci/client"
abcicli "github.com/tendermint/tendermint/abci/client"
"github.com/tendermint/tendermint/abci/example/kvstore"
abci "github.com/tendermint/tendermint/abci/types"
bc "github.com/tendermint/tendermint/blockchain"
@@ -27,16 +27,16 @@ import (
"github.com/tendermint/tendermint/types"
)
func init() {
config = ResetConfig("consensus_reactor_test")
}
//----------------------------------------------
// in-process testnets
func startConsensusNet(t *testing.T, css []*ConsensusState, N int) ([]*ConsensusReactor, []chan interface{}, []*types.EventBus) {
func startConsensusNet(t *testing.T, css []*ConsensusState, N int) (
[]*ConsensusReactor,
[]types.Subscription,
[]*types.EventBus,
) {
reactors := make([]*ConsensusReactor, N)
eventChans := make([]chan interface{}, N)
blocksSubs := make([]types.Subscription, 0)
eventBuses := make([]*types.EventBus, N)
for i := 0; i < N; i++ {
/*logger, err := tmflags.ParseLogLevel("consensus:info,*:error", logger, "info")
@@ -48,9 +48,9 @@ func startConsensusNet(t *testing.T, css []*ConsensusState, N int) ([]*Consensus
eventBuses[i] = css[i].eventBus
reactors[i].SetEventBus(eventBuses[i])
eventChans[i] = make(chan interface{}, 1)
err := eventBuses[i].Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock, eventChans[i])
blocksSub, err := eventBuses[i].Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock)
require.NoError(t, err)
blocksSubs = append(blocksSubs, blocksSub)
}
// make connected switches and start all reactors
p2p.MakeConnectedSwitches(config.P2P, N, func(i int, s *p2p.Switch) *p2p.Switch {
@@ -67,7 +67,7 @@ func startConsensusNet(t *testing.T, css []*ConsensusState, N int) ([]*Consensus
s := reactors[i].conS.GetState()
reactors[i].SwitchToConsensus(s, 0)
}
return reactors, eventChans, eventBuses
return reactors, blocksSubs, eventBuses
}
func stopConsensusNet(logger log.Logger, reactors []*ConsensusReactor, eventBuses []*types.EventBus) {
@@ -86,12 +86,13 @@ func stopConsensusNet(logger log.Logger, reactors []*ConsensusReactor, eventBuse
// Ensure a testnet makes blocks
func TestReactorBasic(t *testing.T) {
N := 4
css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
css, cleanup := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
defer cleanup()
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, N)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
// wait till everyone makes the first new block
timeoutWaitGroup(t, N, func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
}, css)
}
@@ -116,6 +117,7 @@ func TestReactorWithEvidence(t *testing.T) {
stateDB := dbm.NewMemDB() // each state needs its own db
state, _ := sm.LoadStateFromDBOrGenesisDoc(stateDB, genDoc)
thisConfig := ResetConfig(fmt.Sprintf("%s_%d", testName, i))
defer os.RemoveAll(thisConfig.RootDir)
ensureDir(path.Dir(thisConfig.Consensus.WalFile()), 0700) // dir for wal
app := appFunc()
vals := types.TM2PB.ValidatorUpdates(state.Validators)
@@ -163,20 +165,20 @@ func TestReactorWithEvidence(t *testing.T) {
css[i] = cs
}
reactors, eventChans, eventBuses := startConsensusNet(t, css, nValidators)
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, nValidators)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
// wait till everyone makes the first new block with no evidence
timeoutWaitGroup(t, nValidators, func(j int) {
blockI := <-eventChans[j]
block := blockI.(types.EventDataNewBlock).Block
msg := <-blocksSubs[j].Out()
block := msg.Data().(types.EventDataNewBlock).Block
assert.True(t, len(block.Evidence.Evidence) == 0)
}, css)
// second block should have evidence
timeoutWaitGroup(t, nValidators, func(j int) {
blockI := <-eventChans[j]
block := blockI.(types.EventDataNewBlock).Block
msg := <-blocksSubs[j].Out()
block := msg.Data().(types.EventDataNewBlock).Block
assert.True(t, len(block.Evidence.Evidence) > 0)
}, css)
}
@@ -211,17 +213,19 @@ func (m *mockEvidencePool) Update(block *types.Block, state sm.State) {
}
m.height++
}
func (m *mockEvidencePool) IsCommitted(types.Evidence) bool { return false }
//------------------------------------
// Ensure a testnet makes blocks when there are txs
func TestReactorCreatesBlockWhenEmptyBlocksFalse(t *testing.T) {
N := 4
css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter,
css, cleanup := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter,
func(c *cfg.Config) {
c.Consensus.CreateEmptyBlocks = false
})
reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
defer cleanup()
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, N)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
// send a tx
@@ -231,20 +235,21 @@ func TestReactorCreatesBlockWhenEmptyBlocksFalse(t *testing.T) {
// wait till everyone makes the first new block
timeoutWaitGroup(t, N, func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
}, css)
}
// Test we record stats about votes and block parts from other peers.
func TestReactorRecordsVotesAndBlockParts(t *testing.T) {
N := 4
css := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
reactors, eventChans, eventBuses := startConsensusNet(t, css, N)
css, cleanup := randConsensusNet(N, "consensus_reactor_test", newMockTickerFunc(true), newCounter)
defer cleanup()
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, N)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
// wait till everyone makes the first new block
timeoutWaitGroup(t, N, func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
}, css)
// Get peer
@@ -262,8 +267,9 @@ func TestReactorRecordsVotesAndBlockParts(t *testing.T) {
func TestReactorVotingPowerChange(t *testing.T) {
nVals := 4
logger := log.TestingLogger()
css := randConsensusNet(nVals, "consensus_voting_power_changes_test", newMockTickerFunc(true), newPersistentKVStore)
reactors, eventChans, eventBuses := startConsensusNet(t, css, nVals)
css, cleanup := randConsensusNet(nVals, "consensus_voting_power_changes_test", newMockTickerFunc(true), newPersistentKVStore)
defer cleanup()
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, nVals)
defer stopConsensusNet(logger, reactors, eventBuses)
// map of active validators
@@ -275,7 +281,7 @@ func TestReactorVotingPowerChange(t *testing.T) {
// wait till everyone makes block 1
timeoutWaitGroup(t, nVals, func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
}, css)
//---------------------------------------------------------------------------
@@ -286,10 +292,10 @@ func TestReactorVotingPowerChange(t *testing.T) {
updateValidatorTx := kvstore.MakeValSetChangeTx(val1PubKeyABCI, 25)
previousTotalVotingPower := css[0].GetRoundState().LastValidators.TotalVotingPower()
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css, updateValidatorTx)
waitForAndValidateBlockWithTx(t, nVals, activeVals, eventChans, css, updateValidatorTx)
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css)
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css, updateValidatorTx)
waitForAndValidateBlockWithTx(t, nVals, activeVals, blocksSubs, css, updateValidatorTx)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css)
if css[0].GetRoundState().LastValidators.TotalVotingPower() == previousTotalVotingPower {
t.Fatalf("expected voting power to change (before: %d, after: %d)", previousTotalVotingPower, css[0].GetRoundState().LastValidators.TotalVotingPower())
@@ -298,10 +304,10 @@ func TestReactorVotingPowerChange(t *testing.T) {
updateValidatorTx = kvstore.MakeValSetChangeTx(val1PubKeyABCI, 2)
previousTotalVotingPower = css[0].GetRoundState().LastValidators.TotalVotingPower()
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css, updateValidatorTx)
waitForAndValidateBlockWithTx(t, nVals, activeVals, eventChans, css, updateValidatorTx)
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css)
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css, updateValidatorTx)
waitForAndValidateBlockWithTx(t, nVals, activeVals, blocksSubs, css, updateValidatorTx)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css)
if css[0].GetRoundState().LastValidators.TotalVotingPower() == previousTotalVotingPower {
t.Fatalf("expected voting power to change (before: %d, after: %d)", previousTotalVotingPower, css[0].GetRoundState().LastValidators.TotalVotingPower())
@@ -310,10 +316,10 @@ func TestReactorVotingPowerChange(t *testing.T) {
updateValidatorTx = kvstore.MakeValSetChangeTx(val1PubKeyABCI, 26)
previousTotalVotingPower = css[0].GetRoundState().LastValidators.TotalVotingPower()
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css, updateValidatorTx)
waitForAndValidateBlockWithTx(t, nVals, activeVals, eventChans, css, updateValidatorTx)
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css)
waitForAndValidateBlock(t, nVals, activeVals, eventChans, css)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css, updateValidatorTx)
waitForAndValidateBlockWithTx(t, nVals, activeVals, blocksSubs, css, updateValidatorTx)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css)
waitForAndValidateBlock(t, nVals, activeVals, blocksSubs, css)
if css[0].GetRoundState().LastValidators.TotalVotingPower() == previousTotalVotingPower {
t.Fatalf("expected voting power to change (before: %d, after: %d)", previousTotalVotingPower, css[0].GetRoundState().LastValidators.TotalVotingPower())
@@ -323,11 +329,11 @@ func TestReactorVotingPowerChange(t *testing.T) {
func TestReactorValidatorSetChanges(t *testing.T) {
nPeers := 7
nVals := 4
css := randConsensusNetWithPeers(nVals, nPeers, "consensus_val_set_changes_test", newMockTickerFunc(true), newPersistentKVStore)
css, cleanup := randConsensusNetWithPeers(nVals, nPeers, "consensus_val_set_changes_test", newMockTickerFunc(true), newPersistentKVStore)
defer cleanup()
logger := log.TestingLogger()
reactors, eventChans, eventBuses := startConsensusNet(t, css, nPeers)
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, nPeers)
defer stopConsensusNet(logger, reactors, eventBuses)
// map of active validators
@@ -339,7 +345,7 @@ func TestReactorValidatorSetChanges(t *testing.T) {
// wait till everyone makes block 1
timeoutWaitGroup(t, nPeers, func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
}, css)
//---------------------------------------------------------------------------
@@ -352,22 +358,22 @@ func TestReactorValidatorSetChanges(t *testing.T) {
// wait till everyone makes block 2
// ensure the commit includes all validators
// send newValTx to change vals in block 3
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, newValidatorTx1)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css, newValidatorTx1)
// wait till everyone makes block 3.
// it includes the commit for block 2, which is by the original validator set
waitForAndValidateBlockWithTx(t, nPeers, activeVals, eventChans, css, newValidatorTx1)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, blocksSubs, css, newValidatorTx1)
// wait till everyone makes block 4.
// it includes the commit for block 3, which is by the original validator set
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css)
// the commits for block 4 should be with the updated validator set
activeVals[string(newValidatorPubKey1.Address())] = struct{}{}
// wait till everyone makes block 5
// it includes the commit for block 4, which should have the updated validator set
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, eventChans, css)
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, blocksSubs, css)
//---------------------------------------------------------------------------
logger.Info("---------------------------- Testing changing the voting power of one validator")
@@ -377,10 +383,10 @@ func TestReactorValidatorSetChanges(t *testing.T) {
updateValidatorTx1 := kvstore.MakeValSetChangeTx(updatePubKey1ABCI, 25)
previousTotalVotingPower := css[nVals].GetRoundState().LastValidators.TotalVotingPower()
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, updateValidatorTx1)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, eventChans, css, updateValidatorTx1)
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css)
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, eventChans, css)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css, updateValidatorTx1)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, blocksSubs, css, updateValidatorTx1)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css)
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, blocksSubs, css)
if css[nVals].GetRoundState().LastValidators.TotalVotingPower() == previousTotalVotingPower {
t.Errorf("expected voting power to change (before: %d, after: %d)", previousTotalVotingPower, css[nVals].GetRoundState().LastValidators.TotalVotingPower())
@@ -397,12 +403,12 @@ func TestReactorValidatorSetChanges(t *testing.T) {
newVal3ABCI := types.TM2PB.PubKey(newValidatorPubKey3)
newValidatorTx3 := kvstore.MakeValSetChangeTx(newVal3ABCI, testMinPower)
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, newValidatorTx2, newValidatorTx3)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, eventChans, css, newValidatorTx2, newValidatorTx3)
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css, newValidatorTx2, newValidatorTx3)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, blocksSubs, css, newValidatorTx2, newValidatorTx3)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css)
activeVals[string(newValidatorPubKey2.Address())] = struct{}{}
activeVals[string(newValidatorPubKey3.Address())] = struct{}{}
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, eventChans, css)
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, blocksSubs, css)
//---------------------------------------------------------------------------
logger.Info("---------------------------- Testing removing two validators at once")
@@ -410,40 +416,45 @@ func TestReactorValidatorSetChanges(t *testing.T) {
removeValidatorTx2 := kvstore.MakeValSetChangeTx(newVal2ABCI, 0)
removeValidatorTx3 := kvstore.MakeValSetChangeTx(newVal3ABCI, 0)
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, removeValidatorTx2, removeValidatorTx3)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, eventChans, css, removeValidatorTx2, removeValidatorTx3)
waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css, removeValidatorTx2, removeValidatorTx3)
waitForAndValidateBlockWithTx(t, nPeers, activeVals, blocksSubs, css, removeValidatorTx2, removeValidatorTx3)
waitForAndValidateBlock(t, nPeers, activeVals, blocksSubs, css)
delete(activeVals, string(newValidatorPubKey2.Address()))
delete(activeVals, string(newValidatorPubKey3.Address()))
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, eventChans, css)
waitForBlockWithUpdatedValsAndValidateIt(t, nPeers, activeVals, blocksSubs, css)
}
// Check we can make blocks with skip_timeout_commit=false
func TestReactorWithTimeoutCommit(t *testing.T) {
N := 4
css := randConsensusNet(N, "consensus_reactor_with_timeout_commit_test", newMockTickerFunc(false), newCounter)
css, cleanup := randConsensusNet(N, "consensus_reactor_with_timeout_commit_test", newMockTickerFunc(false), newCounter)
defer cleanup()
// override default SkipTimeoutCommit == true for tests
for i := 0; i < N; i++ {
css[i].config.SkipTimeoutCommit = false
}
reactors, eventChans, eventBuses := startConsensusNet(t, css, N-1)
reactors, blocksSubs, eventBuses := startConsensusNet(t, css, N-1)
defer stopConsensusNet(log.TestingLogger(), reactors, eventBuses)
// wait till everyone makes the first new block
timeoutWaitGroup(t, N-1, func(j int) {
<-eventChans[j]
<-blocksSubs[j].Out()
}, css)
}
func waitForAndValidateBlock(t *testing.T, n int, activeVals map[string]struct{}, eventChans []chan interface{}, css []*ConsensusState, txs ...[]byte) {
func waitForAndValidateBlock(
t *testing.T,
n int,
activeVals map[string]struct{},
blocksSubs []types.Subscription,
css []*ConsensusState,
txs ...[]byte,
) {
timeoutWaitGroup(t, n, func(j int) {
css[j].Logger.Debug("waitForAndValidateBlock")
newBlockI, ok := <-eventChans[j]
if !ok {
return
}
newBlock := newBlockI.(types.EventDataNewBlock).Block
msg := <-blocksSubs[j].Out()
newBlock := msg.Data().(types.EventDataNewBlock).Block
css[j].Logger.Debug("waitForAndValidateBlock: Got block", "height", newBlock.Height)
err := validateBlock(newBlock, activeVals)
assert.Nil(t, err)
@@ -454,17 +465,21 @@ func waitForAndValidateBlock(t *testing.T, n int, activeVals map[string]struct{}
}, css)
}
func waitForAndValidateBlockWithTx(t *testing.T, n int, activeVals map[string]struct{}, eventChans []chan interface{}, css []*ConsensusState, txs ...[]byte) {
func waitForAndValidateBlockWithTx(
t *testing.T,
n int,
activeVals map[string]struct{},
blocksSubs []types.Subscription,
css []*ConsensusState,
txs ...[]byte,
) {
timeoutWaitGroup(t, n, func(j int) {
ntxs := 0
BLOCK_TX_LOOP:
for {
css[j].Logger.Debug("waitForAndValidateBlockWithTx", "ntxs", ntxs)
newBlockI, ok := <-eventChans[j]
if !ok {
return
}
newBlock := newBlockI.(types.EventDataNewBlock).Block
msg := <-blocksSubs[j].Out()
newBlock := msg.Data().(types.EventDataNewBlock).Block
css[j].Logger.Debug("waitForAndValidateBlockWithTx: Got block", "height", newBlock.Height)
err := validateBlock(newBlock, activeVals)
assert.Nil(t, err)
@@ -485,18 +500,21 @@ func waitForAndValidateBlockWithTx(t *testing.T, n int, activeVals map[string]st
}, css)
}
func waitForBlockWithUpdatedValsAndValidateIt(t *testing.T, n int, updatedVals map[string]struct{}, eventChans []chan interface{}, css []*ConsensusState) {
func waitForBlockWithUpdatedValsAndValidateIt(
t *testing.T,
n int,
updatedVals map[string]struct{},
blocksSubs []types.Subscription,
css []*ConsensusState,
) {
timeoutWaitGroup(t, n, func(j int) {
var newBlock *types.Block
LOOP:
for {
css[j].Logger.Debug("waitForBlockWithUpdatedValsAndValidateIt")
newBlockI, ok := <-eventChans[j]
if !ok {
return
}
newBlock = newBlockI.(types.EventDataNewBlock).Block
msg := <-blocksSubs[j].Out()
newBlock = msg.Data().(types.EventDataNewBlock).Block
if newBlock.LastCommit.Size() == len(updatedVals) {
css[j].Logger.Debug("waitForBlockWithUpdatedValsAndValidateIt: Got block", "height", newBlock.Height)
break LOOP

View File

@@ -6,6 +6,7 @@ import (
"hash/crc32"
"io"
"reflect"
//"strconv"
//"strings"
"time"
@@ -41,7 +42,7 @@ var crc32c = crc32.MakeTable(crc32.Castagnoli)
// Unmarshal and apply a single message to the consensus state as if it were
// received in receiveRoutine. Lines that start with "#" are ignored.
// NOTE: receiveRoutine should not be running.
func (cs *ConsensusState) readReplayMessage(msg *TimedWALMessage, newStepCh chan interface{}) error {
func (cs *ConsensusState) readReplayMessage(msg *TimedWALMessage, newStepSub types.Subscription) error {
// Skip meta messages which exist for demarcating boundaries.
if _, ok := msg.Msg.(EndHeightMessage); ok {
return nil
@@ -53,15 +54,17 @@ func (cs *ConsensusState) readReplayMessage(msg *TimedWALMessage, newStepCh chan
cs.Logger.Info("Replay: New Step", "height", m.Height, "round", m.Round, "step", m.Step)
// these are playback checks
ticker := time.After(time.Second * 2)
if newStepCh != nil {
if newStepSub != nil {
select {
case mi := <-newStepCh:
m2 := mi.(types.EventDataRoundState)
case stepMsg := <-newStepSub.Out():
m2 := stepMsg.Data().(types.EventDataRoundState)
if m.Height != m2.Height || m.Round != m2.Round || m.Step != m2.Step {
return fmt.Errorf("RoundState mismatch. Got %v; Expected %v", m2, m)
}
case <-newStepSub.Cancelled():
return fmt.Errorf("Failed to read off newStepSub.Out(). newStepSub was cancelled")
case <-ticker:
return fmt.Errorf("Failed to read off newStepCh")
return fmt.Errorf("Failed to read off newStepSub.Out()")
}
}
case msgInfo:
@@ -143,8 +146,8 @@ func (cs *ConsensusState) catchupReplay(csHeight int64) error {
if err == io.EOF {
break
} else if IsDataCorruptionError(err) {
cs.Logger.Debug("data has been corrupted in last height of consensus WAL", "err", err, "height", csHeight)
panic(fmt.Sprintf("data has been corrupted (%v) in last height %d of consensus WAL", err, csHeight))
cs.Logger.Error("data has been corrupted in last height of consensus WAL", "err", err, "height", csHeight)
return err
} else if err != nil {
return err
}
@@ -196,6 +199,7 @@ type Handshaker struct {
stateDB dbm.DB
initialState sm.State
store sm.BlockStore
eventBus types.BlockEventPublisher
genDoc *types.GenesisDoc
logger log.Logger
@@ -209,6 +213,7 @@ func NewHandshaker(stateDB dbm.DB, state sm.State,
stateDB: stateDB,
initialState: state,
store: store,
eventBus: types.NopEventBus{},
genDoc: genDoc,
logger: log.NewNopLogger(),
nBlocks: 0,
@@ -219,6 +224,12 @@ func (h *Handshaker) SetLogger(l log.Logger) {
h.logger = l
}
// SetEventBus - sets the event bus for publishing block related events.
// If not called, it defaults to types.NopEventBus.
func (h *Handshaker) SetEventBus(eventBus types.BlockEventPublisher) {
h.eventBus = eventBus
}
func (h *Handshaker) NBlocks() int {
return h.nBlocks
}
@@ -313,7 +324,7 @@ func (h *Handshaker) ReplayBlocks(
}
if res.ConsensusParams != nil {
state.ConsensusParams = types.PB2TM.ConsensusParams(res.ConsensusParams)
state.ConsensusParams = types.PB2TM.ConsensusParams(res.ConsensusParams, state.ConsensusParams.Block.TimeIotaMs)
}
sm.SaveState(h.stateDB, state)
}
@@ -325,7 +336,7 @@ func (h *Handshaker) ReplayBlocks(
} else if storeBlockHeight < appBlockHeight {
// the app should never be ahead of the store (but this is under app's control)
return appHash, sm.ErrAppBlockHeightTooHigh{storeBlockHeight, appBlockHeight}
return appHash, sm.ErrAppBlockHeightTooHigh{CoreHeight: storeBlockHeight, AppHeight: appBlockHeight}
} else if storeBlockHeight < stateBlockHeight {
// the state should never be ahead of the store (this is under tendermint's control)
@@ -432,6 +443,7 @@ func (h *Handshaker) replayBlock(state sm.State, height int64, proxyApp proxy.Ap
meta := h.store.LoadBlockMeta(height)
blockExec := sm.NewBlockExecutor(h.stateDB, h.logger, proxyApp, sm.MockMempool{}, sm.MockEvidencePool{})
blockExec.SetEventBus(h.eventBus)
var err error
state, err = blockExec.ApplyBlock(state, meta.BlockID, block)

View File

@@ -51,25 +51,13 @@ func (cs *ConsensusState) ReplayFile(file string, console bool) error {
cs.startForReplay()
// ensure all new step events are regenerated as expected
newStepCh := make(chan interface{}, 1)
ctx := context.Background()
err := cs.eventBus.Subscribe(ctx, subscriber, types.EventQueryNewRoundStep, newStepCh)
newStepSub, err := cs.eventBus.Subscribe(ctx, subscriber, types.EventQueryNewRoundStep)
if err != nil {
return errors.Errorf("failed to subscribe %s to %v", subscriber, types.EventQueryNewRoundStep)
}
defer func() {
// drain newStepCh to make sure we don't block
LOOP:
for {
select {
case <-newStepCh:
default:
break LOOP
}
}
cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
}()
defer cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
// just open the file for reading, no need to use wal
fp, err := os.OpenFile(file, os.O_RDONLY, 0600)
@@ -94,7 +82,7 @@ func (cs *ConsensusState) ReplayFile(file string, console bool) error {
return err
}
if err := pb.cs.readReplayMessage(msg, newStepCh); err != nil {
if err := pb.cs.readReplayMessage(msg, newStepSub); err != nil {
return err
}
@@ -103,7 +91,6 @@ func (cs *ConsensusState) ReplayFile(file string, console bool) error {
}
pb.count++
}
return nil
}
//------------------------------------------------
@@ -132,7 +119,7 @@ func newPlayback(fileName string, fp *os.File, cs *ConsensusState, genState sm.S
}
// go back count steps by resetting the state and running (pb.count - count) steps
func (pb *playback) replayReset(count int, newStepCh chan interface{}) error {
func (pb *playback) replayReset(count int, newStepSub types.Subscription) error {
pb.cs.Stop()
pb.cs.Wait()
@@ -162,7 +149,7 @@ func (pb *playback) replayReset(count int, newStepCh chan interface{}) error {
} else if err != nil {
return err
}
if err := pb.cs.readReplayMessage(msg, newStepCh); err != nil {
if err := pb.cs.readReplayMessage(msg, newStepSub); err != nil {
return err
}
pb.count++
@@ -226,27 +213,15 @@ func (pb *playback) replayConsoleLoop() int {
ctx := context.Background()
// ensure all new step events are regenerated as expected
newStepCh := make(chan interface{}, 1)
err := pb.cs.eventBus.Subscribe(ctx, subscriber, types.EventQueryNewRoundStep, newStepCh)
newStepSub, err := pb.cs.eventBus.Subscribe(ctx, subscriber, types.EventQueryNewRoundStep)
if err != nil {
cmn.Exit(fmt.Sprintf("failed to subscribe %s to %v", subscriber, types.EventQueryNewRoundStep))
}
defer func() {
// drain newStepCh to make sure we don't block
LOOP:
for {
select {
case <-newStepCh:
default:
break LOOP
}
}
pb.cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
}()
defer pb.cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep)
if len(tokens) == 1 {
if err := pb.replayReset(1, newStepCh); err != nil {
if err := pb.replayReset(1, newStepSub); err != nil {
pb.cs.Logger.Error("Replay reset error", "err", err)
}
} else {
@@ -256,7 +231,7 @@ func (pb *playback) replayConsoleLoop() int {
} else if i > pb.count {
fmt.Printf("argument to back must not be larger than the current count (%d)\n", pb.count)
} else {
if err := pb.replayReset(i, newStepCh); err != nil {
if err := pb.replayReset(i, newStepSub); err != nil {
pb.cs.Logger.Error("Replay reset error", "err", err)
}
}
@@ -295,7 +270,6 @@ func (pb *playback) replayConsoleLoop() int {
fmt.Println(pb.count)
}
}
return 0
}
//--------------------------------------------------------------------------------
@@ -326,17 +300,18 @@ func newConsensusStateForReplay(config cfg.BaseConfig, csConfig *cfg.ConsensusCo
cmn.Exit(fmt.Sprintf("Error starting proxy app conns: %v", err))
}
handshaker := NewHandshaker(stateDB, state, blockStore, gdoc)
err = handshaker.Handshake(proxyApp)
if err != nil {
cmn.Exit(fmt.Sprintf("Error on handshake: %v", err))
}
eventBus := types.NewEventBus()
if err := eventBus.Start(); err != nil {
cmn.Exit(fmt.Sprintf("Failed to start event bus: %v", err))
}
handshaker := NewHandshaker(stateDB, state, blockStore, gdoc)
handshaker.SetEventBus(eventBus)
err = handshaker.Handshake(proxyApp)
if err != nil {
cmn.Exit(fmt.Sprintf("Error on handshake: %v", err))
}
mempool, evpool := sm.MockMempool{}, sm.MockEvidencePool{}
blockExec := sm.NewBlockExecutor(stateDB, log.TestingLogger(), proxyApp.Consensus(), mempool, evpool)

View File

@@ -17,23 +17,30 @@ import (
"github.com/tendermint/tendermint/abci/example/kvstore"
abci "github.com/tendermint/tendermint/abci/types"
"github.com/tendermint/tendermint/crypto"
auto "github.com/tendermint/tendermint/libs/autofile"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/version"
cfg "github.com/tendermint/tendermint/config"
"github.com/tendermint/tendermint/crypto"
dbm "github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/libs/log"
"github.com/tendermint/tendermint/privval"
"github.com/tendermint/tendermint/proxy"
sm "github.com/tendermint/tendermint/state"
"github.com/tendermint/tendermint/types"
"github.com/tendermint/tendermint/version"
)
var consensusReplayConfig *cfg.Config
func init() {
func TestMain(m *testing.M) {
config = ResetConfig("consensus_reactor_test")
consensusReplayConfig = ResetConfig("consensus_replay_test")
configStateTest := ResetConfig("consensus_state_test")
configMempoolTest := ResetConfig("consensus_mempool_test")
configByzantineTest := ResetConfig("consensus_byzantine_test")
code := m.Run()
os.RemoveAll(config.RootDir)
os.RemoveAll(consensusReplayConfig.RootDir)
os.RemoveAll(configStateTest.RootDir)
os.RemoveAll(configMempoolTest.RootDir)
os.RemoveAll(configByzantineTest.RootDir)
os.Exit(code)
}
// These tests ensure we can always recover from failure at any part of the consensus process.
@@ -51,7 +58,8 @@ func init() {
// and which ones we need the wal for - then we'd also be able to only flush the
// wal writer when we need to, instead of with every message.
func startNewConsensusStateAndWaitForBlock(t *testing.T, lastBlockHeight int64, blockDB dbm.DB, stateDB dbm.DB) {
func startNewConsensusStateAndWaitForBlock(t *testing.T, consensusReplayConfig *cfg.Config,
lastBlockHeight int64, blockDB dbm.DB, stateDB dbm.DB) {
logger := log.TestingLogger()
state, _ := sm.LoadStateFromDBOrGenesisFile(stateDB, consensusReplayConfig.GenesisFile())
privValidator := loadPrivValidator(consensusReplayConfig)
@@ -59,7 +67,6 @@ func startNewConsensusStateAndWaitForBlock(t *testing.T, lastBlockHeight int64,
cs.SetLogger(logger)
bytes, _ := ioutil.ReadFile(cs.config.WalFile())
// fmt.Printf("====== WAL: \n\r%s\n", bytes)
t.Logf("====== WAL: \n\r%X\n", bytes)
err := cs.Start()
@@ -70,13 +77,14 @@ func startNewConsensusStateAndWaitForBlock(t *testing.T, lastBlockHeight int64,
// in the WAL itself. Assuming the consensus state is running, replay of any
// WAL, including the empty one, should eventually be followed by a new
// block, or else something is wrong.
newBlockCh := make(chan interface{}, 1)
err = cs.eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock, newBlockCh)
newBlockSub, err := cs.eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock)
require.NoError(t, err)
select {
case <-newBlockCh:
case <-time.After(60 * time.Second):
t.Fatalf("Timed out waiting for new block (see trace above)")
case <-newBlockSub.Out():
case <-newBlockSub.Cancelled():
t.Fatal("newBlockSub was cancelled")
case <-time.After(120 * time.Second):
t.Fatal("Timed out waiting for new block (see trace above)")
}
}
@@ -110,21 +118,22 @@ func TestWALCrash(t *testing.T) {
3},
}
for _, tc := range testCases {
for i, tc := range testCases {
consensusReplayConfig := ResetConfig(fmt.Sprintf("%s_%d", t.Name(), i))
t.Run(tc.name, func(t *testing.T) {
crashWALandCheckLiveness(t, tc.initFn, tc.heightToStop)
crashWALandCheckLiveness(t, consensusReplayConfig, tc.initFn, tc.heightToStop)
})
}
}
func crashWALandCheckLiveness(t *testing.T, initFn func(dbm.DB, *ConsensusState, context.Context), heightToStop int64) {
walPaniced := make(chan error)
crashingWal := &crashingWAL{panicCh: walPaniced, heightToStop: heightToStop}
func crashWALandCheckLiveness(t *testing.T, consensusReplayConfig *cfg.Config,
initFn func(dbm.DB, *ConsensusState, context.Context), heightToStop int64) {
walPanicked := make(chan error)
crashingWal := &crashingWAL{panicCh: walPanicked, heightToStop: heightToStop}
i := 1
LOOP:
for {
// fmt.Printf("====== LOOP %d\n", i)
t.Logf("====== LOOP %d\n", i)
// create consensus state from a clean slate
@@ -159,11 +168,11 @@ LOOP:
i++
select {
case err := <-walPaniced:
t.Logf("WAL paniced: %v", err)
case err := <-walPanicked:
t.Logf("WAL panicked: %v", err)
// make sure we can make blocks after a crash
startNewConsensusStateAndWaitForBlock(t, cs.Height, blockDB, stateDB)
startNewConsensusStateAndWaitForBlock(t, consensusReplayConfig, cs.Height, blockDB, stateDB)
// stop consensus state and transactions sender (initFn)
cs.Stop()
@@ -181,16 +190,18 @@ LOOP:
// crashingWAL is a WAL which crashes or rather simulates a crash during Save
// (before and after). It remembers a message for which we last panicked
// (lastPanicedForMsgIndex), so we don't panic for it in subsequent iterations.
// (lastPanickedForMsgIndex), so we don't panic for it in subsequent iterations.
type crashingWAL struct {
next WAL
panicCh chan error
heightToStop int64
msgIndex int // current message index
lastPanicedForMsgIndex int // last message for which we panicked
msgIndex int // current message index
lastPanickedForMsgIndex int // last message for which we panicked
}
var _ WAL = &crashingWAL{}
// WALWriteError indicates a WAL crash.
type WALWriteError struct {
msg string
@@ -223,8 +234,8 @@ func (w *crashingWAL) Write(m WALMessage) {
return
}
if w.msgIndex > w.lastPanicedForMsgIndex {
w.lastPanicedForMsgIndex = w.msgIndex
if w.msgIndex > w.lastPanickedForMsgIndex {
w.lastPanickedForMsgIndex = w.msgIndex
_, file, line, _ := runtime.Caller(1)
w.panicCh <- WALWriteError{fmt.Sprintf("failed to write %T to WAL (fileline: %s:%d)", m, file, line)}
runtime.Goexit()
@@ -238,8 +249,9 @@ func (w *crashingWAL) WriteSync(m WALMessage) {
w.Write(m)
}
func (w *crashingWAL) Group() *auto.Group { return w.next.Group() }
func (w *crashingWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (gr *auto.GroupReader, found bool, err error) {
func (w *crashingWAL) FlushAndSync() error { return w.next.FlushAndSync() }
func (w *crashingWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (rd io.ReadCloser, found bool, err error) {
return w.next.SearchForEndHeight(height, options)
}
@@ -269,29 +281,37 @@ var modes = []uint{0, 1, 2}
// Sync from scratch
func TestHandshakeReplayAll(t *testing.T) {
for _, m := range modes {
testHandshakeReplay(t, 0, m)
for i, m := range modes {
config := ResetConfig(fmt.Sprintf("%s_%v", t.Name(), i))
defer os.RemoveAll(config.RootDir)
testHandshakeReplay(t, config, 0, m)
}
}
// Sync many, not from scratch
func TestHandshakeReplaySome(t *testing.T) {
for _, m := range modes {
testHandshakeReplay(t, 1, m)
for i, m := range modes {
config := ResetConfig(fmt.Sprintf("%s_%v", t.Name(), i))
defer os.RemoveAll(config.RootDir)
testHandshakeReplay(t, config, 1, m)
}
}
// Sync from lagging by one
func TestHandshakeReplayOne(t *testing.T) {
for _, m := range modes {
testHandshakeReplay(t, NUM_BLOCKS-1, m)
for i, m := range modes {
config := ResetConfig(fmt.Sprintf("%s_%v", t.Name(), i))
defer os.RemoveAll(config.RootDir)
testHandshakeReplay(t, config, NUM_BLOCKS-1, m)
}
}
// Sync from caught up
func TestHandshakeReplayNone(t *testing.T) {
for _, m := range modes {
testHandshakeReplay(t, NUM_BLOCKS, m)
for i, m := range modes {
config := ResetConfig(fmt.Sprintf("%s_%v", t.Name(), i))
defer os.RemoveAll(config.RootDir)
testHandshakeReplay(t, config, NUM_BLOCKS, m)
}
}
@@ -311,10 +331,8 @@ func tempWALWithData(data []byte) string {
}
// Make some blocks. Start a fresh app and apply nBlocks blocks. Then restart the app and sync it up with the remaining blocks
func testHandshakeReplay(t *testing.T, nBlocks int, mode uint) {
config := ResetConfig("proxy_test_")
walBody, err := WALWithNBlocks(NUM_BLOCKS)
func testHandshakeReplay(t *testing.T, config *cfg.Config, nBlocks int, mode uint) {
walBody, err := WALWithNBlocks(t, NUM_BLOCKS)
require.NoError(t, err)
walFile := tempWALWithData(walBody)
config.Consensus.SetWalFile(walFile)
@@ -537,10 +555,8 @@ func makeBlockchainFromWAL(wal WAL) ([]*types.Block, []*types.Commit, error) {
}
case *types.Vote:
if p.Type == types.PrecommitType {
thisBlockCommit = &types.Commit{
BlockID: p.BlockID,
Precommits: []*types.Vote{p},
}
commitSigs := []*types.CommitSig{p.CommitSig()}
thisBlockCommit = types.NewCommit(p.BlockID, commitSigs)
}
}
}
@@ -633,6 +649,7 @@ func TestInitChainUpdateValidators(t *testing.T) {
clientCreator := proxy.NewLocalClientCreator(app)
config := ResetConfig("proxy_test_")
defer os.RemoveAll(config.RootDir)
privVal := privval.LoadFilePV(config.PrivValidatorKeyFile(), config.PrivValidatorStateFile())
stateDB, state, store := stateAndStore(config, privVal.GetPubKey(), 0x0)

View File

@@ -94,8 +94,7 @@ type ConsensusState struct {
// internal state
mtx sync.RWMutex
cstypes.RoundState
triggeredTimeoutPrecommit bool
state sm.State // State until height-1.
state sm.State // State until height-1.
// state changes may be triggered by: msgs from peers,
// msgs from ourself, or by timeouts
@@ -308,6 +307,23 @@ func (cs *ConsensusState) OnStart() error {
// reload from consensus log to catchup
if cs.doWALCatchup {
if err := cs.catchupReplay(cs.Height); err != nil {
// don't try to recover from data corruption error
if IsDataCorruptionError(err) {
cs.Logger.Error("Encountered corrupt WAL file", "err", err.Error())
cs.Logger.Error("Please repair the WAL file before restarting")
fmt.Println(`You can attempt to repair the WAL as follows:
----
WALFILE=~/.tendermint/data/cs.wal/wal
cp $WALFILE ${WALFILE}.bak # backup the file
go run scripts/wal2json/main.go $WALFILE > wal.json # this will panic, but can be ignored
rm $WALFILE # remove the corrupt file
go run scripts/json2wal/main.go wal.json $WALFILE # rebuild the file without corruption
----`)
return err
}
cs.Logger.Error("Error on catchup replay. Proceeding to start ConsensusState anyway", "err", err.Error())
// NOTE: if we ever do return an error here,
// make sure to stop the timeoutTicker
@@ -438,7 +454,7 @@ func (cs *ConsensusState) updateRoundStep(round int, step cstypes.RoundStepType)
// enterNewRound(height, 0) at cs.StartTime.
func (cs *ConsensusState) scheduleRound0(rs *cstypes.RoundState) {
//cs.Logger.Info("scheduleRound0", "now", tmtime.Now(), "startTime", cs.StartTime)
sleepDuration := rs.StartTime.Sub(tmtime.Now()) // nolint: gotype, gosimple
sleepDuration := rs.StartTime.Sub(tmtime.Now())
cs.scheduleTimeout(sleepDuration, rs.Height, 0, cstypes.RoundStepNewHeight)
}
@@ -473,7 +489,7 @@ func (cs *ConsensusState) reconstructLastCommit(state sm.State) {
if precommit == nil {
continue
}
added, err := lastPrecommits.AddVote(precommit)
added, err := lastPrecommits.AddVote(seenCommit.ToVote(precommit))
if !added || err != nil {
cmn.PanicCrisis(fmt.Sprintf("Failed to reconstruct LastCommit: %v", err))
}
@@ -550,6 +566,7 @@ func (cs *ConsensusState) updateToState(state sm.State) {
cs.CommitRound = -1
cs.LastCommit = lastPrecommits
cs.LastValidators = state.LastValidators
cs.TriggeredTimeoutPrecommit = false
cs.state = state
@@ -625,6 +642,15 @@ func (cs *ConsensusState) receiveRoutine(maxSteps int) {
cs.handleMsg(mi)
case mi = <-cs.internalMsgQueue:
cs.wal.WriteSync(mi) // NOTE: fsync
if _, ok := mi.Msg.(*VoteMessage); ok {
// we actually want to simulate failing during
// the previous WriteSync, but this isn't easy to do.
// Equivalent would be to fail here and manually remove
// some bytes from the end of the wal.
fail.Fail() // XXX
}
// handles proposals, block parts, votes
cs.handleMsg(mi)
case ti := <-cs.timeoutTicker.Chan(): // tockChan:
@@ -644,7 +670,10 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
cs.mtx.Lock()
defer cs.mtx.Unlock()
var err error
var (
added bool
err error
)
msg, peerID := mi.Msg, mi.PeerID
switch msg := msg.(type) {
case *ProposalMessage:
@@ -653,7 +682,7 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
err = cs.setProposal(msg.Proposal)
case *BlockPartMessage:
// if the proposal is complete, we'll enterPrevote or tryFinalizeCommit
added, err := cs.addProposalBlockPart(msg, peerID)
added, err = cs.addProposalBlockPart(msg, peerID)
if added {
cs.statsMsgQueue <- mi
}
@@ -665,7 +694,7 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
case *VoteMessage:
// attempt to add the vote and dupeout the validator if its a duplicate signature
// if the vote gives us a 2/3-any or 2/3-one, we transition
added, err := cs.tryAddVote(msg.Vote, peerID)
added, err = cs.tryAddVote(msg.Vote, peerID)
if added {
cs.statsMsgQueue <- mi
}
@@ -685,10 +714,15 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
// the peer is sending us CatchupCommit precommits.
// We could make note of this and help filter in broadcastHasVoteMessage().
default:
cs.Logger.Error("Unknown msg type", reflect.TypeOf(msg))
cs.Logger.Error("Unknown msg type", "type", reflect.TypeOf(msg))
return
}
if err != nil {
cs.Logger.Error("Error with msg", "height", cs.Height, "round", cs.Round, "type", reflect.TypeOf(msg), "peer", peerID, "err", err, "msg", msg)
// Causes TestReactorValidatorSetChanges to timeout
// https://github.com/tendermint/tendermint/issues/3406
// cs.Logger.Error("Error with msg", "height", cs.Height, "round", cs.Round,
// "peer", peerID, "err", err, "msg", msg)
}
}
@@ -732,6 +766,7 @@ func (cs *ConsensusState) handleTxsAvailable() {
cs.mtx.Lock()
defer cs.mtx.Unlock()
// we only need to do this for round 0
cs.enterNewRound(cs.Height, 0)
cs.enterPropose(cs.Height, 0)
}
@@ -782,7 +817,7 @@ func (cs *ConsensusState) enterNewRound(height int64, round int) {
cs.ProposalBlockParts = nil
}
cs.Votes.SetRound(round + 1) // also track next round (round+1) to allow round-skipping
cs.triggeredTimeoutPrecommit = false
cs.TriggeredTimeoutPrecommit = false
cs.eventBus.PublishEventNewRound(cs.NewRoundEvent())
cs.metrics.Rounds.Set(float64(round))
@@ -882,8 +917,11 @@ func (cs *ConsensusState) defaultDecideProposal(height int64, round int) {
}
}
// Flush the WAL. Otherwise, we may not recompute the same proposal to sign, and the privValidator will refuse to sign anything.
cs.wal.FlushAndSync()
// Make proposal
propBlockId := types.BlockID{block.Hash(), blockParts.Header()}
propBlockId := types.BlockID{Hash: block.Hash(), PartsHeader: blockParts.Header()}
proposal := types.NewProposal(height, round, cs.ValidRound, propBlockId)
if err := cs.privValidator.SignProposal(cs.state.ChainID, proposal); err == nil {
@@ -928,7 +966,7 @@ func (cs *ConsensusState) createProposalBlock() (block *types.Block, blockParts
if cs.Height == 1 {
// We're creating a proposal for the first block.
// The commit is empty, but not nil.
commit = &types.Commit{}
commit = types.NewCommit(types.BlockID{}, nil)
} else if cs.LastCommit.HasTwoThirdsMajority() {
// Make the commit from LastCommit
commit = cs.LastCommit.MakeCommit()
@@ -1128,12 +1166,12 @@ func (cs *ConsensusState) enterPrecommit(height int64, round int) {
func (cs *ConsensusState) enterPrecommitWait(height int64, round int) {
logger := cs.Logger.With("height", height, "round", round)
if cs.Height != height || round < cs.Round || (cs.Round == round && cs.triggeredTimeoutPrecommit) {
if cs.Height != height || round < cs.Round || (cs.Round == round && cs.TriggeredTimeoutPrecommit) {
logger.Debug(
fmt.Sprintf(
"enterPrecommitWait(%v/%v): Invalid args. "+
"Current state is Height/Round: %v/%v/, triggeredTimeoutPrecommit:%v",
height, round, cs.Height, cs.Round, cs.triggeredTimeoutPrecommit))
"Current state is Height/Round: %v/%v/, TriggeredTimeoutPrecommit:%v",
height, round, cs.Height, cs.Round, cs.TriggeredTimeoutPrecommit))
return
}
if !cs.Votes.Precommits(round).HasTwoThirdsAny() {
@@ -1143,7 +1181,7 @@ func (cs *ConsensusState) enterPrecommitWait(height int64, round int) {
defer func() {
// Done enterPrecommitWait:
cs.triggeredTimeoutPrecommit = true
cs.TriggeredTimeoutPrecommit = true
cs.newStep()
}()
@@ -1294,7 +1332,7 @@ func (cs *ConsensusState) finalizeCommit(height int64) {
// Execute and commit the block, update and save the state, and update the mempool.
// NOTE The block.AppHash wont reflect these txs until the next block.
var err error
stateCopy, err = cs.blockExec.ApplyBlock(stateCopy, types.BlockID{block.Hash(), blockParts.Header()}, block)
stateCopy, err = cs.blockExec.ApplyBlock(stateCopy, types.BlockID{Hash: block.Hash(), PartsHeader: blockParts.Header()}, block)
if err != nil {
cs.Logger.Error("Error on ApplyBlock. Did the application crash? Please restart tendermint", "err", err)
err := cmn.Kill()
@@ -1330,7 +1368,7 @@ func (cs *ConsensusState) recordMetrics(height int64, block *types.Block) {
missingValidators := 0
missingValidatorsPower := int64(0)
for i, val := range cs.Validators.Validators {
var vote *types.Vote
var vote *types.CommitSig
if i < len(block.LastCommit.Precommits) {
vote = block.LastCommit.Precommits[i]
}
@@ -1429,7 +1467,7 @@ func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p
_, err = cdc.UnmarshalBinaryLengthPrefixedReader(
cs.ProposalBlockParts.GetReader(),
&cs.ProposalBlock,
int64(cs.state.ConsensusParams.BlockSize.MaxBytes),
int64(cs.state.ConsensusParams.Block.MaxBytes),
)
if err != nil {
return added, err
@@ -1517,7 +1555,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
}
cs.Logger.Info(fmt.Sprintf("Added to lastPrecommits: %v", cs.LastCommit.StringShort()))
cs.eventBus.PublishEventVote(types.EventDataVote{vote})
cs.eventBus.PublishEventVote(types.EventDataVote{Vote: vote})
cs.evsw.FireEvent(types.EventVote, vote)
// if we can skip timeoutCommit and have all the votes now,
@@ -1545,7 +1583,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
return
}
cs.eventBus.PublishEventVote(types.EventDataVote{vote})
cs.eventBus.PublishEventVote(types.EventDataVote{Vote: vote})
cs.evsw.FireEvent(types.EventVote, vote)
switch vote.Type {
@@ -1647,6 +1685,9 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
}
func (cs *ConsensusState) signVote(type_ types.SignedMsgType, hash []byte, header types.PartSetHeader) (*types.Vote, error) {
// Flush the WAL. Otherwise, we may not recompute the same vote to sign, and the privValidator will refuse to sign anything.
cs.wal.FlushAndSync()
addr := cs.privValidator.GetPubKey().Address()
valIndex, _ := cs.Validators.GetByAddress(addr)
@@ -1657,7 +1698,7 @@ func (cs *ConsensusState) signVote(type_ types.SignedMsgType, hash []byte, heade
Round: cs.Round,
Timestamp: cs.voteTime(),
Type: type_,
BlockID: types.BlockID{hash, header},
BlockID: types.BlockID{Hash: hash, PartsHeader: header},
}
err := cs.privValidator.SignVote(cs.state.ChainID, vote)
return vote, err
@@ -1668,10 +1709,12 @@ func (cs *ConsensusState) voteTime() time.Time {
minVoteTime := now
// TODO: We should remove next line in case we don't vote for v in case cs.ProposalBlock == nil,
// even if cs.LockedBlock != nil. See https://github.com/tendermint/spec.
timeIotaMs := time.Duration(cs.state.ConsensusParams.Block.TimeIotaMs) * time.Millisecond
if cs.LockedBlock != nil {
minVoteTime = cs.config.MinValidVoteTime(cs.LockedBlock.Time)
// See the BFT time spec https://tendermint.com/docs/spec/consensus/bft-time.html
minVoteTime = cs.LockedBlock.Time.Add(timeIotaMs)
} else if cs.ProposalBlock != nil {
minVoteTime = cs.config.MinValidVoteTime(cs.ProposalBlock.Time)
minVoteTime = cs.ProposalBlock.Time.Add(timeIotaMs)
}
if now.After(minVoteTime) {

View File

@@ -18,14 +18,6 @@ import (
"github.com/tendermint/tendermint/types"
)
func init() {
config = ResetConfig("consensus_state_test")
}
func ensureProposeTimeout(timeoutPropose time.Duration) time.Duration {
return time.Duration(timeoutPropose.Nanoseconds()*2) * time.Nanosecond
}
/*
ProposeSuite
@@ -1279,6 +1271,130 @@ func TestCommitFromPreviousRound(t *testing.T) {
ensureNewRound(newRoundCh, height+1, 0)
}
type fakeTxNotifier struct {
ch chan struct{}
}
func (n *fakeTxNotifier) TxsAvailable() <-chan struct{} {
return n.ch
}
func (n *fakeTxNotifier) Notify() {
n.ch <- struct{}{}
}
func TestStartNextHeightCorrectly(t *testing.T) {
config.Consensus.SkipTimeoutCommit = false
cs1, vss := randConsensusState(4)
cs1.txNotifier = &fakeTxNotifier{ch: make(chan struct{})}
vs2, vs3, vs4 := vss[1], vss[2], vss[3]
height, round := cs1.Height, cs1.Round
proposalCh := subscribe(cs1.eventBus, types.EventQueryCompleteProposal)
timeoutProposeCh := subscribe(cs1.eventBus, types.EventQueryTimeoutPropose)
newRoundCh := subscribe(cs1.eventBus, types.EventQueryNewRound)
newBlockHeader := subscribe(cs1.eventBus, types.EventQueryNewBlockHeader)
addr := cs1.privValidator.GetPubKey().Address()
voteCh := subscribeToVoter(cs1, addr)
// start round and wait for propose and prevote
startTestRound(cs1, height, round)
ensureNewRound(newRoundCh, height, round)
ensureNewProposal(proposalCh, height, round)
rs := cs1.GetRoundState()
theBlockHash := rs.ProposalBlock.Hash()
theBlockParts := rs.ProposalBlockParts.Header()
ensurePrevote(voteCh, height, round)
validatePrevote(t, cs1, round, vss[0], theBlockHash)
signAddVotes(cs1, types.PrevoteType, theBlockHash, theBlockParts, vs2, vs3, vs4)
ensurePrecommit(voteCh, height, round)
// the proposed block should now be locked and our precommit added
validatePrecommit(t, cs1, round, round, vss[0], theBlockHash, theBlockHash)
rs = cs1.GetRoundState()
// add precommits
signAddVotes(cs1, types.PrecommitType, nil, types.PartSetHeader{}, vs2)
signAddVotes(cs1, types.PrecommitType, theBlockHash, theBlockParts, vs3)
time.Sleep(5 * time.Millisecond)
signAddVotes(cs1, types.PrecommitType, theBlockHash, theBlockParts, vs4)
rs = cs1.GetRoundState()
assert.True(t, rs.TriggeredTimeoutPrecommit)
ensureNewBlockHeader(newBlockHeader, height, theBlockHash)
cs1.txNotifier.(*fakeTxNotifier).Notify()
ensureNewTimeout(timeoutProposeCh, height+1, round, cs1.config.TimeoutPropose.Nanoseconds())
rs = cs1.GetRoundState()
assert.False(t, rs.TriggeredTimeoutPrecommit, "triggeredTimeoutPrecommit should be false at the beginning of each round")
}
func TestResetTimeoutPrecommitUponNewHeight(t *testing.T) {
config.Consensus.SkipTimeoutCommit = false
cs1, vss := randConsensusState(4)
vs2, vs3, vs4 := vss[1], vss[2], vss[3]
height, round := cs1.Height, cs1.Round
partSize := types.BlockPartSizeBytes
proposalCh := subscribe(cs1.eventBus, types.EventQueryCompleteProposal)
newRoundCh := subscribe(cs1.eventBus, types.EventQueryNewRound)
newBlockHeader := subscribe(cs1.eventBus, types.EventQueryNewBlockHeader)
addr := cs1.privValidator.GetPubKey().Address()
voteCh := subscribeToVoter(cs1, addr)
// start round and wait for propose and prevote
startTestRound(cs1, height, round)
ensureNewRound(newRoundCh, height, round)
ensureNewProposal(proposalCh, height, round)
rs := cs1.GetRoundState()
theBlockHash := rs.ProposalBlock.Hash()
theBlockParts := rs.ProposalBlockParts.Header()
ensurePrevote(voteCh, height, round)
validatePrevote(t, cs1, round, vss[0], theBlockHash)
signAddVotes(cs1, types.PrevoteType, theBlockHash, theBlockParts, vs2, vs3, vs4)
ensurePrecommit(voteCh, height, round)
validatePrecommit(t, cs1, round, round, vss[0], theBlockHash, theBlockHash)
rs = cs1.GetRoundState()
// add precommits
signAddVotes(cs1, types.PrecommitType, nil, types.PartSetHeader{}, vs2)
signAddVotes(cs1, types.PrecommitType, theBlockHash, theBlockParts, vs3)
time.Sleep(5 * time.Millisecond)
signAddVotes(cs1, types.PrecommitType, theBlockHash, theBlockParts, vs4)
rs = cs1.GetRoundState()
assert.True(t, rs.TriggeredTimeoutPrecommit)
ensureNewBlockHeader(newBlockHeader, height, theBlockHash)
prop, propBlock := decideProposal(cs1, vs2, height+1, 0)
propBlockParts := propBlock.MakePartSet(partSize)
if err := cs1.SetProposalAndBlock(prop, propBlock, propBlockParts, "some peer"); err != nil {
t.Fatal(err)
}
ensureNewProposal(proposalCh, height+1, 0)
rs = cs1.GetRoundState()
assert.False(t, rs.TriggeredTimeoutPrecommit, "triggeredTimeoutPrecommit should be false at the beginning of each height")
}
//------------------------------------------------------------------------------------------
// SlashingSuite
// TODO: Slashing
@@ -1504,11 +1620,10 @@ func TestStateOutputVoteStats(t *testing.T) {
}
// subscribe subscribes test client to the given query and returns a channel with cap = 1.
func subscribe(eventBus *types.EventBus, q tmpubsub.Query) <-chan interface{} {
out := make(chan interface{}, 1)
err := eventBus.Subscribe(context.Background(), testSubscriber, q, out)
func subscribe(eventBus *types.EventBus, q tmpubsub.Query) <-chan tmpubsub.Message {
sub, err := eventBus.Subscribe(context.Background(), testSubscriber, q)
if err != nil {
panic(fmt.Sprintf("failed to subscribe %s to %v", testSubscriber, q))
}
return out
return sub.Out()
}

View File

@@ -2,6 +2,7 @@ package types
import (
"fmt"
"os"
"testing"
cfg "github.com/tendermint/tendermint/config"
@@ -11,8 +12,11 @@ import (
var config *cfg.Config // NOTE: must be reset for each _test.go file
func init() {
func TestMain(m *testing.M) {
config = cfg.ResetTestRoot("consensus_height_vote_set_test")
code := m.Run()
os.RemoveAll(config.RootDir)
os.Exit(code)
}
func TestPeerCatchupRounds(t *testing.T) {
@@ -64,7 +68,6 @@ func makeVoteHR(t *testing.T, height int64, round int, privVals []types.PrivVali
err := privVal.SignVote(chainID, vote)
if err != nil {
panic(fmt.Sprintf("Error signing vote: %v", err))
return nil
}
return vote
}

View File

@@ -65,25 +65,26 @@ func (rs RoundStepType) String() string {
// NOTE: Not thread safe. Should only be manipulated by functions downstream
// of the cs.receiveRoutine
type RoundState struct {
Height int64 `json:"height"` // Height we are working on
Round int `json:"round"`
Step RoundStepType `json:"step"`
StartTime time.Time `json:"start_time"`
CommitTime time.Time `json:"commit_time"` // Subjective time when +2/3 precommits for Block at Round were found
Validators *types.ValidatorSet `json:"validators"`
Proposal *types.Proposal `json:"proposal"`
ProposalBlock *types.Block `json:"proposal_block"`
ProposalBlockParts *types.PartSet `json:"proposal_block_parts"`
LockedRound int `json:"locked_round"`
LockedBlock *types.Block `json:"locked_block"`
LockedBlockParts *types.PartSet `json:"locked_block_parts"`
ValidRound int `json:"valid_round"` // Last known round with POL for non-nil valid block.
ValidBlock *types.Block `json:"valid_block"` // Last known block of POL mentioned above.
ValidBlockParts *types.PartSet `json:"valid_block_parts"` // Last known block parts of POL metnioned above.
Votes *HeightVoteSet `json:"votes"`
CommitRound int `json:"commit_round"` //
LastCommit *types.VoteSet `json:"last_commit"` // Last precommits at Height-1
LastValidators *types.ValidatorSet `json:"last_validators"`
Height int64 `json:"height"` // Height we are working on
Round int `json:"round"`
Step RoundStepType `json:"step"`
StartTime time.Time `json:"start_time"`
CommitTime time.Time `json:"commit_time"` // Subjective time when +2/3 precommits for Block at Round were found
Validators *types.ValidatorSet `json:"validators"`
Proposal *types.Proposal `json:"proposal"`
ProposalBlock *types.Block `json:"proposal_block"`
ProposalBlockParts *types.PartSet `json:"proposal_block_parts"`
LockedRound int `json:"locked_round"`
LockedBlock *types.Block `json:"locked_block"`
LockedBlockParts *types.PartSet `json:"locked_block_parts"`
ValidRound int `json:"valid_round"` // Last known round with POL for non-nil valid block.
ValidBlock *types.Block `json:"valid_block"` // Last known block of POL mentioned above.
ValidBlockParts *types.PartSet `json:"valid_block_parts"` // Last known block parts of POL metnioned above.
Votes *HeightVoteSet `json:"votes"`
CommitRound int `json:"commit_round"` //
LastCommit *types.VoteSet `json:"last_commit"` // Last precommits at Height-1
LastValidators *types.ValidatorSet `json:"last_validators"`
TriggeredTimeoutPrecommit bool `json:"triggered_timeout_precommit"`
}
// Compressed version of the RoundState for use in RPC
@@ -147,14 +148,10 @@ func (rs *RoundState) CompleteProposalEvent() types.EventDataCompleteProposal {
// RoundStateEvent returns the H/R/S of the RoundState as an event.
func (rs *RoundState) RoundStateEvent() types.EventDataRoundState {
// copy the RoundState.
// TODO: if we want to avoid this, we may need synchronous events after all
rsCopy := *rs
return types.EventDataRoundState{
Height: rs.Height,
Round: rs.Round,
Step: rs.Step.String(),
RoundState: &rsCopy,
Height: rs.Height,
Round: rs.Round,
Step: rs.Step.String(),
}
}

View File

@@ -3,7 +3,7 @@ package types
import (
"testing"
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/tendermint/crypto/ed25519"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/types"
@@ -16,7 +16,7 @@ func BenchmarkRoundStateDeepCopy(b *testing.B) {
// Random validators
nval, ntxs := 100, 100
vset, _ := types.RandValidatorSet(nval, 1)
precommits := make([]*types.Vote, nval)
precommits := make([]*types.CommitSig, nval)
blockID := types.BlockID{
Hash: cmn.RandBytes(20),
PartsHeader: types.PartSetHeader{
@@ -25,12 +25,12 @@ func BenchmarkRoundStateDeepCopy(b *testing.B) {
}
sig := make([]byte, ed25519.SignatureSize)
for i := 0; i < nval; i++ {
precommits[i] = &types.Vote{
precommits[i] = (&types.Vote{
ValidatorAddress: types.Address(cmn.RandBytes(20)),
Timestamp: tmtime.Now(),
BlockID: blockID,
Signature: sig,
}
}).CommitSig()
}
txs := make([]types.Tx, ntxs)
for i := 0; i < ntxs; i++ {
@@ -53,11 +53,8 @@ func BenchmarkRoundStateDeepCopy(b *testing.B) {
Data: types.Data{
Txs: txs,
},
Evidence: types.EvidenceData{},
LastCommit: &types.Commit{
BlockID: blockID,
Precommits: precommits,
},
Evidence: types.EvidenceData{},
LastCommit: types.NewCommit(blockID, precommits),
}
parts := block.MakePartSet(4096)
// Random Proposal

View File

@@ -21,6 +21,9 @@ import (
const (
// must be greater than types.BlockPartSizeBytes + a few bytes
maxMsgSizeBytes = 1024 * 1024 // 1MB
// how often the WAL should be sync'd during period sync'ing
walDefaultFlushInterval = 2 * time.Second
)
//--------------------------------------------------------
@@ -54,26 +57,36 @@ func RegisterWALMessages(cdc *amino.Codec) {
type WAL interface {
Write(WALMessage)
WriteSync(WALMessage)
Group() *auto.Group
SearchForEndHeight(height int64, options *WALSearchOptions) (gr *auto.GroupReader, found bool, err error)
FlushAndSync() error
SearchForEndHeight(height int64, options *WALSearchOptions) (rd io.ReadCloser, found bool, err error)
// service methods
Start() error
Stop() error
Wait()
}
// Write ahead logger writes msgs to disk before they are processed.
// Can be used for crash-recovery and deterministic replay
// TODO: currently the wal is overwritten during replay catchup
// give it a mode so it's either reading or appending - must read to end to start appending again
// Can be used for crash-recovery and deterministic replay.
// TODO: currently the wal is overwritten during replay catchup, give it a mode
// so it's either reading or appending - must read to end to start appending
// again.
type baseWAL struct {
cmn.BaseService
group *auto.Group
enc *WALEncoder
flushTicker *time.Ticker
flushInterval time.Duration
}
var _ WAL = &baseWAL{}
// NewWAL returns a new write-ahead logger based on `baseWAL`, which implements
// WAL. It's flushed and synced to disk every 2s and once when stopped.
func NewWAL(walFile string, groupOptions ...func(*auto.Group)) (*baseWAL, error) {
err := cmn.EnsureDir(filepath.Dir(walFile), 0700)
if err != nil {
@@ -85,13 +98,19 @@ func NewWAL(walFile string, groupOptions ...func(*auto.Group)) (*baseWAL, error)
return nil, err
}
wal := &baseWAL{
group: group,
enc: NewWALEncoder(group),
group: group,
enc: NewWALEncoder(group),
flushInterval: walDefaultFlushInterval,
}
wal.BaseService = *cmn.NewBaseService(nil, "baseWAL", wal)
return wal, nil
}
// SetFlushInterval allows us to override the periodic flush interval for the WAL.
func (wal *baseWAL) SetFlushInterval(i time.Duration) {
wal.flushInterval = i
}
func (wal *baseWAL) Group() *auto.Group {
return wal.group
}
@@ -109,14 +128,49 @@ func (wal *baseWAL) OnStart() error {
wal.WriteSync(EndHeightMessage{0})
}
err = wal.group.Start()
return err
if err != nil {
return err
}
wal.flushTicker = time.NewTicker(wal.flushInterval)
go wal.processFlushTicks()
return nil
}
func (wal *baseWAL) processFlushTicks() {
for {
select {
case <-wal.flushTicker.C:
if err := wal.FlushAndSync(); err != nil {
wal.Logger.Error("Periodic WAL flush failed", "err", err)
}
case <-wal.Quit():
return
}
}
}
// FlushAndSync flushes and fsync's the underlying group's data to disk.
// See auto#FlushAndSync
func (wal *baseWAL) FlushAndSync() error {
return wal.group.FlushAndSync()
}
// Stop the underlying autofile group.
// Use Wait() to ensure it's finished shutting down
// before cleaning up files.
func (wal *baseWAL) OnStop() {
wal.flushTicker.Stop()
wal.FlushAndSync()
wal.group.Stop()
wal.group.Close()
}
// Wait for the underlying autofile group to finish shutting down
// so it's safe to cleanup files.
func (wal *baseWAL) Wait() {
wal.group.Wait()
}
// Write is called in newStep and for each receive on the
// peerMsgQueue and the timeoutTicker.
// NOTE: does not call fsync()
@@ -140,7 +194,7 @@ func (wal *baseWAL) WriteSync(msg WALMessage) {
}
wal.Write(msg)
if err := wal.group.Flush(); err != nil {
if err := wal.FlushAndSync(); err != nil {
panic(fmt.Sprintf("Error flushing consensus wal buf to file. Error: %v \n", err))
}
}
@@ -156,14 +210,17 @@ type WALSearchOptions struct {
// Group reader will be nil if found equals false.
//
// CONTRACT: caller must close group reader.
func (wal *baseWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (gr *auto.GroupReader, found bool, err error) {
var msg *TimedWALMessage
func (wal *baseWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (rd io.ReadCloser, found bool, err error) {
var (
msg *TimedWALMessage
gr *auto.GroupReader
)
lastHeightFound := int64(-1)
// NOTE: starting from the last file in the group because we're usually
// searching for the last height. See replay.go
min, max := wal.group.MinIndex(), wal.group.MaxIndex()
wal.Logger.Debug("Searching for height", "height", height, "min", min, "max", max)
wal.Logger.Info("Searching for height", "height", height, "min", min, "max", max)
for index := max; index >= min; index-- {
gr, err = wal.group.NewReader(index)
if err != nil {
@@ -183,7 +240,7 @@ func (wal *baseWAL) SearchForEndHeight(height int64, options *WALSearchOptions)
break
}
if options.IgnoreDataCorruptionErrors && IsDataCorruptionError(err) {
wal.Logger.Debug("Corrupted entry. Skipping...", "err", err)
wal.Logger.Error("Corrupted entry. Skipping...", "err", err)
// do nothing
continue
} else if err != nil {
@@ -194,7 +251,7 @@ func (wal *baseWAL) SearchForEndHeight(height int64, options *WALSearchOptions)
if m, ok := msg.Msg.(EndHeightMessage); ok {
lastHeightFound = m.Height
if m.Height == height { // found
wal.Logger.Debug("Found", "height", height, "index", index)
wal.Logger.Info("Found", "height", height, "index", index)
return gr, true, nil
}
}
@@ -219,12 +276,17 @@ func NewWALEncoder(wr io.Writer) *WALEncoder {
return &WALEncoder{wr}
}
// Encode writes the custom encoding of v to the stream.
// Encode writes the custom encoding of v to the stream. It returns an error if
// the amino-encoded size of v is greater than 1MB. Any error encountered
// during the write is also returned.
func (enc *WALEncoder) Encode(v *TimedWALMessage) error {
data := cdc.MustMarshalBinaryBare(v)
crc := crc32.Checksum(data, crc32c)
length := uint32(len(data))
if length > maxMsgSizeBytes {
return fmt.Errorf("Msg is too big: %d bytes, max: %d bytes", length, maxMsgSizeBytes)
}
totalLength := 8 + int(length)
msg := make([]byte, totalLength)
@@ -281,31 +343,31 @@ func (dec *WALDecoder) Decode() (*TimedWALMessage, error) {
return nil, err
}
if err != nil {
return nil, fmt.Errorf("failed to read checksum: %v", err)
return nil, DataCorruptionError{fmt.Errorf("failed to read checksum: %v", err)}
}
crc := binary.BigEndian.Uint32(b)
b = make([]byte, 4)
_, err = dec.rd.Read(b)
if err != nil {
return nil, fmt.Errorf("failed to read length: %v", err)
return nil, DataCorruptionError{fmt.Errorf("failed to read length: %v", err)}
}
length := binary.BigEndian.Uint32(b)
if length > maxMsgSizeBytes {
return nil, fmt.Errorf("length %d exceeded maximum possible value of %d bytes", length, maxMsgSizeBytes)
return nil, DataCorruptionError{fmt.Errorf("length %d exceeded maximum possible value of %d bytes", length, maxMsgSizeBytes)}
}
data := make([]byte, length)
_, err = dec.rd.Read(data)
n, err := dec.rd.Read(data)
if err != nil {
return nil, fmt.Errorf("failed to read data: %v", err)
return nil, DataCorruptionError{fmt.Errorf("failed to read data: %v (read: %d, wanted: %d)", err, n, length)}
}
// check checksum before decoding data
actualCRC := crc32.Checksum(data, crc32c)
if actualCRC != crc {
return nil, DataCorruptionError{fmt.Errorf("checksums do not match: (read: %v, actual: %v)", crc, actualCRC)}
return nil, DataCorruptionError{fmt.Errorf("checksums do not match: read: %v, actual: %v", crc, actualCRC)}
}
var res = new(TimedWALMessage) // nolint: gosimple
@@ -319,10 +381,12 @@ func (dec *WALDecoder) Decode() (*TimedWALMessage, error) {
type nilWAL struct{}
var _ WAL = nilWAL{}
func (nilWAL) Write(m WALMessage) {}
func (nilWAL) WriteSync(m WALMessage) {}
func (nilWAL) Group() *auto.Group { return nil }
func (nilWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (gr *auto.GroupReader, found bool, err error) {
func (nilWAL) FlushAndSync() error { return nil }
func (nilWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (rd io.ReadCloser, found bool, err error) {
return nil, false, nil
}
func (nilWAL) Start() error { return nil }

View File

@@ -5,16 +5,14 @@ import (
"bytes"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/pkg/errors"
"github.com/tendermint/tendermint/abci/example/kvstore"
bc "github.com/tendermint/tendermint/blockchain"
cfg "github.com/tendermint/tendermint/config"
auto "github.com/tendermint/tendermint/libs/autofile"
cmn "github.com/tendermint/tendermint/libs/common"
"github.com/tendermint/tendermint/libs/db"
"github.com/tendermint/tendermint/libs/log"
@@ -24,12 +22,12 @@ import (
"github.com/tendermint/tendermint/types"
)
// WALGenerateNBlocks generates a consensus WAL. It does this by spining up a
// WALGenerateNBlocks generates a consensus WAL. It does this by spinning up a
// stripped down version of node (proxy app, event bus, consensus state) with a
// persistent kvstore application and special consensus wal instance
// (byteBufferWAL) and waits until numBlocks are created. If the node fails to produce given numBlocks, it returns an error.
func WALGenerateNBlocks(wr io.Writer, numBlocks int) (err error) {
config := getConfig()
func WALGenerateNBlocks(t *testing.T, wr io.Writer, numBlocks int) (err error) {
config := getConfig(t)
app := kvstore.NewPersistentKVStoreApplication(filepath.Join(config.DBDir(), "wal_generator"))
@@ -102,11 +100,11 @@ func WALGenerateNBlocks(wr io.Writer, numBlocks int) (err error) {
}
//WALWithNBlocks returns a WAL content with numBlocks.
func WALWithNBlocks(numBlocks int) (data []byte, err error) {
func WALWithNBlocks(t *testing.T, numBlocks int) (data []byte, err error) {
var b bytes.Buffer
wr := bufio.NewWriter(&b)
if err := WALGenerateNBlocks(wr, numBlocks); err != nil {
if err := WALGenerateNBlocks(t, wr, numBlocks); err != nil {
return []byte{}, err
}
@@ -114,18 +112,6 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
return b.Bytes(), nil
}
// f**ing long, but unique for each test
func makePathname() string {
// get path
p, err := os.Getwd()
if err != nil {
panic(err)
}
// fmt.Println(p)
sep := string(filepath.Separator)
return strings.Replace(p, sep, "_", -1)
}
func randPort() int {
// returns between base and base + spread
base, spread := 20000, 20000
@@ -140,9 +126,8 @@ func makeAddrs() (string, string, string) {
}
// getConfig returns a config for test cases
func getConfig() *cfg.Config {
pathname := makePathname()
c := cfg.ResetTestRoot(fmt.Sprintf("%s_%d", pathname, cmn.RandInt()))
func getConfig(t *testing.T) *cfg.Config {
c := cfg.ResetTestRoot(t.Name())
// and we use random ports to run in parallel
tm, rpc, grpc := makeAddrs()
@@ -206,10 +191,9 @@ func (w *byteBufferWAL) WriteSync(m WALMessage) {
w.Write(m)
}
func (w *byteBufferWAL) Group() *auto.Group {
panic("not implemented")
}
func (w *byteBufferWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (gr *auto.GroupReader, found bool, err error) {
func (w *byteBufferWAL) FlushAndSync() error { return nil }
func (w *byteBufferWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (rd io.ReadCloser, found bool, err error) {
return nil, false, nil
}

View File

@@ -3,10 +3,10 @@ package consensus
import (
"bytes"
"crypto/rand"
"fmt"
"io/ioutil"
"os"
"path/filepath"
// "sync"
"testing"
"time"
@@ -21,6 +21,10 @@ import (
"github.com/stretchr/testify/require"
)
const (
walTestFlushInterval = time.Duration(100) * time.Millisecond
)
func TestWALTruncate(t *testing.T) {
walDir, err := ioutil.TempDir("", "wal")
require.NoError(t, err)
@@ -28,8 +32,10 @@ func TestWALTruncate(t *testing.T) {
walFile := filepath.Join(walDir, "wal")
//this magic number 4K can truncate the content when RotateFile. defaultHeadSizeLimit(10M) is hard to simulate.
//this magic number 1 * time.Millisecond make RotateFile check frequently. defaultGroupCheckDuration(5s) is hard to simulate.
// this magic number 4K can truncate the content when RotateFile.
// defaultHeadSizeLimit(10M) is hard to simulate.
// this magic number 1 * time.Millisecond make RotateFile check frequently.
// defaultGroupCheckDuration(5s) is hard to simulate.
wal, err := NewWAL(walFile,
autofile.GroupHeadSizeLimit(4096),
autofile.GroupCheckDuration(1*time.Millisecond),
@@ -38,22 +44,28 @@ func TestWALTruncate(t *testing.T) {
wal.SetLogger(log.TestingLogger())
err = wal.Start()
require.NoError(t, err)
defer wal.Stop()
defer func() {
wal.Stop()
// wait for the wal to finish shutting down so we
// can safely remove the directory
wal.Wait()
}()
//60 block's size nearly 70K, greater than group's headBuf size(4096 * 10), when headBuf is full, truncate content will Flush to the file.
//at this time, RotateFile is called, truncate content exist in each file.
err = WALGenerateNBlocks(wal.Group(), 60)
// 60 block's size nearly 70K, greater than group's headBuf size(4096 * 10),
// when headBuf is full, truncate content will Flush to the file. at this
// time, RotateFile is called, truncate content exist in each file.
err = WALGenerateNBlocks(t, wal.Group(), 60)
require.NoError(t, err)
time.Sleep(1 * time.Millisecond) //wait groupCheckDuration, make sure RotateFile run
wal.Group().Flush()
wal.FlushAndSync()
h := int64(50)
gr, found, err := wal.SearchForEndHeight(h, &WALSearchOptions{})
assert.NoError(t, err, fmt.Sprintf("expected not to err on height %d", h))
assert.True(t, found, fmt.Sprintf("expected to find end height for %d", h))
assert.NotNil(t, gr, "expected group not to be nil")
assert.NoError(t, err, "expected not to err on height %d", h)
assert.True(t, found, "expected to find end height for %d", h)
assert.NotNil(t, gr)
defer gr.Close()
dec := NewWALDecoder(gr)
@@ -61,14 +73,14 @@ func TestWALTruncate(t *testing.T) {
assert.NoError(t, err, "expected to decode a message")
rs, ok := msg.Msg.(tmtypes.EventDataRoundState)
assert.True(t, ok, "expected message of type EventDataRoundState")
assert.Equal(t, rs.Height, h+1, fmt.Sprintf("wrong height"))
assert.Equal(t, rs.Height, h+1, "wrong height")
}
func TestWALEncoderDecoder(t *testing.T) {
now := tmtime.Now()
msgs := []TimedWALMessage{
TimedWALMessage{Time: now, Msg: EndHeightMessage{0}},
TimedWALMessage{Time: now, Msg: timeoutInfo{Duration: time.Second, Height: 1, Round: 1, Step: types.RoundStepPropose}},
{Time: now, Msg: EndHeightMessage{0}},
{Time: now, Msg: timeoutInfo{Duration: time.Second, Height: 1, Round: 1, Step: types.RoundStepPropose}},
}
b := new(bytes.Buffer)
@@ -89,8 +101,28 @@ func TestWALEncoderDecoder(t *testing.T) {
}
}
func TestWALWritePanicsIfMsgIsTooBig(t *testing.T) {
walDir, err := ioutil.TempDir("", "wal")
require.NoError(t, err)
defer os.RemoveAll(walDir)
walFile := filepath.Join(walDir, "wal")
wal, err := NewWAL(walFile)
require.NoError(t, err)
err = wal.Start()
require.NoError(t, err)
defer func() {
wal.Stop()
// wait for the wal to finish shutting down so we
// can safely remove the directory
wal.Wait()
}()
assert.Panics(t, func() { wal.Write(make([]byte, maxMsgSizeBytes+1)) })
}
func TestWALSearchForEndHeight(t *testing.T) {
walBody, err := WALWithNBlocks(6)
walBody, err := WALWithNBlocks(t, 6)
if err != nil {
t.Fatal(err)
}
@@ -102,9 +134,9 @@ func TestWALSearchForEndHeight(t *testing.T) {
h := int64(3)
gr, found, err := wal.SearchForEndHeight(h, &WALSearchOptions{})
assert.NoError(t, err, fmt.Sprintf("expected not to err on height %d", h))
assert.True(t, found, fmt.Sprintf("expected to find end height for %d", h))
assert.NotNil(t, gr, "expected group not to be nil")
assert.NoError(t, err, "expected not to err on height %d", h)
assert.True(t, found, "expected to find end height for %d", h)
assert.NotNil(t, gr)
defer gr.Close()
dec := NewWALDecoder(gr)
@@ -112,7 +144,47 @@ func TestWALSearchForEndHeight(t *testing.T) {
assert.NoError(t, err, "expected to decode a message")
rs, ok := msg.Msg.(tmtypes.EventDataRoundState)
assert.True(t, ok, "expected message of type EventDataRoundState")
assert.Equal(t, rs.Height, h+1, fmt.Sprintf("wrong height"))
assert.Equal(t, rs.Height, h+1, "wrong height")
}
func TestWALPeriodicSync(t *testing.T) {
walDir, err := ioutil.TempDir("", "wal")
require.NoError(t, err)
defer os.RemoveAll(walDir)
walFile := filepath.Join(walDir, "wal")
wal, err := NewWAL(walFile, autofile.GroupCheckDuration(1*time.Millisecond))
require.NoError(t, err)
wal.SetFlushInterval(walTestFlushInterval)
wal.SetLogger(log.TestingLogger())
// Generate some data
err = WALGenerateNBlocks(t, wal.Group(), 5)
require.NoError(t, err)
// We should have data in the buffer now
assert.NotZero(t, wal.Group().Buffered())
require.NoError(t, wal.Start())
defer func() {
wal.Stop()
wal.Wait()
}()
time.Sleep(walTestFlushInterval + (10 * time.Millisecond))
// The data should have been flushed by the periodic sync
assert.Zero(t, wal.Group().Buffered())
h := int64(4)
gr, found, err := wal.SearchForEndHeight(h, &WALSearchOptions{})
assert.NoError(t, err, "expected not to err on height %d", h)
assert.True(t, found, "expected to find end height for %d", h)
assert.NotNil(t, gr)
if gr != nil {
gr.Close()
}
}
/*

View File

@@ -1,7 +1,7 @@
package consensus
import (
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
"github.com/tendermint/tendermint/types"
)

View File

@@ -25,9 +25,8 @@ func checkAminoBinary(t *testing.T, src, dst interface{}, size int) {
assert.Equal(t, byterSrc.Bytes(), bz, "Amino binary vs Bytes() mismatch")
}
// Make sure we have the expected length.
if size != -1 {
assert.Equal(t, size, len(bz), "Amino binary size mismatch")
}
assert.Equal(t, size, len(bz), "Amino binary size mismatch")
// Unmarshal.
err = cdc.UnmarshalBinaryBare(bz, dst)
require.Nil(t, err, "%+v", err)
@@ -48,6 +47,8 @@ func checkAminoJSON(t *testing.T, src interface{}, dst interface{}, isNil bool)
require.Nil(t, err, "%+v", err)
}
// ExamplePrintRegisteredTypes refers to unknown identifier: PrintRegisteredTypes
//nolint:govet
func ExamplePrintRegisteredTypes() {
cdc.PrintTypes(os.Stdout)
// Output: | Type | Name | Prefix | Length | Notes |

21
crypto/merkle/hash.go Normal file
View File

@@ -0,0 +1,21 @@
package merkle
import (
"github.com/tendermint/tendermint/crypto/tmhash"
)
// TODO: make these have a large predefined capacity
var (
leafPrefix = []byte{0}
innerPrefix = []byte{1}
)
// returns tmhash(0x00 || leaf)
func leafHash(leaf []byte) []byte {
return tmhash.Sum(append(leafPrefix, leaf...))
}
// returns tmhash(0x01 || left || right)
func innerHash(left []byte, right []byte) []byte {
return tmhash.Sum(append(innerPrefix, append(left, right...)...))
}

View File

@@ -39,7 +39,7 @@ func (m *ProofOp) Reset() { *m = ProofOp{} }
func (m *ProofOp) String() string { return proto.CompactTextString(m) }
func (*ProofOp) ProtoMessage() {}
func (*ProofOp) Descriptor() ([]byte, []int) {
return fileDescriptor_merkle_5d3f6051907285da, []int{0}
return fileDescriptor_merkle_24be8bc4e405ac66, []int{0}
}
func (m *ProofOp) XXX_Unmarshal(b []byte) error {
return m.Unmarshal(b)
@@ -101,7 +101,7 @@ func (m *Proof) Reset() { *m = Proof{} }
func (m *Proof) String() string { return proto.CompactTextString(m) }
func (*Proof) ProtoMessage() {}
func (*Proof) Descriptor() ([]byte, []int) {
return fileDescriptor_merkle_5d3f6051907285da, []int{1}
return fileDescriptor_merkle_24be8bc4e405ac66, []int{1}
}
func (m *Proof) XXX_Unmarshal(b []byte) error {
return m.Unmarshal(b)
@@ -395,6 +395,9 @@ func encodeVarintPopulateMerkle(dAtA []byte, v uint64) []byte {
return dAtA
}
func (m *ProofOp) Size() (n int) {
if m == nil {
return 0
}
var l int
_ = l
l = len(m.Type)
@@ -416,6 +419,9 @@ func (m *ProofOp) Size() (n int) {
}
func (m *Proof) Size() (n int) {
if m == nil {
return 0
}
var l int
_ = l
if len(m.Ops) > 0 {
@@ -772,9 +778,9 @@ var (
ErrIntOverflowMerkle = fmt.Errorf("proto: integer overflow")
)
func init() { proto.RegisterFile("crypto/merkle/merkle.proto", fileDescriptor_merkle_5d3f6051907285da) }
func init() { proto.RegisterFile("crypto/merkle/merkle.proto", fileDescriptor_merkle_24be8bc4e405ac66) }
var fileDescriptor_merkle_5d3f6051907285da = []byte{
var fileDescriptor_merkle_24be8bc4e405ac66 = []byte{
// 200 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x4a, 0x2e, 0xaa, 0x2c,
0x28, 0xc9, 0xd7, 0xcf, 0x4d, 0x2d, 0xca, 0xce, 0x49, 0x85, 0x52, 0x7a, 0x05, 0x45, 0xf9, 0x25,

View File

@@ -98,7 +98,7 @@ func (prt *ProofRuntime) Decode(pop ProofOp) (ProofOperator, error) {
}
func (prt *ProofRuntime) DecodeProof(proof *Proof) (ProofOperators, error) {
var poz ProofOperators
poz := make(ProofOperators, 0, len(proof.Ops))
for _, pop := range proof.Ops {
operator, err := prt.Decode(pop)
if err != nil {

View File

@@ -1,6 +1,8 @@
package merkle
import (
// it is ok to use math/rand here: we do not need a cryptographically secure random
// number generator here and we can run the tests a bit faster
"math/rand"
"testing"
@@ -24,7 +26,7 @@ func TestKeyPath(t *testing.T) {
keys[i][j] = alphanum[rand.Intn(len(alphanum))]
}
case KeyEncodingHex:
rand.Read(keys[i])
rand.Read(keys[i]) //nolint: gosec
default:
panic("Unexpected encoding")
}

View File

@@ -71,11 +71,11 @@ func (op SimpleValueOp) Run(args [][]byte) ([][]byte, error) {
hasher.Write(value) // does not error
vhash := hasher.Sum(nil)
bz := new(bytes.Buffer)
// Wrap <op.Key, vhash> to hash the KVPair.
hasher = tmhash.New()
encodeByteSlice(hasher, []byte(op.key)) // does not error
encodeByteSlice(hasher, []byte(vhash)) // does not error
kvhash := hasher.Sum(nil)
encodeByteSlice(bz, []byte(op.key)) // does not error
encodeByteSlice(bz, []byte(vhash)) // does not error
kvhash := leafHash(bz.Bytes())
if !bytes.Equal(kvhash, op.Proof.LeafHash) {
return nil, cmn.NewError("leaf hash mismatch: want %X got %X", op.Proof.LeafHash, kvhash)

View File

@@ -4,7 +4,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
cmn "github.com/tendermint/tendermint/libs/common"
)
@@ -26,6 +26,7 @@ func NewDominoOp(key, input, output string) DominoOp {
}
}
//nolint:unused
func DominoOpDecoder(pop ProofOp) (ProofOperator, error) {
if pop.Type != ProofOpDomino {
panic("unexpected proof op type")

View File

@@ -0,0 +1,97 @@
package merkle
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// These tests were taken from https://github.com/google/trillian/blob/master/merkle/rfc6962/rfc6962_test.go,
// and consequently fall under the above license.
import (
"bytes"
"encoding/hex"
"testing"
"github.com/tendermint/tendermint/crypto/tmhash"
)
func TestRFC6962Hasher(t *testing.T) {
_, leafHashTrail := trailsFromByteSlices([][]byte{[]byte("L123456")})
leafHash := leafHashTrail.Hash
_, leafHashTrail = trailsFromByteSlices([][]byte{{}})
emptyLeafHash := leafHashTrail.Hash
for _, tc := range []struct {
desc string
got []byte
want string
}{
// Since creating a merkle tree of no leaves is unsupported here, we skip
// the corresponding trillian test vector.
// Check that the empty hash is not the same as the hash of an empty leaf.
// echo -n 00 | xxd -r -p | sha256sum
{
desc: "RFC6962 Empty Leaf",
want: "6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d"[:tmhash.Size*2],
got: emptyLeafHash,
},
// echo -n 004C313233343536 | xxd -r -p | sha256sum
{
desc: "RFC6962 Leaf",
want: "395aa064aa4c29f7010acfe3f25db9485bbd4b91897b6ad7ad547639252b4d56"[:tmhash.Size*2],
got: leafHash,
},
// echo -n 014E3132334E343536 | xxd -r -p | sha256sum
{
desc: "RFC6962 Node",
want: "aa217fe888e47007fa15edab33c2b492a722cb106c64667fc2b044444de66bbb"[:tmhash.Size*2],
got: innerHash([]byte("N123"), []byte("N456")),
},
} {
t.Run(tc.desc, func(t *testing.T) {
wantBytes, err := hex.DecodeString(tc.want)
if err != nil {
t.Fatalf("hex.DecodeString(%x): %v", tc.want, err)
}
if got, want := tc.got, wantBytes; !bytes.Equal(got, want) {
t.Errorf("got %x, want %x", got, want)
}
})
}
}
func TestRFC6962HasherCollisions(t *testing.T) {
// Check that different leaves have different hashes.
leaf1, leaf2 := []byte("Hello"), []byte("World")
_, leafHashTrail := trailsFromByteSlices([][]byte{leaf1})
hash1 := leafHashTrail.Hash
_, leafHashTrail = trailsFromByteSlices([][]byte{leaf2})
hash2 := leafHashTrail.Hash
if bytes.Equal(hash1, hash2) {
t.Errorf("Leaf hashes should differ, but both are %x", hash1)
}
// Compute an intermediate subtree hash.
_, subHash1Trail := trailsFromByteSlices([][]byte{hash1, hash2})
subHash1 := subHash1Trail.Hash
// Check that this is not the same as a leaf hash of their concatenation.
preimage := append(hash1, hash2...)
_, forgedHashTrail := trailsFromByteSlices([][]byte{preimage})
forgedHash := forgedHashTrail.Hash
if bytes.Equal(subHash1, forgedHash) {
t.Errorf("Hasher is not second-preimage resistant")
}
// Swap the order of nodes and check that the hash is different.
_, subHash2Trail := trailsFromByteSlices([][]byte{hash2, hash1})
subHash2 := subHash2Trail.Hash
if bytes.Equal(subHash1, subHash2) {
t.Errorf("Subtree hash does not depend on the order of leaves")
}
}

View File

@@ -13,14 +13,14 @@ func TestSimpleMap(t *testing.T) {
values []string // each string gets converted to []byte in test
want string
}{
{[]string{"key1"}, []string{"value1"}, "321d150de16dceb51c72981b432b115045383259b1a550adf8dc80f927508967"},
{[]string{"key1"}, []string{"value2"}, "2a9e4baf321eac99f6eecc3406603c14bc5e85bb7b80483cbfc75b3382d24a2f"},
{[]string{"key1"}, []string{"value1"}, "a44d3cc7daba1a4600b00a2434b30f8b970652169810d6dfa9fb1793a2189324"},
{[]string{"key1"}, []string{"value2"}, "0638e99b3445caec9d95c05e1a3fc1487b4ddec6a952ff337080360b0dcc078c"},
// swap order with 2 keys
{[]string{"key1", "key2"}, []string{"value1", "value2"}, "c4d8913ab543ba26aa970646d4c99a150fd641298e3367cf68ca45fb45a49881"},
{[]string{"key2", "key1"}, []string{"value2", "value1"}, "c4d8913ab543ba26aa970646d4c99a150fd641298e3367cf68ca45fb45a49881"},
{[]string{"key1", "key2"}, []string{"value1", "value2"}, "8fd19b19e7bb3f2b3ee0574027d8a5a4cec370464ea2db2fbfa5c7d35bb0cff3"},
{[]string{"key2", "key1"}, []string{"value2", "value1"}, "8fd19b19e7bb3f2b3ee0574027d8a5a4cec370464ea2db2fbfa5c7d35bb0cff3"},
// swap order with 3 keys
{[]string{"key1", "key2", "key3"}, []string{"value1", "value2", "value3"}, "b23cef00eda5af4548a213a43793f2752d8d9013b3f2b64bc0523a4791196268"},
{[]string{"key1", "key3", "key2"}, []string{"value1", "value3", "value2"}, "b23cef00eda5af4548a213a43793f2752d8d9013b3f2b64bc0523a4791196268"},
{[]string{"key1", "key2", "key3"}, []string{"value1", "value2", "value3"}, "1dd674ec6782a0d586a903c9c63326a41cbe56b3bba33ed6ff5b527af6efb3dc"},
{[]string{"key1", "key3", "key2"}, []string{"value1", "value3", "value2"}, "1dd674ec6782a0d586a903c9c63326a41cbe56b3bba33ed6ff5b527af6efb3dc"},
}
for i, tc := range tests {
db := newSimpleMap()

View File

@@ -5,7 +5,6 @@ import (
"errors"
"fmt"
"github.com/tendermint/tendermint/crypto/tmhash"
cmn "github.com/tendermint/tendermint/libs/common"
)
@@ -67,7 +66,8 @@ func SimpleProofsFromMap(m map[string][]byte) (rootHash []byte, proofs map[strin
// Verify that the SimpleProof proves the root hash.
// Check sp.Index/sp.Total manually if needed
func (sp *SimpleProof) Verify(rootHash []byte, leafHash []byte) error {
func (sp *SimpleProof) Verify(rootHash []byte, leaf []byte) error {
leafHash := leafHash(leaf)
if sp.Total < 0 {
return errors.New("Proof total must be positive")
}
@@ -128,19 +128,19 @@ func computeHashFromAunts(index int, total int, leafHash []byte, innerHashes [][
if len(innerHashes) == 0 {
return nil
}
numLeft := (total + 1) / 2
numLeft := getSplitPoint(total)
if index < numLeft {
leftHash := computeHashFromAunts(index, numLeft, leafHash, innerHashes[:len(innerHashes)-1])
if leftHash == nil {
return nil
}
return simpleHashFromTwoHashes(leftHash, innerHashes[len(innerHashes)-1])
return innerHash(leftHash, innerHashes[len(innerHashes)-1])
}
rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1])
if rightHash == nil {
return nil
}
return simpleHashFromTwoHashes(innerHashes[len(innerHashes)-1], rightHash)
return innerHash(innerHashes[len(innerHashes)-1], rightHash)
}
}
@@ -182,12 +182,13 @@ func trailsFromByteSlices(items [][]byte) (trails []*SimpleProofNode, root *Simp
case 0:
return nil, nil
case 1:
trail := &SimpleProofNode{tmhash.Sum(items[0]), nil, nil, nil}
trail := &SimpleProofNode{leafHash(items[0]), nil, nil, nil}
return []*SimpleProofNode{trail}, trail
default:
lefts, leftRoot := trailsFromByteSlices(items[:(len(items)+1)/2])
rights, rightRoot := trailsFromByteSlices(items[(len(items)+1)/2:])
rootHash := simpleHashFromTwoHashes(leftRoot.Hash, rightRoot.Hash)
k := getSplitPoint(len(items))
lefts, leftRoot := trailsFromByteSlices(items[:k])
rights, rightRoot := trailsFromByteSlices(items[k:])
rootHash := innerHash(leftRoot.Hash, rightRoot.Hash)
root := &SimpleProofNode{rootHash, nil, nil, nil}
leftRoot.Parent = root
leftRoot.Right = rightRoot

View File

@@ -1,23 +1,9 @@
package merkle
import (
"github.com/tendermint/tendermint/crypto/tmhash"
"math/bits"
)
// simpleHashFromTwoHashes is the basic operation of the Merkle tree: Hash(left | right).
func simpleHashFromTwoHashes(left, right []byte) []byte {
var hasher = tmhash.New()
err := encodeByteSlice(hasher, left)
if err != nil {
panic(err)
}
err = encodeByteSlice(hasher, right)
if err != nil {
panic(err)
}
return hasher.Sum(nil)
}
// SimpleHashFromByteSlices computes a Merkle tree where the leaves are the byte slice,
// in the provided order.
func SimpleHashFromByteSlices(items [][]byte) []byte {
@@ -25,11 +11,12 @@ func SimpleHashFromByteSlices(items [][]byte) []byte {
case 0:
return nil
case 1:
return tmhash.Sum(items[0])
return leafHash(items[0])
default:
left := SimpleHashFromByteSlices(items[:(len(items)+1)/2])
right := SimpleHashFromByteSlices(items[(len(items)+1)/2:])
return simpleHashFromTwoHashes(left, right)
k := getSplitPoint(len(items))
left := SimpleHashFromByteSlices(items[:k])
right := SimpleHashFromByteSlices(items[k:])
return innerHash(left, right)
}
}
@@ -44,3 +31,17 @@ func SimpleHashFromMap(m map[string][]byte) []byte {
}
return sm.Hash()
}
// getSplitPoint returns the largest power of 2 less than length
func getSplitPoint(length int) int {
if length < 1 {
panic("Trying to split a tree with size < 1")
}
uLength := uint(length)
bitlen := bits.Len(uLength)
k := 1 << uint(bitlen-1)
if k == length {
k >>= 1
}
return k
}

View File

@@ -34,7 +34,6 @@ func TestSimpleProof(t *testing.T) {
// For each item, check the trail.
for i, item := range items {
itemHash := tmhash.Sum(item)
proof := proofs[i]
// Check total/index
@@ -43,30 +42,53 @@ func TestSimpleProof(t *testing.T) {
require.Equal(t, proof.Total, total, "Unmatched totals: %d vs %d", proof.Total, total)
// Verify success
err := proof.Verify(rootHash, itemHash)
require.NoError(t, err, "Verificatior failed: %v.", err)
err := proof.Verify(rootHash, item)
require.NoError(t, err, "Verification failed: %v.", err)
// Trail too long should make it fail
origAunts := proof.Aunts
proof.Aunts = append(proof.Aunts, cmn.RandBytes(32))
err = proof.Verify(rootHash, itemHash)
err = proof.Verify(rootHash, item)
require.Error(t, err, "Expected verification to fail for wrong trail length")
proof.Aunts = origAunts
// Trail too short should make it fail
proof.Aunts = proof.Aunts[0 : len(proof.Aunts)-1]
err = proof.Verify(rootHash, itemHash)
err = proof.Verify(rootHash, item)
require.Error(t, err, "Expected verification to fail for wrong trail length")
proof.Aunts = origAunts
// Mutating the itemHash should make it fail.
err = proof.Verify(rootHash, MutateByteSlice(itemHash))
err = proof.Verify(rootHash, MutateByteSlice(item))
require.Error(t, err, "Expected verification to fail for mutated leaf hash")
// Mutating the rootHash should make it fail.
err = proof.Verify(MutateByteSlice(rootHash), itemHash)
err = proof.Verify(MutateByteSlice(rootHash), item)
require.Error(t, err, "Expected verification to fail for mutated root hash")
}
}
func Test_getSplitPoint(t *testing.T) {
tests := []struct {
length int
want int
}{
{1, 0},
{2, 1},
{3, 2},
{4, 2},
{5, 4},
{10, 8},
{20, 16},
{100, 64},
{255, 128},
{256, 128},
{257, 256},
}
for _, tt := range tests {
got := getSplitPoint(tt.length)
require.Equal(t, tt.want, got, "getSplitPoint(%d) = %v, want %v", tt.length, got, tt.want)
}
}

View File

@@ -1,7 +1,7 @@
package merkle
import (
"github.com/tendermint/go-amino"
amino "github.com/tendermint/go-amino"
)
var cdc *amino.Codec

View File

@@ -1,42 +1,11 @@
package crypto
import (
"crypto/cipher"
crand "crypto/rand"
"crypto/sha256"
"encoding/hex"
"io"
"sync"
"golang.org/x/crypto/chacha20poly1305"
)
// NOTE: This is ignored for now until we have time
// to properly review the MixEntropy function - https://github.com/tendermint/tendermint/issues/2099.
//
// The randomness here is derived from xoring a chacha20 keystream with
// output from crypto/rand's OS Entropy Reader. (Due to fears of the OS'
// entropy being backdoored)
//
// For forward secrecy of produced randomness, the internal chacha key is hashed
// and thereby rotated after each call.
var gRandInfo *randInfo
func init() {
gRandInfo = &randInfo{}
// TODO: uncomment after reviewing MixEntropy -
// https://github.com/tendermint/tendermint/issues/2099
// gRandInfo.MixEntropy(randBytes(32)) // Init
}
// WARNING: This function needs review - https://github.com/tendermint/tendermint/issues/2099.
// Mix additional bytes of randomness, e.g. from hardware, user-input, etc.
// It is OK to call it multiple times. It does not diminish security.
func MixEntropy(seedBytes []byte) {
gRandInfo.MixEntropy(seedBytes)
}
// This only uses the OS's randomness
func randBytes(numBytes int) []byte {
b := make([]byte, numBytes)
@@ -52,19 +21,6 @@ func CRandBytes(numBytes int) []byte {
return randBytes(numBytes)
}
/* TODO: uncomment after reviewing MixEntropy - https://github.com/tendermint/tendermint/issues/2099
// This uses the OS and the Seed(s).
func CRandBytes(numBytes int) []byte {
return randBytes(numBytes)
b := make([]byte, numBytes)
_, err := gRandInfo.Read(b)
if err != nil {
panic(err)
}
return b
}
*/
// CRandHex returns a hex encoded string that's floor(numDigits/2) * 2 long.
//
// Note: CRandHex(24) gives 96 bits of randomness that
@@ -77,63 +33,3 @@ func CRandHex(numDigits int) string {
func CReader() io.Reader {
return crand.Reader
}
/* TODO: uncomment after reviewing MixEntropy - https://github.com/tendermint/tendermint/issues/2099
// Returns a crand.Reader mixed with user-supplied entropy
func CReader() io.Reader {
return gRandInfo
}
*/
//--------------------------------------------------------------------------------
type randInfo struct {
mtx sync.Mutex
seedBytes [chacha20poly1305.KeySize]byte
chacha cipher.AEAD
reader io.Reader
}
// You can call this as many times as you'd like.
// XXX/TODO: review - https://github.com/tendermint/tendermint/issues/2099
func (ri *randInfo) MixEntropy(seedBytes []byte) {
ri.mtx.Lock()
defer ri.mtx.Unlock()
// Make new ri.seedBytes using passed seedBytes and current ri.seedBytes:
// ri.seedBytes = sha256( seedBytes || ri.seedBytes )
h := sha256.New()
h.Write(seedBytes)
h.Write(ri.seedBytes[:])
hashBytes := h.Sum(nil)
copy(ri.seedBytes[:], hashBytes)
chacha, err := chacha20poly1305.New(ri.seedBytes[:])
if err != nil {
panic("Initializing chacha20 failed")
}
ri.chacha = chacha
// Create new reader
ri.reader = &cipher.StreamReader{S: ri, R: crand.Reader}
}
func (ri *randInfo) XORKeyStream(dst, src []byte) {
// nonce being 0 is safe due to never re-using a key.
emptyNonce := make([]byte, 12)
tmpDst := ri.chacha.Seal([]byte{}, emptyNonce, src, []byte{0})
// this removes the poly1305 tag as well, since chacha is a stream cipher
// and we truncate at input length.
copy(dst, tmpDst[:len(src)])
// hash seedBytes for forward secrecy, and initialize new chacha instance
newSeed := sha256.Sum256(ri.seedBytes[:])
chacha, err := chacha20poly1305.New(newSeed[:])
if err != nil {
panic("Initializing chacha20 failed")
}
ri.chacha = chacha
}
func (ri *randInfo) Read(b []byte) (n int, err error) {
ri.mtx.Lock()
n, err = ri.reader.Read(b)
ri.mtx.Unlock()
return
}

View File

@@ -0,0 +1,24 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*~

View File

@@ -0,0 +1,31 @@
Copyright (c) 2010 The Go Authors. All rights reserved.
Copyright (c) 2011 ThePiachu. All rights reserved.
Copyright (c) 2015 Jeffrey Wilcke. All rights reserved.
Copyright (c) 2015 Felix Lange. All rights reserved.
Copyright (c) 2015 Gustav Simonsson. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of the copyright holder. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -0,0 +1,3 @@
This package is copied from https://github.com/ethereum/go-ethereum/tree/729bf365b5f17325be9107b63b233da54100eec6/crypto/secp256k1
Unlike the rest of go-ethereum it is MIT licensed so compatible with our Apache2.0 license. We opt to copy in here rather than depend on go-ethereum to avoid issues with vendoring of the GPL parts of that repository by downstream.

View File

@@ -0,0 +1,325 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Copyright 2011 ThePiachu. All rights reserved.
// Copyright 2015 Jeffrey Wilcke, Felix Lange, Gustav Simonsson. All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
// * The name of ThePiachu may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
package secp256k1
import (
"crypto/elliptic"
"math/big"
"unsafe"
)
/*
#include "libsecp256k1/include/secp256k1.h"
extern int secp256k1_ext_scalar_mul(const secp256k1_context* ctx, const unsigned char *point, const unsigned char *scalar);
*/
import "C"
const (
// number of bits in a big.Word
wordBits = 32 << (uint64(^big.Word(0)) >> 63)
// number of bytes in a big.Word
wordBytes = wordBits / 8
)
// readBits encodes the absolute value of bigint as big-endian bytes. Callers
// must ensure that buf has enough space. If buf is too short the result will
// be incomplete.
func readBits(bigint *big.Int, buf []byte) {
i := len(buf)
for _, d := range bigint.Bits() {
for j := 0; j < wordBytes && i > 0; j++ {
i--
buf[i] = byte(d)
d >>= 8
}
}
}
// This code is from https://github.com/ThePiachu/GoBit and implements
// several Koblitz elliptic curves over prime fields.
//
// The curve methods, internally, on Jacobian coordinates. For a given
// (x, y) position on the curve, the Jacobian coordinates are (x1, y1,
// z1) where x = x1/z1² and y = y1/z1³. The greatest speedups come
// when the whole calculation can be performed within the transform
// (as in ScalarMult and ScalarBaseMult). But even for Add and Double,
// it's faster to apply and reverse the transform than to operate in
// affine coordinates.
// A BitCurve represents a Koblitz Curve with a=0.
// See http://www.hyperelliptic.org/EFD/g1p/auto-shortw.html
type BitCurve struct {
P *big.Int // the order of the underlying field
N *big.Int // the order of the base point
B *big.Int // the constant of the BitCurve equation
Gx, Gy *big.Int // (x,y) of the base point
BitSize int // the size of the underlying field
}
func (BitCurve *BitCurve) Params() *elliptic.CurveParams {
return &elliptic.CurveParams{
P: BitCurve.P,
N: BitCurve.N,
B: BitCurve.B,
Gx: BitCurve.Gx,
Gy: BitCurve.Gy,
BitSize: BitCurve.BitSize,
}
}
// IsOnCurve returns true if the given (x,y) lies on the BitCurve.
func (BitCurve *BitCurve) IsOnCurve(x, y *big.Int) bool {
// y² = x³ + b
y2 := new(big.Int).Mul(y, y) //y²
y2.Mod(y2, BitCurve.P) //y²%P
x3 := new(big.Int).Mul(x, x) //x²
x3.Mul(x3, x) //x³
x3.Add(x3, BitCurve.B) //x³+B
x3.Mod(x3, BitCurve.P) //(x³+B)%P
return x3.Cmp(y2) == 0
}
//TODO: double check if the function is okay
// affineFromJacobian reverses the Jacobian transform. See the comment at the
// top of the file.
func (BitCurve *BitCurve) affineFromJacobian(x, y, z *big.Int) (xOut, yOut *big.Int) {
zinv := new(big.Int).ModInverse(z, BitCurve.P)
zinvsq := new(big.Int).Mul(zinv, zinv)
xOut = new(big.Int).Mul(x, zinvsq)
xOut.Mod(xOut, BitCurve.P)
zinvsq.Mul(zinvsq, zinv)
yOut = new(big.Int).Mul(y, zinvsq)
yOut.Mod(yOut, BitCurve.P)
return
}
// Add returns the sum of (x1,y1) and (x2,y2)
func (BitCurve *BitCurve) Add(x1, y1, x2, y2 *big.Int) (*big.Int, *big.Int) {
z := new(big.Int).SetInt64(1)
return BitCurve.affineFromJacobian(BitCurve.addJacobian(x1, y1, z, x2, y2, z))
}
// addJacobian takes two points in Jacobian coordinates, (x1, y1, z1) and
// (x2, y2, z2) and returns their sum, also in Jacobian form.
func (BitCurve *BitCurve) addJacobian(x1, y1, z1, x2, y2, z2 *big.Int) (*big.Int, *big.Int, *big.Int) {
// See http://hyperelliptic.org/EFD/g1p/auto-shortw-jacobian-0.html#addition-add-2007-bl
z1z1 := new(big.Int).Mul(z1, z1)
z1z1.Mod(z1z1, BitCurve.P)
z2z2 := new(big.Int).Mul(z2, z2)
z2z2.Mod(z2z2, BitCurve.P)
u1 := new(big.Int).Mul(x1, z2z2)
u1.Mod(u1, BitCurve.P)
u2 := new(big.Int).Mul(x2, z1z1)
u2.Mod(u2, BitCurve.P)
h := new(big.Int).Sub(u2, u1)
if h.Sign() == -1 {
h.Add(h, BitCurve.P)
}
i := new(big.Int).Lsh(h, 1)
i.Mul(i, i)
j := new(big.Int).Mul(h, i)
s1 := new(big.Int).Mul(y1, z2)
s1.Mul(s1, z2z2)
s1.Mod(s1, BitCurve.P)
s2 := new(big.Int).Mul(y2, z1)
s2.Mul(s2, z1z1)
s2.Mod(s2, BitCurve.P)
r := new(big.Int).Sub(s2, s1)
if r.Sign() == -1 {
r.Add(r, BitCurve.P)
}
r.Lsh(r, 1)
v := new(big.Int).Mul(u1, i)
x3 := new(big.Int).Set(r)
x3.Mul(x3, x3)
x3.Sub(x3, j)
x3.Sub(x3, v)
x3.Sub(x3, v)
x3.Mod(x3, BitCurve.P)
y3 := new(big.Int).Set(r)
v.Sub(v, x3)
y3.Mul(y3, v)
s1.Mul(s1, j)
s1.Lsh(s1, 1)
y3.Sub(y3, s1)
y3.Mod(y3, BitCurve.P)
z3 := new(big.Int).Add(z1, z2)
z3.Mul(z3, z3)
z3.Sub(z3, z1z1)
if z3.Sign() == -1 {
z3.Add(z3, BitCurve.P)
}
z3.Sub(z3, z2z2)
if z3.Sign() == -1 {
z3.Add(z3, BitCurve.P)
}
z3.Mul(z3, h)
z3.Mod(z3, BitCurve.P)
return x3, y3, z3
}
// Double returns 2*(x,y)
func (BitCurve *BitCurve) Double(x1, y1 *big.Int) (*big.Int, *big.Int) {
z1 := new(big.Int).SetInt64(1)
return BitCurve.affineFromJacobian(BitCurve.doubleJacobian(x1, y1, z1))
}
// doubleJacobian takes a point in Jacobian coordinates, (x, y, z), and
// returns its double, also in Jacobian form.
func (BitCurve *BitCurve) doubleJacobian(x, y, z *big.Int) (*big.Int, *big.Int, *big.Int) {
// See http://hyperelliptic.org/EFD/g1p/auto-shortw-jacobian-0.html#doubling-dbl-2009-l
a := new(big.Int).Mul(x, x) //X1²
b := new(big.Int).Mul(y, y) //Y1²
c := new(big.Int).Mul(b, b) //B²
d := new(big.Int).Add(x, b) //X1+B
d.Mul(d, d) //(X1+B)²
d.Sub(d, a) //(X1+B)²-A
d.Sub(d, c) //(X1+B)²-A-C
d.Mul(d, big.NewInt(2)) //2*((X1+B)²-A-C)
e := new(big.Int).Mul(big.NewInt(3), a) //3*A
f := new(big.Int).Mul(e, e) //E²
x3 := new(big.Int).Mul(big.NewInt(2), d) //2*D
x3.Sub(f, x3) //F-2*D
x3.Mod(x3, BitCurve.P)
y3 := new(big.Int).Sub(d, x3) //D-X3
y3.Mul(e, y3) //E*(D-X3)
y3.Sub(y3, new(big.Int).Mul(big.NewInt(8), c)) //E*(D-X3)-8*C
y3.Mod(y3, BitCurve.P)
z3 := new(big.Int).Mul(y, z) //Y1*Z1
z3.Mul(big.NewInt(2), z3) //3*Y1*Z1
z3.Mod(z3, BitCurve.P)
return x3, y3, z3
}
func (BitCurve *BitCurve) ScalarMult(Bx, By *big.Int, scalar []byte) (*big.Int, *big.Int) {
// Ensure scalar is exactly 32 bytes. We pad always, even if
// scalar is 32 bytes long, to avoid a timing side channel.
if len(scalar) > 32 {
panic("can't handle scalars > 256 bits")
}
// NOTE: potential timing issue
padded := make([]byte, 32)
copy(padded[32-len(scalar):], scalar)
scalar = padded
// Do the multiplication in C, updating point.
point := make([]byte, 64)
readBits(Bx, point[:32])
readBits(By, point[32:])
pointPtr := (*C.uchar)(unsafe.Pointer(&point[0]))
scalarPtr := (*C.uchar)(unsafe.Pointer(&scalar[0]))
res := C.secp256k1_ext_scalar_mul(context, pointPtr, scalarPtr)
// Unpack the result and clear temporaries.
x := new(big.Int).SetBytes(point[:32])
y := new(big.Int).SetBytes(point[32:])
for i := range point {
point[i] = 0
}
for i := range padded {
scalar[i] = 0
}
if res != 1 {
return nil, nil
}
return x, y
}
// ScalarBaseMult returns k*G, where G is the base point of the group and k is
// an integer in big-endian form.
func (BitCurve *BitCurve) ScalarBaseMult(k []byte) (*big.Int, *big.Int) {
return BitCurve.ScalarMult(BitCurve.Gx, BitCurve.Gy, k)
}
// Marshal converts a point into the form specified in section 4.3.6 of ANSI
// X9.62.
func (BitCurve *BitCurve) Marshal(x, y *big.Int) []byte {
byteLen := (BitCurve.BitSize + 7) >> 3
ret := make([]byte, 1+2*byteLen)
ret[0] = 4 // uncompressed point flag
readBits(x, ret[1:1+byteLen])
readBits(y, ret[1+byteLen:])
return ret
}
// Unmarshal converts a point, serialised by Marshal, into an x, y pair. On
// error, x = nil.
func (BitCurve *BitCurve) Unmarshal(data []byte) (x, y *big.Int) {
byteLen := (BitCurve.BitSize + 7) >> 3
if len(data) != 1+2*byteLen {
return
}
if data[0] != 4 { // uncompressed form
return
}
x = new(big.Int).SetBytes(data[1 : 1+byteLen])
y = new(big.Int).SetBytes(data[1+byteLen:])
return
}
var theCurve = new(BitCurve)
func init() {
// See SEC 2 section 2.7.1
// curve parameters taken from:
// http://www.secg.org/sec2-v2.pdf
theCurve.P, _ = new(big.Int).SetString("0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F", 0)
theCurve.N, _ = new(big.Int).SetString("0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141", 0)
theCurve.B, _ = new(big.Int).SetString("0x0000000000000000000000000000000000000000000000000000000000000007", 0)
theCurve.Gx, _ = new(big.Int).SetString("0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798", 0)
theCurve.Gy, _ = new(big.Int).SetString("0x483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8", 0)
theCurve.BitSize = 256
}
// S256 returns a BitCurve which implements secp256k1.
func S256() *BitCurve {
return theCurve
}

View File

@@ -0,0 +1,130 @@
// Copyright 2015 Jeffrey Wilcke, Felix Lange, Gustav Simonsson. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be found in
// the LICENSE file.
// secp256k1_context_create_sign_verify creates a context for signing and signature verification.
static secp256k1_context* secp256k1_context_create_sign_verify() {
return secp256k1_context_create(SECP256K1_CONTEXT_SIGN | SECP256K1_CONTEXT_VERIFY);
}
// secp256k1_ext_ecdsa_recover recovers the public key of an encoded compact signature.
//
// Returns: 1: recovery was successful
// 0: recovery was not successful
// Args: ctx: pointer to a context object (cannot be NULL)
// Out: pubkey_out: the serialized 65-byte public key of the signer (cannot be NULL)
// In: sigdata: pointer to a 65-byte signature with the recovery id at the end (cannot be NULL)
// msgdata: pointer to a 32-byte message (cannot be NULL)
static int secp256k1_ext_ecdsa_recover(
const secp256k1_context* ctx,
unsigned char *pubkey_out,
const unsigned char *sigdata,
const unsigned char *msgdata
) {
secp256k1_ecdsa_recoverable_signature sig;
secp256k1_pubkey pubkey;
if (!secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &sig, sigdata, (int)sigdata[64])) {
return 0;
}
if (!secp256k1_ecdsa_recover(ctx, &pubkey, &sig, msgdata)) {
return 0;
}
size_t outputlen = 65;
return secp256k1_ec_pubkey_serialize(ctx, pubkey_out, &outputlen, &pubkey, SECP256K1_EC_UNCOMPRESSED);
}
// secp256k1_ext_ecdsa_verify verifies an encoded compact signature.
//
// Returns: 1: signature is valid
// 0: signature is invalid
// Args: ctx: pointer to a context object (cannot be NULL)
// In: sigdata: pointer to a 64-byte signature (cannot be NULL)
// msgdata: pointer to a 32-byte message (cannot be NULL)
// pubkeydata: pointer to public key data (cannot be NULL)
// pubkeylen: length of pubkeydata
static int secp256k1_ext_ecdsa_verify(
const secp256k1_context* ctx,
const unsigned char *sigdata,
const unsigned char *msgdata,
const unsigned char *pubkeydata,
size_t pubkeylen
) {
secp256k1_ecdsa_signature sig;
secp256k1_pubkey pubkey;
if (!secp256k1_ecdsa_signature_parse_compact(ctx, &sig, sigdata)) {
return 0;
}
if (!secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeydata, pubkeylen)) {
return 0;
}
return secp256k1_ecdsa_verify(ctx, &sig, msgdata, &pubkey);
}
// secp256k1_ext_reencode_pubkey decodes then encodes a public key. It can be used to
// convert between public key formats. The input/output formats are chosen depending on the
// length of the input/output buffers.
//
// Returns: 1: conversion successful
// 0: conversion unsuccessful
// Args: ctx: pointer to a context object (cannot be NULL)
// Out: out: output buffer that will contain the reencoded key (cannot be NULL)
// In: outlen: length of out (33 for compressed keys, 65 for uncompressed keys)
// pubkeydata: the input public key (cannot be NULL)
// pubkeylen: length of pubkeydata
static int secp256k1_ext_reencode_pubkey(
const secp256k1_context* ctx,
unsigned char *out,
size_t outlen,
const unsigned char *pubkeydata,
size_t pubkeylen
) {
secp256k1_pubkey pubkey;
if (!secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeydata, pubkeylen)) {
return 0;
}
unsigned int flag = (outlen == 33) ? SECP256K1_EC_COMPRESSED : SECP256K1_EC_UNCOMPRESSED;
return secp256k1_ec_pubkey_serialize(ctx, out, &outlen, &pubkey, flag);
}
// secp256k1_ext_scalar_mul multiplies a point by a scalar in constant time.
//
// Returns: 1: multiplication was successful
// 0: scalar was invalid (zero or overflow)
// Args: ctx: pointer to a context object (cannot be NULL)
// Out: point: the multiplied point (usually secret)
// In: point: pointer to a 64-byte public point,
// encoded as two 256bit big-endian numbers.
// scalar: a 32-byte scalar with which to multiply the point
int secp256k1_ext_scalar_mul(const secp256k1_context* ctx, unsigned char *point, const unsigned char *scalar) {
int ret = 0;
int overflow = 0;
secp256k1_fe feX, feY;
secp256k1_gej res;
secp256k1_ge ge;
secp256k1_scalar s;
ARG_CHECK(point != NULL);
ARG_CHECK(scalar != NULL);
(void)ctx;
secp256k1_fe_set_b32(&feX, point);
secp256k1_fe_set_b32(&feY, point+32);
secp256k1_ge_set_xy(&ge, &feX, &feY);
secp256k1_scalar_set_b32(&s, scalar, &overflow);
if (overflow || secp256k1_scalar_is_zero(&s)) {
ret = 0;
} else {
secp256k1_ecmult_const(&res, &ge, &s);
secp256k1_ge_set_gej(&ge, &res);
/* Note: can't use secp256k1_pubkey_save here because it is not constant time. */
secp256k1_fe_normalize(&ge.x);
secp256k1_fe_normalize(&ge.y);
secp256k1_fe_get_b32(point, &ge.x);
secp256k1_fe_get_b32(point+32, &ge.y);
ret = 1;
}
secp256k1_scalar_clear(&s);
return ret;
}

View File

@@ -0,0 +1,49 @@
bench_inv
bench_ecdh
bench_sign
bench_verify
bench_schnorr_verify
bench_recover
bench_internal
tests
exhaustive_tests
gen_context
*.exe
*.so
*.a
!.gitignore
Makefile
configure
.libs/
Makefile.in
aclocal.m4
autom4te.cache/
config.log
config.status
*.tar.gz
*.la
libtool
.deps/
.dirstamp
*.lo
*.o
*~
src/libsecp256k1-config.h
src/libsecp256k1-config.h.in
src/ecmult_static_context.h
build-aux/config.guess
build-aux/config.sub
build-aux/depcomp
build-aux/install-sh
build-aux/ltmain.sh
build-aux/m4/libtool.m4
build-aux/m4/lt~obsolete.m4
build-aux/m4/ltoptions.m4
build-aux/m4/ltsugar.m4
build-aux/m4/ltversion.m4
build-aux/missing
build-aux/compile
build-aux/test-driver
src/stamp-h1
libsecp256k1.pc

View File

@@ -0,0 +1,69 @@
language: c
sudo: false
addons:
apt:
packages: libgmp-dev
compiler:
- clang
- gcc
cache:
directories:
- src/java/guava/
env:
global:
- FIELD=auto BIGNUM=auto SCALAR=auto ENDOMORPHISM=no STATICPRECOMPUTATION=yes ASM=no BUILD=check EXTRAFLAGS= HOST= ECDH=no RECOVERY=no EXPERIMENTAL=no
- GUAVA_URL=https://search.maven.org/remotecontent?filepath=com/google/guava/guava/18.0/guava-18.0.jar GUAVA_JAR=src/java/guava/guava-18.0.jar
matrix:
- SCALAR=32bit RECOVERY=yes
- SCALAR=32bit FIELD=32bit ECDH=yes EXPERIMENTAL=yes
- SCALAR=64bit
- FIELD=64bit RECOVERY=yes
- FIELD=64bit ENDOMORPHISM=yes
- FIELD=64bit ENDOMORPHISM=yes ECDH=yes EXPERIMENTAL=yes
- FIELD=64bit ASM=x86_64
- FIELD=64bit ENDOMORPHISM=yes ASM=x86_64
- FIELD=32bit ENDOMORPHISM=yes
- BIGNUM=no
- BIGNUM=no ENDOMORPHISM=yes RECOVERY=yes EXPERIMENTAL=yes
- BIGNUM=no STATICPRECOMPUTATION=no
- BUILD=distcheck
- EXTRAFLAGS=CPPFLAGS=-DDETERMINISTIC
- EXTRAFLAGS=CFLAGS=-O0
- BUILD=check-java ECDH=yes EXPERIMENTAL=yes
matrix:
fast_finish: true
include:
- compiler: clang
env: HOST=i686-linux-gnu ENDOMORPHISM=yes
addons:
apt:
packages:
- gcc-multilib
- libgmp-dev:i386
- compiler: clang
env: HOST=i686-linux-gnu
addons:
apt:
packages:
- gcc-multilib
- compiler: gcc
env: HOST=i686-linux-gnu ENDOMORPHISM=yes
addons:
apt:
packages:
- gcc-multilib
- compiler: gcc
env: HOST=i686-linux-gnu
addons:
apt:
packages:
- gcc-multilib
- libgmp-dev:i386
before_install: mkdir -p `dirname $GUAVA_JAR`
install: if [ ! -f $GUAVA_JAR ]; then wget $GUAVA_URL -O $GUAVA_JAR; fi
before_script: ./autogen.sh
script:
- if [ -n "$HOST" ]; then export USE_HOST="--host=$HOST"; fi
- if [ "x$HOST" = "xi686-linux-gnu" ]; then export CC="$CC -m32"; fi
- ./configure --enable-experimental=$EXPERIMENTAL --enable-endomorphism=$ENDOMORPHISM --with-field=$FIELD --with-bignum=$BIGNUM --with-scalar=$SCALAR --enable-ecmult-static-precomputation=$STATICPRECOMPUTATION --enable-module-ecdh=$ECDH --enable-module-recovery=$RECOVERY $EXTRAFLAGS $USE_HOST && make -j2 $BUILD
os: linux

View File

@@ -0,0 +1,19 @@
Copyright (c) 2013 Pieter Wuille
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View File

@@ -0,0 +1,177 @@
ACLOCAL_AMFLAGS = -I build-aux/m4
lib_LTLIBRARIES = libsecp256k1.la
if USE_JNI
JNI_LIB = libsecp256k1_jni.la
noinst_LTLIBRARIES = $(JNI_LIB)
else
JNI_LIB =
endif
include_HEADERS = include/secp256k1.h
noinst_HEADERS =
noinst_HEADERS += src/scalar.h
noinst_HEADERS += src/scalar_4x64.h
noinst_HEADERS += src/scalar_8x32.h
noinst_HEADERS += src/scalar_low.h
noinst_HEADERS += src/scalar_impl.h
noinst_HEADERS += src/scalar_4x64_impl.h
noinst_HEADERS += src/scalar_8x32_impl.h
noinst_HEADERS += src/scalar_low_impl.h
noinst_HEADERS += src/group.h
noinst_HEADERS += src/group_impl.h
noinst_HEADERS += src/num_gmp.h
noinst_HEADERS += src/num_gmp_impl.h
noinst_HEADERS += src/ecdsa.h
noinst_HEADERS += src/ecdsa_impl.h
noinst_HEADERS += src/eckey.h
noinst_HEADERS += src/eckey_impl.h
noinst_HEADERS += src/ecmult.h
noinst_HEADERS += src/ecmult_impl.h
noinst_HEADERS += src/ecmult_const.h
noinst_HEADERS += src/ecmult_const_impl.h
noinst_HEADERS += src/ecmult_gen.h
noinst_HEADERS += src/ecmult_gen_impl.h
noinst_HEADERS += src/num.h
noinst_HEADERS += src/num_impl.h
noinst_HEADERS += src/field_10x26.h
noinst_HEADERS += src/field_10x26_impl.h
noinst_HEADERS += src/field_5x52.h
noinst_HEADERS += src/field_5x52_impl.h
noinst_HEADERS += src/field_5x52_int128_impl.h
noinst_HEADERS += src/field_5x52_asm_impl.h
noinst_HEADERS += src/java/org_bitcoin_NativeSecp256k1.h
noinst_HEADERS += src/java/org_bitcoin_Secp256k1Context.h
noinst_HEADERS += src/util.h
noinst_HEADERS += src/testrand.h
noinst_HEADERS += src/testrand_impl.h
noinst_HEADERS += src/hash.h
noinst_HEADERS += src/hash_impl.h
noinst_HEADERS += src/field.h
noinst_HEADERS += src/field_impl.h
noinst_HEADERS += src/bench.h
noinst_HEADERS += contrib/lax_der_parsing.h
noinst_HEADERS += contrib/lax_der_parsing.c
noinst_HEADERS += contrib/lax_der_privatekey_parsing.h
noinst_HEADERS += contrib/lax_der_privatekey_parsing.c
if USE_EXTERNAL_ASM
COMMON_LIB = libsecp256k1_common.la
noinst_LTLIBRARIES = $(COMMON_LIB)
else
COMMON_LIB =
endif
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libsecp256k1.pc
if USE_EXTERNAL_ASM
if USE_ASM_ARM
libsecp256k1_common_la_SOURCES = src/asm/field_10x26_arm.s
endif
endif
libsecp256k1_la_SOURCES = src/secp256k1.c
libsecp256k1_la_CPPFLAGS = -DSECP256K1_BUILD -I$(top_srcdir)/include -I$(top_srcdir)/src $(SECP_INCLUDES)
libsecp256k1_la_LIBADD = $(JNI_LIB) $(SECP_LIBS) $(COMMON_LIB)
libsecp256k1_jni_la_SOURCES = src/java/org_bitcoin_NativeSecp256k1.c src/java/org_bitcoin_Secp256k1Context.c
libsecp256k1_jni_la_CPPFLAGS = -DSECP256K1_BUILD $(JNI_INCLUDES)
noinst_PROGRAMS =
if USE_BENCHMARK
noinst_PROGRAMS += bench_verify bench_sign bench_internal
bench_verify_SOURCES = src/bench_verify.c
bench_verify_LDADD = libsecp256k1.la $(SECP_LIBS) $(SECP_TEST_LIBS) $(COMMON_LIB)
bench_sign_SOURCES = src/bench_sign.c
bench_sign_LDADD = libsecp256k1.la $(SECP_LIBS) $(SECP_TEST_LIBS) $(COMMON_LIB)
bench_internal_SOURCES = src/bench_internal.c
bench_internal_LDADD = $(SECP_LIBS) $(COMMON_LIB)
bench_internal_CPPFLAGS = -DSECP256K1_BUILD $(SECP_INCLUDES)
endif
TESTS =
if USE_TESTS
noinst_PROGRAMS += tests
tests_SOURCES = src/tests.c
tests_CPPFLAGS = -DSECP256K1_BUILD -I$(top_srcdir)/src -I$(top_srcdir)/include $(SECP_INCLUDES) $(SECP_TEST_INCLUDES)
if !ENABLE_COVERAGE
tests_CPPFLAGS += -DVERIFY
endif
tests_LDADD = $(SECP_LIBS) $(SECP_TEST_LIBS) $(COMMON_LIB)
tests_LDFLAGS = -static
TESTS += tests
endif
if USE_EXHAUSTIVE_TESTS
noinst_PROGRAMS += exhaustive_tests
exhaustive_tests_SOURCES = src/tests_exhaustive.c
exhaustive_tests_CPPFLAGS = -DSECP256K1_BUILD -I$(top_srcdir)/src $(SECP_INCLUDES)
if !ENABLE_COVERAGE
exhaustive_tests_CPPFLAGS += -DVERIFY
endif
exhaustive_tests_LDADD = $(SECP_LIBS)
exhaustive_tests_LDFLAGS = -static
TESTS += exhaustive_tests
endif
JAVAROOT=src/java
JAVAORG=org/bitcoin
JAVA_GUAVA=$(srcdir)/$(JAVAROOT)/guava/guava-18.0.jar
CLASSPATH_ENV=CLASSPATH=$(JAVA_GUAVA)
JAVA_FILES= \
$(JAVAROOT)/$(JAVAORG)/NativeSecp256k1.java \
$(JAVAROOT)/$(JAVAORG)/NativeSecp256k1Test.java \
$(JAVAROOT)/$(JAVAORG)/NativeSecp256k1Util.java \
$(JAVAROOT)/$(JAVAORG)/Secp256k1Context.java
if USE_JNI
$(JAVA_GUAVA):
@echo Guava is missing. Fetch it via: \
wget https://search.maven.org/remotecontent?filepath=com/google/guava/guava/18.0/guava-18.0.jar -O $(@)
@false
.stamp-java: $(JAVA_FILES)
@echo Compiling $^
$(AM_V_at)$(CLASSPATH_ENV) javac $^
@touch $@
if USE_TESTS
check-java: libsecp256k1.la $(JAVA_GUAVA) .stamp-java
$(AM_V_at)java -Djava.library.path="./:./src:./src/.libs:.libs/" -cp "$(JAVA_GUAVA):$(JAVAROOT)" $(JAVAORG)/NativeSecp256k1Test
endif
endif
if USE_ECMULT_STATIC_PRECOMPUTATION
CPPFLAGS_FOR_BUILD +=-I$(top_srcdir)
CFLAGS_FOR_BUILD += -Wall -Wextra -Wno-unused-function
gen_context_OBJECTS = gen_context.o
gen_context_BIN = gen_context$(BUILD_EXEEXT)
gen_%.o: src/gen_%.c
$(CC_FOR_BUILD) $(CPPFLAGS_FOR_BUILD) $(CFLAGS_FOR_BUILD) -c $< -o $@
$(gen_context_BIN): $(gen_context_OBJECTS)
$(CC_FOR_BUILD) $^ -o $@
$(libsecp256k1_la_OBJECTS): src/ecmult_static_context.h
$(tests_OBJECTS): src/ecmult_static_context.h
$(bench_internal_OBJECTS): src/ecmult_static_context.h
src/ecmult_static_context.h: $(gen_context_BIN)
./$(gen_context_BIN)
CLEANFILES = $(gen_context_BIN) src/ecmult_static_context.h $(JAVAROOT)/$(JAVAORG)/*.class .stamp-java
endif
EXTRA_DIST = autogen.sh src/gen_context.c src/basic-config.h $(JAVA_FILES)
if ENABLE_MODULE_ECDH
include src/modules/ecdh/Makefile.am.include
endif
if ENABLE_MODULE_RECOVERY
include src/modules/recovery/Makefile.am.include
endif

View File

@@ -0,0 +1,61 @@
libsecp256k1
============
[![Build Status](https://travis-ci.org/bitcoin-core/secp256k1.svg?branch=master)](https://travis-ci.org/bitcoin-core/secp256k1)
Optimized C library for EC operations on curve secp256k1.
This library is a work in progress and is being used to research best practices. Use at your own risk.
Features:
* secp256k1 ECDSA signing/verification and key generation.
* Adding/multiplying private/public keys.
* Serialization/parsing of private keys, public keys, signatures.
* Constant time, constant memory access signing and pubkey generation.
* Derandomized DSA (via RFC6979 or with a caller provided function.)
* Very efficient implementation.
Implementation details
----------------------
* General
* No runtime heap allocation.
* Extensive testing infrastructure.
* Structured to facilitate review and analysis.
* Intended to be portable to any system with a C89 compiler and uint64_t support.
* Expose only higher level interfaces to minimize the API surface and improve application security. ("Be difficult to use insecurely.")
* Field operations
* Optimized implementation of arithmetic modulo the curve's field size (2^256 - 0x1000003D1).
* Using 5 52-bit limbs (including hand-optimized assembly for x86_64, by Diederik Huys).
* Using 10 26-bit limbs.
* Field inverses and square roots using a sliding window over blocks of 1s (by Peter Dettman).
* Scalar operations
* Optimized implementation without data-dependent branches of arithmetic modulo the curve's order.
* Using 4 64-bit limbs (relying on __int128 support in the compiler).
* Using 8 32-bit limbs.
* Group operations
* Point addition formula specifically simplified for the curve equation (y^2 = x^3 + 7).
* Use addition between points in Jacobian and affine coordinates where possible.
* Use a unified addition/doubling formula where necessary to avoid data-dependent branches.
* Point/x comparison without a field inversion by comparison in the Jacobian coordinate space.
* Point multiplication for verification (a*P + b*G).
* Use wNAF notation for point multiplicands.
* Use a much larger window for multiples of G, using precomputed multiples.
* Use Shamir's trick to do the multiplication with the public key and the generator simultaneously.
* Optionally (off by default) use secp256k1's efficiently-computable endomorphism to split the P multiplicand into 2 half-sized ones.
* Point multiplication for signing
* Use a precomputed table of multiples of powers of 16 multiplied with the generator, so general multiplication becomes a series of additions.
* Access the table with branch-free conditional moves so memory access is uniform.
* No data-dependent branches
* The precomputed tables add and eventually subtract points for which no known scalar (private key) is known, preventing even an attacker with control over the private key used to control the data internally.
Build steps
-----------
libsecp256k1 is built using autotools:
$ ./autogen.sh
$ ./configure
$ make
$ ./tests
$ sudo make install # optional

View File

@@ -0,0 +1,3 @@
* Unit tests for fieldelem/groupelem, including ones intended to
trigger fieldelem's boundary cases.
* Complete constant-time operations for signing/keygen

View File

@@ -0,0 +1,3 @@
#!/bin/sh
set -e
autoreconf -if --warnings=all

View File

@@ -0,0 +1,140 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_jni_include_dir.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_JNI_INCLUDE_DIR
#
# DESCRIPTION
#
# AX_JNI_INCLUDE_DIR finds include directories needed for compiling
# programs using the JNI interface.
#
# JNI include directories are usually in the Java distribution. This is
# deduced from the value of $JAVA_HOME, $JAVAC, or the path to "javac", in
# that order. When this macro completes, a list of directories is left in
# the variable JNI_INCLUDE_DIRS.
#
# Example usage follows:
#
# AX_JNI_INCLUDE_DIR
#
# for JNI_INCLUDE_DIR in $JNI_INCLUDE_DIRS
# do
# CPPFLAGS="$CPPFLAGS -I$JNI_INCLUDE_DIR"
# done
#
# If you want to force a specific compiler:
#
# - at the configure.in level, set JAVAC=yourcompiler before calling
# AX_JNI_INCLUDE_DIR
#
# - at the configure level, setenv JAVAC
#
# Note: This macro can work with the autoconf M4 macros for Java programs.
# This particular macro is not part of the original set of macros.
#
# LICENSE
#
# Copyright (c) 2008 Don Anderson <dda@sleepycat.com>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 10
AU_ALIAS([AC_JNI_INCLUDE_DIR], [AX_JNI_INCLUDE_DIR])
AC_DEFUN([AX_JNI_INCLUDE_DIR],[
JNI_INCLUDE_DIRS=""
if test "x$JAVA_HOME" != x; then
_JTOPDIR="$JAVA_HOME"
else
if test "x$JAVAC" = x; then
JAVAC=javac
fi
AC_PATH_PROG([_ACJNI_JAVAC], [$JAVAC], [no])
if test "x$_ACJNI_JAVAC" = xno; then
AC_MSG_WARN([cannot find JDK; try setting \$JAVAC or \$JAVA_HOME])
fi
_ACJNI_FOLLOW_SYMLINKS("$_ACJNI_JAVAC")
_JTOPDIR=`echo "$_ACJNI_FOLLOWED" | sed -e 's://*:/:g' -e 's:/[[^/]]*$::'`
fi
case "$host_os" in
darwin*) _JTOPDIR=`echo "$_JTOPDIR" | sed -e 's:/[[^/]]*$::'`
_JINC="$_JTOPDIR/Headers";;
*) _JINC="$_JTOPDIR/include";;
esac
_AS_ECHO_LOG([_JTOPDIR=$_JTOPDIR])
_AS_ECHO_LOG([_JINC=$_JINC])
# On Mac OS X 10.6.4, jni.h is a symlink:
# /System/Library/Frameworks/JavaVM.framework/Versions/Current/Headers/jni.h
# -> ../../CurrentJDK/Headers/jni.h.
AC_CACHE_CHECK(jni headers, ac_cv_jni_header_path,
[
if test -f "$_JINC/jni.h"; then
ac_cv_jni_header_path="$_JINC"
JNI_INCLUDE_DIRS="$JNI_INCLUDE_DIRS $ac_cv_jni_header_path"
else
_JTOPDIR=`echo "$_JTOPDIR" | sed -e 's:/[[^/]]*$::'`
if test -f "$_JTOPDIR/include/jni.h"; then
ac_cv_jni_header_path="$_JTOPDIR/include"
JNI_INCLUDE_DIRS="$JNI_INCLUDE_DIRS $ac_cv_jni_header_path"
else
ac_cv_jni_header_path=none
fi
fi
])
# get the likely subdirectories for system specific java includes
case "$host_os" in
bsdi*) _JNI_INC_SUBDIRS="bsdos";;
darwin*) _JNI_INC_SUBDIRS="darwin";;
freebsd*) _JNI_INC_SUBDIRS="freebsd";;
linux*) _JNI_INC_SUBDIRS="linux genunix";;
osf*) _JNI_INC_SUBDIRS="alpha";;
solaris*) _JNI_INC_SUBDIRS="solaris";;
mingw*) _JNI_INC_SUBDIRS="win32";;
cygwin*) _JNI_INC_SUBDIRS="win32";;
*) _JNI_INC_SUBDIRS="genunix";;
esac
if test "x$ac_cv_jni_header_path" != "xnone"; then
# add any subdirectories that are present
for JINCSUBDIR in $_JNI_INC_SUBDIRS
do
if test -d "$_JTOPDIR/include/$JINCSUBDIR"; then
JNI_INCLUDE_DIRS="$JNI_INCLUDE_DIRS $_JTOPDIR/include/$JINCSUBDIR"
fi
done
fi
])
# _ACJNI_FOLLOW_SYMLINKS <path>
# Follows symbolic links on <path>,
# finally setting variable _ACJNI_FOLLOWED
# ----------------------------------------
AC_DEFUN([_ACJNI_FOLLOW_SYMLINKS],[
# find the include directory relative to the javac executable
_cur="$1"
while ls -ld "$_cur" 2>/dev/null | grep " -> " >/dev/null; do
AC_MSG_CHECKING([symlink for $_cur])
_slink=`ls -ld "$_cur" | sed 's/.* -> //'`
case "$_slink" in
/*) _cur="$_slink";;
# 'X' avoids triggering unwanted echo options.
*) _cur=`echo "X$_cur" | sed -e 's/^X//' -e 's:[[^/]]*$::'`"$_slink";;
esac
AC_MSG_RESULT([$_cur])
done
_ACJNI_FOLLOWED="$_cur"
])# _ACJNI

View File

@@ -0,0 +1,125 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_prog_cc_for_build.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_PROG_CC_FOR_BUILD
#
# DESCRIPTION
#
# This macro searches for a C compiler that generates native executables,
# that is a C compiler that surely is not a cross-compiler. This can be
# useful if you have to generate source code at compile-time like for
# example GCC does.
#
# The macro sets the CC_FOR_BUILD and CPP_FOR_BUILD macros to anything
# needed to compile or link (CC_FOR_BUILD) and preprocess (CPP_FOR_BUILD).
# The value of these variables can be overridden by the user by specifying
# a compiler with an environment variable (like you do for standard CC).
#
# It also sets BUILD_EXEEXT and BUILD_OBJEXT to the executable and object
# file extensions for the build platform, and GCC_FOR_BUILD to `yes' if
# the compiler we found is GCC. All these variables but GCC_FOR_BUILD are
# substituted in the Makefile.
#
# LICENSE
#
# Copyright (c) 2008 Paolo Bonzini <bonzini@gnu.org>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 8
AU_ALIAS([AC_PROG_CC_FOR_BUILD], [AX_PROG_CC_FOR_BUILD])
AC_DEFUN([AX_PROG_CC_FOR_BUILD], [dnl
AC_REQUIRE([AC_PROG_CC])dnl
AC_REQUIRE([AC_PROG_CPP])dnl
AC_REQUIRE([AC_EXEEXT])dnl
AC_REQUIRE([AC_CANONICAL_HOST])dnl
dnl Use the standard macros, but make them use other variable names
dnl
pushdef([ac_cv_prog_CPP], ac_cv_build_prog_CPP)dnl
pushdef([ac_cv_prog_gcc], ac_cv_build_prog_gcc)dnl
pushdef([ac_cv_prog_cc_works], ac_cv_build_prog_cc_works)dnl
pushdef([ac_cv_prog_cc_cross], ac_cv_build_prog_cc_cross)dnl
pushdef([ac_cv_prog_cc_g], ac_cv_build_prog_cc_g)dnl
pushdef([ac_cv_exeext], ac_cv_build_exeext)dnl
pushdef([ac_cv_objext], ac_cv_build_objext)dnl
pushdef([ac_exeext], ac_build_exeext)dnl
pushdef([ac_objext], ac_build_objext)dnl
pushdef([CC], CC_FOR_BUILD)dnl
pushdef([CPP], CPP_FOR_BUILD)dnl
pushdef([CFLAGS], CFLAGS_FOR_BUILD)dnl
pushdef([CPPFLAGS], CPPFLAGS_FOR_BUILD)dnl
pushdef([LDFLAGS], LDFLAGS_FOR_BUILD)dnl
pushdef([host], build)dnl
pushdef([host_alias], build_alias)dnl
pushdef([host_cpu], build_cpu)dnl
pushdef([host_vendor], build_vendor)dnl
pushdef([host_os], build_os)dnl
pushdef([ac_cv_host], ac_cv_build)dnl
pushdef([ac_cv_host_alias], ac_cv_build_alias)dnl
pushdef([ac_cv_host_cpu], ac_cv_build_cpu)dnl
pushdef([ac_cv_host_vendor], ac_cv_build_vendor)dnl
pushdef([ac_cv_host_os], ac_cv_build_os)dnl
pushdef([ac_cpp], ac_build_cpp)dnl
pushdef([ac_compile], ac_build_compile)dnl
pushdef([ac_link], ac_build_link)dnl
save_cross_compiling=$cross_compiling
save_ac_tool_prefix=$ac_tool_prefix
cross_compiling=no
ac_tool_prefix=
AC_PROG_CC
AC_PROG_CPP
AC_EXEEXT
ac_tool_prefix=$save_ac_tool_prefix
cross_compiling=$save_cross_compiling
dnl Restore the old definitions
dnl
popdef([ac_link])dnl
popdef([ac_compile])dnl
popdef([ac_cpp])dnl
popdef([ac_cv_host_os])dnl
popdef([ac_cv_host_vendor])dnl
popdef([ac_cv_host_cpu])dnl
popdef([ac_cv_host_alias])dnl
popdef([ac_cv_host])dnl
popdef([host_os])dnl
popdef([host_vendor])dnl
popdef([host_cpu])dnl
popdef([host_alias])dnl
popdef([host])dnl
popdef([LDFLAGS])dnl
popdef([CPPFLAGS])dnl
popdef([CFLAGS])dnl
popdef([CPP])dnl
popdef([CC])dnl
popdef([ac_objext])dnl
popdef([ac_exeext])dnl
popdef([ac_cv_objext])dnl
popdef([ac_cv_exeext])dnl
popdef([ac_cv_prog_cc_g])dnl
popdef([ac_cv_prog_cc_cross])dnl
popdef([ac_cv_prog_cc_works])dnl
popdef([ac_cv_prog_gcc])dnl
popdef([ac_cv_prog_CPP])dnl
dnl Finally, set Makefile variables
dnl
BUILD_EXEEXT=$ac_build_exeext
BUILD_OBJEXT=$ac_build_objext
AC_SUBST(BUILD_EXEEXT)dnl
AC_SUBST(BUILD_OBJEXT)dnl
AC_SUBST([CFLAGS_FOR_BUILD])dnl
AC_SUBST([CPPFLAGS_FOR_BUILD])dnl
AC_SUBST([LDFLAGS_FOR_BUILD])dnl
])

View File

@@ -0,0 +1,69 @@
dnl libsecp25k1 helper checks
AC_DEFUN([SECP_INT128_CHECK],[
has_int128=$ac_cv_type___int128
])
dnl escape "$0x" below using the m4 quadrigaph @S|@, and escape it again with a \ for the shell.
AC_DEFUN([SECP_64BIT_ASM_CHECK],[
AC_MSG_CHECKING(for x86_64 assembly availability)
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <stdint.h>]],[[
uint64_t a = 11, tmp;
__asm__ __volatile__("movq \@S|@0x100000000,%1; mulq %%rsi" : "+a"(a) : "S"(tmp) : "cc", "%rdx");
]])],[has_64bit_asm=yes],[has_64bit_asm=no])
AC_MSG_RESULT([$has_64bit_asm])
])
dnl
AC_DEFUN([SECP_OPENSSL_CHECK],[
has_libcrypto=no
m4_ifdef([PKG_CHECK_MODULES],[
PKG_CHECK_MODULES([CRYPTO], [libcrypto], [has_libcrypto=yes],[has_libcrypto=no])
if test x"$has_libcrypto" = x"yes"; then
TEMP_LIBS="$LIBS"
LIBS="$LIBS $CRYPTO_LIBS"
AC_CHECK_LIB(crypto, main,[AC_DEFINE(HAVE_LIBCRYPTO,1,[Define this symbol if libcrypto is installed])],[has_libcrypto=no])
LIBS="$TEMP_LIBS"
fi
])
if test x$has_libcrypto = xno; then
AC_CHECK_HEADER(openssl/crypto.h,[
AC_CHECK_LIB(crypto, main,[
has_libcrypto=yes
CRYPTO_LIBS=-lcrypto
AC_DEFINE(HAVE_LIBCRYPTO,1,[Define this symbol if libcrypto is installed])
])
])
LIBS=
fi
if test x"$has_libcrypto" = x"yes" && test x"$has_openssl_ec" = x; then
AC_MSG_CHECKING(for EC functions in libcrypto)
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <openssl/ec.h>
#include <openssl/ecdsa.h>
#include <openssl/obj_mac.h>]],[[
EC_KEY *eckey = EC_KEY_new_by_curve_name(NID_secp256k1);
ECDSA_sign(0, NULL, 0, NULL, NULL, eckey);
ECDSA_verify(0, NULL, 0, NULL, 0, eckey);
EC_KEY_free(eckey);
ECDSA_SIG *sig_openssl;
sig_openssl = ECDSA_SIG_new();
(void)sig_openssl->r;
ECDSA_SIG_free(sig_openssl);
]])],[has_openssl_ec=yes],[has_openssl_ec=no])
AC_MSG_RESULT([$has_openssl_ec])
fi
])
dnl
AC_DEFUN([SECP_GMP_CHECK],[
if test x"$has_gmp" != x"yes"; then
CPPFLAGS_TEMP="$CPPFLAGS"
CPPFLAGS="$GMP_CPPFLAGS $CPPFLAGS"
LIBS_TEMP="$LIBS"
LIBS="$GMP_LIBS $LIBS"
AC_CHECK_HEADER(gmp.h,[AC_CHECK_LIB(gmp, __gmpz_init,[has_gmp=yes; GMP_LIBS="$GMP_LIBS -lgmp"; AC_DEFINE(HAVE_LIBGMP,1,[Define this symbol if libgmp is installed])])])
CPPFLAGS="$CPPFLAGS_TEMP"
LIBS="$LIBS_TEMP"
fi
])

View File

@@ -0,0 +1,493 @@
AC_PREREQ([2.60])
AC_INIT([libsecp256k1],[0.1])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_MACRO_DIR([build-aux/m4])
AC_CANONICAL_HOST
AH_TOP([#ifndef LIBSECP256K1_CONFIG_H])
AH_TOP([#define LIBSECP256K1_CONFIG_H])
AH_BOTTOM([#endif /*LIBSECP256K1_CONFIG_H*/])
AM_INIT_AUTOMAKE([foreign subdir-objects])
LT_INIT
dnl make the compilation flags quiet unless V=1 is used
m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
PKG_PROG_PKG_CONFIG
AC_PATH_TOOL(AR, ar)
AC_PATH_TOOL(RANLIB, ranlib)
AC_PATH_TOOL(STRIP, strip)
AX_PROG_CC_FOR_BUILD
if test "x$CFLAGS" = "x"; then
CFLAGS="-g"
fi
AM_PROG_CC_C_O
AC_PROG_CC_C89
if test x"$ac_cv_prog_cc_c89" = x"no"; then
AC_MSG_ERROR([c89 compiler support required])
fi
AM_PROG_AS
case $host_os in
*darwin*)
if test x$cross_compiling != xyes; then
AC_PATH_PROG([BREW],brew,)
if test x$BREW != x; then
dnl These Homebrew packages may be keg-only, meaning that they won't be found
dnl in expected paths because they may conflict with system files. Ask
dnl Homebrew where each one is located, then adjust paths accordingly.
openssl_prefix=`$BREW --prefix openssl 2>/dev/null`
gmp_prefix=`$BREW --prefix gmp 2>/dev/null`
if test x$openssl_prefix != x; then
PKG_CONFIG_PATH="$openssl_prefix/lib/pkgconfig:$PKG_CONFIG_PATH"
export PKG_CONFIG_PATH
fi
if test x$gmp_prefix != x; then
GMP_CPPFLAGS="-I$gmp_prefix/include"
GMP_LIBS="-L$gmp_prefix/lib"
fi
else
AC_PATH_PROG([PORT],port,)
dnl if homebrew isn't installed and macports is, add the macports default paths
dnl as a last resort.
if test x$PORT != x; then
CPPFLAGS="$CPPFLAGS -isystem /opt/local/include"
LDFLAGS="$LDFLAGS -L/opt/local/lib"
fi
fi
fi
;;
esac
CFLAGS="$CFLAGS -W"
warn_CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wno-unused-function -Wno-long-long -Wno-overlength-strings"
saved_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS $warn_CFLAGS"
AC_MSG_CHECKING([if ${CC} supports ${warn_CFLAGS}])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[char foo;]])],
[ AC_MSG_RESULT([yes]) ],
[ AC_MSG_RESULT([no])
CFLAGS="$saved_CFLAGS"
])
saved_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -fvisibility=hidden"
AC_MSG_CHECKING([if ${CC} supports -fvisibility=hidden])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[char foo;]])],
[ AC_MSG_RESULT([yes]) ],
[ AC_MSG_RESULT([no])
CFLAGS="$saved_CFLAGS"
])
AC_ARG_ENABLE(benchmark,
AS_HELP_STRING([--enable-benchmark],[compile benchmark (default is no)]),
[use_benchmark=$enableval],
[use_benchmark=no])
AC_ARG_ENABLE(coverage,
AS_HELP_STRING([--enable-coverage],[enable compiler flags to support kcov coverage analysis]),
[enable_coverage=$enableval],
[enable_coverage=no])
AC_ARG_ENABLE(tests,
AS_HELP_STRING([--enable-tests],[compile tests (default is yes)]),
[use_tests=$enableval],
[use_tests=yes])
AC_ARG_ENABLE(openssl_tests,
AS_HELP_STRING([--enable-openssl-tests],[enable OpenSSL tests, if OpenSSL is available (default is auto)]),
[enable_openssl_tests=$enableval],
[enable_openssl_tests=auto])
AC_ARG_ENABLE(experimental,
AS_HELP_STRING([--enable-experimental],[allow experimental configure options (default is no)]),
[use_experimental=$enableval],
[use_experimental=no])
AC_ARG_ENABLE(exhaustive_tests,
AS_HELP_STRING([--enable-exhaustive-tests],[compile exhaustive tests (default is yes)]),
[use_exhaustive_tests=$enableval],
[use_exhaustive_tests=yes])
AC_ARG_ENABLE(endomorphism,
AS_HELP_STRING([--enable-endomorphism],[enable endomorphism (default is no)]),
[use_endomorphism=$enableval],
[use_endomorphism=no])
AC_ARG_ENABLE(ecmult_static_precomputation,
AS_HELP_STRING([--enable-ecmult-static-precomputation],[enable precomputed ecmult table for signing (default is yes)]),
[use_ecmult_static_precomputation=$enableval],
[use_ecmult_static_precomputation=auto])
AC_ARG_ENABLE(module_ecdh,
AS_HELP_STRING([--enable-module-ecdh],[enable ECDH shared secret computation (experimental)]),
[enable_module_ecdh=$enableval],
[enable_module_ecdh=no])
AC_ARG_ENABLE(module_recovery,
AS_HELP_STRING([--enable-module-recovery],[enable ECDSA pubkey recovery module (default is no)]),
[enable_module_recovery=$enableval],
[enable_module_recovery=no])
AC_ARG_ENABLE(jni,
AS_HELP_STRING([--enable-jni],[enable libsecp256k1_jni (default is auto)]),
[use_jni=$enableval],
[use_jni=auto])
AC_ARG_WITH([field], [AS_HELP_STRING([--with-field=64bit|32bit|auto],
[Specify Field Implementation. Default is auto])],[req_field=$withval], [req_field=auto])
AC_ARG_WITH([bignum], [AS_HELP_STRING([--with-bignum=gmp|no|auto],
[Specify Bignum Implementation. Default is auto])],[req_bignum=$withval], [req_bignum=auto])
AC_ARG_WITH([scalar], [AS_HELP_STRING([--with-scalar=64bit|32bit|auto],
[Specify scalar implementation. Default is auto])],[req_scalar=$withval], [req_scalar=auto])
AC_ARG_WITH([asm], [AS_HELP_STRING([--with-asm=x86_64|arm|no|auto]
[Specify assembly optimizations to use. Default is auto (experimental: arm)])],[req_asm=$withval], [req_asm=auto])
AC_CHECK_TYPES([__int128])
AC_MSG_CHECKING([for __builtin_expect])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[void myfunc() {__builtin_expect(0,0);}]])],
[ AC_MSG_RESULT([yes]);AC_DEFINE(HAVE_BUILTIN_EXPECT,1,[Define this symbol if __builtin_expect is available]) ],
[ AC_MSG_RESULT([no])
])
if test x"$enable_coverage" = x"yes"; then
AC_DEFINE(COVERAGE, 1, [Define this symbol to compile out all VERIFY code])
CFLAGS="$CFLAGS -O0 --coverage"
LDFLAGS="--coverage"
else
CFLAGS="$CFLAGS -O3"
fi
if test x"$use_ecmult_static_precomputation" != x"no"; then
save_cross_compiling=$cross_compiling
cross_compiling=no
TEMP_CC="$CC"
CC="$CC_FOR_BUILD"
AC_MSG_CHECKING([native compiler: ${CC_FOR_BUILD}])
AC_RUN_IFELSE(
[AC_LANG_PROGRAM([], [return 0])],
[working_native_cc=yes],
[working_native_cc=no],[dnl])
CC="$TEMP_CC"
cross_compiling=$save_cross_compiling
if test x"$working_native_cc" = x"no"; then
set_precomp=no
if test x"$use_ecmult_static_precomputation" = x"yes"; then
AC_MSG_ERROR([${CC_FOR_BUILD} does not produce working binaries. Please set CC_FOR_BUILD])
else
AC_MSG_RESULT([${CC_FOR_BUILD} does not produce working binaries. Please set CC_FOR_BUILD])
fi
else
AC_MSG_RESULT([ok])
set_precomp=yes
fi
else
set_precomp=no
fi
if test x"$req_asm" = x"auto"; then
SECP_64BIT_ASM_CHECK
if test x"$has_64bit_asm" = x"yes"; then
set_asm=x86_64
fi
if test x"$set_asm" = x; then
set_asm=no
fi
else
set_asm=$req_asm
case $set_asm in
x86_64)
SECP_64BIT_ASM_CHECK
if test x"$has_64bit_asm" != x"yes"; then
AC_MSG_ERROR([x86_64 assembly optimization requested but not available])
fi
;;
arm)
;;
no)
;;
*)
AC_MSG_ERROR([invalid assembly optimization selection])
;;
esac
fi
if test x"$req_field" = x"auto"; then
if test x"set_asm" = x"x86_64"; then
set_field=64bit
fi
if test x"$set_field" = x; then
SECP_INT128_CHECK
if test x"$has_int128" = x"yes"; then
set_field=64bit
fi
fi
if test x"$set_field" = x; then
set_field=32bit
fi
else
set_field=$req_field
case $set_field in
64bit)
if test x"$set_asm" != x"x86_64"; then
SECP_INT128_CHECK
if test x"$has_int128" != x"yes"; then
AC_MSG_ERROR([64bit field explicitly requested but neither __int128 support or x86_64 assembly available])
fi
fi
;;
32bit)
;;
*)
AC_MSG_ERROR([invalid field implementation selection])
;;
esac
fi
if test x"$req_scalar" = x"auto"; then
SECP_INT128_CHECK
if test x"$has_int128" = x"yes"; then
set_scalar=64bit
fi
if test x"$set_scalar" = x; then
set_scalar=32bit
fi
else
set_scalar=$req_scalar
case $set_scalar in
64bit)
SECP_INT128_CHECK
if test x"$has_int128" != x"yes"; then
AC_MSG_ERROR([64bit scalar explicitly requested but __int128 support not available])
fi
;;
32bit)
;;
*)
AC_MSG_ERROR([invalid scalar implementation selected])
;;
esac
fi
if test x"$req_bignum" = x"auto"; then
SECP_GMP_CHECK
if test x"$has_gmp" = x"yes"; then
set_bignum=gmp
fi
if test x"$set_bignum" = x; then
set_bignum=no
fi
else
set_bignum=$req_bignum
case $set_bignum in
gmp)
SECP_GMP_CHECK
if test x"$has_gmp" != x"yes"; then
AC_MSG_ERROR([gmp bignum explicitly requested but libgmp not available])
fi
;;
no)
;;
*)
AC_MSG_ERROR([invalid bignum implementation selection])
;;
esac
fi
# select assembly optimization
use_external_asm=no
case $set_asm in
x86_64)
AC_DEFINE(USE_ASM_X86_64, 1, [Define this symbol to enable x86_64 assembly optimizations])
;;
arm)
use_external_asm=yes
;;
no)
;;
*)
AC_MSG_ERROR([invalid assembly optimizations])
;;
esac
# select field implementation
case $set_field in
64bit)
AC_DEFINE(USE_FIELD_5X52, 1, [Define this symbol to use the FIELD_5X52 implementation])
;;
32bit)
AC_DEFINE(USE_FIELD_10X26, 1, [Define this symbol to use the FIELD_10X26 implementation])
;;
*)
AC_MSG_ERROR([invalid field implementation])
;;
esac
# select bignum implementation
case $set_bignum in
gmp)
AC_DEFINE(HAVE_LIBGMP, 1, [Define this symbol if libgmp is installed])
AC_DEFINE(USE_NUM_GMP, 1, [Define this symbol to use the gmp implementation for num])
AC_DEFINE(USE_FIELD_INV_NUM, 1, [Define this symbol to use the num-based field inverse implementation])
AC_DEFINE(USE_SCALAR_INV_NUM, 1, [Define this symbol to use the num-based scalar inverse implementation])
;;
no)
AC_DEFINE(USE_NUM_NONE, 1, [Define this symbol to use no num implementation])
AC_DEFINE(USE_FIELD_INV_BUILTIN, 1, [Define this symbol to use the native field inverse implementation])
AC_DEFINE(USE_SCALAR_INV_BUILTIN, 1, [Define this symbol to use the native scalar inverse implementation])
;;
*)
AC_MSG_ERROR([invalid bignum implementation])
;;
esac
#select scalar implementation
case $set_scalar in
64bit)
AC_DEFINE(USE_SCALAR_4X64, 1, [Define this symbol to use the 4x64 scalar implementation])
;;
32bit)
AC_DEFINE(USE_SCALAR_8X32, 1, [Define this symbol to use the 8x32 scalar implementation])
;;
*)
AC_MSG_ERROR([invalid scalar implementation])
;;
esac
if test x"$use_tests" = x"yes"; then
SECP_OPENSSL_CHECK
if test x"$has_openssl_ec" = x"yes"; then
if test x"$enable_openssl_tests" != x"no"; then
AC_DEFINE(ENABLE_OPENSSL_TESTS, 1, [Define this symbol if OpenSSL EC functions are available])
SECP_TEST_INCLUDES="$SSL_CFLAGS $CRYPTO_CFLAGS"
SECP_TEST_LIBS="$CRYPTO_LIBS"
case $host in
*mingw*)
SECP_TEST_LIBS="$SECP_TEST_LIBS -lgdi32"
;;
esac
fi
else
if test x"$enable_openssl_tests" = x"yes"; then
AC_MSG_ERROR([OpenSSL tests requested but OpenSSL with EC support is not available])
fi
fi
else
if test x"$enable_openssl_tests" = x"yes"; then
AC_MSG_ERROR([OpenSSL tests requested but tests are not enabled])
fi
fi
if test x"$use_jni" != x"no"; then
AX_JNI_INCLUDE_DIR
have_jni_dependencies=yes
if test x"$enable_module_ecdh" = x"no"; then
have_jni_dependencies=no
fi
if test "x$JNI_INCLUDE_DIRS" = "x"; then
have_jni_dependencies=no
fi
if test "x$have_jni_dependencies" = "xno"; then
if test x"$use_jni" = x"yes"; then
AC_MSG_ERROR([jni support explicitly requested but headers/dependencies were not found. Enable ECDH and try again.])
fi
AC_MSG_WARN([jni headers/dependencies not found. jni support disabled])
use_jni=no
else
use_jni=yes
for JNI_INCLUDE_DIR in $JNI_INCLUDE_DIRS; do
JNI_INCLUDES="$JNI_INCLUDES -I$JNI_INCLUDE_DIR"
done
fi
fi
if test x"$set_bignum" = x"gmp"; then
SECP_LIBS="$SECP_LIBS $GMP_LIBS"
SECP_INCLUDES="$SECP_INCLUDES $GMP_CPPFLAGS"
fi
if test x"$use_endomorphism" = x"yes"; then
AC_DEFINE(USE_ENDOMORPHISM, 1, [Define this symbol to use endomorphism optimization])
fi
if test x"$set_precomp" = x"yes"; then
AC_DEFINE(USE_ECMULT_STATIC_PRECOMPUTATION, 1, [Define this symbol to use a statically generated ecmult table])
fi
if test x"$enable_module_ecdh" = x"yes"; then
AC_DEFINE(ENABLE_MODULE_ECDH, 1, [Define this symbol to enable the ECDH module])
fi
if test x"$enable_module_recovery" = x"yes"; then
AC_DEFINE(ENABLE_MODULE_RECOVERY, 1, [Define this symbol to enable the ECDSA pubkey recovery module])
fi
AC_C_BIGENDIAN()
if test x"$use_external_asm" = x"yes"; then
AC_DEFINE(USE_EXTERNAL_ASM, 1, [Define this symbol if an external (non-inline) assembly implementation is used])
fi
AC_MSG_NOTICE([Using static precomputation: $set_precomp])
AC_MSG_NOTICE([Using assembly optimizations: $set_asm])
AC_MSG_NOTICE([Using field implementation: $set_field])
AC_MSG_NOTICE([Using bignum implementation: $set_bignum])
AC_MSG_NOTICE([Using scalar implementation: $set_scalar])
AC_MSG_NOTICE([Using endomorphism optimizations: $use_endomorphism])
AC_MSG_NOTICE([Building for coverage analysis: $enable_coverage])
AC_MSG_NOTICE([Building ECDH module: $enable_module_ecdh])
AC_MSG_NOTICE([Building ECDSA pubkey recovery module: $enable_module_recovery])
AC_MSG_NOTICE([Using jni: $use_jni])
if test x"$enable_experimental" = x"yes"; then
AC_MSG_NOTICE([******])
AC_MSG_NOTICE([WARNING: experimental build])
AC_MSG_NOTICE([Experimental features do not have stable APIs or properties, and may not be safe for production use.])
AC_MSG_NOTICE([Building ECDH module: $enable_module_ecdh])
AC_MSG_NOTICE([******])
else
if test x"$enable_module_ecdh" = x"yes"; then
AC_MSG_ERROR([ECDH module is experimental. Use --enable-experimental to allow.])
fi
if test x"$set_asm" = x"arm"; then
AC_MSG_ERROR([ARM assembly optimization is experimental. Use --enable-experimental to allow.])
fi
fi
AC_CONFIG_HEADERS([src/libsecp256k1-config.h])
AC_CONFIG_FILES([Makefile libsecp256k1.pc])
AC_SUBST(JNI_INCLUDES)
AC_SUBST(SECP_INCLUDES)
AC_SUBST(SECP_LIBS)
AC_SUBST(SECP_TEST_LIBS)
AC_SUBST(SECP_TEST_INCLUDES)
AM_CONDITIONAL([ENABLE_COVERAGE], [test x"$enable_coverage" = x"yes"])
AM_CONDITIONAL([USE_TESTS], [test x"$use_tests" != x"no"])
AM_CONDITIONAL([USE_EXHAUSTIVE_TESTS], [test x"$use_exhaustive_tests" != x"no"])
AM_CONDITIONAL([USE_BENCHMARK], [test x"$use_benchmark" = x"yes"])
AM_CONDITIONAL([USE_ECMULT_STATIC_PRECOMPUTATION], [test x"$set_precomp" = x"yes"])
AM_CONDITIONAL([ENABLE_MODULE_ECDH], [test x"$enable_module_ecdh" = x"yes"])
AM_CONDITIONAL([ENABLE_MODULE_RECOVERY], [test x"$enable_module_recovery" = x"yes"])
AM_CONDITIONAL([USE_JNI], [test x"$use_jni" == x"yes"])
AM_CONDITIONAL([USE_EXTERNAL_ASM], [test x"$use_external_asm" = x"yes"])
AM_CONDITIONAL([USE_ASM_ARM], [test x"$set_asm" = x"arm"])
dnl make sure nothing new is exported so that we don't break the cache
PKGCONFIG_PATH_TEMP="$PKG_CONFIG_PATH"
unset PKG_CONFIG_PATH
PKG_CONFIG_PATH="$PKGCONFIG_PATH_TEMP"
AC_OUTPUT

View File

@@ -0,0 +1,150 @@
/**********************************************************************
* Copyright (c) 2015 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
#include <string.h>
#include <secp256k1.h>
#include "lax_der_parsing.h"
int ecdsa_signature_parse_der_lax(const secp256k1_context* ctx, secp256k1_ecdsa_signature* sig, const unsigned char *input, size_t inputlen) {
size_t rpos, rlen, spos, slen;
size_t pos = 0;
size_t lenbyte;
unsigned char tmpsig[64] = {0};
int overflow = 0;
/* Hack to initialize sig with a correctly-parsed but invalid signature. */
secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig);
/* Sequence tag byte */
if (pos == inputlen || input[pos] != 0x30) {
return 0;
}
pos++;
/* Sequence length bytes */
if (pos == inputlen) {
return 0;
}
lenbyte = input[pos++];
if (lenbyte & 0x80) {
lenbyte -= 0x80;
if (pos + lenbyte > inputlen) {
return 0;
}
pos += lenbyte;
}
/* Integer tag byte for R */
if (pos == inputlen || input[pos] != 0x02) {
return 0;
}
pos++;
/* Integer length for R */
if (pos == inputlen) {
return 0;
}
lenbyte = input[pos++];
if (lenbyte & 0x80) {
lenbyte -= 0x80;
if (pos + lenbyte > inputlen) {
return 0;
}
while (lenbyte > 0 && input[pos] == 0) {
pos++;
lenbyte--;
}
if (lenbyte >= sizeof(size_t)) {
return 0;
}
rlen = 0;
while (lenbyte > 0) {
rlen = (rlen << 8) + input[pos];
pos++;
lenbyte--;
}
} else {
rlen = lenbyte;
}
if (rlen > inputlen - pos) {
return 0;
}
rpos = pos;
pos += rlen;
/* Integer tag byte for S */
if (pos == inputlen || input[pos] != 0x02) {
return 0;
}
pos++;
/* Integer length for S */
if (pos == inputlen) {
return 0;
}
lenbyte = input[pos++];
if (lenbyte & 0x80) {
lenbyte -= 0x80;
if (pos + lenbyte > inputlen) {
return 0;
}
while (lenbyte > 0 && input[pos] == 0) {
pos++;
lenbyte--;
}
if (lenbyte >= sizeof(size_t)) {
return 0;
}
slen = 0;
while (lenbyte > 0) {
slen = (slen << 8) + input[pos];
pos++;
lenbyte--;
}
} else {
slen = lenbyte;
}
if (slen > inputlen - pos) {
return 0;
}
spos = pos;
pos += slen;
/* Ignore leading zeroes in R */
while (rlen > 0 && input[rpos] == 0) {
rlen--;
rpos++;
}
/* Copy R value */
if (rlen > 32) {
overflow = 1;
} else {
memcpy(tmpsig + 32 - rlen, input + rpos, rlen);
}
/* Ignore leading zeroes in S */
while (slen > 0 && input[spos] == 0) {
slen--;
spos++;
}
/* Copy S value */
if (slen > 32) {
overflow = 1;
} else {
memcpy(tmpsig + 64 - slen, input + spos, slen);
}
if (!overflow) {
overflow = !secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig);
}
if (overflow) {
memset(tmpsig, 0, 64);
secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig);
}
return 1;
}

View File

@@ -0,0 +1,91 @@
/**********************************************************************
* Copyright (c) 2015 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
/****
* Please do not link this file directly. It is not part of the libsecp256k1
* project and does not promise any stability in its API, functionality or
* presence. Projects which use this code should instead copy this header
* and its accompanying .c file directly into their codebase.
****/
/* This file defines a function that parses DER with various errors and
* violations. This is not a part of the library itself, because the allowed
* violations are chosen arbitrarily and do not follow or establish any
* standard.
*
* In many places it matters that different implementations do not only accept
* the same set of valid signatures, but also reject the same set of signatures.
* The only means to accomplish that is by strictly obeying a standard, and not
* accepting anything else.
*
* Nonetheless, sometimes there is a need for compatibility with systems that
* use signatures which do not strictly obey DER. The snippet below shows how
* certain violations are easily supported. You may need to adapt it.
*
* Do not use this for new systems. Use well-defined DER or compact signatures
* instead if you have the choice (see secp256k1_ecdsa_signature_parse_der and
* secp256k1_ecdsa_signature_parse_compact).
*
* The supported violations are:
* - All numbers are parsed as nonnegative integers, even though X.609-0207
* section 8.3.3 specifies that integers are always encoded as two's
* complement.
* - Integers can have length 0, even though section 8.3.1 says they can't.
* - Integers with overly long padding are accepted, violation section
* 8.3.2.
* - 127-byte long length descriptors are accepted, even though section
* 8.1.3.5.c says that they are not.
* - Trailing garbage data inside or after the signature is ignored.
* - The length descriptor of the sequence is ignored.
*
* Compared to for example OpenSSL, many violations are NOT supported:
* - Using overly long tag descriptors for the sequence or integers inside,
* violating section 8.1.2.2.
* - Encoding primitive integers as constructed values, violating section
* 8.3.1.
*/
#ifndef _SECP256K1_CONTRIB_LAX_DER_PARSING_H_
#define _SECP256K1_CONTRIB_LAX_DER_PARSING_H_
#include <secp256k1.h>
# ifdef __cplusplus
extern "C" {
# endif
/** Parse a signature in "lax DER" format
*
* Returns: 1 when the signature could be parsed, 0 otherwise.
* Args: ctx: a secp256k1 context object
* Out: sig: a pointer to a signature object
* In: input: a pointer to the signature to be parsed
* inputlen: the length of the array pointed to be input
*
* This function will accept any valid DER encoded signature, even if the
* encoded numbers are out of range. In addition, it will accept signatures
* which violate the DER spec in various ways. Its purpose is to allow
* validation of the Bitcoin blockchain, which includes non-DER signatures
* from before the network rules were updated to enforce DER. Note that
* the set of supported violations is a strict subset of what OpenSSL will
* accept.
*
* After the call, sig will always be initialized. If parsing failed or the
* encoded numbers are out of range, signature validation with it is
* guaranteed to fail for every message and public key.
*/
int ecdsa_signature_parse_der_lax(
const secp256k1_context* ctx,
secp256k1_ecdsa_signature* sig,
const unsigned char *input,
size_t inputlen
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
#ifdef __cplusplus
}
#endif
#endif

View File

@@ -0,0 +1,113 @@
/**********************************************************************
* Copyright (c) 2014, 2015 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
#include <string.h>
#include <secp256k1.h>
#include "lax_der_privatekey_parsing.h"
int ec_privkey_import_der(const secp256k1_context* ctx, unsigned char *out32, const unsigned char *privkey, size_t privkeylen) {
const unsigned char *end = privkey + privkeylen;
int lenb = 0;
int len = 0;
memset(out32, 0, 32);
/* sequence header */
if (end < privkey+1 || *privkey != 0x30) {
return 0;
}
privkey++;
/* sequence length constructor */
if (end < privkey+1 || !(*privkey & 0x80)) {
return 0;
}
lenb = *privkey & ~0x80; privkey++;
if (lenb < 1 || lenb > 2) {
return 0;
}
if (end < privkey+lenb) {
return 0;
}
/* sequence length */
len = privkey[lenb-1] | (lenb > 1 ? privkey[lenb-2] << 8 : 0);
privkey += lenb;
if (end < privkey+len) {
return 0;
}
/* sequence element 0: version number (=1) */
if (end < privkey+3 || privkey[0] != 0x02 || privkey[1] != 0x01 || privkey[2] != 0x01) {
return 0;
}
privkey += 3;
/* sequence element 1: octet string, up to 32 bytes */
if (end < privkey+2 || privkey[0] != 0x04 || privkey[1] > 0x20 || end < privkey+2+privkey[1]) {
return 0;
}
memcpy(out32 + 32 - privkey[1], privkey + 2, privkey[1]);
if (!secp256k1_ec_seckey_verify(ctx, out32)) {
memset(out32, 0, 32);
return 0;
}
return 1;
}
int ec_privkey_export_der(const secp256k1_context *ctx, unsigned char *privkey, size_t *privkeylen, const unsigned char *key32, int compressed) {
secp256k1_pubkey pubkey;
size_t pubkeylen = 0;
if (!secp256k1_ec_pubkey_create(ctx, &pubkey, key32)) {
*privkeylen = 0;
return 0;
}
if (compressed) {
static const unsigned char begin[] = {
0x30,0x81,0xD3,0x02,0x01,0x01,0x04,0x20
};
static const unsigned char middle[] = {
0xA0,0x81,0x85,0x30,0x81,0x82,0x02,0x01,0x01,0x30,0x2C,0x06,0x07,0x2A,0x86,0x48,
0xCE,0x3D,0x01,0x01,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F,0x30,0x06,0x04,0x01,0x00,0x04,0x01,0x07,0x04,
0x21,0x02,0x79,0xBE,0x66,0x7E,0xF9,0xDC,0xBB,0xAC,0x55,0xA0,0x62,0x95,0xCE,0x87,
0x0B,0x07,0x02,0x9B,0xFC,0xDB,0x2D,0xCE,0x28,0xD9,0x59,0xF2,0x81,0x5B,0x16,0xF8,
0x17,0x98,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
0xFF,0xFF,0xFF,0xFF,0xFE,0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B,0xBF,0xD2,0x5E,
0x8C,0xD0,0x36,0x41,0x41,0x02,0x01,0x01,0xA1,0x24,0x03,0x22,0x00
};
unsigned char *ptr = privkey;
memcpy(ptr, begin, sizeof(begin)); ptr += sizeof(begin);
memcpy(ptr, key32, 32); ptr += 32;
memcpy(ptr, middle, sizeof(middle)); ptr += sizeof(middle);
pubkeylen = 33;
secp256k1_ec_pubkey_serialize(ctx, ptr, &pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED);
ptr += pubkeylen;
*privkeylen = ptr - privkey;
} else {
static const unsigned char begin[] = {
0x30,0x82,0x01,0x13,0x02,0x01,0x01,0x04,0x20
};
static const unsigned char middle[] = {
0xA0,0x81,0xA5,0x30,0x81,0xA2,0x02,0x01,0x01,0x30,0x2C,0x06,0x07,0x2A,0x86,0x48,
0xCE,0x3D,0x01,0x01,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F,0x30,0x06,0x04,0x01,0x00,0x04,0x01,0x07,0x04,
0x41,0x04,0x79,0xBE,0x66,0x7E,0xF9,0xDC,0xBB,0xAC,0x55,0xA0,0x62,0x95,0xCE,0x87,
0x0B,0x07,0x02,0x9B,0xFC,0xDB,0x2D,0xCE,0x28,0xD9,0x59,0xF2,0x81,0x5B,0x16,0xF8,
0x17,0x98,0x48,0x3A,0xDA,0x77,0x26,0xA3,0xC4,0x65,0x5D,0xA4,0xFB,0xFC,0x0E,0x11,
0x08,0xA8,0xFD,0x17,0xB4,0x48,0xA6,0x85,0x54,0x19,0x9C,0x47,0xD0,0x8F,0xFB,0x10,
0xD4,0xB8,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,
0xFF,0xFF,0xFF,0xFF,0xFE,0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B,0xBF,0xD2,0x5E,
0x8C,0xD0,0x36,0x41,0x41,0x02,0x01,0x01,0xA1,0x44,0x03,0x42,0x00
};
unsigned char *ptr = privkey;
memcpy(ptr, begin, sizeof(begin)); ptr += sizeof(begin);
memcpy(ptr, key32, 32); ptr += 32;
memcpy(ptr, middle, sizeof(middle)); ptr += sizeof(middle);
pubkeylen = 65;
secp256k1_ec_pubkey_serialize(ctx, ptr, &pubkeylen, &pubkey, SECP256K1_EC_UNCOMPRESSED);
ptr += pubkeylen;
*privkeylen = ptr - privkey;
}
return 1;
}

View File

@@ -0,0 +1,90 @@
/**********************************************************************
* Copyright (c) 2014, 2015 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
/****
* Please do not link this file directly. It is not part of the libsecp256k1
* project and does not promise any stability in its API, functionality or
* presence. Projects which use this code should instead copy this header
* and its accompanying .c file directly into their codebase.
****/
/* This file contains code snippets that parse DER private keys with
* various errors and violations. This is not a part of the library
* itself, because the allowed violations are chosen arbitrarily and
* do not follow or establish any standard.
*
* It also contains code to serialize private keys in a compatible
* manner.
*
* These functions are meant for compatibility with applications
* that require BER encoded keys. When working with secp256k1-specific
* code, the simple 32-byte private keys normally used by the
* library are sufficient.
*/
#ifndef _SECP256K1_CONTRIB_BER_PRIVATEKEY_H_
#define _SECP256K1_CONTRIB_BER_PRIVATEKEY_H_
#include <secp256k1.h>
# ifdef __cplusplus
extern "C" {
# endif
/** Export a private key in DER format.
*
* Returns: 1 if the private key was valid.
* Args: ctx: pointer to a context object, initialized for signing (cannot
* be NULL)
* Out: privkey: pointer to an array for storing the private key in BER.
* Should have space for 279 bytes, and cannot be NULL.
* privkeylen: Pointer to an int where the length of the private key in
* privkey will be stored.
* In: seckey: pointer to a 32-byte secret key to export.
* compressed: 1 if the key should be exported in
* compressed format, 0 otherwise
*
* This function is purely meant for compatibility with applications that
* require BER encoded keys. When working with secp256k1-specific code, the
* simple 32-byte private keys are sufficient.
*
* Note that this function does not guarantee correct DER output. It is
* guaranteed to be parsable by secp256k1_ec_privkey_import_der
*/
SECP256K1_WARN_UNUSED_RESULT int ec_privkey_export_der(
const secp256k1_context* ctx,
unsigned char *privkey,
size_t *privkeylen,
const unsigned char *seckey,
int compressed
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Import a private key in DER format.
* Returns: 1 if a private key was extracted.
* Args: ctx: pointer to a context object (cannot be NULL).
* Out: seckey: pointer to a 32-byte array for storing the private key.
* (cannot be NULL).
* In: privkey: pointer to a private key in DER format (cannot be NULL).
* privkeylen: length of the DER private key pointed to be privkey.
*
* This function will accept more than just strict DER, and even allow some BER
* violations. The public key stored inside the DER-encoded private key is not
* verified for correctness, nor are the curve parameters. Use this function
* only if you know in advance it is supposed to contain a secp256k1 private
* key.
*/
SECP256K1_WARN_UNUSED_RESULT int ec_privkey_import_der(
const secp256k1_context* ctx,
unsigned char *seckey,
const unsigned char *privkey,
size_t privkeylen
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
#ifdef __cplusplus
}
#endif
#endif

View File

@@ -0,0 +1,577 @@
#ifndef _SECP256K1_
# define _SECP256K1_
# ifdef __cplusplus
extern "C" {
# endif
#include <stddef.h>
/* These rules specify the order of arguments in API calls:
*
* 1. Context pointers go first, followed by output arguments, combined
* output/input arguments, and finally input-only arguments.
* 2. Array lengths always immediately the follow the argument whose length
* they describe, even if this violates rule 1.
* 3. Within the OUT/OUTIN/IN groups, pointers to data that is typically generated
* later go first. This means: signatures, public nonces, private nonces,
* messages, public keys, secret keys, tweaks.
* 4. Arguments that are not data pointers go last, from more complex to less
* complex: function pointers, algorithm names, messages, void pointers,
* counts, flags, booleans.
* 5. Opaque data pointers follow the function pointer they are to be passed to.
*/
/** Opaque data structure that holds context information (precomputed tables etc.).
*
* The purpose of context structures is to cache large precomputed data tables
* that are expensive to construct, and also to maintain the randomization data
* for blinding.
*
* Do not create a new context object for each operation, as construction is
* far slower than all other API calls (~100 times slower than an ECDSA
* verification).
*
* A constructed context can safely be used from multiple threads
* simultaneously, but API call that take a non-const pointer to a context
* need exclusive access to it. In particular this is the case for
* secp256k1_context_destroy and secp256k1_context_randomize.
*
* Regarding randomization, either do it once at creation time (in which case
* you do not need any locking for the other calls), or use a read-write lock.
*/
typedef struct secp256k1_context_struct secp256k1_context;
/** Opaque data structure that holds a parsed and valid public key.
*
* The exact representation of data inside is implementation defined and not
* guaranteed to be portable between different platforms or versions. It is
* however guaranteed to be 64 bytes in size, and can be safely copied/moved.
* If you need to convert to a format suitable for storage, transmission, or
* comparison, use secp256k1_ec_pubkey_serialize and secp256k1_ec_pubkey_parse.
*/
typedef struct {
unsigned char data[64];
} secp256k1_pubkey;
/** Opaque data structured that holds a parsed ECDSA signature.
*
* The exact representation of data inside is implementation defined and not
* guaranteed to be portable between different platforms or versions. It is
* however guaranteed to be 64 bytes in size, and can be safely copied/moved.
* If you need to convert to a format suitable for storage, transmission, or
* comparison, use the secp256k1_ecdsa_signature_serialize_* and
* secp256k1_ecdsa_signature_serialize_* functions.
*/
typedef struct {
unsigned char data[64];
} secp256k1_ecdsa_signature;
/** A pointer to a function to deterministically generate a nonce.
*
* Returns: 1 if a nonce was successfully generated. 0 will cause signing to fail.
* Out: nonce32: pointer to a 32-byte array to be filled by the function.
* In: msg32: the 32-byte message hash being verified (will not be NULL)
* key32: pointer to a 32-byte secret key (will not be NULL)
* algo16: pointer to a 16-byte array describing the signature
* algorithm (will be NULL for ECDSA for compatibility).
* data: Arbitrary data pointer that is passed through.
* attempt: how many iterations we have tried to find a nonce.
* This will almost always be 0, but different attempt values
* are required to result in a different nonce.
*
* Except for test cases, this function should compute some cryptographic hash of
* the message, the algorithm, the key and the attempt.
*/
typedef int (*secp256k1_nonce_function)(
unsigned char *nonce32,
const unsigned char *msg32,
const unsigned char *key32,
const unsigned char *algo16,
void *data,
unsigned int attempt
);
# if !defined(SECP256K1_GNUC_PREREQ)
# if defined(__GNUC__)&&defined(__GNUC_MINOR__)
# define SECP256K1_GNUC_PREREQ(_maj,_min) \
((__GNUC__<<16)+__GNUC_MINOR__>=((_maj)<<16)+(_min))
# else
# define SECP256K1_GNUC_PREREQ(_maj,_min) 0
# endif
# endif
# if (!defined(__STDC_VERSION__) || (__STDC_VERSION__ < 199901L) )
# if SECP256K1_GNUC_PREREQ(2,7)
# define SECP256K1_INLINE __inline__
# elif (defined(_MSC_VER))
# define SECP256K1_INLINE __inline
# else
# define SECP256K1_INLINE
# endif
# else
# define SECP256K1_INLINE inline
# endif
#ifndef SECP256K1_API
# if defined(_WIN32)
# ifdef SECP256K1_BUILD
# define SECP256K1_API __declspec(dllexport)
# else
# define SECP256K1_API
# endif
# elif defined(__GNUC__) && defined(SECP256K1_BUILD)
# define SECP256K1_API __attribute__ ((visibility ("default")))
# else
# define SECP256K1_API
# endif
#endif
/**Warning attributes
* NONNULL is not used if SECP256K1_BUILD is set to avoid the compiler optimizing out
* some paranoid null checks. */
# if defined(__GNUC__) && SECP256K1_GNUC_PREREQ(3, 4)
# define SECP256K1_WARN_UNUSED_RESULT __attribute__ ((__warn_unused_result__))
# else
# define SECP256K1_WARN_UNUSED_RESULT
# endif
# if !defined(SECP256K1_BUILD) && defined(__GNUC__) && SECP256K1_GNUC_PREREQ(3, 4)
# define SECP256K1_ARG_NONNULL(_x) __attribute__ ((__nonnull__(_x)))
# else
# define SECP256K1_ARG_NONNULL(_x)
# endif
/** All flags' lower 8 bits indicate what they're for. Do not use directly. */
#define SECP256K1_FLAGS_TYPE_MASK ((1 << 8) - 1)
#define SECP256K1_FLAGS_TYPE_CONTEXT (1 << 0)
#define SECP256K1_FLAGS_TYPE_COMPRESSION (1 << 1)
/** The higher bits contain the actual data. Do not use directly. */
#define SECP256K1_FLAGS_BIT_CONTEXT_VERIFY (1 << 8)
#define SECP256K1_FLAGS_BIT_CONTEXT_SIGN (1 << 9)
#define SECP256K1_FLAGS_BIT_COMPRESSION (1 << 8)
/** Flags to pass to secp256k1_context_create. */
#define SECP256K1_CONTEXT_VERIFY (SECP256K1_FLAGS_TYPE_CONTEXT | SECP256K1_FLAGS_BIT_CONTEXT_VERIFY)
#define SECP256K1_CONTEXT_SIGN (SECP256K1_FLAGS_TYPE_CONTEXT | SECP256K1_FLAGS_BIT_CONTEXT_SIGN)
#define SECP256K1_CONTEXT_NONE (SECP256K1_FLAGS_TYPE_CONTEXT)
/** Flag to pass to secp256k1_ec_pubkey_serialize and secp256k1_ec_privkey_export. */
#define SECP256K1_EC_COMPRESSED (SECP256K1_FLAGS_TYPE_COMPRESSION | SECP256K1_FLAGS_BIT_COMPRESSION)
#define SECP256K1_EC_UNCOMPRESSED (SECP256K1_FLAGS_TYPE_COMPRESSION)
/** Create a secp256k1 context object.
*
* Returns: a newly created context object.
* In: flags: which parts of the context to initialize.
*/
SECP256K1_API secp256k1_context* secp256k1_context_create(
unsigned int flags
) SECP256K1_WARN_UNUSED_RESULT;
/** Copies a secp256k1 context object.
*
* Returns: a newly created context object.
* Args: ctx: an existing context to copy (cannot be NULL)
*/
SECP256K1_API secp256k1_context* secp256k1_context_clone(
const secp256k1_context* ctx
) SECP256K1_ARG_NONNULL(1) SECP256K1_WARN_UNUSED_RESULT;
/** Destroy a secp256k1 context object.
*
* The context pointer may not be used afterwards.
* Args: ctx: an existing context to destroy (cannot be NULL)
*/
SECP256K1_API void secp256k1_context_destroy(
secp256k1_context* ctx
);
/** Set a callback function to be called when an illegal argument is passed to
* an API call. It will only trigger for violations that are mentioned
* explicitly in the header.
*
* The philosophy is that these shouldn't be dealt with through a
* specific return value, as calling code should not have branches to deal with
* the case that this code itself is broken.
*
* On the other hand, during debug stage, one would want to be informed about
* such mistakes, and the default (crashing) may be inadvisable.
* When this callback is triggered, the API function called is guaranteed not
* to cause a crash, though its return value and output arguments are
* undefined.
*
* Args: ctx: an existing context object (cannot be NULL)
* In: fun: a pointer to a function to call when an illegal argument is
* passed to the API, taking a message and an opaque pointer
* (NULL restores a default handler that calls abort).
* data: the opaque pointer to pass to fun above.
*/
SECP256K1_API void secp256k1_context_set_illegal_callback(
secp256k1_context* ctx,
void (*fun)(const char* message, void* data),
const void* data
) SECP256K1_ARG_NONNULL(1);
/** Set a callback function to be called when an internal consistency check
* fails. The default is crashing.
*
* This can only trigger in case of a hardware failure, miscompilation,
* memory corruption, serious bug in the library, or other error would can
* otherwise result in undefined behaviour. It will not trigger due to mere
* incorrect usage of the API (see secp256k1_context_set_illegal_callback
* for that). After this callback returns, anything may happen, including
* crashing.
*
* Args: ctx: an existing context object (cannot be NULL)
* In: fun: a pointer to a function to call when an internal error occurs,
* taking a message and an opaque pointer (NULL restores a default
* handler that calls abort).
* data: the opaque pointer to pass to fun above.
*/
SECP256K1_API void secp256k1_context_set_error_callback(
secp256k1_context* ctx,
void (*fun)(const char* message, void* data),
const void* data
) SECP256K1_ARG_NONNULL(1);
/** Parse a variable-length public key into the pubkey object.
*
* Returns: 1 if the public key was fully valid.
* 0 if the public key could not be parsed or is invalid.
* Args: ctx: a secp256k1 context object.
* Out: pubkey: pointer to a pubkey object. If 1 is returned, it is set to a
* parsed version of input. If not, its value is undefined.
* In: input: pointer to a serialized public key
* inputlen: length of the array pointed to by input
*
* This function supports parsing compressed (33 bytes, header byte 0x02 or
* 0x03), uncompressed (65 bytes, header byte 0x04), or hybrid (65 bytes, header
* byte 0x06 or 0x07) format public keys.
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_parse(
const secp256k1_context* ctx,
secp256k1_pubkey* pubkey,
const unsigned char *input,
size_t inputlen
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Serialize a pubkey object into a serialized byte sequence.
*
* Returns: 1 always.
* Args: ctx: a secp256k1 context object.
* Out: output: a pointer to a 65-byte (if compressed==0) or 33-byte (if
* compressed==1) byte array to place the serialized key
* in.
* In/Out: outputlen: a pointer to an integer which is initially set to the
* size of output, and is overwritten with the written
* size.
* In: pubkey: a pointer to a secp256k1_pubkey containing an
* initialized public key.
* flags: SECP256K1_EC_COMPRESSED if serialization should be in
* compressed format, otherwise SECP256K1_EC_UNCOMPRESSED.
*/
SECP256K1_API int secp256k1_ec_pubkey_serialize(
const secp256k1_context* ctx,
unsigned char *output,
size_t *outputlen,
const secp256k1_pubkey* pubkey,
unsigned int flags
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Parse an ECDSA signature in compact (64 bytes) format.
*
* Returns: 1 when the signature could be parsed, 0 otherwise.
* Args: ctx: a secp256k1 context object
* Out: sig: a pointer to a signature object
* In: input64: a pointer to the 64-byte array to parse
*
* The signature must consist of a 32-byte big endian R value, followed by a
* 32-byte big endian S value. If R or S fall outside of [0..order-1], the
* encoding is invalid. R and S with value 0 are allowed in the encoding.
*
* After the call, sig will always be initialized. If parsing failed or R or
* S are zero, the resulting sig value is guaranteed to fail validation for any
* message and public key.
*/
SECP256K1_API int secp256k1_ecdsa_signature_parse_compact(
const secp256k1_context* ctx,
secp256k1_ecdsa_signature* sig,
const unsigned char *input64
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Parse a DER ECDSA signature.
*
* Returns: 1 when the signature could be parsed, 0 otherwise.
* Args: ctx: a secp256k1 context object
* Out: sig: a pointer to a signature object
* In: input: a pointer to the signature to be parsed
* inputlen: the length of the array pointed to be input
*
* This function will accept any valid DER encoded signature, even if the
* encoded numbers are out of range.
*
* After the call, sig will always be initialized. If parsing failed or the
* encoded numbers are out of range, signature validation with it is
* guaranteed to fail for every message and public key.
*/
SECP256K1_API int secp256k1_ecdsa_signature_parse_der(
const secp256k1_context* ctx,
secp256k1_ecdsa_signature* sig,
const unsigned char *input,
size_t inputlen
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Serialize an ECDSA signature in DER format.
*
* Returns: 1 if enough space was available to serialize, 0 otherwise
* Args: ctx: a secp256k1 context object
* Out: output: a pointer to an array to store the DER serialization
* In/Out: outputlen: a pointer to a length integer. Initially, this integer
* should be set to the length of output. After the call
* it will be set to the length of the serialization (even
* if 0 was returned).
* In: sig: a pointer to an initialized signature object
*/
SECP256K1_API int secp256k1_ecdsa_signature_serialize_der(
const secp256k1_context* ctx,
unsigned char *output,
size_t *outputlen,
const secp256k1_ecdsa_signature* sig
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Serialize an ECDSA signature in compact (64 byte) format.
*
* Returns: 1
* Args: ctx: a secp256k1 context object
* Out: output64: a pointer to a 64-byte array to store the compact serialization
* In: sig: a pointer to an initialized signature object
*
* See secp256k1_ecdsa_signature_parse_compact for details about the encoding.
*/
SECP256K1_API int secp256k1_ecdsa_signature_serialize_compact(
const secp256k1_context* ctx,
unsigned char *output64,
const secp256k1_ecdsa_signature* sig
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Verify an ECDSA signature.
*
* Returns: 1: correct signature
* 0: incorrect or unparseable signature
* Args: ctx: a secp256k1 context object, initialized for verification.
* In: sig: the signature being verified (cannot be NULL)
* msg32: the 32-byte message hash being verified (cannot be NULL)
* pubkey: pointer to an initialized public key to verify with (cannot be NULL)
*
* To avoid accepting malleable signatures, only ECDSA signatures in lower-S
* form are accepted.
*
* If you need to accept ECDSA signatures from sources that do not obey this
* rule, apply secp256k1_ecdsa_signature_normalize to the signature prior to
* validation, but be aware that doing so results in malleable signatures.
*
* For details, see the comments for that function.
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ecdsa_verify(
const secp256k1_context* ctx,
const secp256k1_ecdsa_signature *sig,
const unsigned char *msg32,
const secp256k1_pubkey *pubkey
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Convert a signature to a normalized lower-S form.
*
* Returns: 1 if sigin was not normalized, 0 if it already was.
* Args: ctx: a secp256k1 context object
* Out: sigout: a pointer to a signature to fill with the normalized form,
* or copy if the input was already normalized. (can be NULL if
* you're only interested in whether the input was already
* normalized).
* In: sigin: a pointer to a signature to check/normalize (cannot be NULL,
* can be identical to sigout)
*
* With ECDSA a third-party can forge a second distinct signature of the same
* message, given a single initial signature, but without knowing the key. This
* is done by negating the S value modulo the order of the curve, 'flipping'
* the sign of the random point R which is not included in the signature.
*
* Forgery of the same message isn't universally problematic, but in systems
* where message malleability or uniqueness of signatures is important this can
* cause issues. This forgery can be blocked by all verifiers forcing signers
* to use a normalized form.
*
* The lower-S form reduces the size of signatures slightly on average when
* variable length encodings (such as DER) are used and is cheap to verify,
* making it a good choice. Security of always using lower-S is assured because
* anyone can trivially modify a signature after the fact to enforce this
* property anyway.
*
* The lower S value is always between 0x1 and
* 0x7FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5D576E7357A4501DDFE92F46681B20A0,
* inclusive.
*
* No other forms of ECDSA malleability are known and none seem likely, but
* there is no formal proof that ECDSA, even with this additional restriction,
* is free of other malleability. Commonly used serialization schemes will also
* accept various non-unique encodings, so care should be taken when this
* property is required for an application.
*
* The secp256k1_ecdsa_sign function will by default create signatures in the
* lower-S form, and secp256k1_ecdsa_verify will not accept others. In case
* signatures come from a system that cannot enforce this property,
* secp256k1_ecdsa_signature_normalize must be called before verification.
*/
SECP256K1_API int secp256k1_ecdsa_signature_normalize(
const secp256k1_context* ctx,
secp256k1_ecdsa_signature *sigout,
const secp256k1_ecdsa_signature *sigin
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(3);
/** An implementation of RFC6979 (using HMAC-SHA256) as nonce generation function.
* If a data pointer is passed, it is assumed to be a pointer to 32 bytes of
* extra entropy.
*/
SECP256K1_API extern const secp256k1_nonce_function secp256k1_nonce_function_rfc6979;
/** A default safe nonce generation function (currently equal to secp256k1_nonce_function_rfc6979). */
SECP256K1_API extern const secp256k1_nonce_function secp256k1_nonce_function_default;
/** Create an ECDSA signature.
*
* Returns: 1: signature created
* 0: the nonce generation function failed, or the private key was invalid.
* Args: ctx: pointer to a context object, initialized for signing (cannot be NULL)
* Out: sig: pointer to an array where the signature will be placed (cannot be NULL)
* In: msg32: the 32-byte message hash being signed (cannot be NULL)
* seckey: pointer to a 32-byte secret key (cannot be NULL)
* noncefp:pointer to a nonce generation function. If NULL, secp256k1_nonce_function_default is used
* ndata: pointer to arbitrary data used by the nonce generation function (can be NULL)
*
* The created signature is always in lower-S form. See
* secp256k1_ecdsa_signature_normalize for more details.
*/
SECP256K1_API int secp256k1_ecdsa_sign(
const secp256k1_context* ctx,
secp256k1_ecdsa_signature *sig,
const unsigned char *msg32,
const unsigned char *seckey,
secp256k1_nonce_function noncefp,
const void *ndata
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Verify an ECDSA secret key.
*
* Returns: 1: secret key is valid
* 0: secret key is invalid
* Args: ctx: pointer to a context object (cannot be NULL)
* In: seckey: pointer to a 32-byte secret key (cannot be NULL)
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_seckey_verify(
const secp256k1_context* ctx,
const unsigned char *seckey
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2);
/** Compute the public key for a secret key.
*
* Returns: 1: secret was valid, public key stores
* 0: secret was invalid, try again
* Args: ctx: pointer to a context object, initialized for signing (cannot be NULL)
* Out: pubkey: pointer to the created public key (cannot be NULL)
* In: seckey: pointer to a 32-byte private key (cannot be NULL)
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_create(
const secp256k1_context* ctx,
secp256k1_pubkey *pubkey,
const unsigned char *seckey
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Tweak a private key by adding tweak to it.
* Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for
* uniformly random 32-byte arrays, or if the resulting private key
* would be invalid (only when the tweak is the complement of the
* private key). 1 otherwise.
* Args: ctx: pointer to a context object (cannot be NULL).
* In/Out: seckey: pointer to a 32-byte private key.
* In: tweak: pointer to a 32-byte tweak.
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_privkey_tweak_add(
const secp256k1_context* ctx,
unsigned char *seckey,
const unsigned char *tweak
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Tweak a public key by adding tweak times the generator to it.
* Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for
* uniformly random 32-byte arrays, or if the resulting public key
* would be invalid (only when the tweak is the complement of the
* corresponding private key). 1 otherwise.
* Args: ctx: pointer to a context object initialized for validation
* (cannot be NULL).
* In/Out: pubkey: pointer to a public key object.
* In: tweak: pointer to a 32-byte tweak.
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_tweak_add(
const secp256k1_context* ctx,
secp256k1_pubkey *pubkey,
const unsigned char *tweak
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Tweak a private key by multiplying it by a tweak.
* Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for
* uniformly random 32-byte arrays, or equal to zero. 1 otherwise.
* Args: ctx: pointer to a context object (cannot be NULL).
* In/Out: seckey: pointer to a 32-byte private key.
* In: tweak: pointer to a 32-byte tweak.
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_privkey_tweak_mul(
const secp256k1_context* ctx,
unsigned char *seckey,
const unsigned char *tweak
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Tweak a public key by multiplying it by a tweak value.
* Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for
* uniformly random 32-byte arrays, or equal to zero. 1 otherwise.
* Args: ctx: pointer to a context object initialized for validation
* (cannot be NULL).
* In/Out: pubkey: pointer to a public key obkect.
* In: tweak: pointer to a 32-byte tweak.
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_tweak_mul(
const secp256k1_context* ctx,
secp256k1_pubkey *pubkey,
const unsigned char *tweak
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Updates the context randomization.
* Returns: 1: randomization successfully updated
* 0: error
* Args: ctx: pointer to a context object (cannot be NULL)
* In: seed32: pointer to a 32-byte random seed (NULL resets to initial state)
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_context_randomize(
secp256k1_context* ctx,
const unsigned char *seed32
) SECP256K1_ARG_NONNULL(1);
/** Add a number of public keys together.
* Returns: 1: the sum of the public keys is valid.
* 0: the sum of the public keys is not valid.
* Args: ctx: pointer to a context object
* Out: out: pointer to a public key object for placing the resulting public key
* (cannot be NULL)
* In: ins: pointer to array of pointers to public keys (cannot be NULL)
* n: the number of public keys to add together (must be at least 1)
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_combine(
const secp256k1_context* ctx,
secp256k1_pubkey *out,
const secp256k1_pubkey * const * ins,
size_t n
) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
# ifdef __cplusplus
}
# endif
#endif

View File

@@ -0,0 +1,31 @@
#ifndef _SECP256K1_ECDH_
# define _SECP256K1_ECDH_
# include "secp256k1.h"
# ifdef __cplusplus
extern "C" {
# endif
/** Compute an EC Diffie-Hellman secret in constant time
* Returns: 1: exponentiation was successful
* 0: scalar was invalid (zero or overflow)
* Args: ctx: pointer to a context object (cannot be NULL)
* Out: result: a 32-byte array which will be populated by an ECDH
* secret computed from the point and scalar
* In: pubkey: a pointer to a secp256k1_pubkey containing an
* initialized public key
* privkey: a 32-byte scalar with which to multiply the point
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ecdh(
const secp256k1_context* ctx,
unsigned char *result,
const secp256k1_pubkey *pubkey,
const unsigned char *privkey
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
# ifdef __cplusplus
}
# endif
#endif

View File

@@ -0,0 +1,110 @@
#ifndef _SECP256K1_RECOVERY_
# define _SECP256K1_RECOVERY_
# include "secp256k1.h"
# ifdef __cplusplus
extern "C" {
# endif
/** Opaque data structured that holds a parsed ECDSA signature,
* supporting pubkey recovery.
*
* The exact representation of data inside is implementation defined and not
* guaranteed to be portable between different platforms or versions. It is
* however guaranteed to be 65 bytes in size, and can be safely copied/moved.
* If you need to convert to a format suitable for storage or transmission, use
* the secp256k1_ecdsa_signature_serialize_* and
* secp256k1_ecdsa_signature_parse_* functions.
*
* Furthermore, it is guaranteed that identical signatures (including their
* recoverability) will have identical representation, so they can be
* memcmp'ed.
*/
typedef struct {
unsigned char data[65];
} secp256k1_ecdsa_recoverable_signature;
/** Parse a compact ECDSA signature (64 bytes + recovery id).
*
* Returns: 1 when the signature could be parsed, 0 otherwise
* Args: ctx: a secp256k1 context object
* Out: sig: a pointer to a signature object
* In: input64: a pointer to a 64-byte compact signature
* recid: the recovery id (0, 1, 2 or 3)
*/
SECP256K1_API int secp256k1_ecdsa_recoverable_signature_parse_compact(
const secp256k1_context* ctx,
secp256k1_ecdsa_recoverable_signature* sig,
const unsigned char *input64,
int recid
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Convert a recoverable signature into a normal signature.
*
* Returns: 1
* Out: sig: a pointer to a normal signature (cannot be NULL).
* In: sigin: a pointer to a recoverable signature (cannot be NULL).
*/
SECP256K1_API int secp256k1_ecdsa_recoverable_signature_convert(
const secp256k1_context* ctx,
secp256k1_ecdsa_signature* sig,
const secp256k1_ecdsa_recoverable_signature* sigin
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
/** Serialize an ECDSA signature in compact format (64 bytes + recovery id).
*
* Returns: 1
* Args: ctx: a secp256k1 context object
* Out: output64: a pointer to a 64-byte array of the compact signature (cannot be NULL)
* recid: a pointer to an integer to hold the recovery id (can be NULL).
* In: sig: a pointer to an initialized signature object (cannot be NULL)
*/
SECP256K1_API int secp256k1_ecdsa_recoverable_signature_serialize_compact(
const secp256k1_context* ctx,
unsigned char *output64,
int *recid,
const secp256k1_ecdsa_recoverable_signature* sig
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Create a recoverable ECDSA signature.
*
* Returns: 1: signature created
* 0: the nonce generation function failed, or the private key was invalid.
* Args: ctx: pointer to a context object, initialized for signing (cannot be NULL)
* Out: sig: pointer to an array where the signature will be placed (cannot be NULL)
* In: msg32: the 32-byte message hash being signed (cannot be NULL)
* seckey: pointer to a 32-byte secret key (cannot be NULL)
* noncefp:pointer to a nonce generation function. If NULL, secp256k1_nonce_function_default is used
* ndata: pointer to arbitrary data used by the nonce generation function (can be NULL)
*/
SECP256K1_API int secp256k1_ecdsa_sign_recoverable(
const secp256k1_context* ctx,
secp256k1_ecdsa_recoverable_signature *sig,
const unsigned char *msg32,
const unsigned char *seckey,
secp256k1_nonce_function noncefp,
const void *ndata
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
/** Recover an ECDSA public key from a signature.
*
* Returns: 1: public key successfully recovered (which guarantees a correct signature).
* 0: otherwise.
* Args: ctx: pointer to a context object, initialized for verification (cannot be NULL)
* Out: pubkey: pointer to the recovered public key (cannot be NULL)
* In: sig: pointer to initialized signature that supports pubkey recovery (cannot be NULL)
* msg32: the 32-byte message hash assumed to be signed (cannot be NULL)
*/
SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ecdsa_recover(
const secp256k1_context* ctx,
secp256k1_pubkey *pubkey,
const secp256k1_ecdsa_recoverable_signature *sig,
const unsigned char *msg32
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
# ifdef __cplusplus
}
# endif
#endif

View File

@@ -0,0 +1,13 @@
prefix=@prefix@
exec_prefix=@exec_prefix@
libdir=@libdir@
includedir=@includedir@
Name: libsecp256k1
Description: Optimized C library for EC operations on curve secp256k1
URL: https://github.com/bitcoin-core/secp256k1
Version: @PACKAGE_VERSION@
Cflags: -I${includedir}
Libs.private: @SECP_LIBS@
Libs: -L${libdir} -lsecp256k1

View File

@@ -0,0 +1,322 @@
# This code supports verifying group implementations which have branches
# or conditional statements (like cmovs), by allowing each execution path
# to independently set assumptions on input or intermediary variables.
#
# The general approach is:
# * A constraint is a tuple of two sets of of symbolic expressions:
# the first of which are required to evaluate to zero, the second of which
# are required to evaluate to nonzero.
# - A constraint is said to be conflicting if any of its nonzero expressions
# is in the ideal with basis the zero expressions (in other words: when the
# zero expressions imply that one of the nonzero expressions are zero).
# * There is a list of laws that describe the intended behaviour, including
# laws for addition and doubling. Each law is called with the symbolic point
# coordinates as arguments, and returns:
# - A constraint describing the assumptions under which it is applicable,
# called "assumeLaw"
# - A constraint describing the requirements of the law, called "require"
# * Implementations are transliterated into functions that operate as well on
# algebraic input points, and are called once per combination of branches
# exectured. Each execution returns:
# - A constraint describing the assumptions this implementation requires
# (such as Z1=1), called "assumeFormula"
# - A constraint describing the assumptions this specific branch requires,
# but which is by construction guaranteed to cover the entire space by
# merging the results from all branches, called "assumeBranch"
# - The result of the computation
# * All combinations of laws with implementation branches are tried, and:
# - If the combination of assumeLaw, assumeFormula, and assumeBranch results
# in a conflict, it means this law does not apply to this branch, and it is
# skipped.
# - For others, we try to prove the require constraints hold, assuming the
# information in assumeLaw + assumeFormula + assumeBranch, and if this does
# not succeed, we fail.
# + To prove an expression is zero, we check whether it belongs to the
# ideal with the assumed zero expressions as basis. This test is exact.
# + To prove an expression is nonzero, we check whether each of its
# factors is contained in the set of nonzero assumptions' factors.
# This test is not exact, so various combinations of original and
# reduced expressions' factors are tried.
# - If we succeed, we print out the assumptions from assumeFormula that
# weren't implied by assumeLaw already. Those from assumeBranch are skipped,
# as we assume that all constraints in it are complementary with each other.
#
# Based on the sage verification scripts used in the Explicit-Formulas Database
# by Tanja Lange and others, see http://hyperelliptic.org/EFD
class fastfrac:
"""Fractions over rings."""
def __init__(self,R,top,bot=1):
"""Construct a fractional, given a ring, a numerator, and denominator."""
self.R = R
if parent(top) == ZZ or parent(top) == R:
self.top = R(top)
self.bot = R(bot)
elif top.__class__ == fastfrac:
self.top = top.top
self.bot = top.bot * bot
else:
self.top = R(numerator(top))
self.bot = R(denominator(top)) * bot
def iszero(self,I):
"""Return whether this fraction is zero given an ideal."""
return self.top in I and self.bot not in I
def reduce(self,assumeZero):
zero = self.R.ideal(map(numerator, assumeZero))
return fastfrac(self.R, zero.reduce(self.top)) / fastfrac(self.R, zero.reduce(self.bot))
def __add__(self,other):
"""Add two fractions."""
if parent(other) == ZZ:
return fastfrac(self.R,self.top + self.bot * other,self.bot)
if other.__class__ == fastfrac:
return fastfrac(self.R,self.top * other.bot + self.bot * other.top,self.bot * other.bot)
return NotImplemented
def __sub__(self,other):
"""Subtract two fractions."""
if parent(other) == ZZ:
return fastfrac(self.R,self.top - self.bot * other,self.bot)
if other.__class__ == fastfrac:
return fastfrac(self.R,self.top * other.bot - self.bot * other.top,self.bot * other.bot)
return NotImplemented
def __neg__(self):
"""Return the negation of a fraction."""
return fastfrac(self.R,-self.top,self.bot)
def __mul__(self,other):
"""Multiply two fractions."""
if parent(other) == ZZ:
return fastfrac(self.R,self.top * other,self.bot)
if other.__class__ == fastfrac:
return fastfrac(self.R,self.top * other.top,self.bot * other.bot)
return NotImplemented
def __rmul__(self,other):
"""Multiply something else with a fraction."""
return self.__mul__(other)
def __div__(self,other):
"""Divide two fractions."""
if parent(other) == ZZ:
return fastfrac(self.R,self.top,self.bot * other)
if other.__class__ == fastfrac:
return fastfrac(self.R,self.top * other.bot,self.bot * other.top)
return NotImplemented
def __pow__(self,other):
"""Compute a power of a fraction."""
if parent(other) == ZZ:
if other < 0:
# Negative powers require flipping top and bottom
return fastfrac(self.R,self.bot ^ (-other),self.top ^ (-other))
else:
return fastfrac(self.R,self.top ^ other,self.bot ^ other)
return NotImplemented
def __str__(self):
return "fastfrac((" + str(self.top) + ") / (" + str(self.bot) + "))"
def __repr__(self):
return "%s" % self
def numerator(self):
return self.top
class constraints:
"""A set of constraints, consisting of zero and nonzero expressions.
Constraints can either be used to express knowledge or a requirement.
Both the fields zero and nonzero are maps from expressions to description
strings. The expressions that are the keys in zero are required to be zero,
and the expressions that are the keys in nonzero are required to be nonzero.
Note that (a != 0) and (b != 0) is the same as (a*b != 0), so all keys in
nonzero could be multiplied into a single key. This is often much less
efficient to work with though, so we keep them separate inside the
constraints. This allows higher-level code to do fast checks on the individual
nonzero elements, or combine them if needed for stronger checks.
We can't multiply the different zero elements, as it would suffice for one of
the factors to be zero, instead of all of them. Instead, the zero elements are
typically combined into an ideal first.
"""
def __init__(self, **kwargs):
if 'zero' in kwargs:
self.zero = dict(kwargs['zero'])
else:
self.zero = dict()
if 'nonzero' in kwargs:
self.nonzero = dict(kwargs['nonzero'])
else:
self.nonzero = dict()
def negate(self):
return constraints(zero=self.nonzero, nonzero=self.zero)
def __add__(self, other):
zero = self.zero.copy()
zero.update(other.zero)
nonzero = self.nonzero.copy()
nonzero.update(other.nonzero)
return constraints(zero=zero, nonzero=nonzero)
def __str__(self):
return "constraints(zero=%s,nonzero=%s)" % (self.zero, self.nonzero)
def __repr__(self):
return "%s" % self
def conflicts(R, con):
"""Check whether any of the passed non-zero assumptions is implied by the zero assumptions"""
zero = R.ideal(map(numerator, con.zero))
if 1 in zero:
return True
# First a cheap check whether any of the individual nonzero terms conflict on
# their own.
for nonzero in con.nonzero:
if nonzero.iszero(zero):
return True
# It can be the case that entries in the nonzero set do not individually
# conflict with the zero set, but their combination does. For example, knowing
# that either x or y is zero is equivalent to having x*y in the zero set.
# Having x or y individually in the nonzero set is not a conflict, but both
# simultaneously is, so that is the right thing to check for.
if reduce(lambda a,b: a * b, con.nonzero, fastfrac(R, 1)).iszero(zero):
return True
return False
def get_nonzero_set(R, assume):
"""Calculate a simple set of nonzero expressions"""
zero = R.ideal(map(numerator, assume.zero))
nonzero = set()
for nz in map(numerator, assume.nonzero):
for (f,n) in nz.factor():
nonzero.add(f)
rnz = zero.reduce(nz)
for (f,n) in rnz.factor():
nonzero.add(f)
return nonzero
def prove_nonzero(R, exprs, assume):
"""Check whether an expression is provably nonzero, given assumptions"""
zero = R.ideal(map(numerator, assume.zero))
nonzero = get_nonzero_set(R, assume)
expl = set()
ok = True
for expr in exprs:
if numerator(expr) in zero:
return (False, [exprs[expr]])
allexprs = reduce(lambda a,b: numerator(a)*numerator(b), exprs, 1)
for (f, n) in allexprs.factor():
if f not in nonzero:
ok = False
if ok:
return (True, None)
ok = True
for (f, n) in zero.reduce(numerator(allexprs)).factor():
if f not in nonzero:
ok = False
if ok:
return (True, None)
ok = True
for expr in exprs:
for (f,n) in numerator(expr).factor():
if f not in nonzero:
ok = False
if ok:
return (True, None)
ok = True
for expr in exprs:
for (f,n) in zero.reduce(numerator(expr)).factor():
if f not in nonzero:
expl.add(exprs[expr])
if expl:
return (False, list(expl))
else:
return (True, None)
def prove_zero(R, exprs, assume):
"""Check whether all of the passed expressions are provably zero, given assumptions"""
r, e = prove_nonzero(R, dict(map(lambda x: (fastfrac(R, x.bot, 1), exprs[x]), exprs)), assume)
if not r:
return (False, map(lambda x: "Possibly zero denominator: %s" % x, e))
zero = R.ideal(map(numerator, assume.zero))
nonzero = prod(x for x in assume.nonzero)
expl = []
for expr in exprs:
if not expr.iszero(zero):
expl.append(exprs[expr])
if not expl:
return (True, None)
return (False, expl)
def describe_extra(R, assume, assumeExtra):
"""Describe what assumptions are added, given existing assumptions"""
zerox = assume.zero.copy()
zerox.update(assumeExtra.zero)
zero = R.ideal(map(numerator, assume.zero))
zeroextra = R.ideal(map(numerator, zerox))
nonzero = get_nonzero_set(R, assume)
ret = set()
# Iterate over the extra zero expressions
for base in assumeExtra.zero:
if base not in zero:
add = []
for (f, n) in numerator(base).factor():
if f not in nonzero:
add += ["%s" % f]
if add:
ret.add((" * ".join(add)) + " = 0 [%s]" % assumeExtra.zero[base])
# Iterate over the extra nonzero expressions
for nz in assumeExtra.nonzero:
nzr = zeroextra.reduce(numerator(nz))
if nzr not in zeroextra:
for (f,n) in nzr.factor():
if zeroextra.reduce(f) not in nonzero:
ret.add("%s != 0" % zeroextra.reduce(f))
return ", ".join(x for x in ret)
def check_symbolic(R, assumeLaw, assumeAssert, assumeBranch, require):
"""Check a set of zero and nonzero requirements, given a set of zero and nonzero assumptions"""
assume = assumeLaw + assumeAssert + assumeBranch
if conflicts(R, assume):
# This formula does not apply
return None
describe = describe_extra(R, assumeLaw + assumeBranch, assumeAssert)
ok, msg = prove_zero(R, require.zero, assume)
if not ok:
return "FAIL, %s fails (assuming %s)" % (str(msg), describe)
res, expl = prove_nonzero(R, require.nonzero, assume)
if not res:
return "FAIL, %s fails (assuming %s)" % (str(expl), describe)
if describe != "":
return "OK (assuming %s)" % describe
else:
return "OK"
def concrete_verify(c):
for k in c.zero:
if k != 0:
return (False, c.zero[k])
for k in c.nonzero:
if k == 0:
return (False, c.nonzero[k])
return (True, None)

View File

@@ -0,0 +1,306 @@
# Test libsecp256k1' group operation implementations using prover.sage
import sys
load("group_prover.sage")
load("weierstrass_prover.sage")
def formula_secp256k1_gej_double_var(a):
"""libsecp256k1's secp256k1_gej_double_var, used by various addition functions"""
rz = a.Z * a.Y
rz = rz * 2
t1 = a.X^2
t1 = t1 * 3
t2 = t1^2
t3 = a.Y^2
t3 = t3 * 2
t4 = t3^2
t4 = t4 * 2
t3 = t3 * a.X
rx = t3
rx = rx * 4
rx = -rx
rx = rx + t2
t2 = -t2
t3 = t3 * 6
t3 = t3 + t2
ry = t1 * t3
t2 = -t4
ry = ry + t2
return jacobianpoint(rx, ry, rz)
def formula_secp256k1_gej_add_var(branch, a, b):
"""libsecp256k1's secp256k1_gej_add_var"""
if branch == 0:
return (constraints(), constraints(nonzero={a.Infinity : 'a_infinite'}), b)
if branch == 1:
return (constraints(), constraints(zero={a.Infinity : 'a_finite'}, nonzero={b.Infinity : 'b_infinite'}), a)
z22 = b.Z^2
z12 = a.Z^2
u1 = a.X * z22
u2 = b.X * z12
s1 = a.Y * z22
s1 = s1 * b.Z
s2 = b.Y * z12
s2 = s2 * a.Z
h = -u1
h = h + u2
i = -s1
i = i + s2
if branch == 2:
r = formula_secp256k1_gej_double_var(a)
return (constraints(), constraints(zero={h : 'h=0', i : 'i=0', a.Infinity : 'a_finite', b.Infinity : 'b_finite'}), r)
if branch == 3:
return (constraints(), constraints(zero={h : 'h=0', a.Infinity : 'a_finite', b.Infinity : 'b_finite'}, nonzero={i : 'i!=0'}), point_at_infinity())
i2 = i^2
h2 = h^2
h3 = h2 * h
h = h * b.Z
rz = a.Z * h
t = u1 * h2
rx = t
rx = rx * 2
rx = rx + h3
rx = -rx
rx = rx + i2
ry = -rx
ry = ry + t
ry = ry * i
h3 = h3 * s1
h3 = -h3
ry = ry + h3
return (constraints(), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite'}, nonzero={h : 'h!=0'}), jacobianpoint(rx, ry, rz))
def formula_secp256k1_gej_add_ge_var(branch, a, b):
"""libsecp256k1's secp256k1_gej_add_ge_var, which assume bz==1"""
if branch == 0:
return (constraints(zero={b.Z - 1 : 'b.z=1'}), constraints(nonzero={a.Infinity : 'a_infinite'}), b)
if branch == 1:
return (constraints(zero={b.Z - 1 : 'b.z=1'}), constraints(zero={a.Infinity : 'a_finite'}, nonzero={b.Infinity : 'b_infinite'}), a)
z12 = a.Z^2
u1 = a.X
u2 = b.X * z12
s1 = a.Y
s2 = b.Y * z12
s2 = s2 * a.Z
h = -u1
h = h + u2
i = -s1
i = i + s2
if (branch == 2):
r = formula_secp256k1_gej_double_var(a)
return (constraints(zero={b.Z - 1 : 'b.z=1'}), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite', h : 'h=0', i : 'i=0'}), r)
if (branch == 3):
return (constraints(zero={b.Z - 1 : 'b.z=1'}), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite', h : 'h=0'}, nonzero={i : 'i!=0'}), point_at_infinity())
i2 = i^2
h2 = h^2
h3 = h * h2
rz = a.Z * h
t = u1 * h2
rx = t
rx = rx * 2
rx = rx + h3
rx = -rx
rx = rx + i2
ry = -rx
ry = ry + t
ry = ry * i
h3 = h3 * s1
h3 = -h3
ry = ry + h3
return (constraints(zero={b.Z - 1 : 'b.z=1'}), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite'}, nonzero={h : 'h!=0'}), jacobianpoint(rx, ry, rz))
def formula_secp256k1_gej_add_zinv_var(branch, a, b):
"""libsecp256k1's secp256k1_gej_add_zinv_var"""
bzinv = b.Z^(-1)
if branch == 0:
return (constraints(), constraints(nonzero={b.Infinity : 'b_infinite'}), a)
if branch == 1:
bzinv2 = bzinv^2
bzinv3 = bzinv2 * bzinv
rx = b.X * bzinv2
ry = b.Y * bzinv3
rz = 1
return (constraints(), constraints(zero={b.Infinity : 'b_finite'}, nonzero={a.Infinity : 'a_infinite'}), jacobianpoint(rx, ry, rz))
azz = a.Z * bzinv
z12 = azz^2
u1 = a.X
u2 = b.X * z12
s1 = a.Y
s2 = b.Y * z12
s2 = s2 * azz
h = -u1
h = h + u2
i = -s1
i = i + s2
if branch == 2:
r = formula_secp256k1_gej_double_var(a)
return (constraints(), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite', h : 'h=0', i : 'i=0'}), r)
if branch == 3:
return (constraints(), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite', h : 'h=0'}, nonzero={i : 'i!=0'}), point_at_infinity())
i2 = i^2
h2 = h^2
h3 = h * h2
rz = a.Z
rz = rz * h
t = u1 * h2
rx = t
rx = rx * 2
rx = rx + h3
rx = -rx
rx = rx + i2
ry = -rx
ry = ry + t
ry = ry * i
h3 = h3 * s1
h3 = -h3
ry = ry + h3
return (constraints(), constraints(zero={a.Infinity : 'a_finite', b.Infinity : 'b_finite'}, nonzero={h : 'h!=0'}), jacobianpoint(rx, ry, rz))
def formula_secp256k1_gej_add_ge(branch, a, b):
"""libsecp256k1's secp256k1_gej_add_ge"""
zeroes = {}
nonzeroes = {}
a_infinity = False
if (branch & 4) != 0:
nonzeroes.update({a.Infinity : 'a_infinite'})
a_infinity = True
else:
zeroes.update({a.Infinity : 'a_finite'})
zz = a.Z^2
u1 = a.X
u2 = b.X * zz
s1 = a.Y
s2 = b.Y * zz
s2 = s2 * a.Z
t = u1
t = t + u2
m = s1
m = m + s2
rr = t^2
m_alt = -u2
tt = u1 * m_alt
rr = rr + tt
degenerate = (branch & 3) == 3
if (branch & 1) != 0:
zeroes.update({m : 'm_zero'})
else:
nonzeroes.update({m : 'm_nonzero'})
if (branch & 2) != 0:
zeroes.update({rr : 'rr_zero'})
else:
nonzeroes.update({rr : 'rr_nonzero'})
rr_alt = s1
rr_alt = rr_alt * 2
m_alt = m_alt + u1
if not degenerate:
rr_alt = rr
m_alt = m
n = m_alt^2
q = n * t
n = n^2
if degenerate:
n = m
t = rr_alt^2
rz = a.Z * m_alt
infinity = False
if (branch & 8) != 0:
if not a_infinity:
infinity = True
zeroes.update({rz : 'r.z=0'})
else:
nonzeroes.update({rz : 'r.z!=0'})
rz = rz * 2
q = -q
t = t + q
rx = t
t = t * 2
t = t + q
t = t * rr_alt
t = t + n
ry = -t
rx = rx * 4
ry = ry * 4
if a_infinity:
rx = b.X
ry = b.Y
rz = 1
if infinity:
return (constraints(zero={b.Z - 1 : 'b.z=1', b.Infinity : 'b_finite'}), constraints(zero=zeroes, nonzero=nonzeroes), point_at_infinity())
return (constraints(zero={b.Z - 1 : 'b.z=1', b.Infinity : 'b_finite'}), constraints(zero=zeroes, nonzero=nonzeroes), jacobianpoint(rx, ry, rz))
def formula_secp256k1_gej_add_ge_old(branch, a, b):
"""libsecp256k1's old secp256k1_gej_add_ge, which fails when ay+by=0 but ax!=bx"""
a_infinity = (branch & 1) != 0
zero = {}
nonzero = {}
if a_infinity:
nonzero.update({a.Infinity : 'a_infinite'})
else:
zero.update({a.Infinity : 'a_finite'})
zz = a.Z^2
u1 = a.X
u2 = b.X * zz
s1 = a.Y
s2 = b.Y * zz
s2 = s2 * a.Z
z = a.Z
t = u1
t = t + u2
m = s1
m = m + s2
n = m^2
q = n * t
n = n^2
rr = t^2
t = u1 * u2
t = -t
rr = rr + t
t = rr^2
rz = m * z
infinity = False
if (branch & 2) != 0:
if not a_infinity:
infinity = True
else:
return (constraints(zero={b.Z - 1 : 'b.z=1', b.Infinity : 'b_finite'}), constraints(nonzero={z : 'conflict_a'}, zero={z : 'conflict_b'}), point_at_infinity())
zero.update({rz : 'r.z=0'})
else:
nonzero.update({rz : 'r.z!=0'})
rz = rz * (0 if a_infinity else 2)
rx = t
q = -q
rx = rx + q
q = q * 3
t = t * 2
t = t + q
t = t * rr
t = t + n
ry = -t
rx = rx * (0 if a_infinity else 4)
ry = ry * (0 if a_infinity else 4)
t = b.X
t = t * (1 if a_infinity else 0)
rx = rx + t
t = b.Y
t = t * (1 if a_infinity else 0)
ry = ry + t
t = (1 if a_infinity else 0)
rz = rz + t
if infinity:
return (constraints(zero={b.Z - 1 : 'b.z=1', b.Infinity : 'b_finite'}), constraints(zero=zero, nonzero=nonzero), point_at_infinity())
return (constraints(zero={b.Z - 1 : 'b.z=1', b.Infinity : 'b_finite'}), constraints(zero=zero, nonzero=nonzero), jacobianpoint(rx, ry, rz))
if __name__ == "__main__":
check_symbolic_jacobian_weierstrass("secp256k1_gej_add_var", 0, 7, 5, formula_secp256k1_gej_add_var)
check_symbolic_jacobian_weierstrass("secp256k1_gej_add_ge_var", 0, 7, 5, formula_secp256k1_gej_add_ge_var)
check_symbolic_jacobian_weierstrass("secp256k1_gej_add_zinv_var", 0, 7, 5, formula_secp256k1_gej_add_zinv_var)
check_symbolic_jacobian_weierstrass("secp256k1_gej_add_ge", 0, 7, 16, formula_secp256k1_gej_add_ge)
check_symbolic_jacobian_weierstrass("secp256k1_gej_add_ge_old [should fail]", 0, 7, 4, formula_secp256k1_gej_add_ge_old)
if len(sys.argv) >= 2 and sys.argv[1] == "--exhaustive":
check_exhaustive_jacobian_weierstrass("secp256k1_gej_add_var", 0, 7, 5, formula_secp256k1_gej_add_var, 43)
check_exhaustive_jacobian_weierstrass("secp256k1_gej_add_ge_var", 0, 7, 5, formula_secp256k1_gej_add_ge_var, 43)
check_exhaustive_jacobian_weierstrass("secp256k1_gej_add_zinv_var", 0, 7, 5, formula_secp256k1_gej_add_zinv_var, 43)
check_exhaustive_jacobian_weierstrass("secp256k1_gej_add_ge", 0, 7, 16, formula_secp256k1_gej_add_ge, 43)
check_exhaustive_jacobian_weierstrass("secp256k1_gej_add_ge_old [should fail]", 0, 7, 4, formula_secp256k1_gej_add_ge_old, 43)

View File

@@ -0,0 +1,264 @@
# Prover implementation for Weierstrass curves of the form
# y^2 = x^3 + A * x + B, specifically with a = 0 and b = 7, with group laws
# operating on affine and Jacobian coordinates, including the point at infinity
# represented by a 4th variable in coordinates.
load("group_prover.sage")
class affinepoint:
def __init__(self, x, y, infinity=0):
self.x = x
self.y = y
self.infinity = infinity
def __str__(self):
return "affinepoint(x=%s,y=%s,inf=%s)" % (self.x, self.y, self.infinity)
class jacobianpoint:
def __init__(self, x, y, z, infinity=0):
self.X = x
self.Y = y
self.Z = z
self.Infinity = infinity
def __str__(self):
return "jacobianpoint(X=%s,Y=%s,Z=%s,inf=%s)" % (self.X, self.Y, self.Z, self.Infinity)
def point_at_infinity():
return jacobianpoint(1, 1, 1, 1)
def negate(p):
if p.__class__ == affinepoint:
return affinepoint(p.x, -p.y)
if p.__class__ == jacobianpoint:
return jacobianpoint(p.X, -p.Y, p.Z)
assert(False)
def on_weierstrass_curve(A, B, p):
"""Return a set of zero-expressions for an affine point to be on the curve"""
return constraints(zero={p.x^3 + A*p.x + B - p.y^2: 'on_curve'})
def tangential_to_weierstrass_curve(A, B, p12, p3):
"""Return a set of zero-expressions for ((x12,y12),(x3,y3)) to be a line that is tangential to the curve at (x12,y12)"""
return constraints(zero={
(p12.y - p3.y) * (p12.y * 2) - (p12.x^2 * 3 + A) * (p12.x - p3.x): 'tangential_to_curve'
})
def colinear(p1, p2, p3):
"""Return a set of zero-expressions for ((x1,y1),(x2,y2),(x3,y3)) to be collinear"""
return constraints(zero={
(p1.y - p2.y) * (p1.x - p3.x) - (p1.y - p3.y) * (p1.x - p2.x): 'colinear_1',
(p2.y - p3.y) * (p2.x - p1.x) - (p2.y - p1.y) * (p2.x - p3.x): 'colinear_2',
(p3.y - p1.y) * (p3.x - p2.x) - (p3.y - p2.y) * (p3.x - p1.x): 'colinear_3'
})
def good_affine_point(p):
return constraints(nonzero={p.x : 'nonzero_x', p.y : 'nonzero_y'})
def good_jacobian_point(p):
return constraints(nonzero={p.X : 'nonzero_X', p.Y : 'nonzero_Y', p.Z^6 : 'nonzero_Z'})
def good_point(p):
return constraints(nonzero={p.Z^6 : 'nonzero_X'})
def finite(p, *affine_fns):
con = good_point(p) + constraints(zero={p.Infinity : 'finite_point'})
if p.Z != 0:
return con + reduce(lambda a, b: a + b, (f(affinepoint(p.X / p.Z^2, p.Y / p.Z^3)) for f in affine_fns), con)
else:
return con
def infinite(p):
return constraints(nonzero={p.Infinity : 'infinite_point'})
def law_jacobian_weierstrass_add(A, B, pa, pb, pA, pB, pC):
"""Check whether the passed set of coordinates is a valid Jacobian add, given assumptions"""
assumeLaw = (good_affine_point(pa) +
good_affine_point(pb) +
good_jacobian_point(pA) +
good_jacobian_point(pB) +
on_weierstrass_curve(A, B, pa) +
on_weierstrass_curve(A, B, pb) +
finite(pA) +
finite(pB) +
constraints(nonzero={pa.x - pb.x : 'different_x'}))
require = (finite(pC, lambda pc: on_weierstrass_curve(A, B, pc) +
colinear(pa, pb, negate(pc))))
return (assumeLaw, require)
def law_jacobian_weierstrass_double(A, B, pa, pb, pA, pB, pC):
"""Check whether the passed set of coordinates is a valid Jacobian doubling, given assumptions"""
assumeLaw = (good_affine_point(pa) +
good_affine_point(pb) +
good_jacobian_point(pA) +
good_jacobian_point(pB) +
on_weierstrass_curve(A, B, pa) +
on_weierstrass_curve(A, B, pb) +
finite(pA) +
finite(pB) +
constraints(zero={pa.x - pb.x : 'equal_x', pa.y - pb.y : 'equal_y'}))
require = (finite(pC, lambda pc: on_weierstrass_curve(A, B, pc) +
tangential_to_weierstrass_curve(A, B, pa, negate(pc))))
return (assumeLaw, require)
def law_jacobian_weierstrass_add_opposites(A, B, pa, pb, pA, pB, pC):
assumeLaw = (good_affine_point(pa) +
good_affine_point(pb) +
good_jacobian_point(pA) +
good_jacobian_point(pB) +
on_weierstrass_curve(A, B, pa) +
on_weierstrass_curve(A, B, pb) +
finite(pA) +
finite(pB) +
constraints(zero={pa.x - pb.x : 'equal_x', pa.y + pb.y : 'opposite_y'}))
require = infinite(pC)
return (assumeLaw, require)
def law_jacobian_weierstrass_add_infinite_a(A, B, pa, pb, pA, pB, pC):
assumeLaw = (good_affine_point(pa) +
good_affine_point(pb) +
good_jacobian_point(pA) +
good_jacobian_point(pB) +
on_weierstrass_curve(A, B, pb) +
infinite(pA) +
finite(pB))
require = finite(pC, lambda pc: constraints(zero={pc.x - pb.x : 'c.x=b.x', pc.y - pb.y : 'c.y=b.y'}))
return (assumeLaw, require)
def law_jacobian_weierstrass_add_infinite_b(A, B, pa, pb, pA, pB, pC):
assumeLaw = (good_affine_point(pa) +
good_affine_point(pb) +
good_jacobian_point(pA) +
good_jacobian_point(pB) +
on_weierstrass_curve(A, B, pa) +
infinite(pB) +
finite(pA))
require = finite(pC, lambda pc: constraints(zero={pc.x - pa.x : 'c.x=a.x', pc.y - pa.y : 'c.y=a.y'}))
return (assumeLaw, require)
def law_jacobian_weierstrass_add_infinite_ab(A, B, pa, pb, pA, pB, pC):
assumeLaw = (good_affine_point(pa) +
good_affine_point(pb) +
good_jacobian_point(pA) +
good_jacobian_point(pB) +
infinite(pA) +
infinite(pB))
require = infinite(pC)
return (assumeLaw, require)
laws_jacobian_weierstrass = {
'add': law_jacobian_weierstrass_add,
'double': law_jacobian_weierstrass_double,
'add_opposite': law_jacobian_weierstrass_add_opposites,
'add_infinite_a': law_jacobian_weierstrass_add_infinite_a,
'add_infinite_b': law_jacobian_weierstrass_add_infinite_b,
'add_infinite_ab': law_jacobian_weierstrass_add_infinite_ab
}
def check_exhaustive_jacobian_weierstrass(name, A, B, branches, formula, p):
"""Verify an implementation of addition of Jacobian points on a Weierstrass curve, by executing and validating the result for every possible addition in a prime field"""
F = Integers(p)
print "Formula %s on Z%i:" % (name, p)
points = []
for x in xrange(0, p):
for y in xrange(0, p):
point = affinepoint(F(x), F(y))
r, e = concrete_verify(on_weierstrass_curve(A, B, point))
if r:
points.append(point)
for za in xrange(1, p):
for zb in xrange(1, p):
for pa in points:
for pb in points:
for ia in xrange(2):
for ib in xrange(2):
pA = jacobianpoint(pa.x * F(za)^2, pa.y * F(za)^3, F(za), ia)
pB = jacobianpoint(pb.x * F(zb)^2, pb.y * F(zb)^3, F(zb), ib)
for branch in xrange(0, branches):
assumeAssert, assumeBranch, pC = formula(branch, pA, pB)
pC.X = F(pC.X)
pC.Y = F(pC.Y)
pC.Z = F(pC.Z)
pC.Infinity = F(pC.Infinity)
r, e = concrete_verify(assumeAssert + assumeBranch)
if r:
match = False
for key in laws_jacobian_weierstrass:
assumeLaw, require = laws_jacobian_weierstrass[key](A, B, pa, pb, pA, pB, pC)
r, e = concrete_verify(assumeLaw)
if r:
if match:
print " multiple branches for (%s,%s,%s,%s) + (%s,%s,%s,%s)" % (pA.X, pA.Y, pA.Z, pA.Infinity, pB.X, pB.Y, pB.Z, pB.Infinity)
else:
match = True
r, e = concrete_verify(require)
if not r:
print " failure in branch %i for (%s,%s,%s,%s) + (%s,%s,%s,%s) = (%s,%s,%s,%s): %s" % (branch, pA.X, pA.Y, pA.Z, pA.Infinity, pB.X, pB.Y, pB.Z, pB.Infinity, pC.X, pC.Y, pC.Z, pC.Infinity, e)
print
def check_symbolic_function(R, assumeAssert, assumeBranch, f, A, B, pa, pb, pA, pB, pC):
assumeLaw, require = f(A, B, pa, pb, pA, pB, pC)
return check_symbolic(R, assumeLaw, assumeAssert, assumeBranch, require)
def check_symbolic_jacobian_weierstrass(name, A, B, branches, formula):
"""Verify an implementation of addition of Jacobian points on a Weierstrass curve symbolically"""
R.<ax,bx,ay,by,Az,Bz,Ai,Bi> = PolynomialRing(QQ,8,order='invlex')
lift = lambda x: fastfrac(R,x)
ax = lift(ax)
ay = lift(ay)
Az = lift(Az)
bx = lift(bx)
by = lift(by)
Bz = lift(Bz)
Ai = lift(Ai)
Bi = lift(Bi)
pa = affinepoint(ax, ay, Ai)
pb = affinepoint(bx, by, Bi)
pA = jacobianpoint(ax * Az^2, ay * Az^3, Az, Ai)
pB = jacobianpoint(bx * Bz^2, by * Bz^3, Bz, Bi)
res = {}
for key in laws_jacobian_weierstrass:
res[key] = []
print ("Formula " + name + ":")
count = 0
for branch in xrange(branches):
assumeFormula, assumeBranch, pC = formula(branch, pA, pB)
pC.X = lift(pC.X)
pC.Y = lift(pC.Y)
pC.Z = lift(pC.Z)
pC.Infinity = lift(pC.Infinity)
for key in laws_jacobian_weierstrass:
res[key].append((check_symbolic_function(R, assumeFormula, assumeBranch, laws_jacobian_weierstrass[key], A, B, pa, pb, pA, pB, pC), branch))
for key in res:
print " %s:" % key
val = res[key]
for x in val:
if x[0] is not None:
print " branch %i: %s" % (x[1], x[0])
print

View File

@@ -0,0 +1,919 @@
@ vim: set tabstop=8 softtabstop=8 shiftwidth=8 noexpandtab syntax=armasm:
/**********************************************************************
* Copyright (c) 2014 Wladimir J. van der Laan *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
/*
ARM implementation of field_10x26 inner loops.
Note:
- To avoid unnecessary loads and make use of available registers, two
'passes' have every time been interleaved, with the odd passes accumulating c' and d'
which will be added to c and d respectively in the the even passes
*/
.syntax unified
.arch armv7-a
@ eabi attributes - see readelf -A
.eabi_attribute 8, 1 @ Tag_ARM_ISA_use = yes
.eabi_attribute 9, 0 @ Tag_Thumb_ISA_use = no
.eabi_attribute 10, 0 @ Tag_FP_arch = none
.eabi_attribute 24, 1 @ Tag_ABI_align_needed = 8-byte
.eabi_attribute 25, 1 @ Tag_ABI_align_preserved = 8-byte, except leaf SP
.eabi_attribute 30, 2 @ Tag_ABI_optimization_goals = Agressive Speed
.eabi_attribute 34, 1 @ Tag_CPU_unaligned_access = v6
.text
@ Field constants
.set field_R0, 0x3d10
.set field_R1, 0x400
.set field_not_M, 0xfc000000 @ ~M = ~0x3ffffff
.align 2
.global secp256k1_fe_mul_inner
.type secp256k1_fe_mul_inner, %function
@ Arguments:
@ r0 r Restrict: can overlap with a, not with b
@ r1 a
@ r2 b
@ Stack (total 4+10*4 = 44)
@ sp + #0 saved 'r' pointer
@ sp + #4 + 4*X t0,t1,t2,t3,t4,t5,t6,t7,u8,t9
secp256k1_fe_mul_inner:
stmfd sp!, {r4, r5, r6, r7, r8, r9, r10, r11, r14}
sub sp, sp, #48 @ frame=44 + alignment
str r0, [sp, #0] @ save result address, we need it only at the end
/******************************************
* Main computation code.
******************************************
Allocation:
r0,r14,r7,r8 scratch
r1 a (pointer)
r2 b (pointer)
r3:r4 c
r5:r6 d
r11:r12 c'
r9:r10 d'
Note: do not write to r[] here, it may overlap with a[]
*/
/* A - interleaved with B */
ldr r7, [r1, #0*4] @ a[0]
ldr r8, [r2, #9*4] @ b[9]
ldr r0, [r1, #1*4] @ a[1]
umull r5, r6, r7, r8 @ d = a[0] * b[9]
ldr r14, [r2, #8*4] @ b[8]
umull r9, r10, r0, r8 @ d' = a[1] * b[9]
ldr r7, [r1, #2*4] @ a[2]
umlal r5, r6, r0, r14 @ d += a[1] * b[8]
ldr r8, [r2, #7*4] @ b[7]
umlal r9, r10, r7, r14 @ d' += a[2] * b[8]
ldr r0, [r1, #3*4] @ a[3]
umlal r5, r6, r7, r8 @ d += a[2] * b[7]
ldr r14, [r2, #6*4] @ b[6]
umlal r9, r10, r0, r8 @ d' += a[3] * b[7]
ldr r7, [r1, #4*4] @ a[4]
umlal r5, r6, r0, r14 @ d += a[3] * b[6]
ldr r8, [r2, #5*4] @ b[5]
umlal r9, r10, r7, r14 @ d' += a[4] * b[6]
ldr r0, [r1, #5*4] @ a[5]
umlal r5, r6, r7, r8 @ d += a[4] * b[5]
ldr r14, [r2, #4*4] @ b[4]
umlal r9, r10, r0, r8 @ d' += a[5] * b[5]
ldr r7, [r1, #6*4] @ a[6]
umlal r5, r6, r0, r14 @ d += a[5] * b[4]
ldr r8, [r2, #3*4] @ b[3]
umlal r9, r10, r7, r14 @ d' += a[6] * b[4]
ldr r0, [r1, #7*4] @ a[7]
umlal r5, r6, r7, r8 @ d += a[6] * b[3]
ldr r14, [r2, #2*4] @ b[2]
umlal r9, r10, r0, r8 @ d' += a[7] * b[3]
ldr r7, [r1, #8*4] @ a[8]
umlal r5, r6, r0, r14 @ d += a[7] * b[2]
ldr r8, [r2, #1*4] @ b[1]
umlal r9, r10, r7, r14 @ d' += a[8] * b[2]
ldr r0, [r1, #9*4] @ a[9]
umlal r5, r6, r7, r8 @ d += a[8] * b[1]
ldr r14, [r2, #0*4] @ b[0]
umlal r9, r10, r0, r8 @ d' += a[9] * b[1]
ldr r7, [r1, #0*4] @ a[0]
umlal r5, r6, r0, r14 @ d += a[9] * b[0]
@ r7,r14 used in B
bic r0, r5, field_not_M @ t9 = d & M
str r0, [sp, #4 + 4*9]
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
/* B */
umull r3, r4, r7, r14 @ c = a[0] * b[0]
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u0 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u0 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t0 = c & M
str r14, [sp, #4 + 0*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u0 * R1
umlal r3, r4, r0, r14
/* C - interleaved with D */
ldr r7, [r1, #0*4] @ a[0]
ldr r8, [r2, #2*4] @ b[2]
ldr r14, [r2, #1*4] @ b[1]
umull r11, r12, r7, r8 @ c' = a[0] * b[2]
ldr r0, [r1, #1*4] @ a[1]
umlal r3, r4, r7, r14 @ c += a[0] * b[1]
ldr r8, [r2, #0*4] @ b[0]
umlal r11, r12, r0, r14 @ c' += a[1] * b[1]
ldr r7, [r1, #2*4] @ a[2]
umlal r3, r4, r0, r8 @ c += a[1] * b[0]
ldr r14, [r2, #9*4] @ b[9]
umlal r11, r12, r7, r8 @ c' += a[2] * b[0]
ldr r0, [r1, #3*4] @ a[3]
umlal r5, r6, r7, r14 @ d += a[2] * b[9]
ldr r8, [r2, #8*4] @ b[8]
umull r9, r10, r0, r14 @ d' = a[3] * b[9]
ldr r7, [r1, #4*4] @ a[4]
umlal r5, r6, r0, r8 @ d += a[3] * b[8]
ldr r14, [r2, #7*4] @ b[7]
umlal r9, r10, r7, r8 @ d' += a[4] * b[8]
ldr r0, [r1, #5*4] @ a[5]
umlal r5, r6, r7, r14 @ d += a[4] * b[7]
ldr r8, [r2, #6*4] @ b[6]
umlal r9, r10, r0, r14 @ d' += a[5] * b[7]
ldr r7, [r1, #6*4] @ a[6]
umlal r5, r6, r0, r8 @ d += a[5] * b[6]
ldr r14, [r2, #5*4] @ b[5]
umlal r9, r10, r7, r8 @ d' += a[6] * b[6]
ldr r0, [r1, #7*4] @ a[7]
umlal r5, r6, r7, r14 @ d += a[6] * b[5]
ldr r8, [r2, #4*4] @ b[4]
umlal r9, r10, r0, r14 @ d' += a[7] * b[5]
ldr r7, [r1, #8*4] @ a[8]
umlal r5, r6, r0, r8 @ d += a[7] * b[4]
ldr r14, [r2, #3*4] @ b[3]
umlal r9, r10, r7, r8 @ d' += a[8] * b[4]
ldr r0, [r1, #9*4] @ a[9]
umlal r5, r6, r7, r14 @ d += a[8] * b[3]
ldr r8, [r2, #2*4] @ b[2]
umlal r9, r10, r0, r14 @ d' += a[9] * b[3]
umlal r5, r6, r0, r8 @ d += a[9] * b[2]
bic r0, r5, field_not_M @ u1 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u1 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t1 = c & M
str r14, [sp, #4 + 1*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u1 * R1
umlal r3, r4, r0, r14
/* D */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u2 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u2 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t2 = c & M
str r14, [sp, #4 + 2*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u2 * R1
umlal r3, r4, r0, r14
/* E - interleaved with F */
ldr r7, [r1, #0*4] @ a[0]
ldr r8, [r2, #4*4] @ b[4]
umull r11, r12, r7, r8 @ c' = a[0] * b[4]
ldr r8, [r2, #3*4] @ b[3]
umlal r3, r4, r7, r8 @ c += a[0] * b[3]
ldr r7, [r1, #1*4] @ a[1]
umlal r11, r12, r7, r8 @ c' += a[1] * b[3]
ldr r8, [r2, #2*4] @ b[2]
umlal r3, r4, r7, r8 @ c += a[1] * b[2]
ldr r7, [r1, #2*4] @ a[2]
umlal r11, r12, r7, r8 @ c' += a[2] * b[2]
ldr r8, [r2, #1*4] @ b[1]
umlal r3, r4, r7, r8 @ c += a[2] * b[1]
ldr r7, [r1, #3*4] @ a[3]
umlal r11, r12, r7, r8 @ c' += a[3] * b[1]
ldr r8, [r2, #0*4] @ b[0]
umlal r3, r4, r7, r8 @ c += a[3] * b[0]
ldr r7, [r1, #4*4] @ a[4]
umlal r11, r12, r7, r8 @ c' += a[4] * b[0]
ldr r8, [r2, #9*4] @ b[9]
umlal r5, r6, r7, r8 @ d += a[4] * b[9]
ldr r7, [r1, #5*4] @ a[5]
umull r9, r10, r7, r8 @ d' = a[5] * b[9]
ldr r8, [r2, #8*4] @ b[8]
umlal r5, r6, r7, r8 @ d += a[5] * b[8]
ldr r7, [r1, #6*4] @ a[6]
umlal r9, r10, r7, r8 @ d' += a[6] * b[8]
ldr r8, [r2, #7*4] @ b[7]
umlal r5, r6, r7, r8 @ d += a[6] * b[7]
ldr r7, [r1, #7*4] @ a[7]
umlal r9, r10, r7, r8 @ d' += a[7] * b[7]
ldr r8, [r2, #6*4] @ b[6]
umlal r5, r6, r7, r8 @ d += a[7] * b[6]
ldr r7, [r1, #8*4] @ a[8]
umlal r9, r10, r7, r8 @ d' += a[8] * b[6]
ldr r8, [r2, #5*4] @ b[5]
umlal r5, r6, r7, r8 @ d += a[8] * b[5]
ldr r7, [r1, #9*4] @ a[9]
umlal r9, r10, r7, r8 @ d' += a[9] * b[5]
ldr r8, [r2, #4*4] @ b[4]
umlal r5, r6, r7, r8 @ d += a[9] * b[4]
bic r0, r5, field_not_M @ u3 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u3 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t3 = c & M
str r14, [sp, #4 + 3*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u3 * R1
umlal r3, r4, r0, r14
/* F */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u4 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u4 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t4 = c & M
str r14, [sp, #4 + 4*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u4 * R1
umlal r3, r4, r0, r14
/* G - interleaved with H */
ldr r7, [r1, #0*4] @ a[0]
ldr r8, [r2, #6*4] @ b[6]
ldr r14, [r2, #5*4] @ b[5]
umull r11, r12, r7, r8 @ c' = a[0] * b[6]
ldr r0, [r1, #1*4] @ a[1]
umlal r3, r4, r7, r14 @ c += a[0] * b[5]
ldr r8, [r2, #4*4] @ b[4]
umlal r11, r12, r0, r14 @ c' += a[1] * b[5]
ldr r7, [r1, #2*4] @ a[2]
umlal r3, r4, r0, r8 @ c += a[1] * b[4]
ldr r14, [r2, #3*4] @ b[3]
umlal r11, r12, r7, r8 @ c' += a[2] * b[4]
ldr r0, [r1, #3*4] @ a[3]
umlal r3, r4, r7, r14 @ c += a[2] * b[3]
ldr r8, [r2, #2*4] @ b[2]
umlal r11, r12, r0, r14 @ c' += a[3] * b[3]
ldr r7, [r1, #4*4] @ a[4]
umlal r3, r4, r0, r8 @ c += a[3] * b[2]
ldr r14, [r2, #1*4] @ b[1]
umlal r11, r12, r7, r8 @ c' += a[4] * b[2]
ldr r0, [r1, #5*4] @ a[5]
umlal r3, r4, r7, r14 @ c += a[4] * b[1]
ldr r8, [r2, #0*4] @ b[0]
umlal r11, r12, r0, r14 @ c' += a[5] * b[1]
ldr r7, [r1, #6*4] @ a[6]
umlal r3, r4, r0, r8 @ c += a[5] * b[0]
ldr r14, [r2, #9*4] @ b[9]
umlal r11, r12, r7, r8 @ c' += a[6] * b[0]
ldr r0, [r1, #7*4] @ a[7]
umlal r5, r6, r7, r14 @ d += a[6] * b[9]
ldr r8, [r2, #8*4] @ b[8]
umull r9, r10, r0, r14 @ d' = a[7] * b[9]
ldr r7, [r1, #8*4] @ a[8]
umlal r5, r6, r0, r8 @ d += a[7] * b[8]
ldr r14, [r2, #7*4] @ b[7]
umlal r9, r10, r7, r8 @ d' += a[8] * b[8]
ldr r0, [r1, #9*4] @ a[9]
umlal r5, r6, r7, r14 @ d += a[8] * b[7]
ldr r8, [r2, #6*4] @ b[6]
umlal r9, r10, r0, r14 @ d' += a[9] * b[7]
umlal r5, r6, r0, r8 @ d += a[9] * b[6]
bic r0, r5, field_not_M @ u5 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u5 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t5 = c & M
str r14, [sp, #4 + 5*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u5 * R1
umlal r3, r4, r0, r14
/* H */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u6 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u6 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t6 = c & M
str r14, [sp, #4 + 6*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u6 * R1
umlal r3, r4, r0, r14
/* I - interleaved with J */
ldr r8, [r2, #8*4] @ b[8]
ldr r7, [r1, #0*4] @ a[0]
ldr r14, [r2, #7*4] @ b[7]
umull r11, r12, r7, r8 @ c' = a[0] * b[8]
ldr r0, [r1, #1*4] @ a[1]
umlal r3, r4, r7, r14 @ c += a[0] * b[7]
ldr r8, [r2, #6*4] @ b[6]
umlal r11, r12, r0, r14 @ c' += a[1] * b[7]
ldr r7, [r1, #2*4] @ a[2]
umlal r3, r4, r0, r8 @ c += a[1] * b[6]
ldr r14, [r2, #5*4] @ b[5]
umlal r11, r12, r7, r8 @ c' += a[2] * b[6]
ldr r0, [r1, #3*4] @ a[3]
umlal r3, r4, r7, r14 @ c += a[2] * b[5]
ldr r8, [r2, #4*4] @ b[4]
umlal r11, r12, r0, r14 @ c' += a[3] * b[5]
ldr r7, [r1, #4*4] @ a[4]
umlal r3, r4, r0, r8 @ c += a[3] * b[4]
ldr r14, [r2, #3*4] @ b[3]
umlal r11, r12, r7, r8 @ c' += a[4] * b[4]
ldr r0, [r1, #5*4] @ a[5]
umlal r3, r4, r7, r14 @ c += a[4] * b[3]
ldr r8, [r2, #2*4] @ b[2]
umlal r11, r12, r0, r14 @ c' += a[5] * b[3]
ldr r7, [r1, #6*4] @ a[6]
umlal r3, r4, r0, r8 @ c += a[5] * b[2]
ldr r14, [r2, #1*4] @ b[1]
umlal r11, r12, r7, r8 @ c' += a[6] * b[2]
ldr r0, [r1, #7*4] @ a[7]
umlal r3, r4, r7, r14 @ c += a[6] * b[1]
ldr r8, [r2, #0*4] @ b[0]
umlal r11, r12, r0, r14 @ c' += a[7] * b[1]
ldr r7, [r1, #8*4] @ a[8]
umlal r3, r4, r0, r8 @ c += a[7] * b[0]
ldr r14, [r2, #9*4] @ b[9]
umlal r11, r12, r7, r8 @ c' += a[8] * b[0]
ldr r0, [r1, #9*4] @ a[9]
umlal r5, r6, r7, r14 @ d += a[8] * b[9]
ldr r8, [r2, #8*4] @ b[8]
umull r9, r10, r0, r14 @ d' = a[9] * b[9]
umlal r5, r6, r0, r8 @ d += a[9] * b[8]
bic r0, r5, field_not_M @ u7 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u7 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t7 = c & M
str r14, [sp, #4 + 7*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u7 * R1
umlal r3, r4, r0, r14
/* J */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u8 = d & M
str r0, [sp, #4 + 8*4]
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u8 * R0
umlal r3, r4, r0, r14
/******************************************
* compute and write back result
******************************************
Allocation:
r0 r
r3:r4 c
r5:r6 d
r7 t0
r8 t1
r9 t2
r11 u8
r12 t9
r1,r2,r10,r14 scratch
Note: do not read from a[] after here, it may overlap with r[]
*/
ldr r0, [sp, #0]
add r1, sp, #4 + 3*4 @ r[3..7] = t3..7, r11=u8, r12=t9
ldmia r1, {r2,r7,r8,r9,r10,r11,r12}
add r1, r0, #3*4
stmia r1, {r2,r7,r8,r9,r10}
bic r2, r3, field_not_M @ r[8] = c & M
str r2, [r0, #8*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u8 * R1
umlal r3, r4, r11, r14
movw r14, field_R0 @ c += d * R0
umlal r3, r4, r5, r14
adds r3, r3, r12 @ c += t9
adc r4, r4, #0
add r1, sp, #4 + 0*4 @ r7,r8,r9 = t0,t1,t2
ldmia r1, {r7,r8,r9}
ubfx r2, r3, #0, #22 @ r[9] = c & (M >> 4)
str r2, [r0, #9*4]
mov r3, r3, lsr #22 @ c >>= 22
orr r3, r3, r4, asl #10
mov r4, r4, lsr #22
movw r14, field_R1 << 4 @ c += d * (R1 << 4)
umlal r3, r4, r5, r14
movw r14, field_R0 >> 4 @ d = c * (R0 >> 4) + t0 (64x64 multiply+add)
umull r5, r6, r3, r14 @ d = c.lo * (R0 >> 4)
adds r5, r5, r7 @ d.lo += t0
mla r6, r14, r4, r6 @ d.hi += c.hi * (R0 >> 4)
adc r6, r6, 0 @ d.hi += carry
bic r2, r5, field_not_M @ r[0] = d & M
str r2, [r0, #0*4]
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R1 >> 4 @ d += c * (R1 >> 4) + t1 (64x64 multiply+add)
umull r1, r2, r3, r14 @ tmp = c.lo * (R1 >> 4)
adds r5, r5, r8 @ d.lo += t1
adc r6, r6, #0 @ d.hi += carry
adds r5, r5, r1 @ d.lo += tmp.lo
mla r2, r14, r4, r2 @ tmp.hi += c.hi * (R1 >> 4)
adc r6, r6, r2 @ d.hi += carry + tmp.hi
bic r2, r5, field_not_M @ r[1] = d & M
str r2, [r0, #1*4]
mov r5, r5, lsr #26 @ d >>= 26 (ignore hi)
orr r5, r5, r6, asl #6
add r5, r5, r9 @ d += t2
str r5, [r0, #2*4] @ r[2] = d
add sp, sp, #48
ldmfd sp!, {r4, r5, r6, r7, r8, r9, r10, r11, pc}
.size secp256k1_fe_mul_inner, .-secp256k1_fe_mul_inner
.align 2
.global secp256k1_fe_sqr_inner
.type secp256k1_fe_sqr_inner, %function
@ Arguments:
@ r0 r Can overlap with a
@ r1 a
@ Stack (total 4+10*4 = 44)
@ sp + #0 saved 'r' pointer
@ sp + #4 + 4*X t0,t1,t2,t3,t4,t5,t6,t7,u8,t9
secp256k1_fe_sqr_inner:
stmfd sp!, {r4, r5, r6, r7, r8, r9, r10, r11, r14}
sub sp, sp, #48 @ frame=44 + alignment
str r0, [sp, #0] @ save result address, we need it only at the end
/******************************************
* Main computation code.
******************************************
Allocation:
r0,r14,r2,r7,r8 scratch
r1 a (pointer)
r3:r4 c
r5:r6 d
r11:r12 c'
r9:r10 d'
Note: do not write to r[] here, it may overlap with a[]
*/
/* A interleaved with B */
ldr r0, [r1, #1*4] @ a[1]*2
ldr r7, [r1, #0*4] @ a[0]
mov r0, r0, asl #1
ldr r14, [r1, #9*4] @ a[9]
umull r3, r4, r7, r7 @ c = a[0] * a[0]
ldr r8, [r1, #8*4] @ a[8]
mov r7, r7, asl #1
umull r5, r6, r7, r14 @ d = a[0]*2 * a[9]
ldr r7, [r1, #2*4] @ a[2]*2
umull r9, r10, r0, r14 @ d' = a[1]*2 * a[9]
ldr r14, [r1, #7*4] @ a[7]
umlal r5, r6, r0, r8 @ d += a[1]*2 * a[8]
mov r7, r7, asl #1
ldr r0, [r1, #3*4] @ a[3]*2
umlal r9, r10, r7, r8 @ d' += a[2]*2 * a[8]
ldr r8, [r1, #6*4] @ a[6]
umlal r5, r6, r7, r14 @ d += a[2]*2 * a[7]
mov r0, r0, asl #1
ldr r7, [r1, #4*4] @ a[4]*2
umlal r9, r10, r0, r14 @ d' += a[3]*2 * a[7]
ldr r14, [r1, #5*4] @ a[5]
mov r7, r7, asl #1
umlal r5, r6, r0, r8 @ d += a[3]*2 * a[6]
umlal r9, r10, r7, r8 @ d' += a[4]*2 * a[6]
umlal r5, r6, r7, r14 @ d += a[4]*2 * a[5]
umlal r9, r10, r14, r14 @ d' += a[5] * a[5]
bic r0, r5, field_not_M @ t9 = d & M
str r0, [sp, #4 + 9*4]
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
/* B */
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u0 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u0 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t0 = c & M
str r14, [sp, #4 + 0*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u0 * R1
umlal r3, r4, r0, r14
/* C interleaved with D */
ldr r0, [r1, #0*4] @ a[0]*2
ldr r14, [r1, #1*4] @ a[1]
mov r0, r0, asl #1
ldr r8, [r1, #2*4] @ a[2]
umlal r3, r4, r0, r14 @ c += a[0]*2 * a[1]
mov r7, r8, asl #1 @ a[2]*2
umull r11, r12, r14, r14 @ c' = a[1] * a[1]
ldr r14, [r1, #9*4] @ a[9]
umlal r11, r12, r0, r8 @ c' += a[0]*2 * a[2]
ldr r0, [r1, #3*4] @ a[3]*2
ldr r8, [r1, #8*4] @ a[8]
umlal r5, r6, r7, r14 @ d += a[2]*2 * a[9]
mov r0, r0, asl #1
ldr r7, [r1, #4*4] @ a[4]*2
umull r9, r10, r0, r14 @ d' = a[3]*2 * a[9]
ldr r14, [r1, #7*4] @ a[7]
umlal r5, r6, r0, r8 @ d += a[3]*2 * a[8]
mov r7, r7, asl #1
ldr r0, [r1, #5*4] @ a[5]*2
umlal r9, r10, r7, r8 @ d' += a[4]*2 * a[8]
ldr r8, [r1, #6*4] @ a[6]
mov r0, r0, asl #1
umlal r5, r6, r7, r14 @ d += a[4]*2 * a[7]
umlal r9, r10, r0, r14 @ d' += a[5]*2 * a[7]
umlal r5, r6, r0, r8 @ d += a[5]*2 * a[6]
umlal r9, r10, r8, r8 @ d' += a[6] * a[6]
bic r0, r5, field_not_M @ u1 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u1 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t1 = c & M
str r14, [sp, #4 + 1*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u1 * R1
umlal r3, r4, r0, r14
/* D */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u2 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u2 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t2 = c & M
str r14, [sp, #4 + 2*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u2 * R1
umlal r3, r4, r0, r14
/* E interleaved with F */
ldr r7, [r1, #0*4] @ a[0]*2
ldr r0, [r1, #1*4] @ a[1]*2
ldr r14, [r1, #2*4] @ a[2]
mov r7, r7, asl #1
ldr r8, [r1, #3*4] @ a[3]
ldr r2, [r1, #4*4]
umlal r3, r4, r7, r8 @ c += a[0]*2 * a[3]
mov r0, r0, asl #1
umull r11, r12, r7, r2 @ c' = a[0]*2 * a[4]
mov r2, r2, asl #1 @ a[4]*2
umlal r11, r12, r0, r8 @ c' += a[1]*2 * a[3]
ldr r8, [r1, #9*4] @ a[9]
umlal r3, r4, r0, r14 @ c += a[1]*2 * a[2]
ldr r0, [r1, #5*4] @ a[5]*2
umlal r11, r12, r14, r14 @ c' += a[2] * a[2]
ldr r14, [r1, #8*4] @ a[8]
mov r0, r0, asl #1
umlal r5, r6, r2, r8 @ d += a[4]*2 * a[9]
ldr r7, [r1, #6*4] @ a[6]*2
umull r9, r10, r0, r8 @ d' = a[5]*2 * a[9]
mov r7, r7, asl #1
ldr r8, [r1, #7*4] @ a[7]
umlal r5, r6, r0, r14 @ d += a[5]*2 * a[8]
umlal r9, r10, r7, r14 @ d' += a[6]*2 * a[8]
umlal r5, r6, r7, r8 @ d += a[6]*2 * a[7]
umlal r9, r10, r8, r8 @ d' += a[7] * a[7]
bic r0, r5, field_not_M @ u3 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u3 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t3 = c & M
str r14, [sp, #4 + 3*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u3 * R1
umlal r3, r4, r0, r14
/* F */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u4 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u4 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t4 = c & M
str r14, [sp, #4 + 4*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u4 * R1
umlal r3, r4, r0, r14
/* G interleaved with H */
ldr r7, [r1, #0*4] @ a[0]*2
ldr r0, [r1, #1*4] @ a[1]*2
mov r7, r7, asl #1
ldr r8, [r1, #5*4] @ a[5]
ldr r2, [r1, #6*4] @ a[6]
umlal r3, r4, r7, r8 @ c += a[0]*2 * a[5]
ldr r14, [r1, #4*4] @ a[4]
mov r0, r0, asl #1
umull r11, r12, r7, r2 @ c' = a[0]*2 * a[6]
ldr r7, [r1, #2*4] @ a[2]*2
umlal r11, r12, r0, r8 @ c' += a[1]*2 * a[5]
mov r7, r7, asl #1
ldr r8, [r1, #3*4] @ a[3]
umlal r3, r4, r0, r14 @ c += a[1]*2 * a[4]
mov r0, r2, asl #1 @ a[6]*2
umlal r11, r12, r7, r14 @ c' += a[2]*2 * a[4]
ldr r14, [r1, #9*4] @ a[9]
umlal r3, r4, r7, r8 @ c += a[2]*2 * a[3]
ldr r7, [r1, #7*4] @ a[7]*2
umlal r11, r12, r8, r8 @ c' += a[3] * a[3]
mov r7, r7, asl #1
ldr r8, [r1, #8*4] @ a[8]
umlal r5, r6, r0, r14 @ d += a[6]*2 * a[9]
umull r9, r10, r7, r14 @ d' = a[7]*2 * a[9]
umlal r5, r6, r7, r8 @ d += a[7]*2 * a[8]
umlal r9, r10, r8, r8 @ d' += a[8] * a[8]
bic r0, r5, field_not_M @ u5 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u5 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t5 = c & M
str r14, [sp, #4 + 5*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u5 * R1
umlal r3, r4, r0, r14
/* H */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
adds r5, r5, r9 @ d += d'
adc r6, r6, r10
bic r0, r5, field_not_M @ u6 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u6 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t6 = c & M
str r14, [sp, #4 + 6*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u6 * R1
umlal r3, r4, r0, r14
/* I interleaved with J */
ldr r7, [r1, #0*4] @ a[0]*2
ldr r0, [r1, #1*4] @ a[1]*2
mov r7, r7, asl #1
ldr r8, [r1, #7*4] @ a[7]
ldr r2, [r1, #8*4] @ a[8]
umlal r3, r4, r7, r8 @ c += a[0]*2 * a[7]
ldr r14, [r1, #6*4] @ a[6]
mov r0, r0, asl #1
umull r11, r12, r7, r2 @ c' = a[0]*2 * a[8]
ldr r7, [r1, #2*4] @ a[2]*2
umlal r11, r12, r0, r8 @ c' += a[1]*2 * a[7]
ldr r8, [r1, #5*4] @ a[5]
umlal r3, r4, r0, r14 @ c += a[1]*2 * a[6]
ldr r0, [r1, #3*4] @ a[3]*2
mov r7, r7, asl #1
umlal r11, r12, r7, r14 @ c' += a[2]*2 * a[6]
ldr r14, [r1, #4*4] @ a[4]
mov r0, r0, asl #1
umlal r3, r4, r7, r8 @ c += a[2]*2 * a[5]
mov r2, r2, asl #1 @ a[8]*2
umlal r11, r12, r0, r8 @ c' += a[3]*2 * a[5]
umlal r3, r4, r0, r14 @ c += a[3]*2 * a[4]
umlal r11, r12, r14, r14 @ c' += a[4] * a[4]
ldr r8, [r1, #9*4] @ a[9]
umlal r5, r6, r2, r8 @ d += a[8]*2 * a[9]
@ r8 will be used in J
bic r0, r5, field_not_M @ u7 = d & M
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u7 * R0
umlal r3, r4, r0, r14
bic r14, r3, field_not_M @ t7 = c & M
str r14, [sp, #4 + 7*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u7 * R1
umlal r3, r4, r0, r14
/* J */
adds r3, r3, r11 @ c += c'
adc r4, r4, r12
umlal r5, r6, r8, r8 @ d += a[9] * a[9]
bic r0, r5, field_not_M @ u8 = d & M
str r0, [sp, #4 + 8*4]
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R0 @ c += u8 * R0
umlal r3, r4, r0, r14
/******************************************
* compute and write back result
******************************************
Allocation:
r0 r
r3:r4 c
r5:r6 d
r7 t0
r8 t1
r9 t2
r11 u8
r12 t9
r1,r2,r10,r14 scratch
Note: do not read from a[] after here, it may overlap with r[]
*/
ldr r0, [sp, #0]
add r1, sp, #4 + 3*4 @ r[3..7] = t3..7, r11=u8, r12=t9
ldmia r1, {r2,r7,r8,r9,r10,r11,r12}
add r1, r0, #3*4
stmia r1, {r2,r7,r8,r9,r10}
bic r2, r3, field_not_M @ r[8] = c & M
str r2, [r0, #8*4]
mov r3, r3, lsr #26 @ c >>= 26
orr r3, r3, r4, asl #6
mov r4, r4, lsr #26
mov r14, field_R1 @ c += u8 * R1
umlal r3, r4, r11, r14
movw r14, field_R0 @ c += d * R0
umlal r3, r4, r5, r14
adds r3, r3, r12 @ c += t9
adc r4, r4, #0
add r1, sp, #4 + 0*4 @ r7,r8,r9 = t0,t1,t2
ldmia r1, {r7,r8,r9}
ubfx r2, r3, #0, #22 @ r[9] = c & (M >> 4)
str r2, [r0, #9*4]
mov r3, r3, lsr #22 @ c >>= 22
orr r3, r3, r4, asl #10
mov r4, r4, lsr #22
movw r14, field_R1 << 4 @ c += d * (R1 << 4)
umlal r3, r4, r5, r14
movw r14, field_R0 >> 4 @ d = c * (R0 >> 4) + t0 (64x64 multiply+add)
umull r5, r6, r3, r14 @ d = c.lo * (R0 >> 4)
adds r5, r5, r7 @ d.lo += t0
mla r6, r14, r4, r6 @ d.hi += c.hi * (R0 >> 4)
adc r6, r6, 0 @ d.hi += carry
bic r2, r5, field_not_M @ r[0] = d & M
str r2, [r0, #0*4]
mov r5, r5, lsr #26 @ d >>= 26
orr r5, r5, r6, asl #6
mov r6, r6, lsr #26
movw r14, field_R1 >> 4 @ d += c * (R1 >> 4) + t1 (64x64 multiply+add)
umull r1, r2, r3, r14 @ tmp = c.lo * (R1 >> 4)
adds r5, r5, r8 @ d.lo += t1
adc r6, r6, #0 @ d.hi += carry
adds r5, r5, r1 @ d.lo += tmp.lo
mla r2, r14, r4, r2 @ tmp.hi += c.hi * (R1 >> 4)
adc r6, r6, r2 @ d.hi += carry + tmp.hi
bic r2, r5, field_not_M @ r[1] = d & M
str r2, [r0, #1*4]
mov r5, r5, lsr #26 @ d >>= 26 (ignore hi)
orr r5, r5, r6, asl #6
add r5, r5, r9 @ d += t2
str r5, [r0, #2*4] @ r[2] = d
add sp, sp, #48
ldmfd sp!, {r4, r5, r6, r7, r8, r9, r10, r11, pc}
.size secp256k1_fe_sqr_inner, .-secp256k1_fe_sqr_inner

View File

@@ -0,0 +1,32 @@
/**********************************************************************
* Copyright (c) 2013, 2014 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
#ifndef _SECP256K1_BASIC_CONFIG_
#define _SECP256K1_BASIC_CONFIG_
#ifdef USE_BASIC_CONFIG
#undef USE_ASM_X86_64
#undef USE_ENDOMORPHISM
#undef USE_FIELD_10X26
#undef USE_FIELD_5X52
#undef USE_FIELD_INV_BUILTIN
#undef USE_FIELD_INV_NUM
#undef USE_NUM_GMP
#undef USE_NUM_NONE
#undef USE_SCALAR_4X64
#undef USE_SCALAR_8X32
#undef USE_SCALAR_INV_BUILTIN
#undef USE_SCALAR_INV_NUM
#define USE_NUM_NONE 1
#define USE_FIELD_INV_BUILTIN 1
#define USE_SCALAR_INV_BUILTIN 1
#define USE_FIELD_10X26 1
#define USE_SCALAR_8X32 1
#endif // USE_BASIC_CONFIG
#endif // _SECP256K1_BASIC_CONFIG_

View File

@@ -0,0 +1,66 @@
/**********************************************************************
* Copyright (c) 2014 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
#ifndef _SECP256K1_BENCH_H_
#define _SECP256K1_BENCH_H_
#include <stdio.h>
#include <math.h>
#include "sys/time.h"
static double gettimedouble(void) {
struct timeval tv;
gettimeofday(&tv, NULL);
return tv.tv_usec * 0.000001 + tv.tv_sec;
}
void print_number(double x) {
double y = x;
int c = 0;
if (y < 0.0) {
y = -y;
}
while (y < 100.0) {
y *= 10.0;
c++;
}
printf("%.*f", c, x);
}
void run_benchmark(char *name, void (*benchmark)(void*), void (*setup)(void*), void (*teardown)(void*), void* data, int count, int iter) {
int i;
double min = HUGE_VAL;
double sum = 0.0;
double max = 0.0;
for (i = 0; i < count; i++) {
double begin, total;
if (setup != NULL) {
setup(data);
}
begin = gettimedouble();
benchmark(data);
total = gettimedouble() - begin;
if (teardown != NULL) {
teardown(data);
}
if (total < min) {
min = total;
}
if (total > max) {
max = total;
}
sum += total;
}
printf("%s: min ", name);
print_number(min * 1000000.0 / iter);
printf("us / avg ");
print_number((sum / count) * 1000000.0 / iter);
printf("us / max ");
print_number(max * 1000000.0 / iter);
printf("us\n");
}
#endif

View File

@@ -0,0 +1,54 @@
/**********************************************************************
* Copyright (c) 2015 Pieter Wuille, Andrew Poelstra *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
#include <string.h>
#include "include/secp256k1.h"
#include "include/secp256k1_ecdh.h"
#include "util.h"
#include "bench.h"
typedef struct {
secp256k1_context *ctx;
secp256k1_pubkey point;
unsigned char scalar[32];
} bench_ecdh_t;
static void bench_ecdh_setup(void* arg) {
int i;
bench_ecdh_t *data = (bench_ecdh_t*)arg;
const unsigned char point[] = {
0x03,
0x54, 0x94, 0xc1, 0x5d, 0x32, 0x09, 0x97, 0x06,
0xc2, 0x39, 0x5f, 0x94, 0x34, 0x87, 0x45, 0xfd,
0x75, 0x7c, 0xe3, 0x0e, 0x4e, 0x8c, 0x90, 0xfb,
0xa2, 0xba, 0xd1, 0x84, 0xf8, 0x83, 0xc6, 0x9f
};
/* create a context with no capabilities */
data->ctx = secp256k1_context_create(SECP256K1_FLAGS_TYPE_CONTEXT);
for (i = 0; i < 32; i++) {
data->scalar[i] = i + 1;
}
CHECK(secp256k1_ec_pubkey_parse(data->ctx, &data->point, point, sizeof(point)) == 1);
}
static void bench_ecdh(void* arg) {
int i;
unsigned char res[32];
bench_ecdh_t *data = (bench_ecdh_t*)arg;
for (i = 0; i < 20000; i++) {
CHECK(secp256k1_ecdh(data->ctx, res, &data->point, data->scalar) == 1);
}
}
int main(void) {
bench_ecdh_t data;
run_benchmark("ecdh", bench_ecdh, bench_ecdh_setup, NULL, &data, 10, 20000);
return 0;
}

Some files were not shown because too many files have changed in this diff Show More