Thane Thomson dff3deb2a9 cs: sync WAL more frequently (#3300)
As per #3043, this adds a ticker to sync the WAL every 2s while the WAL is running.

* Flush WAL every 2s

This adds a ticker that flushes the WAL every 2s while the WAL is
running. This is related to #3043.

* Fix spelling

* Increase timeout to 2mins for slower build environments

* Make WAL sync interval configurable

* Add TODO to replace testChan with more comprehensive testBus

* Remove extraneous debug statement

* Remove testChan in favour of using system time

As per
https://github.com/tendermint/tendermint/pull/3300#discussion_r255886586,
this removes the `testChan` WAL member and replaces the approach with a
system time-oriented one. In this new approach, we keep track of the
system time at which each flush and periodic flush successfully
occurred.

The naming of the various functions is also updated here to be more
consistent with "flushing" as opposed to "sync'ing".

* Update naming convention and ensure lock for timestamp update

* Add Flush method as part of WAL interface

Adds a `Flush` method as part of the WAL interface to enforce the idea
that we can manually trigger a WAL flush from outside of the WAL. This
is employed in the consensus state management to flush the WAL prior to
signing votes/proposals, as per https://github.com/tendermint/tendermint/issues/3043#issuecomment-453853630

* Update CHANGELOG_PENDING

* Remove mutex approach and replace with DI

The dependency injection approach to dealing with testing concerns could
allow similar effects to some kind of "testing bus"-based approach. This
commit introduces an example of this, where instead of relying on
(potentially fragile) timing of things between the code and the test, we
inject code into the function under test that can signal the test
through a channel.

This allows us to avoid the `time.Sleep()`-based approach previously
employed.

* Update comment on WAL flushing during vote signing

Co-Authored-By: thanethomson <connect@thanethomson.com>

* Simplify flush interval definition

Co-Authored-By: thanethomson <connect@thanethomson.com>

* Expand commentary on WAL disk flushing

Co-Authored-By: thanethomson <connect@thanethomson.com>

* Add broken test to illustrate WAL sync test problem

Removes test-related state (dependency injection code) from the WAL data
structure and adds test code to illustrate the problem with using
`WALGenerateNBlocks` and `wal.SearchForEndHeight` to test periodic
sync'ing.

* Fix test error messages

* Use WAL group buffer size to check for flush

A function is added to `libs/autofile/group.go#Group` in order to return
the size of the buffered data (i.e. data that has not yet been flushed
to disk). The test now checks that, prior to a `time.Sleep`, the group
buffer has data in it. After the `time.Sleep` (during which time the
periodic flush should have been called), the buffer should be empty.

* Remove config root dir removal from #3291

* Add godoc for NewWAL mentioning periodic sync
2019-02-20 09:45:18 +04:00
2019-02-13 08:26:01 +04:00
2019-02-11 16:31:34 +04:00
2019-01-28 16:13:17 +04:00
2019-02-18 13:23:40 +04:00
2019-02-18 13:23:40 +04:00
2019-02-12 08:54:12 +04:00
2019-01-06 11:40:15 +03:00
2019-02-20 09:45:18 +04:00
2019-02-18 13:23:40 +04:00
2019-02-18 13:23:40 +04:00
2018-09-30 12:35:52 -04:00
2019-02-18 13:23:40 +04:00
2019-02-11 16:31:34 +04:00
2018-12-16 23:34:13 -05:00
2019-02-18 13:23:40 +04:00
2018-11-01 02:07:18 -04:00
2019-02-12 08:54:12 +04:00
2019-02-08 18:50:02 -05:00
2017-12-04 15:01:28 -06:00
2019-02-12 08:54:12 +04:00
2017-12-10 20:07:44 -05:00
2018-07-17 17:42:30 +01:00
2018-11-17 16:04:05 -05:00
2016-07-18 11:51:37 -04:00
2019-02-07 20:12:57 -05:00
2019-02-08 18:50:02 -05:00

Tendermint

Byzantine-Fault Tolerant State Machines. Or Blockchain, for short.

version API Reference Go version riot.im license

Branch Tests Coverage
master CircleCI codecov
develop CircleCI codecov

Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines.

For protocol details, see the specification.

For detailed analysis of the consensus protocol, including safety and liveness proofs, see our recent paper, "The latest gossip on BFT consensus".

A Note on Production Readiness

While Tendermint is being used in production in private, permissioned environments, we are still working actively to harden and audit it in preparation for use in public blockchains, such as the Cosmos Network. We are also still making breaking changes to the protocol and the APIs. Thus, we tag the releases as alpha software.

In any case, if you intend to run Tendermint in production, please contact us and join the chat.

Security

To report a security vulnerability, see our bug bounty program

For examples of the kinds of bugs we're looking for, see SECURITY.md

Minimum requirements

Requirement Notes
Go version Go1.11.4 or higher

Documentation

Complete documentation can be found on the website.

Install

See the install instructions

Quick Start

Contributing

Please abide by the Code of Conduct in all interactions, and the contributing guidelines when submitting code.

Join the larger community on the forum and the chat.

To learn more about the structure of the software, watch the Developer Sessions and read some Architectural Decision Records.

Learn more by reading the code and comparing it to the specification.

Versioning

Semantic Versioning

Tendermint uses Semantic Versioning to determine when and how the version changes. According to SemVer, anything in the public API can change at any time before version 1.0.0

To provide some stability to Tendermint users in these 0.X.X days, the MINOR version is used to signal breaking changes across a subset of the total public API. This subset includes all interfaces exposed to other processes (cli, rpc, p2p, etc.), but does not include the in-process Go APIs.

That said, breaking changes in the following packages will be documented in the CHANGELOG even if they don't lead to MINOR version bumps:

  • crypto
  • types
  • rpc/client
  • config
  • node
  • libs
    • bech32
    • common
    • db
    • errors
    • log

Exported objects in these packages that are not covered by the versioning scheme are explicitly marked by // UNSTABLE in their go doc comment and may change at any time without notice. Functions, types, and values in any other package may also change at any time.

Upgrades

In an effort to avoid accumulating technical debt prior to 1.0.0, we do not guarantee that breaking changes (ie. bumps in the MINOR version) will work with existing tendermint blockchains. In these cases you will have to start a new blockchain, or write something custom to get the old data into the new chain.

However, any bump in the PATCH version should be compatible with existing histories (if not please open an issue).

For more information on upgrading, see UPGRADING.md

Resources

Tendermint Core

For details about the blockchain data structures and the p2p protocols, see the Tendermint specification.

For details on using the software, see the documentation which is also hosted at: https://tendermint.com/docs/

Tools

Benchmarking and monitoring is provided by tm-bench and tm-monitor, respectively. Their code is found here and these binaries need to be built seperately. Additional documentation is found here.

Sub-projects

  • Amino, reflection-based proto3, with interfaces
  • IAVL, Merkleized IAVL+ Tree implementation

Applications

Research

Description
No description provided
Readme Apache-2.0 63 MiB
Languages
Go 72.6%
C 17.7%
Shell 3.2%
Python 1.6%
Makefile 1%
Other 3.7%