fluencelabs/tendermint
1
0
Fork 0
mirror of https://github.com/fluencelabs/tendermint synced 2025-05-10 05:37:12 +00:00
Code Issues Projects Releases Wiki Activity

[docs] signal handling

Browse Source 
Refs #1494
This commit is contained in:
Anton Kaliaev 2018-05-28 16:20:15 +04:00
parent 82ded582f2
commit b542dce2e1
No known key found for this signature in database
GPG Key ID: 7B6881D965918214
1 changed files with 27 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
docs/running-in-production.rst
View File

@ -6,13 +6,11 @@ Logging
Default logging level (``main:info,state:info,*:``) should suffice for normal Default logging level (``main:info,state:info,*:``) should suffice for normal
operation mode. Read `this post operation mode. Read `this post
<https://blog.cosmos.network/one-of-the-exciting-new-features-in-0-10-0-release-is-smart-log-level-flag-e2506b4ab756>__` <https://blog.cosmos.network/one-of-the-exciting-new-features-in-0-10-0-release-is-smart-log-level-flag-e2506b4ab756>`__
for details on how to configure ``log_level`` config variable. Some of the for details on how to configure ``log_level`` config variable. Some of the
modules can be found `here <./how-to-read-logs.html#list-of-modules>__`. modules can be found `here <./how-to-read-logs.html#list-of-modules>`__. If
you're trying to debug Tendermint or asked to provide logs with debug logging
If you're trying to debug Tendermint or asked to provide logs with debug level, you can do so by running tendermint with ``--log_level="*:debug"``.
logging level, you can do so by running tendermint with
``--log_level="*:debug"``.
Consensus WAL Consensus WAL
------------- -------------
@ -20,11 +18,11 @@ Consensus WAL
Consensus module writes every message to the WAL (write-ahead log). Consensus module writes every message to the WAL (write-ahead log).
It also issues fsync syscall through `File#Sync It also issues fsync syscall through `File#Sync
<https://golang.org/pkg/os/#File.Sync>__` for messages signed by this node (to <https://golang.org/pkg/os/#File.Sync>`__ for messages signed by this node (to
prevent double signing). prevent double signing).
Under the hood, it uses `autofile.Group Under the hood, it uses `autofile.Group
<https://godoc.org/github.com/tendermint/tmlibs/autofile#Group>__`, which <https://godoc.org/github.com/tendermint/tmlibs/autofile#Group>`__, which
rotates files when those get too big (> 10MB). rotates files when those get too big (> 10MB).
The total maximum size is 1GB. We only need the latest block and the block before it, The total maximum size is 1GB. We only need the latest block and the block before it,
@ -47,15 +45,15 @@ private validator contains the ``LastSignBytes`` and then we’ll replay the
precommit from the WAL. precommit from the WAL.
Make sure to read about `WAL corruption Make sure to read about `WAL corruption
<./specification/corruption.html#wal-corruption>__` and recovery strategies. <./specification/corruption.html#wal-corruption>`__ and recovery strategies.
DOS Exposure and Mitigation DOS Exposure and Mitigation
--------------------------- ---------------------------
Validators are supposed to setup `Sentry Node Architecture Validators are supposed to setup `Sentry Node Architecture
<https://blog.cosmos.network/tendermint-explained-bringing-bft-based-pos-to-the-public-blockchain-domain-f22e274a0fdb>__` <https://blog.cosmos.network/tendermint-explained-bringing-bft-based-pos-to-the-public-blockchain-domain-f22e274a0fdb>`__
to prevent Denial-of-service attacks. You can read more about it `here to prevent Denial-of-service attacks. You can read more about it `here
<https://github.com/tendermint/aib-data/blob/develop/medium/TendermintBFT.md>__`. <https://github.com/tendermint/aib-data/blob/develop/medium/TendermintBFT.md>`__.
Blockchain Reactor Blockchain Reactor
~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~
@ -80,7 +78,7 @@ Sending incorrectly encoded data will result in stopping the peer.
Evidence Reactor Evidence Reactor
~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~
`#1503 <https://github.com/tendermint/tendermint/issues/1503>__` `#1503 <https://github.com/tendermint/tendermint/issues/1503>`__
Sending invalid evidence will result in stopping the peer. Sending invalid evidence will result in stopping the peer.
@ -90,7 +88,7 @@ in stopping the peer.
PEX Reactor PEX Reactor
~~~~~~~~~~~ ~~~~~~~~~~~
Defines only ``SendQueueCapacity``. `#1503 <https://github.com/tendermint/tendermint/issues/1503>__` Defines only ``SendQueueCapacity``. `#1503 <https://github.com/tendermint/tendermint/issues/1503>`__
Implements rate-limiting by enforcing minimal time between two consecutive Implements rate-limiting by enforcing minimal time between two consecutive
``pexRequestMessage`` requests. If the peer sends us addresses we did not ask, ``pexRequestMessage`` requests. If the peer sends us addresses we did not ask,
@ -102,12 +100,12 @@ in stopping the peer.
Mempool Reactor Mempool Reactor
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
`#1503 <https://github.com/tendermint/tendermint/issues/1503>__` `#1503 <https://github.com/tendermint/tendermint/issues/1503>`__
Mempool maintains a cache of the last 10000 transactions to prevent replaying Mempool maintains a cache of the last 10000 transactions to prevent replaying
old transactions (plus transactions coming from other validators, who are old transactions (plus transactions coming from other validators, who are
continually exchanging transactions). Read `Replay Protection continually exchanging transactions). Read `Replay Protection
<./app-development.html#replay-protection>` for details. <./app-development.html#replay-protection>`__ for details.
Sending incorrectly encoded data or data exceeding ``maxMsgSize`` will result Sending incorrectly encoded data or data exceeding ``maxMsgSize`` will result
in stopping the peer. in stopping the peer.
@ -129,15 +127,15 @@ elements (100 max).
Rate-limiting and authentication are another key aspects to help protect Rate-limiting and authentication are another key aspects to help protect
against DOS attacks. While in the future we may implement these features, for against DOS attacks. While in the future we may implement these features, for
now, validators are supposed to use external tools like `NGINX now, validators are supposed to use external tools like `NGINX
<https://www.nginx.com/blog/rate-limiting-nginx/>__` or `traefik <https://www.nginx.com/blog/rate-limiting-nginx/>`__ or `traefik
<https://docs.traefik.io/configuration/commons/#rate-limiting>__` to archive <https://docs.traefik.io/configuration/commons/#rate-limiting>`__ to archive
the same things. the same things.
Debugging Tendermint Debugging Tendermint
-------------------- --------------------
If you ever have to debug Tendermint, the first thing you should probably do is If you ever have to debug Tendermint, the first thing you should probably do is
to check out the logs. See `"How to read logs" <./how-to-read-logs.html>__`, to check out the logs. See `"How to read logs" <./how-to-read-logs.html>`__,
where we explain what certain log statements mean. where we explain what certain log statements mean.
If, after skimming through the logs, things are not clear still, the second If, after skimming through the logs, things are not clear still, the second
@ -159,8 +157,8 @@ $ curl http(s)://{ip}:{rpcPort}/dump_consensus_state
There is a reduced version of this endpoint - `/consensus_state`, which There is a reduced version of this endpoint - `/consensus_state`, which
returns just the votes seen at the current height. returns just the votes seen at the current height.
- `Github Issues <https://github.com/tendermint/tendermint/issues>__` - `Github Issues <https://github.com/tendermint/tendermint/issues>`__
- `StackOverflow questions <https://stackoverflow.com/questions/tagged/tendermint>__` - `StackOverflow questions <https://stackoverflow.com/questions/tagged/tendermint>`__
Monitoring Tendermint Monitoring Tendermint
--------------------- ---------------------
@ -174,16 +172,23 @@ Other useful endpoints include mentioned earlier `/status`, `/net_info` and
We have a small tool, called tm-monitor, which outputs information from the We have a small tool, called tm-monitor, which outputs information from the
endpoints above plus some statistics. The tool can be found `here endpoints above plus some statistics. The tool can be found `here
<https://github.com/tendermint/tools/tree/master/tm-monitor>__`. <https://github.com/tendermint/tools/tree/master/tm-monitor>`__.
What happens when my app die? What happens when my app die?
----------------------------- -----------------------------
You are supposed to run Tendermint under a `process supervisor You are supposed to run Tendermint under a `process supervisor
<https://en.wikipedia.org/wiki/Process_supervision>__` (like systemd or runit). <https://en.wikipedia.org/wiki/Process_supervision>`__ (like systemd or runit).
It will ensure Tendermint is always running (despite possible errors). It will ensure Tendermint is always running (despite possible errors).
Getting back to the original question, if your application dies, Tendermint Getting back to the original question, if your application dies, Tendermint
will panic. After a process supervisor restarts your application, Tendermint will panic. After a process supervisor restarts your application, Tendermint
should be able to reconnect successfully. The order of restart does not matter should be able to reconnect successfully. The order of restart does not matter
for it. for it.
Signal handling
---------------
We catch SIGINT and SIGTERM and try to clean up nicely. For other signals we
use the default behaviour in Go: `Default behavior of signals in Go programs
<https://golang.org/pkg/os/signal/#hdr-Default_behavior_of_signals_in_Go_programs>`__.