consensus: fix addProposalBlockPart

* When create_empty_blocks=false, we don't enterPropose until we * receive a transaction, but if we then receive a complete proposal, * we should enterPrevote. A guard in addProposalBlockPart was checking if * step==Propose before calling enterPrevote, but we need it to be step<=Propose, * since we may not have seen a tx. * This was discovered by disabling mempool broadcast, sending txs to * peers one a time, and observing their consensus logs.
mempool: log hashes, not whole tx
2025-07-16 04:41:59 +00:00 · 2018-06-22 15:08:39 -04:00 · 2018-06-22 15:06:43 -04:00 · 2018-06-21 12:44:31 -07:00 · 2018-06-21 12:47:29 -07:00 · 2018-06-20 00:22:07 -07:00
222 changed files with 9212 additions and 4566 deletions
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -3,7 +3,7 @@ version: 2
 defaults: &defaults
  working_directory: /go/src/github.com/tendermint/tendermint
  docker:
-    - image: circleci/golang:1.10.0
+    - image: circleci/golang:1.10.3
  environment:
    GOBIN: /tmp/workspace/bin

@@ -16,7 +16,7 @@ jobs:
      - checkout
      - restore_cache:
          keys:
-            - v1-pkg-cache
+            - v2-pkg-cache
      - run:
          name: tools
          command: |
@@ -38,11 +38,11 @@ jobs:
            - bin
            - profiles
      - save_cache:
-          key: v1-pkg-cache
+          key: v2-pkg-cache
          paths:
            - /go/pkg
      - save_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
          paths:
            - /go/src/github.com/tendermint/tendermint

@@ -52,9 +52,9 @@ jobs:
      - attach_workspace:
          at: /tmp/workspace
      - restore_cache:
-          key: v1-pkg-cache
+          key: v2-pkg-cache
      - restore_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
      - run:
          name: Checkout abci
          command: |
@@ -77,15 +77,31 @@ jobs:
          paths:
            - "bin/abci*"

+  build_slate:
+    <<: *defaults
+    steps:
+      - attach_workspace:
+          at: /tmp/workspace
+      - restore_cache:
+          key: v2-pkg-cache
+      - restore_cache:
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
+      - run:
+          name: slate docs
+          command: |
+            set -ex
+            export PATH="$GOBIN:$PATH"
+            make build-slate
+
  lint:
    <<: *defaults
    steps:
      - attach_workspace:
          at: /tmp/workspace
      - restore_cache:
-          key: v1-pkg-cache
+          key: v2-pkg-cache
      - restore_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
      - run:
          name: metalinter
          command: |
@@ -99,9 +115,9 @@ jobs:
      - attach_workspace:
          at: /tmp/workspace
      - restore_cache:
-          key: v1-pkg-cache
+          key: v2-pkg-cache
      - restore_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
      - run: sudo apt-get update && sudo apt-get install -y --no-install-recommends bsdmainutils
      - run:
          name: Run tests
@@ -114,21 +130,24 @@ jobs:
      - attach_workspace:
          at: /tmp/workspace
      - restore_cache:
-          key: v1-pkg-cache
+          key: v2-pkg-cache
      - restore_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
+      - run: mkdir -p /tmp/logs
      - run:
          name: Run tests
          command: |
            for pkg in $(go list github.com/tendermint/tendermint/... | grep -v /vendor/ | circleci tests split --split-by=timings); do
              id=$(basename "$pkg")

-              go test -timeout 5m -race -coverprofile=/tmp/workspace/profiles/$id.out -covermode=atomic "$pkg"
+              GOCACHE=off go test -v -timeout 5m -race -coverprofile=/tmp/workspace/profiles/$id.out -covermode=atomic "$pkg" | tee "/tmp/logs/$id-$RANDOM.log"
            done
      - persist_to_workspace:
          root: /tmp/workspace
          paths:
            - "profiles/*"
+      - store_artifacts:
+          path: /tmp/logs

  test_persistence:
    <<: *defaults
@@ -136,9 +155,9 @@ jobs:
      - attach_workspace:
          at: /tmp/workspace
      - restore_cache:
-          key: v1-pkg-cache
+          key: v2-pkg-cache
      - restore_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
      - run:
          name: Run tests
          command: bash test/persist/test_failure_indices.sh
@@ -161,7 +180,7 @@ jobs:
      - attach_workspace:
          at: /tmp/workspace
      - restore_cache:
-          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+          key: v2-tree-{{ .Environment.CIRCLE_SHA1 }}
      - run:
          name: gather
          command: |
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -1,4 +1,4 @@
 # CODEOWNERS: https://help.github.com/articles/about-codeowners/

-# Everything goes through Bucky and Anton. For now.
-*       @ebuchman @melekes
+# Everything goes through Bucky, Anton, Alex. For now.
+*       @ebuchman @melekes @xla
--- a/.github/ISSUE_TEMPLATE
+++ b/.github/ISSUE_TEMPLATE
@@ -35,6 +35,8 @@ in a case of bug.

 **Logs (you can paste a part showing an error or attach the whole file)**:

+**Config (you can paste only the changes you've made)**:
+
 **`/dump_consensus_state` output for consensus bugs**

 **Anything else do we need to know**:
--- a/.gitignore
+++ b/.gitignore
@@ -5,7 +5,6 @@
 .DS_Store
 build/*
 rpc/test/.tendermint
-.debora
 .tendermint
 remote_dump
 .revision
@@ -13,11 +12,10 @@ vendor
 .vagrant
 test/p2p/data/
 test/logs
-.glide
 coverage.txt
 docs/_build
 docs/tools
-docs/abci-spec.rst
+docs/abci-spec.md
 *.log

 scripts/wal2json/wal2json
@@ -25,3 +23,5 @@ scripts/cutWALUntil/cutWALUntil

 .idea/
 *.iml
+
+libs/pubsub/query/fuzz_test/output
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,108 @@
 # Changelog

+## 0.21.0
+
+*June 21th, 2018*
+
+BREAKING CHANGES
+
+- [config] Change default ports from 4665X to 2665X. Ports over 32768 are
+  ephemeral and reserved for use by the kernel.
+- [cmd] `unsafe_reset_all` removes the addrbook.json
+
+IMPROVEMENT
+
+- [pubsub] Set default capacity to 0
+- [docs] Various improvements
+
+BUG FIXES
+
+- [consensus] Fix an issue where we don't make blocks after `fast_sync` when `create_empty_blocks=false`
+- [mempool] Fix #1761 where we don't process txs if `cache_size=0`
+- [rpc] Fix memory leak in Websocket (when using `/subscribe` method)
+- [config] Escape paths in config - fixes config paths on Windows
+
+## 0.20.0
+
+*June 6th, 2018*
+
+This is the first in a series of breaking releases coming to Tendermint after
+soliciting developer feedback and conducting security audits.
+
+This release does not break any blockchain data structures or
+protocols other than the ABCI messages between Tendermint and the application.
+
+Applications that upgrade for ABCI v0.11.0 should be able to continue running Tendermint
+v0.20.0 on blockchains created with v0.19.X
+
+BREAKING CHANGES
+
+- [abci] Upgrade to
+  [v0.11.0](https://github.com/tendermint/abci/blob/master/CHANGELOG.md#0110)
+- [abci] Change Query path for filtering peers by node ID from
+  `p2p/filter/pubkey/<id>` to `p2p/filter/id/<id>`
+
+## 0.19.9
+
+*June 5th, 2018*
+
+BREAKING CHANGES
+
+- [types/priv_validator] Moved to top level `privval` package
+
+FEATURES
+
+- [config] Collapse PeerConfig into P2PConfig
+- [docs] Add quick-install script
+- [docs/spec] Add table of Amino prefixes
+
+BUG FIXES
+
+- [rpc] Return 404 for unknown endpoints
+- [consensus] Flush WAL on stop
+- [evidence] Don't send evidence to peers that are behind
+- [p2p] Fix memory leak on peer disconnects
+- [rpc] Fix panic when `per_page=0`
+
+## 0.19.8
+
+*June 4th, 2018*
+
+BREAKING:
+
+- [p2p] Remove `auth_enc` config option, peer connections are always auth
+  encrypted. Technically a breaking change but seems no one was using it and
+  arguably a bug fix :)
+
+BUG FIXES
+
+- [mempool] Fix deadlock under high load when `skip_timeout_commit=true` and
+  `create_empty_blocks=false`
+
+## 0.19.7
+
+*May 31st, 2018*
+
+BREAKING:
+
+- [libs/pubsub] TagMap#Get returns a string value
+- [libs/pubsub] NewTagMap accepts a map of strings
+
+FEATURES
+
+- [rpc] the RPC documentation is now published to https://tendermint.github.io/slate
+- [p2p] AllowDuplicateIP config option to refuse connections from same IP.
+    - true by default for now, false by default in next breaking release
+- [docs] Add docs for query, tx indexing, events, pubsub
+- [docs] Add some notes about running Tendermint in production
+
+IMPROVEMENTS:
+
+- [consensus] Consensus reactor now receives events from a separate synchronous event bus,
+  which is not dependant on external RPC load
+- [consensus/wal] do not look for height in older files if we've seen height - 1
+- [docs] Various cleanup and link fixes
+
 ## 0.19.6

 *May 29th, 2018*
@@ -8,6 +111,12 @@ BUG FIXES

 - [blockchain] Fix fast-sync deadlock during high peer turnover

+BUG FIX:
+
+- [evidence] Dont send peers evidence from heights they haven't synced to yet
+- [p2p] Refuse connections to more than one peer with the same IP
+- [docs] Various fixes
+
 ## 0.19.5

 *May 20th, 2018*
--- a/DOCKER/Dockerfile
+++ b/DOCKER/Dockerfile
@@ -28,7 +28,7 @@ VOLUME [ $TMHOME ]
 WORKDIR $TMHOME

 # p2p and rpc port
-EXPOSE 46656 46657
+EXPOSE 26656 26657

 ENTRYPOINT ["/usr/bin/tendermint"]
 CMD ["node", "--moniker=`hostname`"]
--- a/DOCKER/Dockerfile.develop
+++ b/DOCKER/Dockerfile.develop
@@ -27,8 +27,8 @@ RUN mkdir -p /go/src/github.com/tendermint/tendermint && \

 VOLUME $DATA_ROOT

-EXPOSE 46656
-EXPOSE 46657
+EXPOSE 26656
+EXPOSE 26657

 ENTRYPOINT ["tendermint"]

--- a/DOCKER/Dockerfile.testing
+++ b/DOCKER/Dockerfile.testing
@@ -13,6 +13,6 @@ RUN echo 'deb http://httpredir.debian.org/debian testing main non-free contrib'

 VOLUME /go

-EXPOSE 46656
-EXPOSE 46657
+EXPOSE 26656
+EXPOSE 26657

--- a/DOCKER/README.md
+++ b/DOCKER/README.md
@@ -1,4 +1,6 @@
-# Supported tags and respective `Dockerfile` links
+# Docker
+
+## Supported tags and respective `Dockerfile` links

 - `0.17.1`, `latest` [(Dockerfile)](https://github.com/tendermint/tendermint/blob/208ac32fa266657bd6c304e84ec828aa252bb0b8/DOCKER/Dockerfile)
 - `0.15.0` [(Dockerfile)](https://github.com/tendermint/tendermint/blob/170777300ea92dc21a8aec1abc16cb51812513a4/DOCKER/Dockerfile)
@@ -14,10 +16,10 @@

 `develop` tag points to the [develop](https://github.com/tendermint/tendermint/tree/develop) branch.

-# Quick reference
+## Quick reference

 * **Where to get help:**
-  https://tendermint.com/community
+  https://cosmos.network/community

 * **Where to file issues:**
  https://github.com/tendermint/tendermint/issues
@@ -25,7 +27,7 @@
 * **Supported Docker versions:**
  [the latest release](https://github.com/moby/moby/releases) (down to 1.6 on a best-effort basis)

-# Tendermint
+## Tendermint

 Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine, written in any programming language, and securely replicates it on many machines.

@@ -33,29 +35,33 @@ For more background, see the [introduction](https://tendermint.readthedocs.io/en

 To get started developing applications, see the [application developers guide](https://tendermint.readthedocs.io/en/master/getting-started.html).

-# How to use this image
+## How to use this image

-## Start one instance of the Tendermint core with the `kvstore` app
+### Start one instance of the Tendermint core with the `kvstore` app

-A very simple example of a built-in app and Tendermint core in one container.
+A quick example of a built-in app and Tendermint core in one container.

 ```
 docker run -it --rm -v "/tmp:/tendermint" tendermint/tendermint init
 docker run -it --rm -v "/tmp:/tendermint" tendermint/tendermint node --proxy_app=kvstore
 ```

-## mintnet-kubernetes
+## Local cluster

-If you want to see many containers talking to each other, consider using [mintnet-kubernetes](https://github.com/tendermint/tools/tree/master/mintnet-kubernetes), which is a tool for running Tendermint-based applications on a Kubernetes cluster.
+To run a 4-node network, see the `Makefile` in the root of [the repo](https://github.com/tendermint/tendermint/master/Makefile) and run:

-# License
+```
+make build-linux
+make build-docker-localnode
+make localnet-start
+```

-View [license information](https://raw.githubusercontent.com/tendermint/tendermint/master/LICENSE) for the software contained in this image.
+Note that this will build and use a different image than the ones provided here.

-# User Feedback
+## License
+
+- Tendermint's license is [Apache 2.0](https://github.com/tendermint/tendermint/master/LICENSE).

 ## Contributing

-You are invited to contribute new features, fixes, or updates, large or small; we are always thrilled to receive pull requests, and do our best to process them as fast as we can.
-
-Before you start to code, we recommend discussing your plans through a [GitHub](https://github.com/tendermint/tendermint/issues) issue, especially for more ambitious contributions. This gives other contributors a chance to point you in the right direction, give you feedback on your design, and help you find out if someone else is working on the same thing.
+Contributions are most welcome! See the [contributing file](https://github.com/tendermint/tendermint/blob/master/CONTRIBUTING.md) for more information.
--- a/Gopkg.lock
+++ b/Gopkg.lock
@@ -5,7 +5,7 @@
  branch = "master"
  name = "github.com/btcsuite/btcd"
  packages = ["btcec"]
-  revision = "675abc5df3c5531bc741b56a765e35623459da6d"
+  revision = "86fed781132ac890ee03e906e4ecd5d6fa180c64"

 [[projects]]
  name = "github.com/davecgh/go-spew"
@@ -82,7 +82,7 @@
  branch = "master"
  name = "github.com/golang/snappy"
  packages = ["."]
-  revision = "553a641470496b2327abcac10b36396bd98e45c9"
+  revision = "2e65f85255dbc3072edf28d6b5b8efc472979f5a"

 [[projects]]
  name = "github.com/gorilla/websocket"
@@ -128,20 +128,20 @@
 [[projects]]
  name = "github.com/magiconair/properties"
  packages = ["."]
-  revision = "c3beff4c2358b44d0493c7dda585e7db7ff28ae6"
-  version = "v1.7.6"
+  revision = "c2353362d570a7bfa228149c62842019201cfb71"
+  version = "v1.8.0"

 [[projects]]
  branch = "master"
  name = "github.com/mitchellh/mapstructure"
  packages = ["."]
-  revision = "00c29f56e2386353d58c599509e8dc3801b0d716"
+  revision = "bb74f1db0675b241733089d5a1faa5dd8b0ef57b"

 [[projects]]
  name = "github.com/pelletier/go-toml"
  packages = ["."]
-  revision = "acdc4509485b587f5e675510c4f2c63e90ff68a8"
-  version = "v1.1.0"
+  revision = "c01d1270ff3e442a8a57cddc1c92dc1138598194"
+  version = "v1.2.0"

 [[projects]]
  name = "github.com/pkg/errors"
@@ -159,7 +159,7 @@
  branch = "master"
  name = "github.com/rcrowley/go-metrics"
  packages = ["."]
-  revision = "d932a24a8ccb8fcadc993e5c6c58f93dac168294"
+  revision = "e2704e165165ec55d062f5919b4b29494e9fa790"

 [[projects]]
  name = "github.com/spf13/afero"
@@ -167,8 +167,8 @@
    ".",
    "mem"
  ]
-  revision = "63644898a8da0bc22138abf860edaf5277b6102e"
-  version = "v1.1.0"
+  revision = "787d034dfe70e44075ccc060d346146ef53270ad"
+  version = "v1.1.1"

 [[projects]]
  name = "github.com/spf13/cast"
@@ -179,8 +179,8 @@
 [[projects]]
  name = "github.com/spf13/cobra"
  packages = ["."]
-  revision = "a1f051bc3eba734da4772d60e2d677f47cf93ef4"
-  version = "v0.0.2"
+  revision = "ef82de70bb3f60c65fb8eebacbb2d122ef517385"
+  version = "v0.0.3"

 [[projects]]
  branch = "master"
@@ -206,8 +206,8 @@
    "assert",
    "require"
  ]
-  revision = "12b6f73e6084dad08a7c6e575284b177ecafbc71"
-  version = "v1.2.1"
+  revision = "f35b8ab0b5a2cef36673838d662e249dd9c94686"
+  version = "v1.2.2"

 [[projects]]
  branch = "master"
@@ -226,7 +226,7 @@
    "leveldb/table",
    "leveldb/util"
  ]
-  revision = "714f901b98fdb3aa954b4193d8cbd64a28d80cad"
+  revision = "e2150783cd35f5b607daca48afd8c57ec54cc995"

 [[projects]]
  name = "github.com/tendermint/abci"
@@ -238,8 +238,8 @@
    "server",
    "types"
  ]
-  revision = "78a8905690ef54f9d57e3b2b0ee7ad3a04ef3f1f"
-  version = "v0.10.3"
+  revision = "198dccf0ddfd1bb176f87657e3286a05a6ed9540"
+  version = "v0.12.0"

 [[projects]]
  branch = "master"
@@ -263,12 +263,6 @@
  revision = "915416979bf70efa4bcbf1c6cd5d64c5fff9fc19"
  version = "v0.6.2"

-[[projects]]
-  name = "github.com/tendermint/go-wire"
-  packages = ["."]
-  revision = "fa721242b042ecd4c6ed1a934ee740db4f74e45c"
-  version = "v0.7.3"
-
 [[projects]]
  name = "github.com/tendermint/tmlibs"
  packages = [
@@ -281,12 +275,10 @@
    "flowrate",
    "log",
    "merkle",
-    "pubsub",
-    "pubsub/query",
    "test"
  ]
-  revision = "cc5f287c4798ffe88c04d02df219ecb6932080fd"
-  version = "v0.8.3-rc0"
+  revision = "692f1d86a6e2c0efa698fd1e4541b68c74ffaf38"
+  version = "v0.8.4"

 [[projects]]
  branch = "master"
@@ -301,7 +293,7 @@
    "ripemd160",
    "salsa20/salsa"
  ]
-  revision = "b0697eccbea9adec5b7ba8008f4c33d98d733388"
+  revision = "8ac0e0d97ce45cd83d1d7243c060cb8461dda5e9"

 [[projects]]
  branch = "master"
@@ -313,16 +305,15 @@
    "http2/hpack",
    "idna",
    "internal/timeseries",
-    "lex/httplex",
    "trace"
  ]
-  revision = "5f9ae10d9af5b1c89ae6904293b14b064d4ada23"
+  revision = "db08ff08e8622530d9ed3a0e8ac279f6d4c02196"

 [[projects]]
  branch = "master"
  name = "golang.org/x/sys"
  packages = ["unix"]
-  revision = "bb9c189858d91f42db229b04d45a4c3d23a7662a"
+  revision = "a9e25c09b96b8870693763211309e213c6ef299d"

 [[projects]]
  name = "golang.org/x/text"
@@ -346,7 +337,6 @@
  version = "v0.3.0"

 [[projects]]
-  branch = "master"
  name = "google.golang.org/genproto"
  packages = ["googleapis/rpc/status"]
  revision = "7fd901a49ba6a7f87732eb344f6e3c5b19d1b200"
@@ -384,6 +374,6 @@
 [solve-meta]
  analyzer-name = "dep"
  analyzer-version = 1
-  inputs-digest = "52a0dcbebdf8714612444914cfce59a3af8c47c4453a2d43c4ccc5ff1a91d8ea"
+  inputs-digest = "62d0626fa963b25411234da915fb3b8bfc7aefffd76824d27ac1647ca2b5d489"
  solver-name = "gps-cdcl"
  solver-version = 1
--- a/Gopkg.toml
+++ b/Gopkg.toml
@@ -71,7 +71,7 @@

 [[constraint]]
  name = "github.com/tendermint/abci"
-  version = "~0.10.3"
+  version = "~0.12.0"

 [[constraint]]
  name = "github.com/tendermint/go-crypto"
@@ -79,16 +79,21 @@

 [[constraint]]
  name = "github.com/tendermint/go-amino"
-  version = "0.9.9"
+  version = "=0.9.9"

 [[override]]
  name = "github.com/tendermint/tmlibs"
-  version = "~0.8.3-rc0"
+  version = "~0.8.4"

 [[constraint]]
  name = "google.golang.org/grpc"
  version = "~1.7.3"

+# this got updated and broke, so locked to an old working commit ...
+[[override]]
+  name = "google.golang.org/genproto"
+  revision = "7fd901a49ba6a7f87732eb344f6e3c5b19d1b200"
+
 [prune]
  go-tests = true
  unused-packages = true
--- a/7
+++ b/7
@@ -226,8 +226,11 @@ sentry-stop:
 	@if [ -z "$(DO_API_TOKEN)" ]; then echo "DO_API_TOKEN environment variable not set." ; false ; fi
 	cd networks/remote/terraform && terraform destroy -var DO_API_TOKEN="$(DO_API_TOKEN)" -var SSH_KEY_FILE="$(HOME)/.ssh/id_rsa.pub"

+# meant for the CI, inspect script & adapt accordingly
+build-slate:
+	bash scripts/slate.sh
+
 # To avoid unintended conflicts with file names, always add to .PHONY
 # unless there is a reason not to.
 # https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html
-.PHONY: check build build_race dist install check_tools get_tools update_tools get_vendor_deps draw_deps test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop
-
+.PHONY: check build build_race dist install check_tools get_tools update_tools get_vendor_deps draw_deps test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@ Or [Blockchain](https://en.wikipedia.org/wiki/Blockchain_(database)) for short.
 https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667
 )](https://godoc.org/github.com/tendermint/tendermint)
 [![Go version](https://img.shields.io/badge/go-1.9.2-blue.svg)](https://github.com/moovweb/gvm)
-[![Rocket.Chat](https://demo.rocket.chat/images/join-chat.svg)](https://cosmos.rocket.chat/)
+[![riot.im](https://img.shields.io/badge/riot.im-JOIN%20CHAT-green.svg)](https://riot.im/app/#/room/#tendermint:matrix.org)
 [![license](https://img.shields.io/github/license/tendermint/tendermint.svg)](https://github.com/tendermint/tendermint/blob/master/LICENSE)
 [![](https://tokei.rs/b1/github/tendermint/tendermint?category=lines)](https://github.com/tendermint/tendermint)

@@ -19,13 +19,22 @@ Branch    | Tests | Coverage
 master    | [![CircleCI](https://circleci.com/gh/tendermint/tendermint/tree/master.svg?style=shield)](https://circleci.com/gh/tendermint/tendermint/tree/master) | [![codecov](https://codecov.io/gh/tendermint/tendermint/branch/master/graph/badge.svg)](https://codecov.io/gh/tendermint/tendermint)
 develop   | [![CircleCI](https://circleci.com/gh/tendermint/tendermint/tree/develop.svg?style=shield)](https://circleci.com/gh/tendermint/tendermint/tree/develop) | [![codecov](https://codecov.io/gh/tendermint/tendermint/branch/develop/graph/badge.svg)](https://codecov.io/gh/tendermint/tendermint)

-_NOTE: This is alpha software. Please contact us if you intend to run it in production._
-
 Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language -
 and securely replicates it on many machines.

 For protocol details, see [the specification](/docs/spec).

+## A Note on Production Readiness
+
+While Tendermint is being used in production in private, permissioned
+environments, we are still working actively to harden and audit it in preparation
+for use in public blockchains, such as the [Cosmos Network](https://cosmos.network/).
+We are also still making breaking changes to the protocol and the APIs.
+Thus we tag the releases as *alpha software*.
+
+In any case, if you intend to run Tendermint in production,
+please [contact us](https://riot.im/app/#/room/#tendermint:matrix.org) :)
+
 ## Security

 To report a security vulnerability, see our [bug bounty
@@ -54,9 +63,13 @@ See the [install instructions](/docs/install.rst)

 ### Tendermint Core

-For more, [Read The Docs](https://tendermint.readthedocs.io/en/master/).
+For details about the blockchain data structures and the p2p protocols, see the
+the [Tendermint specification](/docs/spec).
+
+For details on using the software, [Read The Docs](https://tendermint.readthedocs.io/en/master/).
 Additional information about some - and eventually all - of the sub-projects below, can be found at Read The Docs.

+
 ### Sub-projects

 * [ABCI](http://github.com/tendermint/abci), the Application Blockchain Interface
--- a/benchmarks/simu/counter.go
+++ b/benchmarks/simu/counter.go
@@ -11,7 +11,7 @@ import (
 )

 func main() {
-	wsc := rpcclient.NewWSClient("127.0.0.1:46657", "/websocket")
+	wsc := rpcclient.NewWSClient("127.0.0.1:26657", "/websocket")
 	err := wsc.Start()
 	if err != nil {
 		cmn.Exit(err.Error())
--- a/blockchain/reactor_test.go
+++ b/blockchain/reactor_test.go
@@ -1,6 +1,7 @@
 package blockchain

 import (
+	"net"
 	"testing"

 	cmn "github.com/tendermint/tmlibs/common"
@@ -35,7 +36,7 @@ func newBlockchainReactor(logger log.Logger, maxBlockHeight int64) *BlockchainRe
 	fastSync := true
 	var nilApp proxy.AppConnConsensus
 	blockExec := sm.NewBlockExecutor(dbm.NewMemDB(), log.TestingLogger(), nilApp,
-		types.MockMempool{}, types.MockEvidencePool{})
+		sm.MockMempool{}, sm.MockEvidencePool{})

 	bcReactor := NewBlockchainReactor(state.Copy(), blockExec, blockStore, fastSync)
 	bcReactor.SetLogger(logger.With("module", "blockchain"))
@@ -204,3 +205,4 @@ func (tp *bcrTestPeer) IsOutbound() bool                     { return false }
 func (tp *bcrTestPeer) IsPersistent() bool                   { return true }
 func (tp *bcrTestPeer) Get(s string) interface{}             { return s }
 func (tp *bcrTestPeer) Set(string, interface{})              {}
+func (tp *bcrTestPeer) RemoteIP() net.IP                     { return []byte{127, 0, 0, 1} }
--- a/cmd/priv_val_server/main.go
+++ b/cmd/priv_val_server/main.go
@@ -8,12 +8,12 @@ import (
 	cmn "github.com/tendermint/tmlibs/common"
 	"github.com/tendermint/tmlibs/log"

-	priv_val "github.com/tendermint/tendermint/types/priv_validator"
+	"github.com/tendermint/tendermint/privval"
 )

 func main() {
 	var (
-		addr        = flag.String("addr", ":46659", "Address of client to connect to")
+		addr        = flag.String("addr", ":26659", "Address of client to connect to")
 		chainID     = flag.String("chain-id", "mychain", "chain id")
 		privValPath = flag.String("priv", "", "priv val file path")

@@ -30,13 +30,13 @@ func main() {
 		"privPath", *privValPath,
 	)

-	privVal := priv_val.LoadFilePV(*privValPath)
+	pv := privval.LoadFilePV(*privValPath)

-	rs := priv_val.NewRemoteSigner(
+	rs := privval.NewRemoteSigner(
 		logger,
 		*chainID,
 		*addr,
-		privVal,
+		pv,
 		crypto.GenPrivKeyEd25519(),
 	)
 	err := rs.Start()
--- a/cmd/tendermint/commands/gen_validator.go
+++ b/cmd/tendermint/commands/gen_validator.go
@@ -5,7 +5,7 @@ import (

 	"github.com/spf13/cobra"

-	pvm "github.com/tendermint/tendermint/types/priv_validator"
+	"github.com/tendermint/tendermint/privval"
 )

 // GenValidatorCmd allows the generation of a keypair for a
@@ -17,7 +17,7 @@ var GenValidatorCmd = &cobra.Command{
 }

 func genValidator(cmd *cobra.Command, args []string) {
-	pv := pvm.GenFilePV("")
+	pv := privval.GenFilePV("")
 	jsbz, err := cdc.MarshalJSON(pv)
 	if err != nil {
 		panic(err)
--- a/cmd/tendermint/commands/init.go
+++ b/cmd/tendermint/commands/init.go
@@ -7,8 +7,8 @@ import (

 	cfg "github.com/tendermint/tendermint/config"
 	"github.com/tendermint/tendermint/p2p"
+	"github.com/tendermint/tendermint/privval"
 	"github.com/tendermint/tendermint/types"
-	pvm "github.com/tendermint/tendermint/types/priv_validator"
 	cmn "github.com/tendermint/tmlibs/common"
 )

@@ -26,12 +26,12 @@ func initFiles(cmd *cobra.Command, args []string) error {
 func initFilesWithConfig(config *cfg.Config) error {
 	// private validator
 	privValFile := config.PrivValidatorFile()
-	var pv *pvm.FilePV
+	var pv *privval.FilePV
 	if cmn.FileExists(privValFile) {
-		pv = pvm.LoadFilePV(privValFile)
+		pv = privval.LoadFilePV(privValFile)
 		logger.Info("Found private validator", "path", privValFile)
 	} else {
-		pv = pvm.GenFilePV(privValFile)
+		pv = privval.GenFilePV(privValFile)
 		pv.Save()
 		logger.Info("Generated private validator", "path", privValFile)
 	}
--- a/cmd/tendermint/commands/lite.go
+++ b/cmd/tendermint/commands/lite.go
@@ -35,7 +35,7 @@ var (

 func init() {
 	LiteCmd.Flags().StringVar(&listenAddr, "laddr", "tcp://localhost:8888", "Serve the proxy on the given address")
-	LiteCmd.Flags().StringVar(&nodeAddr, "node", "tcp://localhost:46657", "Connect to a Tendermint node at this address")
+	LiteCmd.Flags().StringVar(&nodeAddr, "node", "tcp://localhost:26657", "Connect to a Tendermint node at this address")
 	LiteCmd.Flags().StringVar(&chainID, "chain-id", "tendermint", "Specify the Tendermint chain ID")
 	LiteCmd.Flags().StringVar(&home, "home-dir", ".tendermint-lite", "Specify the home directory")
 }
--- a/cmd/tendermint/commands/reset_priv_validator.go
+++ b/cmd/tendermint/commands/reset_priv_validator.go
@@ -5,7 +5,7 @@ import (

 	"github.com/spf13/cobra"

-	pvm "github.com/tendermint/tendermint/types/priv_validator"
+	"github.com/tendermint/tendermint/privval"
 	"github.com/tendermint/tmlibs/log"
 )

@@ -13,32 +13,21 @@ import (
 // instance.
 var ResetAllCmd = &cobra.Command{
 	Use:   "unsafe_reset_all",
-	Short: "(unsafe) Remove all the data and WAL, reset this node's validator",
+	Short: "(unsafe) Remove all the data and WAL, reset this node's validator to genesis state",
 	Run:   resetAll,
 }

 // ResetPrivValidatorCmd resets the private validator files.
 var ResetPrivValidatorCmd = &cobra.Command{
 	Use:   "unsafe_reset_priv_validator",
-	Short: "(unsafe) Reset this node's validator",
+	Short: "(unsafe) Reset this node's validator to genesis state",
 	Run:   resetPrivValidator,
 }

-// ResetAll removes the privValidator files.
-// Exported so other CLI tools can use it.
-func ResetAll(dbDir, privValFile string, logger log.Logger) {
-	resetFilePV(privValFile, logger)
-	if err := os.RemoveAll(dbDir); err != nil {
-		logger.Error("Error removing directory", "err", err)
-		return
-	}
-	logger.Info("Removed all data", "dir", dbDir)
-}
-
 // XXX: this is totally unsafe.
 // it's only suitable for testnets.
 func resetAll(cmd *cobra.Command, args []string) {
-	ResetAll(config.DBDir(), config.PrivValidatorFile(), logger)
+	ResetAll(config.DBDir(), config.P2P.AddrBookFile(), config.PrivValidatorFile(), logger)
 }

 // XXX: this is totally unsafe.
@@ -47,15 +36,34 @@ func resetPrivValidator(cmd *cobra.Command, args []string) {
 	resetFilePV(config.PrivValidatorFile(), logger)
 }

-func resetFilePV(privValFile string, logger log.Logger) {
-	// Get PrivValidator
-	if _, err := os.Stat(privValFile); err == nil {
-		pv := pvm.LoadFilePV(privValFile)
-		pv.Reset()
-		logger.Info("Reset PrivValidator", "file", privValFile)
+// ResetAll removes the privValidator and address book files plus all data.
+// Exported so other CLI tools can use it.
+func ResetAll(dbDir, addrBookFile, privValFile string, logger log.Logger) {
+	resetFilePV(privValFile, logger)
+	removeAddrBook(addrBookFile, logger)
+	if err := os.RemoveAll(dbDir); err == nil {
+		logger.Info("Removed all blockchain history", "dir", dbDir)
 	} else {
-		pv := pvm.GenFilePV(privValFile)
-		pv.Save()
-		logger.Info("Generated PrivValidator", "file", privValFile)
+		logger.Error("Error removing all blockchain history", "dir", dbDir, "err", err)
+	}
+}
+
+func resetFilePV(privValFile string, logger log.Logger) {
+	if _, err := os.Stat(privValFile); err == nil {
+		pv := privval.LoadFilePV(privValFile)
+		pv.Reset()
+		logger.Info("Reset private validator file to genesis state", "file", privValFile)
+	} else {
+		pv := privval.GenFilePV(privValFile)
+		pv.Save()
+		logger.Info("Generated private validator file", "file", privValFile)
+	}
+}
+
+func removeAddrBook(addrBookFile string, logger log.Logger) {
+	if err := os.Remove(addrBookFile); err == nil {
+		logger.Info("Removed existing address book", "file", addrBookFile)
+	} else if !os.IsNotExist(err) {
+		logger.Info("Error removing address book", "file", addrBookFile, "err", err)
 	}
 }
--- a/cmd/tendermint/commands/show_validator.go
+++ b/cmd/tendermint/commands/show_validator.go
@@ -5,7 +5,7 @@ import (

 	"github.com/spf13/cobra"

-	privval "github.com/tendermint/tendermint/types/priv_validator"
+	"github.com/tendermint/tendermint/privval"
 )

 // ShowValidatorCmd adds capabilities for showing the validator info.
--- a/cmd/tendermint/commands/testnet.go
+++ b/cmd/tendermint/commands/testnet.go
@@ -12,8 +12,8 @@ import (

 	cfg "github.com/tendermint/tendermint/config"
 	"github.com/tendermint/tendermint/p2p"
+	"github.com/tendermint/tendermint/privval"
 	"github.com/tendermint/tendermint/types"
-	pvm "github.com/tendermint/tendermint/types/priv_validator"
 	cmn "github.com/tendermint/tmlibs/common"
 )

@@ -46,10 +46,10 @@ func init() {
 	TestnetFilesCmd.Flags().BoolVar(&populatePersistentPeers, "populate-persistent-peers", true,
 		"Update config of each node with the list of persistent peers build using either hostname-prefix or starting-ip-address")
 	TestnetFilesCmd.Flags().StringVar(&hostnamePrefix, "hostname-prefix", "node",
-		"Hostname prefix (node results in persistent peers list ID0@node0:46656, ID1@node1:46656, ...)")
+		"Hostname prefix (node results in persistent peers list ID0@node0:26656, ID1@node1:26656, ...)")
 	TestnetFilesCmd.Flags().StringVar(&startingIPAddress, "starting-ip-address", "",
-		"Starting IP address (192.168.0.1 results in persistent peers list ID0@192.168.0.1:46656, ID1@192.168.0.2:46656, ...)")
-	TestnetFilesCmd.Flags().IntVar(&p2pPort, "p2p-port", 46656,
+		"Starting IP address (192.168.0.1 results in persistent peers list ID0@192.168.0.1:26656, ID1@192.168.0.2:26656, ...)")
+	TestnetFilesCmd.Flags().IntVar(&p2pPort, "p2p-port", 26656,
 		"P2P Port")
 }

@@ -89,7 +89,7 @@ func testnetFiles(cmd *cobra.Command, args []string) error {
 		initFilesWithConfig(config)

 		pvFile := filepath.Join(nodeDir, config.BaseConfig.PrivValidator)
-		pv := pvm.LoadFilePV(pvFile)
+		pv := privval.LoadFilePV(pvFile)
 		genVals[i] = types.GenesisValidator{
 			PubKey: pv.GetPubKey(),
 			Power:  1,
--- a/config/config.go
+++ b/config/config.go
@@ -7,6 +7,13 @@ import (
 	"time"
 )

+const (
+	// FuzzModeDrop is a mode in which we randomly drop reads/writes, connections or sleep
+	FuzzModeDrop = iota
+	// FuzzModeDelay is a mode in which we randomly sleep
+	FuzzModeDelay
+)
+
 // NOTE: Most of the structs & relevant comments + the
 // default configuration options were used to manually
 // generate the config.toml. Please reflect any changes
@@ -144,7 +151,7 @@ func DefaultBaseConfig() BaseConfig {
 		PrivValidator:     defaultPrivValPath,
 		NodeKey:           defaultNodeKeyPath,
 		Moniker:           defaultMoniker,
-		ProxyApp:          "tcp://127.0.0.1:46658",
+		ProxyApp:          "tcp://127.0.0.1:26658",
 		ABCI:              "socket",
 		LogLevel:          DefaultPackageLogLevels(),
 		ProfListenAddress: "",
@@ -221,7 +228,7 @@ type RPCConfig struct {
 // DefaultRPCConfig returns a default configuration for the RPC server
 func DefaultRPCConfig() *RPCConfig {
 	return &RPCConfig{
-		ListenAddress:     "tcp://0.0.0.0:46657",
+		ListenAddress:     "tcp://0.0.0.0:26657",
 		GRPCListenAddress: "",
 		Unsafe:            false,
 	}
@@ -287,17 +294,29 @@ type P2PConfig struct {
 	// Does not work if the peer-exchange reactor is disabled.
 	SeedMode bool `mapstructure:"seed_mode"`

-	// Authenticated encryption
-	AuthEnc bool `mapstructure:"auth_enc"`
-
-	// Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
+	// Comma separated list of peer IDs to keep private (will not be gossiped to
+	// other peers)
 	PrivatePeerIDs string `mapstructure:"private_peer_ids"`
+
+	// Toggle to disable guard against peers connecting from the same ip.
+	AllowDuplicateIP bool `mapstructure:"allow_duplicate_ip"`
+
+	// Peer connection configuration.
+	HandshakeTimeout time.Duration `mapstructure:"handshake_timeout"`
+	DialTimeout      time.Duration `mapstructure:"dial_timeout"`
+
+	// Testing params.
+	// Force dial to fail
+	TestDialFail bool `mapstructure:"test_dial_fail"`
+	// FUzz connection
+	TestFuzz       bool            `mapstructure:"test_fuzz"`
+	TestFuzzConfig *FuzzConnConfig `mapstructure:"test_fuzz_config"`
 }

 // DefaultP2PConfig returns a default configuration for the peer-to-peer layer
 func DefaultP2PConfig() *P2PConfig {
 	return &P2PConfig{
-		ListenAddress:           "tcp://0.0.0.0:46656",
+		ListenAddress:           "tcp://0.0.0.0:26656",
 		AddrBook:                defaultAddrBookPath,
 		AddrBookStrict:          true,
 		MaxNumPeers:             50,
@@ -307,7 +326,12 @@ func DefaultP2PConfig() *P2PConfig {
 		RecvRate:                512000, // 500 kB/s
 		PexReactor:              true,
 		SeedMode:                false,
-		AuthEnc:                 true,
+		AllowDuplicateIP:        true, // so non-breaking yet
+		HandshakeTimeout:        20 * time.Second,
+		DialTimeout:             3 * time.Second,
+		TestDialFail:            false,
+		TestFuzz:                false,
+		TestFuzzConfig:          DefaultFuzzConnConfig(),
 	}
 }

@@ -317,6 +341,7 @@ func TestP2PConfig() *P2PConfig {
 	cfg.ListenAddress = "tcp://0.0.0.0:36656"
 	cfg.SkipUPNP = true
 	cfg.FlushThrottleTimeout = 10
+	cfg.AllowDuplicateIP = true
 	return cfg
 }

@@ -325,6 +350,26 @@ func (cfg *P2PConfig) AddrBookFile() string {
 	return rootify(cfg.AddrBook, cfg.RootDir)
 }

+// FuzzConnConfig is a FuzzedConnection configuration.
+type FuzzConnConfig struct {
+	Mode         int
+	MaxDelay     time.Duration
+	ProbDropRW   float64
+	ProbDropConn float64
+	ProbSleep    float64
+}
+
+// DefaultFuzzConnConfig returns the default config.
+func DefaultFuzzConnConfig() *FuzzConnConfig {
+	return &FuzzConnConfig{
+		Mode:         FuzzModeDrop,
+		MaxDelay:     3 * time.Second,
+		ProbDropRW:   0.2,
+		ProbDropConn: 0.00,
+		ProbSleep:    0.00,
+	}
+}
+
 //-----------------------------------------------------------------------------
 // MempoolConfig

--- a/config/toml.go
+++ b/config/toml.go
@@ -81,7 +81,7 @@ fast_sync = {{ .BaseConfig.FastSync }}
 db_backend = "{{ .BaseConfig.DBBackend }}"

 # Database directory
-db_path = "{{ .BaseConfig.DBPath }}"
+db_path = "{{ js .BaseConfig.DBPath }}"

 # Output level for logging, including package level options
 log_level = "{{ .BaseConfig.LogLevel }}"
@@ -89,13 +89,13 @@ log_level = "{{ .BaseConfig.LogLevel }}"
 ##### additional base config options #####

 # Path to the JSON file containing the initial validator set and other meta data
-genesis_file = "{{ .BaseConfig.Genesis }}"
+genesis_file = "{{ js .BaseConfig.Genesis }}"

 # Path to the JSON file containing the private key to use as a validator in the consensus protocol
-priv_validator_file = "{{ .BaseConfig.PrivValidator }}"
+priv_validator_file = "{{ js .BaseConfig.PrivValidator }}"

 # Path to the JSON file containing the private key to use for node authentication in the p2p protocol
-node_key_file = "{{ .BaseConfig.NodeKey}}"
+node_key_file = "{{ js .BaseConfig.NodeKey}}"

 # Mechanism to connect to the ABCI application: socket | grpc
 abci = "{{ .BaseConfig.ABCI }}"
@@ -136,7 +136,7 @@ seeds = "{{ .P2P.Seeds }}"
 persistent_peers = "{{ .P2P.PersistentPeers }}"

 # Path to address book
-addr_book_file = "{{ .P2P.AddrBook }}"
+addr_book_file = "{{ js .P2P.AddrBook }}"

 # Set true for strict address routability rules
 addr_book_strict = {{ .P2P.AddrBookStrict }}
@@ -165,9 +165,6 @@ pex = {{ .P2P.PexReactor }}
 # Does not work if the peer-exchange reactor is disabled.
 seed_mode = {{ .P2P.SeedMode }}

-# Authenticated encryption
-auth_enc = {{ .P2P.AuthEnc }}
-
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = "{{ .P2P.PrivatePeerIDs }}"

@@ -177,7 +174,7 @@ private_peer_ids = "{{ .P2P.PrivatePeerIDs }}"
 recheck = {{ .Mempool.Recheck }}
 recheck_empty = {{ .Mempool.RecheckEmpty }}
 broadcast = {{ .Mempool.Broadcast }}
-wal_dir = "{{ .Mempool.WalPath }}"
+wal_dir = "{{ js .Mempool.WalPath }}"

 # size of the mempool
 size = {{ .Mempool.Size }}
@@ -188,7 +185,7 @@ cache_size = {{ .Mempool.CacheSize }}
 ##### consensus configuration options #####
 [consensus]

-wal_file = "{{ .Consensus.WalPath }}"
+wal_file = "{{ js .Consensus.WalPath }}"

 # All timeouts are in milliseconds
 timeout_propose = {{ .Consensus.TimeoutPropose }}
--- a/consensus/README.md
+++ b/consensus/README.md
@@ -1,18 +1 @@
-# The core consensus algorithm.
-
-* state.go - The state machine as detailed in the whitepaper
-* reactor.go - A reactor that connects the state machine to the gossip network
-
-# Go-routine summary
-
-The reactor runs 2 go-routines for each added peer: gossipDataRoutine and gossipVotesRoutine.
-
-The consensus state runs two persistent go-routines: timeoutRoutine and receiveRoutine.
-Go-routines are also started to trigger timeouts and to avoid blocking when the internalMsgQueue is really backed up.
-
-# Replay/WAL
-
-A write-ahead log is used to record all messages processed by the receiveRoutine,
-which amounts to all inputs to the consensus state machine:
-messages from peers, messages from ourselves, and timeouts.
-They can be played back deterministically at startup or using the replay console. 
+See the [consensus spec](https://github.com/tendermint/tendermint/tree/master/docs/spec/consensus) and the [reactor consensus spec](https://github.com/tendermint/tendermint/tree/master/docs/spec/reactors/consensus) for more information.
--- a/consensus/byzantine_test.go
+++ b/consensus/byzantine_test.go
@@ -27,7 +27,7 @@ func init() {
 // Heal partition and ensure A sees the commit
 func TestByzantine(t *testing.T) {
 	N := 4
-	logger := consensusLogger()
+	logger := consensusLogger().With("test", "byzantine")
 	css := randConsensusNet(N, "consensus_byzantine_test", newMockTickerFunc(false), newCounter)

 	// give the byzantine validator a normal ticker
@@ -58,14 +58,11 @@ func TestByzantine(t *testing.T) {
 			css[i].doPrevote = func(height int64, round int) {}
 		}

-		eventBus := types.NewEventBus()
+		eventBus := css[i].eventBus
 		eventBus.SetLogger(logger.With("module", "events", "validator", i))
-		err := eventBus.Start()
-		require.NoError(t, err)
-		defer eventBus.Stop()

 		eventChans[i] = make(chan interface{}, 1)
-		err = eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock, eventChans[i])
+		err := eventBus.Subscribe(context.Background(), testSubscriber, types.EventQueryNewBlock, eventChans[i])
 		require.NoError(t, err)

 		conR := NewConsensusReactor(css[i], true) // so we dont start the consensus states
@@ -105,15 +102,18 @@ func TestByzantine(t *testing.T) {
 		p2p.Connect2Switches(sws, i, j)
 	})

-	// start the state machines
-	byzR := reactors[0].(*ByzantineReactor)
-	s := byzR.reactor.conS.GetState()
-	byzR.reactor.SwitchToConsensus(s, 0)
+	// start the non-byz state machines.
+	// note these must be started before the byz
 	for i := 1; i < N; i++ {
 		cr := reactors[i].(*ConsensusReactor)
 		cr.SwitchToConsensus(cr.conS.GetState(), 0)
 	}

+	// start the byzantine state machine
+	byzR := reactors[0].(*ByzantineReactor)
+	s := byzR.reactor.conS.GetState()
+	byzR.reactor.SwitchToConsensus(s, 0)
+
 	// byz proposer sends one block to peers[0]
 	// and the other block to peers[1] and peers[2].
 	// note peers and switches order don't match.
--- a/consensus/common_test.go
+++ b/consensus/common_test.go
@@ -19,9 +19,9 @@ import (
 	cstypes "github.com/tendermint/tendermint/consensus/types"
 	mempl "github.com/tendermint/tendermint/mempool"
 	"github.com/tendermint/tendermint/p2p"
+	"github.com/tendermint/tendermint/privval"
 	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
-	pvm "github.com/tendermint/tendermint/types/priv_validator"
 	cmn "github.com/tendermint/tmlibs/common"
 	dbm "github.com/tendermint/tmlibs/db"
 	"github.com/tendermint/tmlibs/log"
@@ -262,9 +262,9 @@ func newConsensusStateWithConfigAndBlockStore(thisConfig *cfg.Config, state sm.S
 	}

 	// mock the evidence pool
-	evpool := types.MockEvidencePool{}
+	evpool := sm.MockEvidencePool{}

-	// Make ConsensusReactor
+	// Make ConsensusState
 	stateDB := dbm.NewMemDB()
 	blockExec := sm.NewBlockExecutor(stateDB, log.TestingLogger(), proxyAppConnCon, mempool, evpool)
 	cs := NewConsensusState(thisConfig.Consensus, state, blockExec, blockStore, mempool, evpool)
@@ -278,10 +278,10 @@ func newConsensusStateWithConfigAndBlockStore(thisConfig *cfg.Config, state sm.S
 	return cs
 }

-func loadPrivValidator(config *cfg.Config) *pvm.FilePV {
+func loadPrivValidator(config *cfg.Config) *privval.FilePV {
 	privValidatorFile := config.PrivValidatorFile()
 	ensureDir(path.Dir(privValidatorFile), 0700)
-	privValidator := pvm.LoadOrGenFilePV(privValidatorFile)
+	privValidator := privval.LoadOrGenFilePV(privValidatorFile)
 	privValidator.Reset()
 	return privValidator
 }
@@ -379,7 +379,7 @@ func randConsensusNetWithPeers(nValidators, nPeers int, testName string, tickerF
 			privVal = privVals[i]
 		} else {
 			_, tempFilePath := cmn.Tempfile("priv_validator_")
-			privVal = pvm.GenFilePV(tempFilePath)
+			privVal = privval.GenFilePV(tempFilePath)
 		}

 		app := appFunc()
--- a/consensus/reactor.go
+++ b/consensus/reactor.go
@@ -1,7 +1,6 @@
 package consensus

 import (
-	"context"
 	"fmt"
 	"reflect"
 	"sync"
@@ -14,6 +13,7 @@ import (
 	"github.com/tendermint/tmlibs/log"

 	cstypes "github.com/tendermint/tendermint/consensus/types"
+	tmevents "github.com/tendermint/tendermint/libs/events"
 	"github.com/tendermint/tendermint/p2p"
 	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
@@ -43,7 +43,8 @@ type ConsensusReactor struct {
 	eventBus *types.EventBus
 }

-// NewConsensusReactor returns a new ConsensusReactor with the given consensusState.
+// NewConsensusReactor returns a new ConsensusReactor with the given
+// consensusState.
 func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *ConsensusReactor {
 	conR := &ConsensusReactor{
 		conS:     consensusState,
@@ -53,17 +54,15 @@ func NewConsensusReactor(consensusState *ConsensusState, fastSync bool) *Consens
 	return conR
 }

-// OnStart implements BaseService.
+// OnStart implements BaseService by subscribing to events, which later will be
+// broadcasted to other peers and starting state if we're not in fast sync.
 func (conR *ConsensusReactor) OnStart() error {
 	conR.Logger.Info("ConsensusReactor ", "fastSync", conR.FastSync())
 	if err := conR.BaseReactor.OnStart(); err != nil {
 		return err
 	}

-	err := conR.startBroadcastRoutine()
-	if err != nil {
-		return err
-	}
+	conR.subscribeToBroadcastEvents()

 	if !conR.FastSync() {
 		err := conR.conS.Start()
@@ -75,9 +74,11 @@ func (conR *ConsensusReactor) OnStart() error {
 	return nil
 }

-// OnStop implements BaseService
+// OnStop implements BaseService by unsubscribing from events and stopping
+// state.
 func (conR *ConsensusReactor) OnStop() {
 	conR.BaseReactor.OnStop()
+	conR.unsubscribeFromBroadcastEvents()
 	conR.conS.Stop()
 }

@@ -101,6 +102,7 @@ func (conR *ConsensusReactor) SwitchToConsensus(state sm.State, blocksSynced int
 	err := conR.conS.Start()
 	if err != nil {
 		conR.Logger.Error("Error starting conS", "err", err)
+		return
 	}
 }

@@ -345,77 +347,40 @@ func (conR *ConsensusReactor) FastSync() bool {

 //--------------------------------------

-// startBroadcastRoutine subscribes for new round steps, votes and proposal
-// heartbeats using the event bus and starts a go routine to broadcasts events
-// to peers upon receiving them.
-func (conR *ConsensusReactor) startBroadcastRoutine() error {
+// subscribeToBroadcastEvents subscribes for new round steps, votes and
+// proposal heartbeats using internal pubsub defined on state to broadcast
+// them to peers upon receiving.
+func (conR *ConsensusReactor) subscribeToBroadcastEvents() {
 	const subscriber = "consensus-reactor"
-	ctx := context.Background()
+	conR.conS.evsw.AddListenerForEvent(subscriber, types.EventNewRoundStep,
+		func(data tmevents.EventData) {
+			conR.broadcastNewRoundStepMessages(data.(*cstypes.RoundState))
+		})

-	// new round steps
-	stepsCh := make(chan interface{})
-	err := conR.eventBus.Subscribe(ctx, subscriber, types.EventQueryNewRoundStep, stepsCh)
-	if err != nil {
-		return errors.Wrapf(err, "failed to subscribe %s to %s", subscriber, types.EventQueryNewRoundStep)
-	}
+	conR.conS.evsw.AddListenerForEvent(subscriber, types.EventVote,
+		func(data tmevents.EventData) {
+			conR.broadcastHasVoteMessage(data.(*types.Vote))
+		})

-	// votes
-	votesCh := make(chan interface{})
-	err = conR.eventBus.Subscribe(ctx, subscriber, types.EventQueryVote, votesCh)
-	if err != nil {
-		return errors.Wrapf(err, "failed to subscribe %s to %s", subscriber, types.EventQueryVote)
-	}
-
-	// proposal heartbeats
-	heartbeatsCh := make(chan interface{})
-	err = conR.eventBus.Subscribe(ctx, subscriber, types.EventQueryProposalHeartbeat, heartbeatsCh)
-	if err != nil {
-		return errors.Wrapf(err, "failed to subscribe %s to %s", subscriber, types.EventQueryProposalHeartbeat)
-	}
-
-	go func() {
-		var data interface{}
-		var ok bool
-		for {
-			select {
-			case data, ok = <-stepsCh:
-				if ok { // a receive from a closed channel returns the zero value immediately
-					edrs := data.(types.EventDataRoundState)
-					conR.broadcastNewRoundStep(edrs.RoundState.(*cstypes.RoundState))
-				}
-			case data, ok = <-votesCh:
-				if ok {
-					edv := data.(types.EventDataVote)
-					conR.broadcastHasVoteMessage(edv.Vote)
-				}
-			case data, ok = <-heartbeatsCh:
-				if ok {
-					edph := data.(types.EventDataProposalHeartbeat)
-					conR.broadcastProposalHeartbeatMessage(edph)
-				}
-			case <-conR.Quit():
-				conR.eventBus.UnsubscribeAll(ctx, subscriber)
-				return
-			}
-			if !ok {
-				conR.eventBus.UnsubscribeAll(ctx, subscriber)
-				return
-			}
-		}
-	}()
-
-	return nil
+	conR.conS.evsw.AddListenerForEvent(subscriber, types.EventProposalHeartbeat,
+		func(data tmevents.EventData) {
+			conR.broadcastProposalHeartbeatMessage(data.(*types.Heartbeat))
+		})
 }

-func (conR *ConsensusReactor) broadcastProposalHeartbeatMessage(heartbeat types.EventDataProposalHeartbeat) {
-	hb := heartbeat.Heartbeat
+func (conR *ConsensusReactor) unsubscribeFromBroadcastEvents() {
+	const subscriber = "consensus-reactor"
+	conR.conS.evsw.RemoveListener(subscriber)
+}
+
+func (conR *ConsensusReactor) broadcastProposalHeartbeatMessage(hb *types.Heartbeat) {
 	conR.Logger.Debug("Broadcasting proposal heartbeat message",
 		"height", hb.Height, "round", hb.Round, "sequence", hb.Sequence)
 	msg := &ProposalHeartbeatMessage{hb}
 	conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(msg))
 }

-func (conR *ConsensusReactor) broadcastNewRoundStep(rs *cstypes.RoundState) {
+func (conR *ConsensusReactor) broadcastNewRoundStepMessages(rs *cstypes.RoundState) {
 	nrsMsg, csMsg := makeRoundStepMessages(rs)
 	if nrsMsg != nil {
 		conR.Switch.Broadcast(StateChannel, cdc.MustMarshalBinaryBare(nrsMsg))
--- a/consensus/reactor_test.go
+++ b/consensus/reactor_test.go
@@ -254,7 +254,8 @@ func TestReactorVotingPowerChange(t *testing.T) {
 	logger.Debug("---------------------------- Testing changing the voting power of one validator a few times")

 	val1PubKey := css[0].privValidator.GetPubKey()
-	updateValidatorTx := kvstore.MakeValSetChangeTx(val1PubKey.Bytes(), 25)
+	val1PubKeyABCI := types.TM2PB.PubKey(val1PubKey)
+	updateValidatorTx := kvstore.MakeValSetChangeTx(val1PubKeyABCI, 25)
 	previousTotalVotingPower := css[0].GetRoundState().LastValidators.TotalVotingPower()

 	waitForAndValidateBlock(t, nVals, activeVals, eventChans, css, updateValidatorTx)
@@ -266,7 +267,7 @@ func TestReactorVotingPowerChange(t *testing.T) {
 		t.Fatalf("expected voting power to change (before: %d, after: %d)", previousTotalVotingPower, css[0].GetRoundState().LastValidators.TotalVotingPower())
 	}

-	updateValidatorTx = kvstore.MakeValSetChangeTx(val1PubKey.Bytes(), 2)
+	updateValidatorTx = kvstore.MakeValSetChangeTx(val1PubKeyABCI, 2)
 	previousTotalVotingPower = css[0].GetRoundState().LastValidators.TotalVotingPower()

 	waitForAndValidateBlock(t, nVals, activeVals, eventChans, css, updateValidatorTx)
@@ -278,7 +279,7 @@ func TestReactorVotingPowerChange(t *testing.T) {
 		t.Fatalf("expected voting power to change (before: %d, after: %d)", previousTotalVotingPower, css[0].GetRoundState().LastValidators.TotalVotingPower())
 	}

-	updateValidatorTx = kvstore.MakeValSetChangeTx(val1PubKey.Bytes(), 26)
+	updateValidatorTx = kvstore.MakeValSetChangeTx(val1PubKeyABCI, 26)
 	previousTotalVotingPower = css[0].GetRoundState().LastValidators.TotalVotingPower()

 	waitForAndValidateBlock(t, nVals, activeVals, eventChans, css, updateValidatorTx)
@@ -316,7 +317,8 @@ func TestReactorValidatorSetChanges(t *testing.T) {
 	logger.Info("---------------------------- Testing adding one validator")

 	newValidatorPubKey1 := css[nVals].privValidator.GetPubKey()
-	newValidatorTx1 := kvstore.MakeValSetChangeTx(newValidatorPubKey1.Bytes(), testMinPower)
+	valPubKey1ABCI := types.TM2PB.PubKey(newValidatorPubKey1)
+	newValidatorTx1 := kvstore.MakeValSetChangeTx(valPubKey1ABCI, testMinPower)

 	// wait till everyone makes block 2
 	// ensure the commit includes all validators
@@ -342,7 +344,8 @@ func TestReactorValidatorSetChanges(t *testing.T) {
 	logger.Info("---------------------------- Testing changing the voting power of one validator")

 	updateValidatorPubKey1 := css[nVals].privValidator.GetPubKey()
-	updateValidatorTx1 := kvstore.MakeValSetChangeTx(updateValidatorPubKey1.Bytes(), 25)
+	updatePubKey1ABCI := types.TM2PB.PubKey(updateValidatorPubKey1)
+	updateValidatorTx1 := kvstore.MakeValSetChangeTx(updatePubKey1ABCI, 25)
 	previousTotalVotingPower := css[nVals].GetRoundState().LastValidators.TotalVotingPower()

 	waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, updateValidatorTx1)
@@ -358,10 +361,12 @@ func TestReactorValidatorSetChanges(t *testing.T) {
 	logger.Info("---------------------------- Testing adding two validators at once")

 	newValidatorPubKey2 := css[nVals+1].privValidator.GetPubKey()
-	newValidatorTx2 := kvstore.MakeValSetChangeTx(newValidatorPubKey2.Bytes(), testMinPower)
+	newVal2ABCI := types.TM2PB.PubKey(newValidatorPubKey2)
+	newValidatorTx2 := kvstore.MakeValSetChangeTx(newVal2ABCI, testMinPower)

 	newValidatorPubKey3 := css[nVals+2].privValidator.GetPubKey()
-	newValidatorTx3 := kvstore.MakeValSetChangeTx(newValidatorPubKey3.Bytes(), testMinPower)
+	newVal3ABCI := types.TM2PB.PubKey(newValidatorPubKey3)
+	newValidatorTx3 := kvstore.MakeValSetChangeTx(newVal3ABCI, testMinPower)

 	waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, newValidatorTx2, newValidatorTx3)
 	waitForAndValidateBlockWithTx(t, nPeers, activeVals, eventChans, css, newValidatorTx2, newValidatorTx3)
@@ -373,8 +378,8 @@ func TestReactorValidatorSetChanges(t *testing.T) {
 	//---------------------------------------------------------------------------
 	logger.Info("---------------------------- Testing removing two validators at once")

-	removeValidatorTx2 := kvstore.MakeValSetChangeTx(newValidatorPubKey2.Bytes(), 0)
-	removeValidatorTx3 := kvstore.MakeValSetChangeTx(newValidatorPubKey3.Bytes(), 0)
+	removeValidatorTx2 := kvstore.MakeValSetChangeTx(newVal2ABCI, 0)
+	removeValidatorTx3 := kvstore.MakeValSetChangeTx(newVal3ABCI, 0)

 	waitForAndValidateBlock(t, nPeers, activeVals, eventChans, css, removeValidatorTx2, removeValidatorTx3)
 	waitForAndValidateBlockWithTx(t, nPeers, activeVals, eventChans, css, removeValidatorTx2, removeValidatorTx3)
--- a/consensus/replay.go
+++ b/consensus/replay.go
@@ -2,7 +2,6 @@ package consensus

 import (
 	"bytes"
-	"encoding/json"
 	"fmt"
 	"hash/crc32"
 	"io"
@@ -196,21 +195,21 @@ func makeHeightSearchFunc(height int64) auto.SearchFunc {
 type Handshaker struct {
 	stateDB      dbm.DB
 	initialState sm.State
-	store        types.BlockStore
-	appState     json.RawMessage
+	store        sm.BlockStore
+	genDoc       *types.GenesisDoc
 	logger       log.Logger

 	nBlocks int // number of blocks applied to the state
 }

 func NewHandshaker(stateDB dbm.DB, state sm.State,
-	store types.BlockStore, appState json.RawMessage) *Handshaker {
+	store sm.BlockStore, genDoc *types.GenesisDoc) *Handshaker {

 	return &Handshaker{
 		stateDB:      stateDB,
 		initialState: state,
 		store:        store,
-		appState:     appState,
+		genDoc:       genDoc,
 		logger:       log.NewNopLogger(),
 		nBlocks:      0,
 	}
@@ -268,14 +267,33 @@ func (h *Handshaker) ReplayBlocks(state sm.State, appHash []byte, appBlockHeight
 	// If appBlockHeight == 0 it means that we are at genesis and hence should send InitChain
 	if appBlockHeight == 0 {
 		validators := types.TM2PB.Validators(state.Validators)
+		csParams := types.TM2PB.ConsensusParams(h.genDoc.ConsensusParams)
 		req := abci.RequestInitChain{
-			Validators:    validators,
-			AppStateBytes: h.appState,
+			Time:            h.genDoc.GenesisTime.Unix(), // TODO
+			ChainId:         h.genDoc.ChainID,
+			ConsensusParams: csParams,
+			Validators:      validators,
+			AppStateBytes:   h.genDoc.AppStateJSON,
 		}
-		_, err := proxyApp.Consensus().InitChainSync(req)
+		res, err := proxyApp.Consensus().InitChainSync(req)
 		if err != nil {
 			return nil, err
 		}
+
+		// if the app returned validators
+		// or consensus params, update the state
+		// with the them
+		if len(res.Validators) > 0 {
+			vals, err := types.PB2TM.Validators(res.Validators)
+			if err != nil {
+				return nil, err
+			}
+			state.Validators = types.NewValidatorSet(vals)
+		}
+		if res.ConsensusParams != nil {
+			state.ConsensusParams = types.PB2TM.ConsensusParams(res.ConsensusParams)
+		}
+		sm.SaveState(h.stateDB, state)
 	}

 	// First handle edge cases and constraints on the storeBlockHeight
@@ -365,7 +383,7 @@ func (h *Handshaker) replayBlocks(state sm.State, proxyApp proxy.AppConns, appBl
 	for i := appBlockHeight + 1; i <= finalBlock; i++ {
 		h.logger.Info("Applying block", "height", i)
 		block := h.store.LoadBlock(i)
-		appHash, err = sm.ExecCommitBlock(proxyApp.Consensus(), block, h.logger)
+		appHash, err = sm.ExecCommitBlock(proxyApp.Consensus(), block, h.logger, state.LastValidators, h.stateDB)
 		if err != nil {
 			return nil, err
 		}
@@ -390,7 +408,7 @@ func (h *Handshaker) replayBlock(state sm.State, height int64, proxyApp proxy.Ap
 	block := h.store.LoadBlock(height)
 	meta := h.store.LoadBlockMeta(height)

-	blockExec := sm.NewBlockExecutor(h.stateDB, h.logger, proxyApp, types.MockMempool{}, types.MockEvidencePool{})
+	blockExec := sm.NewBlockExecutor(h.stateDB, h.logger, proxyApp, sm.MockMempool{}, sm.MockEvidencePool{})

 	var err error
 	state, err = blockExec.ApplyBlock(state, meta.BlockID, block)
--- a/consensus/replay_file.go
+++ b/consensus/replay_file.go
@@ -299,7 +299,7 @@ func newConsensusStateForReplay(config cfg.BaseConfig, csConfig *cfg.ConsensusCo
 	// Create proxyAppConn connection (consensus, mempool, query)
 	clientCreator := proxy.DefaultClientCreator(config.ProxyApp, config.ABCI, config.DBDir())
 	proxyApp := proxy.NewAppConns(clientCreator,
-		NewHandshaker(stateDB, state, blockStore, gdoc.AppState()))
+		NewHandshaker(stateDB, state, blockStore, gdoc))
 	err = proxyApp.Start()
 	if err != nil {
 		cmn.Exit(cmn.Fmt("Error starting proxy app conns: %v", err))
@@ -310,7 +310,7 @@ func newConsensusStateForReplay(config cfg.BaseConfig, csConfig *cfg.ConsensusCo
 		cmn.Exit(cmn.Fmt("Failed to start event bus: %v", err))
 	}

-	mempool, evpool := types.MockMempool{}, types.MockEvidencePool{}
+	mempool, evpool := sm.MockMempool{}, sm.MockEvidencePool{}
 	blockExec := sm.NewBlockExecutor(stateDB, log.TestingLogger(), proxyApp.Consensus(), mempool, evpool)

 	consensusState := NewConsensusState(csConfig, state.Copy(), blockExec,
--- a/consensus/replay_test.go
+++ b/consensus/replay_test.go
@@ -13,6 +13,7 @@ import (
 	"testing"
 	"time"

+	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"

 	"github.com/tendermint/abci/example/kvstore"
@@ -23,10 +24,10 @@ import (
 	dbm "github.com/tendermint/tmlibs/db"

 	cfg "github.com/tendermint/tendermint/config"
+	"github.com/tendermint/tendermint/privval"
 	"github.com/tendermint/tendermint/proxy"
 	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
-	pvm "github.com/tendermint/tendermint/types/priv_validator"
 	"github.com/tendermint/tmlibs/log"
 )

@@ -263,8 +264,8 @@ const (
 )

 var (
-	mempool = types.MockMempool{}
-	evpool  = types.MockEvidencePool{}
+	mempool = sm.MockMempool{}
+	evpool  = sm.MockEvidencePool{}
 )

 //---------------------------------------
@@ -329,7 +330,7 @@ func testHandshakeReplay(t *testing.T, nBlocks int, mode uint) {
 	walFile := tempWALWithData(walBody)
 	config.Consensus.SetWalFile(walFile)

-	privVal := pvm.LoadFilePV(config.PrivValidatorFile())
+	privVal := privval.LoadFilePV(config.PrivValidatorFile())

 	wal, err := NewWAL(walFile)
 	if err != nil {
@@ -366,7 +367,8 @@ func testHandshakeReplay(t *testing.T, nBlocks int, mode uint) {
 	}

 	// now start the app using the handshake - it should sync
-	handshaker := NewHandshaker(stateDB, state, store, nil)
+	genDoc, _ := sm.MakeGenesisDocFromFile(config.GenesisFile())
+	handshaker := NewHandshaker(stateDB, state, store, genDoc)
 	proxyApp := proxy.NewAppConns(clientCreator2, handshaker)
 	if err := proxyApp.Start(); err != nil {
 		t.Fatalf("Error starting proxy app connections: %v", err)
@@ -416,10 +418,10 @@ func buildAppStateFromChain(proxyApp proxy.AppConns, stateDB dbm.DB,
 	}
 	defer proxyApp.Stop()

-	// TODO: get the genesis bytes (https://github.com/tendermint/tendermint/issues/1224)
-	var genesisBytes []byte
 	validators := types.TM2PB.Validators(state.Validators)
-	if _, err := proxyApp.Consensus().InitChainSync(abci.RequestInitChain{validators, genesisBytes}); err != nil {
+	if _, err := proxyApp.Consensus().InitChainSync(abci.RequestInitChain{
+		Validators: validators,
+	}); err != nil {
 		panic(err)
 	}

@@ -453,10 +455,10 @@ func buildTMStateFromChain(config *cfg.Config, stateDB dbm.DB, state sm.State, c
 	}
 	defer proxyApp.Stop()

-	// TODO: get the genesis bytes (https://github.com/tendermint/tendermint/issues/1224)
-	var genesisBytes []byte
 	validators := types.TM2PB.Validators(state.Validators)
-	if _, err := proxyApp.Consensus().InitChainSync(abci.RequestInitChain{validators, genesisBytes}); err != nil {
+	if _, err := proxyApp.Consensus().InitChainSync(abci.RequestInitChain{
+		Validators: validators,
+	}); err != nil {
 		panic(err)
 	}

@@ -633,3 +635,53 @@ func (bs *mockBlockStore) LoadBlockCommit(height int64) *types.Commit {
 func (bs *mockBlockStore) LoadSeenCommit(height int64) *types.Commit {
 	return bs.commits[height-1]
 }
+
+//----------------------------------------
+
+func TestInitChainUpdateValidators(t *testing.T) {
+	val, _ := types.RandValidator(true, 10)
+	vals := types.NewValidatorSet([]*types.Validator{val})
+	app := &initChainApp{vals: types.TM2PB.Validators(vals)}
+	clientCreator := proxy.NewLocalClientCreator(app)
+
+	config := ResetConfig("proxy_test_")
+	privVal := privval.LoadFilePV(config.PrivValidatorFile())
+	stateDB, state, store := stateAndStore(config, privVal.GetPubKey())
+
+	oldValAddr := state.Validators.Validators[0].Address
+
+	// now start the app using the handshake - it should sync
+	genDoc, _ := sm.MakeGenesisDocFromFile(config.GenesisFile())
+	handshaker := NewHandshaker(stateDB, state, store, genDoc)
+	proxyApp := proxy.NewAppConns(clientCreator, handshaker)
+	if err := proxyApp.Start(); err != nil {
+		t.Fatalf("Error starting proxy app connections: %v", err)
+	}
+	defer proxyApp.Stop()
+
+	// reload the state, check the validator set was updated
+	state = sm.LoadState(stateDB)
+
+	newValAddr := state.Validators.Validators[0].Address
+	expectValAddr := val.Address
+	assert.NotEqual(t, oldValAddr, newValAddr)
+	assert.Equal(t, newValAddr, expectValAddr)
+}
+
+func newInitChainApp(vals []abci.Validator) *initChainApp {
+	return &initChainApp{
+		vals: vals,
+	}
+}
+
+// returns the vals on InitChain
+type initChainApp struct {
+	abci.BaseApplication
+	vals []abci.Validator
+}
+
+func (ica *initChainApp) InitChain(req abci.RequestInitChain) abci.ResponseInitChain {
+	return abci.ResponseInitChain{
+		Validators: ica.vals,
+	}
+}
--- a/consensus/state.go
+++ b/consensus/state.go
@@ -15,6 +15,7 @@ import (

 	cfg "github.com/tendermint/tendermint/config"
 	cstypes "github.com/tendermint/tendermint/consensus/types"
+	tmevents "github.com/tendermint/tendermint/libs/events"
 	"github.com/tendermint/tendermint/p2p"
 	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
@@ -75,9 +76,9 @@ type ConsensusState struct {
 	// services for creating and executing blocks
 	// TODO: encapsulate all of this in one "BlockManager"
 	blockExec  *sm.BlockExecutor
-	blockStore types.BlockStore
-	mempool    types.Mempool
-	evpool     types.EvidencePool
+	blockStore sm.BlockStore
+	mempool    sm.Mempool
+	evpool     sm.EvidencePool

 	// internal state
 	mtx sync.Mutex
@@ -110,10 +111,14 @@ type ConsensusState struct {

 	// closed when we finish shutting down
 	done chan struct{}
+
+	// synchronous pubsub between consensus state and reactor.
+	// state only emits EventNewRoundStep, EventVote and EventProposalHeartbeat
+	evsw tmevents.EventSwitch
 }

 // NewConsensusState returns a new ConsensusState.
-func NewConsensusState(config *cfg.ConsensusConfig, state sm.State, blockExec *sm.BlockExecutor, blockStore types.BlockStore, mempool types.Mempool, evpool types.EvidencePool) *ConsensusState {
+func NewConsensusState(config *cfg.ConsensusConfig, state sm.State, blockExec *sm.BlockExecutor, blockStore sm.BlockStore, mempool sm.Mempool, evpool sm.EvidencePool) *ConsensusState {
 	cs := &ConsensusState{
 		config:           config,
 		blockExec:        blockExec,
@@ -126,6 +131,7 @@ func NewConsensusState(config *cfg.ConsensusConfig, state sm.State, blockExec *s
 		doWALCatchup:     true,
 		wal:              nilWAL{},
 		evpool:           evpool,
+		evsw:             tmevents.NewEventSwitch(),
 	}
 	// set function defaults (may be overwritten before calling Start)
 	cs.decideProposal = cs.defaultDecideProposal
@@ -227,6 +233,10 @@ func (cs *ConsensusState) LoadCommit(height int64) *types.Commit {
 // OnStart implements cmn.Service.
 // It loads the latest state via the WAL, and starts the timeout and receive routines.
 func (cs *ConsensusState) OnStart() error {
+	if err := cs.evsw.Start(); err != nil {
+		return err
+	}
+
 	// we may set the WAL in testing before calling Start,
 	// so only OpenWAL if its still the nilWAL
 	if _, ok := cs.wal.(nilWAL); ok {
@@ -244,8 +254,7 @@ func (cs *ConsensusState) OnStart() error {
 	// NOTE: we will get a build up of garbage go routines
 	// firing on the tockChan until the receiveRoutine is started
 	// to deal with them (by that point, at most one will be valid)
-	err := cs.timeoutTicker.Start()
-	if err != nil {
+	if err := cs.timeoutTicker.Start(); err != nil {
 		return err
 	}

@@ -284,6 +293,8 @@ func (cs *ConsensusState) startRoutines(maxSteps int) {
 func (cs *ConsensusState) OnStop() {
 	cs.BaseService.OnStop()

+	cs.evsw.Stop()
+
 	cs.timeoutTicker.Stop()

 	// Make BaseService.Wait() wait until cs.wal.Wait()
@@ -449,9 +460,12 @@ func (cs *ConsensusState) updateToState(state sm.State) {

 	// If state isn't further out than cs.state, just ignore.
 	// This happens when SwitchToConsensus() is called in the reactor.
-	// We don't want to reset e.g. the Votes.
+	// We don't want to reset e.g. the Votes, but we still want to
+	// signal the new round step, because other services (eg. mempool)
+	// depend on having an up-to-date peer state!
 	if !cs.state.IsEmpty() && (state.LastBlockHeight <= cs.state.LastBlockHeight) {
 		cs.Logger.Info("Ignoring updateToState()", "newHeight", state.LastBlockHeight+1, "oldHeight", cs.state.LastBlockHeight+1)
+		cs.newStep()
 		return
 	}

@@ -481,6 +495,7 @@ func (cs *ConsensusState) updateToState(state sm.State) {
 	} else {
 		cs.StartTime = cs.config.Commit(cs.CommitTime)
 	}
+
 	cs.Validators = validators
 	cs.Proposal = nil
 	cs.ProposalBlock = nil
@@ -506,9 +521,10 @@ func (cs *ConsensusState) newStep() {
 	rs := cs.RoundStateEvent()
 	cs.wal.Write(rs)
 	cs.nSteps++
-	// newStep is called by updateToStep in NewConsensusState before the eventBus is set!
+	// newStep is called by updateToState in NewConsensusState before the eventBus is set!
 	if cs.eventBus != nil {
 		cs.eventBus.PublishEventNewRoundStep(rs)
+		cs.evsw.FireEvent(types.EventNewRoundStep, &cs.RoundState)
 	}
 }

@@ -584,7 +600,7 @@ func (cs *ConsensusState) handleMsg(mi msgInfo) {
 		err = cs.setProposal(msg.Proposal)
 	case *BlockPartMessage:
 		// if the proposal is complete, we'll enterPrevote or tryFinalizeCommit
-		_, err = cs.addProposalBlockPart(msg.Height, msg.Part)
+		_, err = cs.addProposalBlockPart(msg, peerID)
 		if err != nil && msg.Round != cs.Round {
 			cs.Logger.Debug("Received block part from wrong round", "height", cs.Height, "csRound", cs.Round, "blockRound", msg.Round)
 			err = nil
@@ -752,6 +768,7 @@ func (cs *ConsensusState) proposalHeartbeat(height int64, round int) {
 		}
 		cs.privValidator.SignHeartbeat(chainID, heartbeat)
 		cs.eventBus.PublishEventProposalHeartbeat(types.EventDataProposalHeartbeat{heartbeat})
+		cs.evsw.FireEvent(types.EventProposalHeartbeat, heartbeat)
 		counter++
 		time.Sleep(proposalHeartbeatIntervalSeconds * time.Second)
 	}
@@ -1316,17 +1333,22 @@ func (cs *ConsensusState) defaultSetProposal(proposal *types.Proposal) error {

 // NOTE: block is not necessarily valid.
 // Asynchronously triggers either enterPrevote (before we timeout of propose) or tryFinalizeCommit, once we have the full block.
-func (cs *ConsensusState) addProposalBlockPart(height int64, part *types.Part) (added bool, err error) {
+func (cs *ConsensusState) addProposalBlockPart(msg *BlockPartMessage, peerID p2p.ID) (added bool, err error) {
+	height, round, part := msg.Height, msg.Round, msg.Part
+
 	// Blocks might be reused, so round mismatch is OK
 	if cs.Height != height {
-		cs.Logger.Debug("Received block part from wrong height", "height", height)
+		cs.Logger.Debug("Received block part from wrong height", "height", height, "round", round)
 		return false, nil
 	}

 	// We're not expecting a block part.
 	if cs.ProposalBlockParts == nil {
-		cs.Logger.Info("Received a block part when we're not expecting any", "height", height)
-		return false, nil // TODO: bad peer? Return error?
+		// NOTE: this can happen when we've gone to a higher round and
+		// then receive parts from the previous round - not necessarily a bad peer.
+		cs.Logger.Info("Received a block part when we're not expecting any",
+			"height", height, "round", round, "index", part.Index, "peer", peerID)
+		return false, nil
 	}

 	added, err = cs.ProposalBlockParts.AddPart(part)
@@ -1360,7 +1382,7 @@ func (cs *ConsensusState) addProposalBlockPart(height int64, part *types.Part) (
 			// procedure at this point.
 		}

-		if cs.Step == cstypes.RoundStepPropose && cs.isProposalComplete() {
+		if cs.Step <= cstypes.RoundStepPropose && cs.isProposalComplete() {
 			// Move onto the next step
 			cs.enterPrevote(height, cs.Round)
 		} else if cs.Step == cstypes.RoundStepCommit {
@@ -1418,6 +1440,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,

 		cs.Logger.Info(cmn.Fmt("Added to lastPrecommits: %v", cs.LastCommit.StringShort()))
 		cs.eventBus.PublishEventVote(types.EventDataVote{vote})
+		cs.evsw.FireEvent(types.EventVote, vote)

 		// if we can skip timeoutCommit and have all the votes now,
 		if cs.config.SkipTimeoutCommit && cs.LastCommit.HasAll() {
@@ -1445,6 +1468,7 @@ func (cs *ConsensusState) addVote(vote *types.Vote, peerID p2p.ID) (added bool,
 	}

 	cs.eventBus.PublishEventVote(types.EventDataVote{vote})
+	cs.evsw.FireEvent(types.EventVote, vote)

 	switch vote.Type {
 	case types.VoteTypePrevote:
--- a/consensus/state_test.go
+++ b/consensus/state_test.go
@@ -8,10 +8,10 @@ import (
 	"time"

 	cstypes "github.com/tendermint/tendermint/consensus/types"
+	tmpubsub "github.com/tendermint/tendermint/libs/pubsub"
 	"github.com/tendermint/tendermint/types"
 	cmn "github.com/tendermint/tmlibs/common"
 	"github.com/tendermint/tmlibs/log"
-	tmpubsub "github.com/tendermint/tmlibs/pubsub"
 )

 func init() {
--- a/consensus/wal.go
+++ b/consensus/wal.go
@@ -106,12 +106,12 @@ func (wal *baseWAL) OnStart() error {
 }

 func (wal *baseWAL) OnStop() {
-	wal.BaseService.OnStop()
 	wal.group.Stop()
+	wal.group.Close()
 }

 // Write is called in newStep and for each receive on the
-// peerMsgQueue and the timoutTicker.
+// peerMsgQueue and the timeoutTicker.
 // NOTE: does not call fsync()
 func (wal *baseWAL) Write(msg WALMessage) {
 	if wal == nil {
@@ -144,13 +144,14 @@ type WALSearchOptions struct {
 	IgnoreDataCorruptionErrors bool
 }

-// SearchForEndHeight searches for the EndHeightMessage with the height and
-// returns an auto.GroupReader, whenever it was found or not and an error.
+// SearchForEndHeight searches for the EndHeightMessage with the given height
+// and returns an auto.GroupReader, whenever it was found or not and an error.
 // Group reader will be nil if found equals false.
 //
 // CONTRACT: caller must close group reader.
 func (wal *baseWAL) SearchForEndHeight(height int64, options *WALSearchOptions) (gr *auto.GroupReader, found bool, err error) {
 	var msg *TimedWALMessage
+	lastHeightFound := int64(-1)

 	// NOTE: starting from the last file in the group because we're usually
 	// searching for the last height. See replay.go
@@ -166,17 +167,25 @@ func (wal *baseWAL) SearchForEndHeight(height int64, options *WALSearchOptions)
 		for {
 			msg, err = dec.Decode()
 			if err == io.EOF {
+				// OPTIMISATION: no need to look for height in older files if we've seen h < height
+				if lastHeightFound > 0 && lastHeightFound < height {
+					gr.Close()
+					return nil, false, nil
+				}
 				// check next file
 				break
 			}
 			if options.IgnoreDataCorruptionErrors && IsDataCorruptionError(err) {
+				wal.Logger.Debug("Corrupted entry. Skipping...", "err", err)
 				// do nothing
+				continue
 			} else if err != nil {
 				gr.Close()
 				return nil, false, err
 			}

 			if m, ok := msg.Msg.(EndHeightMessage); ok {
+				lastHeightFound = m.Height
 				if m.Height == height { // found
 					wal.Logger.Debug("Found", "height", height, "index", index)
 					return gr, true, nil
@@ -271,23 +280,17 @@ func (dec *WALDecoder) Decode() (*TimedWALMessage, error) {

 	b = make([]byte, 4)
 	_, err = dec.rd.Read(b)
-	if err == io.EOF {
-		return nil, err
-	}
 	if err != nil {
 		return nil, fmt.Errorf("failed to read length: %v", err)
 	}
 	length := binary.BigEndian.Uint32(b)

 	if length > maxMsgSizeBytes {
-		return nil, DataCorruptionError{fmt.Errorf("length %d exceeded maximum possible value of %d bytes", length, maxMsgSizeBytes)}
+		return nil, fmt.Errorf("length %d exceeded maximum possible value of %d bytes", length, maxMsgSizeBytes)
 	}

 	data := make([]byte, length)
 	_, err = dec.rd.Read(data)
-	if err == io.EOF {
-		return nil, err
-	}
 	if err != nil {
 		return nil, fmt.Errorf("failed to read data: %v", err)
 	}
--- a/consensus/wal_generator.go
+++ b/consensus/wal_generator.go
@@ -13,10 +13,10 @@ import (
 	"github.com/tendermint/abci/example/kvstore"
 	bc "github.com/tendermint/tendermint/blockchain"
 	cfg "github.com/tendermint/tendermint/config"
+	"github.com/tendermint/tendermint/privval"
 	"github.com/tendermint/tendermint/proxy"
 	sm "github.com/tendermint/tendermint/state"
 	"github.com/tendermint/tendermint/types"
-	pvm "github.com/tendermint/tendermint/types/priv_validator"
 	auto "github.com/tendermint/tmlibs/autofile"
 	cmn "github.com/tendermint/tmlibs/common"
 	"github.com/tendermint/tmlibs/db"
@@ -40,7 +40,7 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
 	// COPY PASTE FROM node.go WITH A FEW MODIFICATIONS
 	// NOTE: we can't import node package because of circular dependency
 	privValidatorFile := config.PrivValidatorFile()
-	privValidator := pvm.LoadOrGenFilePV(privValidatorFile)
+	privValidator := privval.LoadOrGenFilePV(privValidatorFile)
 	genDoc, err := types.GenesisDocFromFile(config.GenesisFile())
 	if err != nil {
 		return nil, errors.Wrap(err, "failed to read genesis file")
@@ -52,7 +52,7 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
 		return nil, errors.Wrap(err, "failed to make genesis state")
 	}
 	blockStore := bc.NewBlockStore(blockStoreDB)
-	handshaker := NewHandshaker(stateDB, state, blockStore, genDoc.AppState())
+	handshaker := NewHandshaker(stateDB, state, blockStore, genDoc)
 	proxyApp := proxy.NewAppConns(proxy.NewLocalClientCreator(app), handshaker)
 	proxyApp.SetLogger(logger.With("module", "proxy"))
 	if err := proxyApp.Start(); err != nil {
@@ -65,8 +65,8 @@ func WALWithNBlocks(numBlocks int) (data []byte, err error) {
 		return nil, errors.Wrap(err, "failed to start event bus")
 	}
 	defer eventBus.Stop()
-	mempool := types.MockMempool{}
-	evpool := types.MockEvidencePool{}
+	mempool := sm.MockMempool{}
+	evpool := sm.MockEvidencePool{}
 	blockExec := sm.NewBlockExecutor(stateDB, log.TestingLogger(), proxyApp.Consensus(), mempool, evpool)
 	consensusState := NewConsensusState(config.Consensus, state.Copy(), blockExec, blockStore, mempool, evpool)
 	consensusState.SetLogger(logger)
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -5,7 +5,7 @@ services:
    container_name: node0
    image: "tendermint/localnode"
    ports:
-      - "46656-46657:46656-46657"
+      - "26656-26657:26656-26657"
    environment:
      - ID=0
      - LOG=$${LOG:-tendermint.log}
@@ -19,7 +19,7 @@ services:
    container_name: node1
    image: "tendermint/localnode"
    ports:
-      - "46659-46660:46656-46657"
+      - "26659-26660:26656-26657"
    environment:
      - ID=1
      - LOG=$${LOG:-tendermint.log}
@@ -36,7 +36,7 @@ services:
      - ID=2
      - LOG=$${LOG:-tendermint.log}
    ports:
-      - "46661-46662:46656-46657"
+      - "26661-26662:26656-26657"
    volumes:
      - ./build:/tendermint:Z
    networks:
@@ -50,7 +50,7 @@ services:
      - ID=3
      - LOG=$${LOG:-tendermint.log}
    ports:
-      - "46663-46664:46656-46657"
+      - "26663-26664:26656-26657"
    volumes:
      - ./build:/tendermint:Z
    networks:
--- a/docs/abci-cli.md
+++ b/docs/abci-cli.md
@@ -0,0 +1,329 @@
+# Using ABCI-CLI
+
+To facilitate testing and debugging of ABCI servers and simple apps, we
+built a CLI, the `abci-cli`, for sending ABCI messages from the command
+line.
+
+## Install
+
+Make sure you [have Go installed](https://golang.org/doc/install).
+
+Next, install the `abci-cli` tool and example applications:
+
+    go get -u github.com/tendermint/abci/cmd/abci-cli
+
+If this fails, you may need to use [dep](https://github.com/golang/dep)
+to get vendored dependencies:
+
+    cd $GOPATH/src/github.com/tendermint/abci
+    make get_tools
+    make get_vendor_deps
+    make install
+
+Now run `abci-cli` to see the list of commands:
+
+    Usage:
+      abci-cli [command]
+
+    Available Commands:
+      batch       Run a batch of abci commands against an application
+      check_tx    Validate a tx
+      commit      Commit the application state and return the Merkle root hash
+      console     Start an interactive abci console for multiple commands
+      counter     ABCI demo example
+      deliver_tx  Deliver a new tx to the application
+      kvstore       ABCI demo example
+      echo        Have the application echo a message
+      help        Help about any command
+      info        Get some info about the application
+      query       Query the application state
+      set_option  Set an options on the application
+
+    Flags:
+          --abci string      socket or grpc (default "socket")
+          --address string   address of application socket (default "tcp://127.0.0.1:26658")
+      -h, --help             help for abci-cli
+      -v, --verbose          print the command and results as if it were a console session
+
+    Use "abci-cli [command] --help" for more information about a command.
+
+## KVStore - First Example
+
+The `abci-cli` tool lets us send ABCI messages to our application, to
+help build and debug them.
+
+The most important messages are `deliver_tx`, `check_tx`, and `commit`,
+but there are others for convenience, configuration, and information
+purposes.
+
+We'll start a kvstore application, which was installed at the same time
+as `abci-cli` above. The kvstore just stores transactions in a merkle
+tree.
+
+Its code can be found
+[here](https://github.com/tendermint/abci/blob/master/cmd/abci-cli/abci-cli.go)
+and looks like:
+
+    func cmdKVStore(cmd *cobra.Command, args []string) error {
+        logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
+    
+        // Create the application - in memory or persisted to disk
+        var app types.Application
+        if flagPersist == "" {
+            app = kvstore.NewKVStoreApplication()
+        } else {
+            app = kvstore.NewPersistentKVStoreApplication(flagPersist)
+            app.(*kvstore.PersistentKVStoreApplication).SetLogger(logger.With("module", "kvstore"))
+        }
+    
+        // Start the listener
+        srv, err := server.NewServer(flagAddrD, flagAbci, app)
+        if err != nil {
+            return err
+        }
+        srv.SetLogger(logger.With("module", "abci-server"))
+        if err := srv.Start(); err != nil {
+            return err
+        }
+    
+        // Wait forever
+        cmn.TrapSignal(func() {
+            // Cleanup
+            srv.Stop()
+        })
+        return nil
+    }
+
+Start by running:
+
+    abci-cli kvstore
+
+And in another terminal, run
+
+    abci-cli echo hello
+    abci-cli info
+
+You'll see something like:
+
+    -> data: hello
+    -> data.hex: 68656C6C6F
+
+and:
+
+    -> data: {"size":0}
+    -> data.hex: 7B2273697A65223A307D
+
+An ABCI application must provide two things:
+
+-   a socket server
+-   a handler for ABCI messages
+
+When we run the `abci-cli` tool we open a new connection to the
+application's socket server, send the given ABCI message, and wait for a
+response.
+
+The server may be generic for a particular language, and we provide a
+[reference implementation in
+Golang](https://github.com/tendermint/abci/tree/master/server). See the
+[list of other ABCI implementations](./ecosystem.html) for servers in
+other languages.
+
+The handler is specific to the application, and may be arbitrary, so
+long as it is deterministic and conforms to the ABCI interface
+specification.
+
+So when we run `abci-cli info`, we open a new connection to the ABCI
+server, which calls the `Info()` method on the application, which tells
+us the number of transactions in our Merkle tree.
+
+Now, since every command opens a new connection, we provide the
+`abci-cli console` and `abci-cli batch` commands, to allow multiple ABCI
+messages to be sent over a single connection.
+
+Running `abci-cli console` should drop you in an interactive console for
+speaking ABCI messages to your application.
+
+Try running these commands:
+
+    > echo hello
+    -> code: OK
+    -> data: hello
+    -> data.hex: 0x68656C6C6F
+
+    > info
+    -> code: OK
+    -> data: {"size":0}
+    -> data.hex: 0x7B2273697A65223A307D
+
+    > commit
+    -> code: OK
+    -> data.hex: 0x0000000000000000
+
+    > deliver_tx "abc"
+    -> code: OK
+
+    > info
+    -> code: OK
+    -> data: {"size":1}
+    -> data.hex: 0x7B2273697A65223A317D
+
+    > commit
+    -> code: OK
+    -> data.hex: 0x0200000000000000
+
+    > query "abc"
+    -> code: OK
+    -> log: exists
+    -> height: 0
+    -> value: abc
+    -> value.hex: 616263
+
+    > deliver_tx "def=xyz"
+    -> code: OK
+
+    > commit
+    -> code: OK
+    -> data.hex: 0x0400000000000000
+
+    > query "def"
+    -> code: OK
+    -> log: exists
+    -> height: 0
+    -> value: xyz
+    -> value.hex: 78797A
+
+Note that if we do `deliver_tx "abc"` it will store `(abc, abc)`, but if
+we do `deliver_tx "abc=efg"` it will store `(abc, efg)`.
+
+Similarly, you could put the commands in a file and run
+`abci-cli --verbose batch < myfile`.
+
+## Counter - Another Example
+
+Now that we've got the hang of it, let's try another application, the
+"counter" app.
+
+Like the kvstore app, its code can be found
+[here](https://github.com/tendermint/abci/blob/master/cmd/abci-cli/abci-cli.go)
+and looks like:
+
+    func cmdCounter(cmd *cobra.Command, args []string) error {
+    
+        app := counter.NewCounterApplication(flagSerial)
+    
+        logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
+    
+        // Start the listener
+        srv, err := server.NewServer(flagAddrC, flagAbci, app)
+        if err != nil {
+            return err
+        }
+        srv.SetLogger(logger.With("module", "abci-server"))
+        if err := srv.Start(); err != nil {
+            return err
+        }
+    
+        // Wait forever
+        cmn.TrapSignal(func() {
+            // Cleanup
+            srv.Stop()
+        })
+        return nil
+    }
+
+The counter app doesn't use a Merkle tree, it just counts how many times
+we've sent a transaction, asked for a hash, or committed the state. The
+result of `commit` is just the number of transactions sent.
+
+This application has two modes: `serial=off` and `serial=on`.
+
+When `serial=on`, transactions must be a big-endian encoded incrementing
+integer, starting at 0.
+
+If `serial=off`, there are no restrictions on transactions.
+
+We can toggle the value of `serial` using the `set_option` ABCI message.
+
+When `serial=on`, some transactions are invalid. In a live blockchain,
+transactions collect in memory before they are committed into blocks. To
+avoid wasting resources on invalid transactions, ABCI provides the
+`check_tx` message, which application developers can use to accept or
+reject transactions, before they are stored in memory or gossipped to
+other peers.
+
+In this instance of the counter app, `check_tx` only allows transactions
+whose integer is greater than the last committed one.
+
+Let's kill the console and the kvstore application, and start the
+counter app:
+
+    abci-cli counter
+
+In another window, start the `abci-cli console`:
+
+    > set_option serial on
+    -> code: OK
+    -> log: OK (SetOption doesn't return anything.)
+
+    > check_tx 0x00
+    -> code: OK
+
+    > check_tx 0xff
+    -> code: OK
+
+    > deliver_tx 0x00
+    -> code: OK
+
+    > check_tx 0x00
+    -> code: BadNonce
+    -> log: Invalid nonce. Expected >= 1, got 0
+
+    > deliver_tx 0x01
+    -> code: OK
+
+    > deliver_tx 0x04
+    -> code: BadNonce
+    -> log: Invalid nonce. Expected 2, got 4
+
+    > info
+    -> code: OK
+    -> data: {"hashes":0,"txs":2}
+    -> data.hex: 0x7B22686173686573223A302C22747873223A327D
+
+This is a very simple application, but between `counter` and `kvstore`,
+its easy to see how you can build out arbitrary application states on
+top of the ABCI. [Hyperledger's
+Burrow](https://github.com/hyperledger/burrow) also runs atop ABCI,
+bringing with it Ethereum-like accounts, the Ethereum virtual-machine,
+Monax's permissioning scheme, and native contracts extensions.
+
+But the ultimate flexibility comes from being able to write the
+application easily in any language.
+
+We have implemented the counter in a number of languages [see the
+example directory](https://github.com/tendermint/abci/tree/master/example).
+
+To run the Node JS version, `cd` to `example/js` and run
+
+    node app.js
+
+(you'll have to kill the other counter application process). In another
+window, run the console and those previous ABCI commands. You should get
+the same results as for the Go version.
+
+## Bounties
+
+Want to write the counter app in your favorite language?! We'd be happy
+to add you to our [ecosystem](https://tendermint.com/ecosystem)! We're
+also offering [bounties](https://hackerone.com/tendermint/) for
+implementations in new languages!
+
+The `abci-cli` is designed strictly for testing and debugging. In a real
+deployment, the role of sending messages is taken by Tendermint, which
+connects to the app using three separate connections, each with its own
+pattern of messages.
+
+For more information, see the [application developers
+guide](./app-development.html). For examples of running an ABCI app with
+Tendermint, see the [getting started guide](./getting-started.html).
+Next is the ABCI specification.
--- a/docs/abci-cli.rst
+++ b/docs/abci-cli.rst
@@ -1,369 +0,0 @@
-Using ABCI-CLI
-==============
-
-To facilitate testing and debugging of ABCI servers and simple apps, we
-built a CLI, the ``abci-cli``, for sending ABCI messages from the
-command line.
-
-Install
-------
-
-Make sure you `have Go installed <https://golang.org/doc/install>`__.
-
-Next, install the ``abci-cli`` tool and example applications:
-
-::
-
-    go get -u github.com/tendermint/abci/cmd/abci-cli
-
-If this fails, you may need to use `dep <https://github.com/golang/dep>`__ to get vendored
-dependencies:
-
-::
-
-    cd $GOPATH/src/github.com/tendermint/abci
-    make get_tools
-    make get_vendor_deps
-    make install
-
-Now run ``abci-cli`` to see the list of commands:
-
-::
-
-    Usage:
-      abci-cli [command]
-
-    Available Commands:
-      batch       Run a batch of abci commands against an application
-      check_tx    Validate a tx
-      commit      Commit the application state and return the Merkle root hash
-      console     Start an interactive abci console for multiple commands
-      counter     ABCI demo example
-      deliver_tx  Deliver a new tx to the application
-      kvstore       ABCI demo example
-      echo        Have the application echo a message
-      help        Help about any command
-      info        Get some info about the application
-      query       Query the application state
-      set_option  Set an options on the application
-
-    Flags:
-          --abci string      socket or grpc (default "socket")
-          --address string   address of application socket (default "tcp://127.0.0.1:46658")
-      -h, --help             help for abci-cli
-      -v, --verbose          print the command and results as if it were a console session
-
-    Use "abci-cli [command] --help" for more information about a command.
-
-
-KVStore - First Example
-----------------------
-
-The ``abci-cli`` tool lets us send ABCI messages to our application, to
-help build and debug them.
-
-The most important messages are ``deliver_tx``, ``check_tx``, and
-``commit``, but there are others for convenience, configuration, and
-information purposes.
-
-We'll start a kvstore application, which was installed at the same time as
-``abci-cli`` above. The kvstore just stores transactions in a merkle tree.
-
-Its code can be found `here <https://github.com/tendermint/abci/blob/master/cmd/abci-cli/abci-cli.go>`__ and looks like:
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide KVStore Example**
-
-    .. code-block:: go
-
-        func cmdKVStore(cmd *cobra.Command, args []string) error {
-        	logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
-        
-        	// Create the application - in memory or persisted to disk
-        	var app types.Application
-        	if flagPersist == "" {
-        		app = kvstore.NewKVStoreApplication()
-        	} else {
-        		app = kvstore.NewPersistentKVStoreApplication(flagPersist)
-        		app.(*kvstore.PersistentKVStoreApplication).SetLogger(logger.With("module", "kvstore"))
-        	}
-        
-        	// Start the listener
-        	srv, err := server.NewServer(flagAddrD, flagAbci, app)
-        	if err != nil {
-        		return err
-        	}
-        	srv.SetLogger(logger.With("module", "abci-server"))
-        	if err := srv.Start(); err != nil {
-        		return err
-        	}
-        
-        	// Wait forever
-        	cmn.TrapSignal(func() {
-        		// Cleanup
-        		srv.Stop()
-        	})
-        	return nil
-        }
-
-Start by running:
-
-::
-
-    abci-cli kvstore
-
-And in another terminal, run
-
-::
-
-    abci-cli echo hello
-    abci-cli info
-
-You'll see something like:
-
-::
-
-    -> data: hello
-    -> data.hex: 68656C6C6F
-
-and:
-
-::
-
-    -> data: {"size":0}
-    -> data.hex: 7B2273697A65223A307D
-
-An ABCI application must provide two things:
-
-  a socket server
-  a handler for ABCI messages
-
-When we run the ``abci-cli`` tool we open a new connection to the
-application's socket server, send the given ABCI message, and wait for a
-response.
-
-The server may be generic for a particular language, and we provide a
-`reference implementation in
-Golang <https://github.com/tendermint/abci/tree/master/server>`__. See
-the `list of other ABCI
-implementations <./ecosystem.html>`__ for servers in
-other languages.
-
-The handler is specific to the application, and may be arbitrary, so
-long as it is deterministic and conforms to the ABCI interface
-specification.
-
-So when we run ``abci-cli info``, we open a new connection to the ABCI
-server, which calls the ``Info()`` method on the application, which
-tells us the number of transactions in our Merkle tree.
-
-Now, since every command opens a new connection, we provide the
-``abci-cli console`` and ``abci-cli batch`` commands, to allow multiple
-ABCI messages to be sent over a single connection.
-
-Running ``abci-cli console`` should drop you in an interactive console
-for speaking ABCI messages to your application.
-
-Try running these commands:
-
-::
-
-    > echo hello
-    -> code: OK
-    -> data: hello
-    -> data.hex: 0x68656C6C6F
-    
-    > info
-    -> code: OK
-    -> data: {"size":0}
-    -> data.hex: 0x7B2273697A65223A307D
-    
-    > commit
-    -> code: OK
-    
-    > deliver_tx "abc"
-    -> code: OK
-    
-    > info
-    -> code: OK
-    -> data: {"size":1}
-    -> data.hex: 0x7B2273697A65223A317D
-    
-    > commit
-    -> code: OK
-    -> data.hex: 0x49DFD15CCDACDEAE9728CB01FBB5E8688CA58B91
-    
-    > query "abc"
-    -> code: OK
-    -> log: exists
-    -> height: 0
-    -> value: abc
-    -> value.hex: 616263
-    
-    > deliver_tx "def=xyz"
-    -> code: OK
-    
-    > commit
-    -> code: OK
-    -> data.hex: 0x70102DB32280373FBF3F9F89DA2A20CE2CD62B0B
-    
-    > query "def"
-    -> code: OK
-    -> log: exists
-    -> height: 0
-    -> value: xyz
-    -> value.hex: 78797A
-
-Note that if we do ``deliver_tx "abc"`` it will store ``(abc, abc)``,
-but if we do ``deliver_tx "abc=efg"`` it will store ``(abc, efg)``.
-
-Similarly, you could put the commands in a file and run
-``abci-cli --verbose batch < myfile``.
-
-Counter - Another Example
-------------------------
-
-Now that we've got the hang of it, let's try another application, the
-"counter" app.
-
-Like the kvstore app, its code can be found `here <https://github.com/tendermint/abci/blob/master/cmd/abci-cli/abci-cli.go>`__ and looks like:
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Counter Example**
-
-    .. code-block:: go
-
-        func cmdCounter(cmd *cobra.Command, args []string) error {
-        
-        	app := counter.NewCounterApplication(flagSerial)
-        
-        	logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
-        
-        	// Start the listener
-        	srv, err := server.NewServer(flagAddrC, flagAbci, app)
-        	if err != nil {
-        		return err
-        	}
-        	srv.SetLogger(logger.With("module", "abci-server"))
-        	if err := srv.Start(); err != nil {
-        		return err
-        	}
-        
-        	// Wait forever
-        	cmn.TrapSignal(func() {
-        		// Cleanup
-        		srv.Stop()
-        	})
-        	return nil
-        }
-
-
-The counter app doesn't use a Merkle tree, it just counts how many times
-we've sent a transaction, asked for a hash, or committed the state. The
-result of ``commit`` is just the number of transactions sent.
-
-This application has two modes: ``serial=off`` and ``serial=on``.
-
-When ``serial=on``, transactions must be a big-endian encoded
-incrementing integer, starting at 0.
-
-If ``serial=off``, there are no restrictions on transactions.
-
-We can toggle the value of ``serial`` using the ``set_option`` ABCI
-message.
-
-When ``serial=on``, some transactions are invalid. In a live blockchain,
-transactions collect in memory before they are committed into blocks. To
-avoid wasting resources on invalid transactions, ABCI provides the
-``check_tx`` message, which application developers can use to accept or
-reject transactions, before they are stored in memory or gossipped to
-other peers.
-
-In this instance of the counter app, ``check_tx`` only allows
-transactions whose integer is greater than the last committed one.
-
-Let's kill the console and the kvstore application, and start the counter
-app:
-
-::
-
-    abci-cli counter
-
-In another window, start the ``abci-cli console``:
-
-::
-
-    > set_option serial on
-    -> code: OK
-    
-    > check_tx 0x00
-    -> code: OK
-    
-    > check_tx 0xff
-    -> code: OK
-    
-    > deliver_tx 0x00
-    -> code: OK
-    
-    > check_tx 0x00
-    -> code: BadNonce
-    -> log: Invalid nonce. Expected >= 1, got 0
-    
-    > deliver_tx 0x01
-    -> code: OK
-    
-    > deliver_tx 0x04
-    -> code: BadNonce
-    -> log: Invalid nonce. Expected 2, got 4
-    
-    > info
-    -> code: OK
-    -> data: {"hashes":0,"txs":2}
-    -> data.hex: 0x7B22686173686573223A302C22747873223A327D
-
-This is a very simple application, but between ``counter`` and
-``kvstore``, its easy to see how you can build out arbitrary application
-states on top of the ABCI. `Hyperledger's
-Burrow <https://github.com/hyperledger/burrow>`__ also runs atop ABCI,
-bringing with it Ethereum-like accounts, the Ethereum virtual-machine,
-Monax's permissioning scheme, and native contracts extensions.
-
-But the ultimate flexibility comes from being able to write the
-application easily in any language.
-
-We have implemented the counter in a number of languages (see the
-`example directory <https://github.com/tendermint/abci/tree/master/example`__).
-
-To run the Node JS version, ``cd`` to ``example/js`` and run
-
-::
-
-    node app.js
-
-(you'll have to kill the other counter application process). In another
-window, run the console and those previous ABCI commands. You should get
-the same results as for the Go version.
-
-Bounties
--------
-
-Want to write the counter app in your favorite language?! We'd be happy
-to add you to our `ecosystem <https://tendermint.com/ecosystem>`__!
-We're also offering `bounties <https://tendermint.com/bounties>`__ for
-implementations in new languages!
-
-The ``abci-cli`` is designed strictly for testing and debugging. In a
-real deployment, the role of sending messages is taken by Tendermint,
-which connects to the app using three separate connections, each with
-its own pattern of messages.
-
-For more information, see the `application developers
-guide <./app-development.html>`__. For examples of running an ABCI
-app with Tendermint, see the `getting started
-guide <./getting-started.html>`__. Next is the ABCI specification.
--- a/docs/app-architecture.md
+++ b/docs/app-architecture.md
@@ -0,0 +1,50 @@
+# Application Architecture Guide
+
+Here we provide a brief guide on the recommended architecture of a
+Tendermint blockchain application.
+
+The following diagram provides a superb example:
+
+<https://drive.google.com/open?id=1yR2XpRi9YCY9H9uMfcw8-RMJpvDyvjz9>
+
+The end-user application here is the Cosmos Voyager, at the bottom left.
+Voyager communicates with a REST API exposed by a local Light-Client
+Daemon. The Light-Client Daemon is an application specific program that
+communicates with Tendermint nodes and verifies Tendermint light-client
+proofs through the Tendermint Core RPC. The Tendermint Core process
+communicates with a local ABCI application, where the user query or
+transaction is actually processed.
+
+The ABCI application must be a deterministic result of the Tendermint
+consensus - any external influence on the application state that didn't
+come through Tendermint could cause a consensus failure. Thus *nothing*
+should communicate with the application except Tendermint via ABCI.
+
+If the application is written in Go, it can be compiled into the
+Tendermint binary. Otherwise, it should use a unix socket to communicate
+with Tendermint. If it's necessary to use TCP, extra care must be taken
+to encrypt and authenticate the connection.
+
+All reads from the app happen through the Tendermint `/abci_query`
+endpoint. All writes to the app happen through the Tendermint
+`/broadcast_tx_*` endpoints.
+
+The Light-Client Daemon is what provides light clients (end users) with
+nearly all the security of a full node. It formats and broadcasts
+transactions, and verifies proofs of queries and transaction results.
+Note that it need not be a daemon - the Light-Client logic could instead
+be implemented in the same process as the end-user application.
+
+Note for those ABCI applications with weaker security requirements, the
+functionality of the Light-Client Daemon can be moved into the ABCI
+application process itself. That said, exposing the application process
+to anything besides Tendermint over ABCI requires extreme caution, as
+all transactions, and possibly all queries, should still pass through
+Tendermint.
+
+See the following for more extensive documentation:
+- [Interchain Standard for the Light-Client REST API](https://github.com/cosmos/cosmos-sdk/pull/1028)
+- [Tendermint RPC Docs](https://tendermint.github.io/slate/)
+- [Tendermint in Production](https://github.com/tendermint/tendermint/pull/1618)
+- [Tendermint Basics](https://tendermint.readthedocs.io/en/master/using-tendermint.html)
+- [ABCI spec](https://github.com/tendermint/abci/blob/develop/specification.md)
--- a/docs/app-architecture.rst
+++ b/docs/app-architecture.rst
@@ -1,133 +0,0 @@
-Application Architecture Guide
-==============================
-
-Overview
--------
-
-A blockchain application is more than the consensus engine and the
-transaction logic (eg. smart contracts, business logic) as implemented
-in the ABCI app. There are also (mobile, web, desktop) clients that will
-need to connect and make use of the app. We will assume for now that you
-have a well designed transactions and database model, but maybe this
-will be the topic of another article. This article is more interested in
-various ways of setting up the "plumbing" and connecting these pieces,
-and demonstrating some evolving best practices.
-
-Security
--------
-
-A very important aspect when constructing a blockchain is security. The
-consensus model can be DoSed (no consensus possible) by corrupting 1/3
-of the validators and exploited (writing arbitrary blocks) by corrupting
-2/3 of the validators. So, while the security is not that of the
-"weakest link", you should take care that the "average link" is
-sufficiently hardened.
-
-One big attack surface on the validators is the communication between
-the ABCI app and the tendermint core. This should be highly protected.
-Ideally, the app and the core are running on the same machine, so no
-external agent can target the communication channel. You can use unix
-sockets (with permissions preventing access from other users), or even
-compile the two apps into one binary if the ABCI app is also writen in
-go. If you are unable to do that due to language support, then the ABCI
-app should bind a TCP connection to localhost (127.0.0.1), which is less
-efficient and secure, but still not reachable from outside. If you must
-run the ABCI app and tendermint core on separate machines, make sure you
-have a secure communication channel (ssh tunnel?)
-
-Now assuming, you have linked together your app and the core securely,
-you must also make sure no one can get on the machine it is hosted on.
-At this point it is basic network security. Run on a secure operating
-system (SELinux?). Limit who has access to the machine (user accounts,
-but also where the physical machine is hosted). Turn off all services
-except for ssh, which should only be accessible by some well-guarded
-public/private key pairs (no password). And maybe even firewall off
-access to the ports used by the validators, so only known validators can
-connect.
-
-There was also a suggestion on slack from @jhon about compiling
-everything together with a unikernel for more security, such as
-`Mirage <https://mirage.io>`__ or
-`UNIK <https://github.com/emc-advanced-dev/unik>`__.
-
-Connecting your client to the blockchain
----------------------------------------
-
-Tendermint Core RPC
-~~~~~~~~~~~~~~~~~~~
-
-The concept is that the ABCI app is completely hidden from the outside
-world and only communicated through a tested and secured `interface
-exposed by the tendermint core <./specification/rpc.html>`__. This interface
-exposes a lot of data on the block header and consensus process, which
-is quite useful for externally verifying the system. It also includes
-3(!) methods to broadcast a transaction (propose it for the blockchain,
-and possibly await a response). And one method to query app-specific
-data from the ABCI application.
-
-Pros:
-
- Server code already written
- Access to block headers to validate merkle proofs (nice for light clients)
- Basic read/write functionality is supported
-
-Cons:
-
- Limited interface to app. All queries must be serialized into []byte (less expressive than JSON over HTTP) and there is no way to push data from ABCI app to the client (eg. notify me if account X receives a transaction)
-
-Custom ABCI server
-~~~~~~~~~~~~~~~~~~
-
-This was proposed by @wolfposd on slack and demonstrated by
-`TMChat <https://github.com/wolfposd/TMChat>`__, a sample app. The
-concept is to write a custom server for your app (with typical REST
-API/websockets/etc for easy use by a mobile app). This custom server is
-in the same binary as the ABCI app and data store, so can easily react
-to complex events there that involve understanding the data format (send
-a message if my balance drops below 500). All "writes" sent to this
-server are proxied via websocket/JSON-RPC to tendermint core. When they
-come back as deliver\_tx over ABCI, they will be written to the data
-store. For "reads", we can do any queries we wish that are supported by
-our architecture, using any web technology that is useful. The general
-architecture is shown in the following diagram:
-
-.. figure:: assets/tm-application-example.png
-
-Pros:
-
- Separates application logic from blockchain logic
- Allows much richer, more flexible client-facing API
- Allows pub-sub, watching certain fields, etc.
-
-Cons:
-
- Access to ABCI app can be dangerous (be VERY careful not to write unless it comes from the validator node)
- No direct access to the blockchain headers to verify tx
- You must write your own API (but maybe that's a pro...)
-
-Hybrid solutions
-~~~~~~~~~~~~~~~~
-
-Likely the least secure but most versatile. The client can access both
-the tendermint node for all blockchain info, as well as a custom app
-server, for complex queries and pub-sub on the abci app.
-
-Pros:
-
- All from both above solutions
-
-Cons:
-
- Even more complexity; even more attack vectors (less
-security)
-
-Scalability
-----------
-
-Read replica using non-validating nodes? They could forward transactions
-to the validators (fewer connections, more security), and locally allow
-all queries in any of the above configurations. Thus, while
-transaction-processing speed is limited by the speed of the abci app and
-the number of validators, one should be able to scale our read
-performance to quite an extent (until the replication process drains too
-many resources from the validator nodes).
--- a/docs/app-development.md
+++ b/docs/app-development.md
@@ -0,0 +1,527 @@
+# Application Development Guide
+
+## ABCI Design
+
+The purpose of ABCI is to provide a clean interface between state
+transition machines on one computer and the mechanics of their
+replication across multiple computers. The former we call 'application
+logic' and the latter the 'consensus engine'. Application logic
+validates transactions and optionally executes transactions against some
+persistent state. A consensus engine ensures all transactions are
+replicated in the same order on every machine. We call each machine in a
+consensus engine a 'validator', and each validator runs the same
+transactions through the same application logic. In particular, we are
+interested in blockchain-style consensus engines, where transactions are
+committed in hash-linked blocks.
+
+The ABCI design has a few distinct components:
+
+-   message protocol
+    -   pairs of request and response messages
+    -   consensus makes requests, application responds
+    -   defined using protobuf
+-   server/client
+    -   consensus engine runs the client
+    -   application runs the server
+    -   two implementations:
+        -   async raw bytes
+        -   grpc
+-   blockchain protocol
+    -   abci is connection oriented
+    -   Tendermint Core maintains three connections:
+        -   [mempool connection](#mempool-connection): for checking if
+            transactions should be relayed before they are committed;
+            only uses `CheckTx`
+        -   [consensus connection](#consensus-connection): for executing
+            transactions that have been committed. Message sequence is
+            -for every block
+            -`BeginBlock, [DeliverTx, ...], EndBlock, Commit`
+        -   [query connection](#query-connection): for querying the
+            application state; only uses Query and Info
+
+The mempool and consensus logic act as clients, and each maintains an
+open ABCI connection with the application, which hosts an ABCI server.
+Shown are the request and response types sent on each connection.
+
+## Message Protocol
+
+The message protocol consists of pairs of requests and responses. Some
+messages have no fields, while others may include byte-arrays, strings,
+or integers. See the `message Request` and `message Response`
+definitions in [the protobuf definition
+file](https://github.com/tendermint/abci/blob/master/types/types.proto),
+and the [protobuf
+documentation](https://developers.google.com/protocol-buffers/docs/overview)
+for more details.
+
+For each request, a server should respond with the corresponding
+response, where order of requests is preserved in the order of
+responses.
+
+## Server
+
+To use ABCI in your programming language of choice, there must be a ABCI
+server in that language. Tendermint supports two kinds of implementation
+of the server:
+
+-   Asynchronous, raw socket server (Tendermint Socket Protocol, also
+    known as TSP or Teaspoon)
+-   GRPC
+
+Both can be tested using the `abci-cli` by setting the `--abci` flag
+appropriately (ie. to `socket` or `grpc`).
+
+See examples, in various stages of maintenance, in
+[Go](https://github.com/tendermint/abci/tree/master/server),
+[JavaScript](https://github.com/tendermint/js-abci),
+[Python](https://github.com/tendermint/abci/tree/master/example/python3/abci),
+[C++](https://github.com/mdyring/cpp-tmsp), and
+[Java](https://github.com/jTendermint/jabci).
+
+### GRPC
+
+If GRPC is available in your language, this is the easiest approach,
+though it will have significant performance overhead.
+
+To get started with GRPC, copy in the [protobuf
+file](https://github.com/tendermint/abci/blob/master/types/types.proto)
+and compile it using the GRPC plugin for your language. For instance,
+for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`.
+See the [grpc documentation for more details](http://www.grpc.io/docs/).
+`protoc` will autogenerate all the necessary code for ABCI client and
+server in your language, including whatever interface your application
+must satisfy to be used by the ABCI server for handling requests.
+
+### TSP
+
+If GRPC is not available in your language, or you require higher
+performance, or otherwise enjoy programming, you may implement your own
+ABCI server using the Tendermint Socket Protocol, known affectionately
+as Teaspoon. The first step is still to auto-generate the relevant data
+types and codec in your language using `protoc`. Messages coming over
+the socket are Protobuf3 encoded, but additionally length-prefixed to
+facilitate use as a streaming protocol. Protobuf3 doesn't have an
+official length-prefix standard, so we use our own. The first byte in
+the prefix represents the length of the Big Endian encoded length. The
+remaining bytes in the prefix are the Big Endian encoded length.
+
+For example, if the Protobuf3 encoded ABCI message is 0xDEADBEEF (4
+bytes), the length-prefixed message is 0x0104DEADBEEF. If the Protobuf3
+encoded ABCI message is 65535 bytes long, the length-prefixed message
+would be like 0x02FFFF....
+
+Note this prefixing does not apply for grpc.
+
+An ABCI server must also be able to support multiple connections, as
+Tendermint uses three connections.
+
+## Client
+
+There are currently two use-cases for an ABCI client. One is a testing
+tool, as in the `abci-cli`, which allows ABCI requests to be sent via
+command line. The other is a consensus engine, such as Tendermint Core,
+which makes requests to the application every time a new transaction is
+received or a block is committed.
+
+It is unlikely that you will need to implement a client. For details of
+our client, see
+[here](https://github.com/tendermint/abci/tree/master/client).
+
+Most of the examples below are from [kvstore
+application](https://github.com/tendermint/abci/blob/master/example/kvstore/kvstore.go),
+which is a part of the abci repo. [persistent_kvstore
+application](https://github.com/tendermint/abci/blob/master/example/kvstore/persistent_kvstore.go)
+is used to show `BeginBlock`, `EndBlock` and `InitChain` example
+implementations.
+
+## Blockchain Protocol
+
+In ABCI, a transaction is simply an arbitrary length byte-array. It is
+the application's responsibility to define the transaction codec as they
+please, and to use it for both CheckTx and DeliverTx.
+
+Note that there are two distinct means for running transactions,
+corresponding to stages of 'awareness' of the transaction in the
+network. The first stage is when a transaction is received by a
+validator from a client into the so-called mempool or transaction pool
+-this is where we use CheckTx. The second is when the transaction is
+successfully committed on more than 2/3 of validators - where we use
+DeliverTx. In the former case, it may not be necessary to run all the
+state transitions associated with the transaction, as the transaction
+may not ultimately be committed until some much later time, when the
+result of its execution will be different. For instance, an Ethereum
+ABCI app would check signatures and amounts in CheckTx, but would not
+actually execute any contract code until the DeliverTx, so as to avoid
+executing state transitions that have not been finalized.
+
+To formalize the distinction further, two explicit ABCI connections are
+made between Tendermint Core and the application: the mempool connection
+and the consensus connection. We also make a third connection, the query
+connection, to query the local state of the app.
+
+### Mempool Connection
+
+The mempool connection is used *only* for CheckTx requests. Transactions
+are run using CheckTx in the same order they were received by the
+validator. If the CheckTx returns `OK`, the transaction is kept in
+memory and relayed to other peers in the same order it was received.
+Otherwise, it is discarded.
+
+CheckTx requests run concurrently with block processing; so they should
+run against a copy of the main application state which is reset after
+every block. This copy is necessary to track transitions made by a
+sequence of CheckTx requests before they are included in a block. When a
+block is committed, the application must ensure to reset the mempool
+state to the latest committed state. Tendermint Core will then filter
+through all transactions in the mempool, removing any that were included
+in the block, and re-run the rest using CheckTx against the post-Commit
+mempool state (this behaviour can be turned off with
+`[mempool] recheck = false`).
+
+In go:
+
+    func (app *KVStoreApplication) CheckTx(tx []byte) types.Result {
+      return types.OK
+    }
+
+In Java:
+
+    ResponseCheckTx requestCheckTx(RequestCheckTx req) {
+        byte[] transaction = req.getTx().toByteArray();
+    
+        // validate transaction
+    
+        if (notValid) {
+            return ResponseCheckTx.newBuilder().setCode(CodeType.BadNonce).setLog("invalid tx").build();
+        } else {
+            return ResponseCheckTx.newBuilder().setCode(CodeType.OK).build();
+        }
+    }
+
+### Replay Protection
+
+To prevent old transactions from being replayed, CheckTx must implement
+replay protection.
+
+Tendermint provides the first defence layer by keeping a lightweight
+in-memory cache of 100k (`[mempool] cache_size`) last transactions in
+the mempool. If Tendermint is just started or the clients sent more than
+100k transactions, old transactions may be sent to the application. So
+it is important CheckTx implements some logic to handle them.
+
+There are cases where a transaction will (or may) become valid in some
+future state, in which case you probably want to disable Tendermint's
+cache. You can do that by setting `[mempool] cache_size = 0` in the
+config.
+
+### Consensus Connection
+
+The consensus connection is used only when a new block is committed, and
+communicates all information from the block in a series of requests:
+`BeginBlock, [DeliverTx, ...], EndBlock, Commit`. That is, when a block
+is committed in the consensus, we send a list of DeliverTx requests (one
+for each transaction) sandwiched by BeginBlock and EndBlock requests,
+and followed by a Commit.
+
+### DeliverTx
+
+DeliverTx is the workhorse of the blockchain. Tendermint sends the
+DeliverTx requests asynchronously but in order, and relies on the
+underlying socket protocol (ie. TCP) to ensure they are received by the
+app in order. They have already been ordered in the global consensus by
+the Tendermint protocol.
+
+DeliverTx returns a abci.Result, which includes a Code, Data, and Log.
+The code may be non-zero (non-OK), meaning the corresponding transaction
+should have been rejected by the mempool, but may have been included in
+a block by a Byzantine proposer.
+
+The block header will be updated (TODO) to include some commitment to
+the results of DeliverTx, be it a bitarray of non-OK transactions, or a
+merkle root of the data returned by the DeliverTx requests, or both.
+
+In go:
+
+    // tx is either "key=value" or just arbitrary bytes
+    func (app *KVStoreApplication) DeliverTx(tx []byte) types.Result {
+      parts := strings.Split(string(tx), "=")
+      if len(parts) == 2 {
+        app.state.Set([]byte(parts[0]), []byte(parts[1]))
+      } else {
+        app.state.Set(tx, tx)
+      }
+      return types.OK
+    }
+
+In Java:
+
+    /**
+     * Using Protobuf types from the protoc compiler, we always start with a byte[]
+     */
+    ResponseDeliverTx deliverTx(RequestDeliverTx request) {
+        byte[] transaction  = request.getTx().toByteArray();
+    
+        // validate your transaction
+    
+        if (notValid) {
+            return ResponseDeliverTx.newBuilder().setCode(CodeType.BadNonce).setLog("transaction was invalid").build();
+        } else {
+            ResponseDeliverTx.newBuilder().setCode(CodeType.OK).build();
+        }
+    
+    }
+
+### Commit
+
+Once all processing of the block is complete, Tendermint sends the
+Commit request and blocks waiting for a response. While the mempool may
+run concurrently with block processing (the BeginBlock, DeliverTxs, and
+EndBlock), it is locked for the Commit request so that its state can be
+safely reset during Commit. This means the app *MUST NOT* do any
+blocking communication with the mempool (ie. broadcast\_tx) during
+Commit, or there will be deadlock. Note also that all remaining
+transactions in the mempool are replayed on the mempool connection
+(CheckTx) following a commit.
+
+The app should respond to the Commit request with a byte array, which is
+the deterministic state root of the application. It is included in the
+header of the next block. It can be used to provide easily verified
+Merkle-proofs of the state of the application.
+
+It is expected that the app will persist state to disk on Commit. The
+option to have all transactions replayed from some previous block is the
+job of the [Handshake](#handshake).
+
+In go:
+
+    func (app *KVStoreApplication) Commit() types.Result {
+      hash := app.state.Hash()
+      return types.NewResultOK(hash, "")
+    }
+
+In Java:
+
+    ResponseCommit requestCommit(RequestCommit requestCommit) {
+    
+        // update the internal app-state
+        byte[] newAppState = calculateAppState();
+    
+        // and return it to the node
+        return ResponseCommit.newBuilder().setCode(CodeType.OK).setData(ByteString.copyFrom(newAppState)).build();
+    }
+
+### BeginBlock
+
+The BeginBlock request can be used to run some code at the beginning of
+every block. It also allows Tendermint to send the current block hash
+and header to the application, before it sends any of the transactions.
+
+The app should remember the latest height and header (ie. from which it
+has run a successful Commit) so that it can tell Tendermint where to
+pick up from when it restarts. See information on the Handshake, below.
+
+In go:
+
+    // Track the block hash and header information
+    func (app *PersistentKVStoreApplication) BeginBlock(params types.RequestBeginBlock) {
+      // update latest block info
+      app.blockHeader = params.Header
+    
+      // reset valset changes
+      app.changes = make([]*types.Validator, 0)
+    }
+
+In Java:
+
+    /*
+     * all types come from protobuf definition
+     */
+    ResponseBeginBlock requestBeginBlock(RequestBeginBlock req) {
+    
+        Header header = req.getHeader();
+        byte[] prevAppHash = header.getAppHash().toByteArray();
+        long prevHeight = header.getHeight();
+        long numTxs = header.getNumTxs();
+    
+        // run your pre-block logic. Maybe prepare a state snapshot, message components, etc
+    
+        return ResponseBeginBlock.newBuilder().build();
+    }
+
+### EndBlock
+
+The EndBlock request can be used to run some code at the end of every
+block. Additionally, the response may contain a list of validators,
+which can be used to update the validator set. To add a new validator or
+update an existing one, simply include them in the list returned in the
+EndBlock response. To remove one, include it in the list with a `power`
+equal to `0`. Tendermint core will take care of updating the validator
+set. Note the change in voting power must be strictly less than 1/3 per
+block if you want a light client to be able to prove the transition
+externally. See the [light client
+docs](https://godoc.org/github.com/tendermint/tendermint/lite#hdr-How_We_Track_Validators)
+for details on how it tracks validators.
+
+In go:
+
+    // Update the validator set
+    func (app *PersistentKVStoreApplication) EndBlock(req types.RequestEndBlock) types.ResponseEndBlock {
+      return types.ResponseEndBlock{ValidatorUpdates: app.ValUpdates}
+    }
+
+In Java:
+
+    /*
+     * Assume that one validator changes. The new validator has a power of 10
+     */
+    ResponseEndBlock requestEndBlock(RequestEndBlock req) {
+        final long currentHeight = req.getHeight();
+        final byte[] validatorPubKey = getValPubKey();
+    
+        ResponseEndBlock.Builder builder = ResponseEndBlock.newBuilder();
+        builder.addDiffs(1, Types.Validator.newBuilder().setPower(10L).setPubKey(ByteString.copyFrom(validatorPubKey)).build());
+    
+        return builder.build();
+    }
+
+### Query Connection
+
+This connection is used to query the application without engaging
+consensus. It's exposed over the tendermint core rpc, so clients can
+query the app without exposing a server on the app itself, but they must
+serialize each query as a single byte array. Additionally, certain
+"standardized" queries may be used to inform local decisions, for
+instance about which peers to connect to.
+
+Tendermint Core currently uses the Query connection to filter peers upon
+connecting, according to IP address or public key. For instance,
+returning non-OK ABCI response to either of the following queries will
+cause Tendermint to not connect to the corresponding peer:
+
+-   `p2p/filter/addr/<addr>`, where `<addr>` is an IP address.
+-   `p2p/filter/pubkey/<pubkey>`, where `<pubkey>` is the hex-encoded
+    ED25519 key of the node (not it's validator key)
+
+Note: these query formats are subject to change!
+
+In go:
+
+    func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery types.ResponseQuery) {
+      if reqQuery.Prove {
+        value, proof, exists := app.state.Proof(reqQuery.Data)
+        resQuery.Index = -1 // TODO make Proof return index
+        resQuery.Key = reqQuery.Data
+        resQuery.Value = value
+        resQuery.Proof = proof
+        if exists {
+          resQuery.Log = "exists"
+        } else {
+          resQuery.Log = "does not exist"
+        }
+        return
+      } else {
+        index, value, exists := app.state.Get(reqQuery.Data)
+        resQuery.Index = int64(index)
+        resQuery.Value = value
+        if exists {
+          resQuery.Log = "exists"
+        } else {
+          resQuery.Log = "does not exist"
+        }
+        return
+      }
+    }
+
+In Java:
+
+    ResponseQuery requestQuery(RequestQuery req) {
+        final boolean isProveQuery = req.getProve();
+        final ResponseQuery.Builder responseBuilder = ResponseQuery.newBuilder();
+    
+        if (isProveQuery) {
+            com.app.example.ProofResult proofResult = generateProof(req.getData().toByteArray());
+            final byte[] proofAsByteArray = proofResult.getAsByteArray();
+    
+            responseBuilder.setProof(ByteString.copyFrom(proofAsByteArray));
+            responseBuilder.setKey(req.getData());
+            responseBuilder.setValue(ByteString.copyFrom(proofResult.getData()));
+            responseBuilder.setLog(result.getLogValue());
+        } else {
+            byte[] queryData = req.getData().toByteArray();
+    
+            final com.app.example.QueryResult result = generateQueryResult(queryData);
+    
+            responseBuilder.setIndex(result.getIndex());
+            responseBuilder.setValue(ByteString.copyFrom(result.getValue()));
+            responseBuilder.setLog(result.getLogValue());
+        }
+    
+        return responseBuilder.build();
+    }
+
+### Handshake
+
+When the app or tendermint restarts, they need to sync to a common
+height. When an ABCI connection is first established, Tendermint will
+call `Info` on the Query connection. The response should contain the
+LastBlockHeight and LastBlockAppHash - the former is the last block for
+which the app ran Commit successfully, the latter is the response from
+that Commit.
+
+Using this information, Tendermint will determine what needs to be
+replayed, if anything, against the app, to ensure both Tendermint and
+the app are synced to the latest block height.
+
+If the app returns a LastBlockHeight of 0, Tendermint will just replay
+all blocks.
+
+In go:
+
+    func (app *KVStoreApplication) Info(req types.RequestInfo) (resInfo types.ResponseInfo) {
+      return types.ResponseInfo{Data: cmn.Fmt("{\"size\":%v}", app.state.Size())}
+    }
+
+In Java:
+
+    ResponseInfo requestInfo(RequestInfo req) {
+        final byte[] lastAppHash = getLastAppHash();
+        final long lastHeight = getLastHeight();
+        return ResponseInfo.newBuilder().setLastBlockAppHash(ByteString.copyFrom(lastAppHash)).setLastBlockHeight(lastHeight).build();
+    }
+
+### Genesis
+
+`InitChain` will be called once upon the genesis. `params` includes the
+initial validator set. Later on, it may be extended to take parts of the
+consensus params.
+
+In go:
+
+    // Save the validators in the merkle tree
+    func (app *PersistentKVStoreApplication) InitChain(params types.RequestInitChain) {
+      for _, v := range params.Validators {
+        r := app.updateValidator(v)
+        if r.IsErr() {
+          app.logger.Error("Error updating validators", "r", r)
+        }
+      }
+    }
+
+In Java:
+
+    /*
+     * all types come from protobuf definition
+     */
+    ResponseInitChain requestInitChain(RequestInitChain req) {
+        final int validatorsCount = req.getValidatorsCount();
+        final List<Types.Validator> validatorsList = req.getValidatorsList();
+    
+        validatorsList.forEach((validator) -> {
+            long power = validator.getPower();
+            byte[] validatorPubKey = validator.getPubKey().toByteArray();
+    
+            // do somehing for validator setup in app
+        });
+    
+        return ResponseInitChain.newBuilder().build();
+    }
--- a/docs/app-development.rst
+++ b/docs/app-development.rst
@@ -1,648 +0,0 @@
-Application Development Guide
-=============================
-
-ABCI Design
-----------
-
-The purpose of ABCI is to provide a clean interface between state
-transition machines on one computer and the mechanics of their
-replication across multiple computers. The former we call 'application
-logic' and the latter the 'consensus engine'. Application logic
-validates transactions and optionally executes transactions against some
-persistent state. A consensus engine ensures all transactions are
-replicated in the same order on every machine. We call each machine in a
-consensus engine a 'validator', and each validator runs the same
-transactions through the same application logic. In particular, we are
-interested in blockchain-style consensus engines, where transactions are
-committed in hash-linked blocks.
-
-The ABCI design has a few distinct components:
-
-  message protocol
-
-   -  pairs of request and response messages
-   -  consensus makes requests, application responds
-   -  defined using protobuf
-
-  server/client
-
-   -  consensus engine runs the client
-   -  application runs the server
-   -  two implementations:
-
-      -  async raw bytes
-      -  grpc
-
-  blockchain protocol
-
-   -  abci is connection oriented
-   -  Tendermint Core maintains three connections:
-
-      -  `mempool connection <#mempool-connection>`__: for checking if
-         transactions should be relayed before they are committed; only
-         uses ``CheckTx``
-      -  `consensus connection <#consensus-connection>`__: for executing
-         transactions that have been committed. Message sequence is -
-         for every block -
-         ``BeginBlock, [DeliverTx, ...], EndBlock, Commit``
-      -  `query connection <#query-connection>`__: for querying the
-         application state; only uses Query and Info
-
-The mempool and consensus logic act as clients, and each maintains an
-open ABCI connection with the application, which hosts an ABCI server.
-Shown are the request and response types sent on each connection.
-
-Message Protocol
----------------
-
-The message protocol consists of pairs of requests and responses. Some
-messages have no fields, while others may include byte-arrays, strings,
-or integers. See the ``message Request`` and ``message Response``
-definitions in `the protobuf definition
-file <https://github.com/tendermint/abci/blob/master/types/types.proto>`__,
-and the `protobuf
-documentation <https://developers.google.com/protocol-buffers/docs/overview>`__
-for more details.
-
-For each request, a server should respond with the corresponding
-response, where order of requests is preserved in the order of
-responses.
-
-Server
------
-
-To use ABCI in your programming language of choice, there must be a ABCI
-server in that language. Tendermint supports two kinds of implementation
-of the server:
-
-  Asynchronous, raw socket server (Tendermint Socket Protocol, also
-   known as TSP or Teaspoon)
-  GRPC
-
-Both can be tested using the ``abci-cli`` by setting the ``--abci`` flag
-appropriately (ie. to ``socket`` or ``grpc``).
-
-See examples, in various stages of maintenance, in
-`Go <https://github.com/tendermint/abci/tree/master/server>`__,
-`JavaScript <https://github.com/tendermint/js-abci>`__,
-`Python <https://github.com/tendermint/abci/tree/master/example/python3/abci>`__,
-`C++ <https://github.com/mdyring/cpp-tmsp>`__, and
-`Java <https://github.com/jTendermint/jabci>`__.
-
-GRPC
-~~~~
-
-If GRPC is available in your language, this is the easiest approach,
-though it will have significant performance overhead.
-
-To get started with GRPC, copy in the `protobuf
-file <https://github.com/tendermint/abci/blob/master/types/types.proto>`__
-and compile it using the GRPC plugin for your language. For instance,
-for golang, the command is
-``protoc --go_out=plugins=grpc:. types.proto``. See the `grpc
-documentation for more details <http://www.grpc.io/docs/>`__. ``protoc``
-will autogenerate all the necessary code for ABCI client and server in
-your language, including whatever interface your application must
-satisfy to be used by the ABCI server for handling requests.
-
-TSP
-~~~
-
-If GRPC is not available in your language, or you require higher
-performance, or otherwise enjoy programming, you may implement your own
-ABCI server using the Tendermint Socket Protocol, known affectionately
-as Teaspoon. The first step is still to auto-generate the relevant data
-types and codec in your language using ``protoc``. Messages coming over
-the socket are Protobuf3 encoded, but additionally length-prefixed to
-facilitate use as a streaming protocol. Protobuf3 doesn't have an
-official length-prefix standard, so we use our own. The first byte in
-the prefix represents the length of the Big Endian encoded length. The
-remaining bytes in the prefix are the Big Endian encoded length.
-
-For example, if the Protobuf3 encoded ABCI message is 0xDEADBEEF (4
-bytes), the length-prefixed message is 0x0104DEADBEEF. If the Protobuf3
-encoded ABCI message is 65535 bytes long, the length-prefixed message
-would be like 0x02FFFF....
-
-Note this prefixing does not apply for grpc.
-
-An ABCI server must also be able to support multiple connections, as
-Tendermint uses three connections.
-
-Client
------
-
-There are currently two use-cases for an ABCI client. One is a testing
-tool, as in the ``abci-cli``, which allows ABCI requests to be sent via
-command line. The other is a consensus engine, such as Tendermint Core,
-which makes requests to the application every time a new transaction is
-received or a block is committed.
-
-It is unlikely that you will need to implement a client. For details of
-our client, see
-`here <https://github.com/tendermint/abci/tree/master/client>`__.
-
-Most of the examples below are from `kvstore application
-<https://github.com/tendermint/abci/blob/master/example/kvstore/kvstore.go>`__,
-which is a part of the abci repo. `persistent_kvstore application
-<https://github.com/tendermint/abci/blob/master/example/kvstore/persistent_kvstore.go>`__
-is used to show ``BeginBlock``, ``EndBlock`` and ``InitChain``
-example implementations.
-
-Blockchain Protocol
-------------------
-
-In ABCI, a transaction is simply an arbitrary length byte-array. It is
-the application's responsibility to define the transaction codec as they
-please, and to use it for both CheckTx and DeliverTx.
-
-Note that there are two distinct means for running transactions,
-corresponding to stages of 'awareness' of the transaction in the
-network. The first stage is when a transaction is received by a
-validator from a client into the so-called mempool or transaction pool -
-this is where we use CheckTx. The second is when the transaction is
-successfully committed on more than 2/3 of validators - where we use
-DeliverTx. In the former case, it may not be necessary to run all the
-state transitions associated with the transaction, as the transaction
-may not ultimately be committed until some much later time, when the
-result of its execution will be different. For instance, an Ethereum
-ABCI app would check signatures and amounts in CheckTx, but would not
-actually execute any contract code until the DeliverTx, so as to avoid
-executing state transitions that have not been finalized.
-
-To formalize the distinction further, two explicit ABCI connections are
-made between Tendermint Core and the application: the mempool connection
-and the consensus connection. We also make a third connection, the query
-connection, to query the local state of the app.
-
-Mempool Connection
-~~~~~~~~~~~~~~~~~~
-
-The mempool connection is used *only* for CheckTx requests.
-Transactions are run using CheckTx in the same order they were
-received by the validator. If the CheckTx returns ``OK``, the
-transaction is kept in memory and relayed to other peers in the same
-order it was received. Otherwise, it is discarded.
-
-CheckTx requests run concurrently with block processing; so they
-should run against a copy of the main application state which is reset
-after every block. This copy is necessary to track transitions made by
-a sequence of CheckTx requests before they are included in a block.
-When a block is committed, the application must ensure to reset the
-mempool state to the latest committed state. Tendermint Core will then
-filter through all transactions in the mempool, removing any that were
-included in the block, and re-run the rest using CheckTx against the
-post-Commit mempool state (this behaviour can be turned off with
-``[mempool] recheck = false``).
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        func (app *KVStoreApplication) CheckTx(tx []byte) types.Result {
-          return types.OK
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        ResponseCheckTx requestCheckTx(RequestCheckTx req) {
-            byte[] transaction = req.getTx().toByteArray();
-
-            // validate transaction
-
-            if (notValid) {
-                return ResponseCheckTx.newBuilder().setCode(CodeType.BadNonce).setLog("invalid tx").build();
-            } else {
-                return ResponseCheckTx.newBuilder().setCode(CodeType.OK).build();
-            }
-        }
-
-Replay Protection
-^^^^^^^^^^^^^^^^^
-To prevent old transactions from being replayed, CheckTx must
-implement replay protection.
-
-Tendermint provides the first defence layer by keeping a lightweight
-in-memory cache of 100k (``[mempool] cache_size``) last transactions in
-the mempool. If Tendermint is just started or the clients sent more
-than 100k transactions, old transactions may be sent to the
-application. So it is important CheckTx implements some logic to
-handle them.
-
-There are cases where a transaction will (or may) become valid in some
-future state, in which case you probably want to disable Tendermint's
-cache. You can do that by setting ``[mempool] cache_size = 0`` in the
-config.
-
-Consensus Connection
-~~~~~~~~~~~~~~~~~~~~
-
-The consensus connection is used only when a new block is committed, and
-communicates all information from the block in a series of requests:
-``BeginBlock, [DeliverTx, ...], EndBlock, Commit``. That is, when a
-block is committed in the consensus, we send a list of DeliverTx
-requests (one for each transaction) sandwiched by BeginBlock and
-EndBlock requests, and followed by a Commit.
-
-DeliverTx
-^^^^^^^^^
-
-DeliverTx is the workhorse of the blockchain. Tendermint sends the
-DeliverTx requests asynchronously but in order, and relies on the
-underlying socket protocol (ie. TCP) to ensure they are received by the
-app in order. They have already been ordered in the global consensus by
-the Tendermint protocol.
-
-DeliverTx returns a abci.Result, which includes a Code, Data, and Log.
-The code may be non-zero (non-OK), meaning the corresponding transaction
-should have been rejected by the mempool, but may have been included in
-a block by a Byzantine proposer.
-
-The block header will be updated (TODO) to include some commitment to
-the results of DeliverTx, be it a bitarray of non-OK transactions, or a
-merkle root of the data returned by the DeliverTx requests, or both.
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        // tx is either "key=value" or just arbitrary bytes
-        func (app *KVStoreApplication) DeliverTx(tx []byte) types.Result {
-          parts := strings.Split(string(tx), "=")
-          if len(parts) == 2 {
-            app.state.Set([]byte(parts[0]), []byte(parts[1]))
-          } else {
-            app.state.Set(tx, tx)
-          }
-          return types.OK
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        /**
-         * Using Protobuf types from the protoc compiler, we always start with a byte[]
-         */
-        ResponseDeliverTx deliverTx(RequestDeliverTx request) {
-            byte[] transaction  = request.getTx().toByteArray();
-
-            // validate your transaction
-
-            if (notValid) {
-                return ResponseDeliverTx.newBuilder().setCode(CodeType.BadNonce).setLog("transaction was invalid").build();
-            } else {
-                ResponseDeliverTx.newBuilder().setCode(CodeType.OK).build();
-            }
-
-        }
-
-Commit
-^^^^^^
-
-Once all processing of the block is complete, Tendermint sends the
-Commit request and blocks waiting for a response. While the mempool may
-run concurrently with block processing (the BeginBlock, DeliverTxs, and
-EndBlock), it is locked for the Commit request so that its state can be
-safely reset during Commit. This means the app *MUST NOT* do any
-blocking communication with the mempool (ie. broadcast\_tx) during
-Commit, or there will be deadlock. Note also that all remaining
-transactions in the mempool are replayed on the mempool connection
-(CheckTx) following a commit.
-
-The app should respond to the Commit request with a byte array, which is the deterministic
-state root of the application. It is included in the header of the next
-block. It can be used to provide easily verified Merkle-proofs of the
-state of the application.
-
-It is expected that the app will persist state to disk on Commit. The
-option to have all transactions replayed from some previous block is the
-job of the `Handshake <#handshake>`__.
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        func (app *KVStoreApplication) Commit() types.Result {
-          hash := app.state.Hash()
-          return types.NewResultOK(hash, "")
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        ResponseCommit requestCommit(RequestCommit requestCommit) {
-
-            // update the internal app-state
-            byte[] newAppState = calculateAppState();
-
-            // and return it to the node
-            return ResponseCommit.newBuilder().setCode(CodeType.OK).setData(ByteString.copyFrom(newAppState)).build();
-        }
-
-BeginBlock
-^^^^^^^^^^
-
-The BeginBlock request can be used to run some code at the beginning of
-every block. It also allows Tendermint to send the current block hash
-and header to the application, before it sends any of the transactions.
-
-The app should remember the latest height and header (ie. from which it
-has run a successful Commit) so that it can tell Tendermint where to
-pick up from when it restarts. See information on the Handshake, below.
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        // Track the block hash and header information
-        func (app *PersistentKVStoreApplication) BeginBlock(params types.RequestBeginBlock) {
-          // update latest block info
-          app.blockHeader = params.Header
-
-          // reset valset changes
-          app.changes = make([]*types.Validator, 0)
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        /*
-         * all types come from protobuf definition
-         */
-        ResponseBeginBlock requestBeginBlock(RequestBeginBlock req) {
-
-            Header header = req.getHeader();
-            byte[] prevAppHash = header.getAppHash().toByteArray();
-            long prevHeight = header.getHeight();
-            long numTxs = header.getNumTxs();
-
-            // run your pre-block logic. Maybe prepare a state snapshot, message components, etc
-
-            return ResponseBeginBlock.newBuilder().build();
-        }
-
-EndBlock
-^^^^^^^^
-
-The EndBlock request can be used to run some code at the end of every block.
-Additionally, the response may contain a list of validators, which can be used
-to update the validator set. To add a new validator or update an existing one,
-simply include them in the list returned in the EndBlock response. To remove
-one, include it in the list with a ``power`` equal to ``0``. Tendermint core
-will take care of updating the validator set. Note the change in voting power
-must be strictly less than 1/3 per block if you want a light client to be able
-to prove the transition externally. See the `light client docs
-<https://godoc.org/github.com/tendermint/tendermint/lite#hdr-How_We_Track_Validators>`__
-for details on how it tracks validators.
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        // Update the validator set
-        func (app *PersistentKVStoreApplication) EndBlock(req types.RequestEndBlock) types.ResponseEndBlock {
-          return types.ResponseEndBlock{ValidatorUpdates: app.ValUpdates}
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        /*
-         * Assume that one validator changes. The new validator has a power of 10
-         */
-        ResponseEndBlock requestEndBlock(RequestEndBlock req) {
-            final long currentHeight = req.getHeight();
-            final byte[] validatorPubKey = getValPubKey();
-
-            ResponseEndBlock.Builder builder = ResponseEndBlock.newBuilder();
-            builder.addDiffs(1, Types.Validator.newBuilder().setPower(10L).setPubKey(ByteString.copyFrom(validatorPubKey)).build());
-
-            return builder.build();
-        }
-
-Query Connection
-~~~~~~~~~~~~~~~~
-
-This connection is used to query the application without engaging
-consensus. It's exposed over the tendermint core rpc, so clients can
-query the app without exposing a server on the app itself, but they must
-serialize each query as a single byte array. Additionally, certain
-"standardized" queries may be used to inform local decisions, for
-instance about which peers to connect to.
-
-Tendermint Core currently uses the Query connection to filter peers upon
-connecting, according to IP address or public key. For instance,
-returning non-OK ABCI response to either of the following queries will
-cause Tendermint to not connect to the corresponding peer:
-
-  ``p2p/filter/addr/<addr>``, where ``<addr>`` is an IP address.
-  ``p2p/filter/pubkey/<pubkey>``, where ``<pubkey>`` is the hex-encoded
-   ED25519 key of the node (not it's validator key)
-
-Note: these query formats are subject to change!
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        func (app *KVStoreApplication) Query(reqQuery types.RequestQuery) (resQuery types.ResponseQuery) {
-          if reqQuery.Prove {
-            value, proof, exists := app.state.Proof(reqQuery.Data)
-            resQuery.Index = -1 // TODO make Proof return index
-            resQuery.Key = reqQuery.Data
-            resQuery.Value = value
-            resQuery.Proof = proof
-            if exists {
-              resQuery.Log = "exists"
-            } else {
-              resQuery.Log = "does not exist"
-            }
-            return
-          } else {
-            index, value, exists := app.state.Get(reqQuery.Data)
-            resQuery.Index = int64(index)
-            resQuery.Value = value
-            if exists {
-              resQuery.Log = "exists"
-            } else {
-              resQuery.Log = "does not exist"
-            }
-            return
-          }
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        ResponseQuery requestQuery(RequestQuery req) {
-            final boolean isProveQuery = req.getProve();
-            final ResponseQuery.Builder responseBuilder = ResponseQuery.newBuilder();
-
-            if (isProveQuery) {
-                com.app.example.ProofResult proofResult = generateProof(req.getData().toByteArray());
-                final byte[] proofAsByteArray = proofResult.getAsByteArray();
-
-                responseBuilder.setProof(ByteString.copyFrom(proofAsByteArray));
-                responseBuilder.setKey(req.getData());
-                responseBuilder.setValue(ByteString.copyFrom(proofResult.getData()));
-                responseBuilder.setLog(result.getLogValue());
-            } else {
-                byte[] queryData = req.getData().toByteArray();
-
-                final com.app.example.QueryResult result = generateQueryResult(queryData);
-
-                responseBuilder.setIndex(result.getIndex());
-                responseBuilder.setValue(ByteString.copyFrom(result.getValue()));
-                responseBuilder.setLog(result.getLogValue());
-            }
-
-            return responseBuilder.build();
-        }
-
-Handshake
-~~~~~~~~~
-
-When the app or tendermint restarts, they need to sync to a common
-height. When an ABCI connection is first established, Tendermint will
-call ``Info`` on the Query connection. The response should contain the
-LastBlockHeight and LastBlockAppHash - the former is the last block for
-which the app ran Commit successfully, the latter is the response
-from that Commit.
-
-Using this information, Tendermint will determine what needs to be
-replayed, if anything, against the app, to ensure both Tendermint and
-the app are synced to the latest block height.
-
-If the app returns a LastBlockHeight of 0, Tendermint will just replay
-all blocks.
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        func (app *KVStoreApplication) Info(req types.RequestInfo) (resInfo types.ResponseInfo) {
-          return types.ResponseInfo{Data: cmn.Fmt("{\"size\":%v}", app.state.Size())}
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        ResponseInfo requestInfo(RequestInfo req) {
-            final byte[] lastAppHash = getLastAppHash();
-            final long lastHeight = getLastHeight();
-            return ResponseInfo.newBuilder().setLastBlockAppHash(ByteString.copyFrom(lastAppHash)).setLastBlockHeight(lastHeight).build();
-        }
-
-Genesis
-~~~~~~~
-
-``InitChain`` will be called once upon the genesis. ``params`` includes the
-initial validator set. Later on, it may be extended to take parts of the
-consensus params.
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Go Example**
-
-    .. code-block:: go
-
-        // Save the validators in the merkle tree
-        func (app *PersistentKVStoreApplication) InitChain(params types.RequestInitChain) {
-          for _, v := range params.Validators {
-            r := app.updateValidator(v)
-            if r.IsErr() {
-              app.logger.Error("Error updating validators", "r", r)
-            }
-          }
-        }
-
-.. container:: toggle
-
-    .. container:: header
-
-        **Show/Hide Java Example**
-
-    .. code-block:: java
-
-        /*
-         * all types come from protobuf definition
-         */
-        ResponseInitChain requestInitChain(RequestInitChain req) {
-            final int validatorsCount = req.getValidatorsCount();
-            final List<Types.Validator> validatorsList = req.getValidatorsList();
-
-            validatorsList.forEach((validator) -> {
-                long power = validator.getPower();
-                byte[] validatorPubKey = validator.getPubKey().toByteArray();
-
-                // do somehing for validator setup in app
-            });
-
-            return ResponseInitChain.newBuilder().build();
-        }
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -41,8 +41,15 @@ templates_path = ['_templates']
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
 #
-#source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
+
+from recommonmark.parser import CommonMarkParser
+
+source_parsers = {
+    '.md': CommonMarkParser,
+}
+
+source_suffix = ['.rst', '.md']
+#source_suffix = '.rst'

 # The master toctree document.
 master_doc = 'index'
@@ -171,27 +178,29 @@ texinfo_documents = [
     'Database'),
 ]

-# ---- customization -------------------------
+# ---------------- customizations ----------------------

+# for Docker README, below
+from shutil import copyfile
+
+# tm-bench and tm-monitor
 tools_repo = "https://raw.githubusercontent.com/tendermint/tools/"
 tools_branch = "master"

 tools_dir = "./tools"
-assets_dir = tools_dir + "/assets"
+

 if os.path.isdir(tools_dir) != True:
    os.mkdir(tools_dir)
-if os.path.isdir(assets_dir) != True:
-    os.mkdir(assets_dir)

-urllib.urlretrieve(tools_repo+tools_branch+'/docker/README.rst', filename=tools_dir+'/docker.rst')
+copyfile('../DOCKER/README.md', tools_dir+'/docker.md')

-urllib.urlretrieve(tools_repo+tools_branch+'/tm-bench/README.rst', filename=tools_dir+'/benchmarking.rst')
-urllib.urlretrieve(tools_repo+tools_branch+'/tm-monitor/README.rst', filename='tools/monitoring.rst')
+urllib.urlretrieve(tools_repo+tools_branch+'/tm-bench/README.md', filename=tools_dir+'/benchmarking.md')
+urllib.urlretrieve(tools_repo+tools_branch+'/tm-monitor/README.md', filename=tools_dir+'/monitoring.md')

 #### abci spec #################################

 abci_repo = "https://raw.githubusercontent.com/tendermint/abci/"
 abci_branch = "develop"

-urllib.urlretrieve(abci_repo+abci_branch+'/specification.rst', filename='abci-spec.rst')
+urllib.urlretrieve(abci_repo+abci_branch+'/specification.md', filename='abci-spec.md')
--- a/docs/deploy-testnets.md
+++ b/docs/deploy-testnets.md
@@ -0,0 +1,68 @@
+# Deploy a Testnet
+
+Now that we've seen how ABCI works, and even played with a few
+applications on a single validator node, it's time to deploy a test
+network to four validator nodes.
+
+## Manual Deployments
+
+It's relatively easy to setup a Tendermint cluster manually. The only
+requirements for a particular Tendermint node are a private key for the
+validator, stored as `priv_validator.json`, a node key, stored as
+`node_key.json` and a list of the public keys of all validators, stored
+as `genesis.json`. These files should be stored in
+`~/.tendermint/config`, or wherever the `$TMHOME` variable might be set
+to.
+
+Here are the steps to setting up a testnet manually:
+
+1)  Provision nodes on your cloud provider of choice
+2)  Install Tendermint and the application of interest on all nodes
+3)  Generate a private key and a node key for each validator using
+    `tendermint init`
+4)  Compile a list of public keys for each validator into a
+    `genesis.json` file and replace the existing file with it.
+5)  Run
+    `tendermint node --proxy_app=kvstore --p2p.persistent_peers=< peer addresses >`
+    on each node, where `< peer addresses >` is a comma separated list
+    of the IP:PORT combination for each node. The default port for
+    Tendermint is `26656`. Thus, if the IP addresses of your nodes were
+    `192.168.0.1, 192.168.0.2, 192.168.0.3, 192.168.0.4`, the command
+    would look like:
+
+
+    tendermint node --proxy_app=kvstore --p2p.persistent_peers=96663a3dd0d7b9d17d4c8211b191af259621c693@192.168.0.1:26656, 429fcf25974313b95673f58d77eacdd434402665@192.168.0.2:26656, 0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@192.168.0.3:26656, f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@192.168.0.4:26656
+
+After a few seconds, all the nodes should connect to each other and
+start making blocks! For more information, see the Tendermint Networks
+section of [the guide to using Tendermint](using-tendermint.html).
+
+But wait! Steps 3 and 4 are quite manual. Instead, use [this
+script](https://github.com/tendermint/tendermint/blob/develop/docs/examples/init_testnet.sh),
+which does the heavy lifting for you. And it gets better.
+
+Instead of the previously linked script to initialize the files required
+for a testnet, we have the `tendermint testnet` command. By default,
+running `tendermint testnet` will create all the required files, just
+like the script. Of course, you'll still need to manually edit some
+fields in the `config.toml`. Alternatively, see the available flags to
+auto-populate the `config.toml` with the fields that would otherwise be
+passed in via flags when running `tendermint node`. As you might
+imagine, this command is useful for manual or automated deployments.
+
+## Automated Deployments
+
+The easiest and fastest way to get a testnet up in less than 5 minutes.
+
+### Local
+
+With `docker` and `docker-compose` installed, run the command:
+
+    make localnet-start
+
+from the root of the tendermint repository. This will spin up a 4-node
+local testnet. Review the target in the Makefile to debug any problems.
+
+### Cloud
+
+See the [next section](./terraform-and-ansible.html) for details.
--- a/docs/deploy-testnets.rst
+++ b/docs/deploy-testnets.rst
@@ -1,64 +0,0 @@
-Deploy a Testnet
-================
-
-Now that we've seen how ABCI works, and even played with a few
-applications on a single validator node, it's time to deploy a test
-network to four validator nodes.
-
-Manual Deployments
------------------
-
-It's relatively easy to setup a Tendermint cluster manually. The only
-requirements for a particular Tendermint node are a private key for the
-validator, stored as ``priv_validator.json``, a node key, stored as
-``node_key.json`` and a list of the public keys of all validators, stored as
-``genesis.json``. These files should be stored in ``~/.tendermint/config``, or
-wherever the ``$TMHOME`` variable might be set to.
-
-Here are the steps to setting up a testnet manually:
-
-1) Provision nodes on your cloud provider of choice
-2) Install Tendermint and the application of interest on all nodes
-3) Generate a private key and a node key for each validator using
-   ``tendermint init``
-4) Compile a list of public keys for each validator into a
-   ``genesis.json`` file and replace the existing file with it.
-5) Run ``tendermint node --proxy_app=kvstore --p2p.persistent_peers=< peer addresses >`` on each node,
-   where ``< peer addresses >`` is a comma separated list of the IP:PORT
-   combination for each node. The default port for Tendermint is
-   ``46656``. Thus, if the IP addresses of your nodes were
-   ``192.168.0.1, 192.168.0.2, 192.168.0.3, 192.168.0.4``, the command
-   would look like:
-
-::
-
-    tendermint node --proxy_app=kvstore --p2p.persistent_peers=96663a3dd0d7b9d17d4c8211b191af259621c693@192.168.0.1:46656, 429fcf25974313b95673f58d77eacdd434402665@192.168.0.2:46656, 0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@192.168.0.3:46656, f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@192.168.0.4:46656
-
-After a few seconds, all the nodes should connect to each other and start
-making blocks! For more information, see the Tendermint Networks section
-of `the guide to using Tendermint <using-tendermint.html>`__.
-
-But wait! Steps 3 and 4 are quite manual. Instead, use `this script <https://github.com/tendermint/tendermint/blob/develop/docs/examples/init_testnet.sh>`__, which does the heavy lifting for you. And it gets better.
-
-Instead of the previously linked script to initialize the files required for a testnet, we have the ``tendermint testnet`` command. By default, running ``tendermint testnet`` will create all the required files, just like the script. Of course, you'll still need to manually edit some fields in the ``config.toml``. Alternatively, see the available flags to auto-populate the ``config.toml`` with the fields that would otherwise be passed in via flags when running ``tendermint node``. As you might imagine, this command is useful for manual or automated deployments.
-
-Automated Deployments
---------------------
-
-The easiest and fastest way to get a testnet up in less than 5 minutes.
-
-Local
-^^^^^
-
-With ``docker`` and ``docker-compose`` installed, run the command:
-
-::
-
-    make localnet-start
-
-from the root of the tendermint repository. This will spin up a 4-node local testnet.
-
-Cloud
-^^^^^
-
-See the `next section <./terraform-and-ansible.html>`__ for details.
--- a/docs/determinism.rst
+++ b/docs/determinism.rst
@@ -1,8 +1,5 @@
-On Determinism
-==============
+# On Determinism

 Arguably, the most difficult part of blockchain programming is determinism - that is, ensuring that sources of indeterminism do not creep into the design of such systems.

-See `this issue <https://github.com/tendermint/abci/issues/56>`__ for more information on the potential sources of indeterminism.
-
-
+See [this issue](https://github.com/tendermint/abci/issues/56) for more information on the potential sources of indeterminism.
--- a/docs/ecosystem.md
+++ b/docs/ecosystem.md
@@ -0,0 +1,21 @@
+# Ecosystem
+
+The growing list of applications built using various pieces of the
+Tendermint stack can be found at:
+
+-   https://tendermint.com/ecosystem
+
+We thank the community for their contributions thus far and welcome the
+addition of new projects. A pull request can be submitted to [this
+file](https://github.com/tendermint/aib-data/blob/master/json/ecosystem.json)
+to include your project.
+
+## Other Tools
+
+See [deploy testnets](./deploy-testnets.html) for information about all
+the tools built by Tendermint. We have Kubernetes, Ansible, and
+Terraform integrations.
+
+For upgrading from older to newer versions of tendermint and to migrate
+your chain data, see [tm-migrator](https://github.com/hxzqlh/tm-tools)
+written by @hxzqlh.
--- a/docs/ecosystem.rst
+++ b/docs/ecosystem.rst
@@ -1,15 +0,0 @@
-Tendermint Ecosystem
-====================
-
-The growing list of applications built using various pieces of the Tendermint stack can be found at:
-
-* https://tendermint.com/ecosystem
-
-We thank the community for their contributions thus far and welcome the addition of new projects. A pull request can be submitted to `this file <https://github.com/tendermint/aib-data/blob/master/json/ecosystem.json>`__ to include your project.
-
-Other Tools
-----------
-
-See `deploy testnets <./deploy-testnets.html>`__ for information about all the tools built by Tendermint. We have Kubernetes, Ansible, and Terraform integrations.
-
-For upgrading from older to newer versions of tendermint and to migrate your chain data, see `tm-migrator <https://github.com/hxzqlh/tm-tools>`__ written by @hxzqlh.
--- a/docs/examples/getting-started.md
+++ b/docs/examples/getting-started.md
@@ -85,7 +85,7 @@ I[01-06|01:45:15.624] Committed state                              module=state
 Check the status with:

 ```
-curl -s localhost:46657/status
+curl -s localhost:26657/status
 ```

 ### Sending Transactions
@@ -93,25 +93,25 @@ curl -s localhost:46657/status
 With the kvstore app running, we can send transactions:

 ```
-curl -s 'localhost:46657/broadcast_tx_commit?tx="abcd"'
+curl -s 'localhost:26657/broadcast_tx_commit?tx="abcd"'
 ```

 and check that it worked with:

 ```
-curl -s 'localhost:46657/abci_query?data="abcd"'
+curl -s 'localhost:26657/abci_query?data="abcd"'
 ```

 We can send transactions with a key and value too:

 ```
-curl -s 'localhost:46657/broadcast_tx_commit?tx="name=satoshi"'
+curl -s 'localhost:26657/broadcast_tx_commit?tx="name=satoshi"'
 ```

 and query the key:

 ```
-curl -s 'localhost:46657/abci_query?data="name"'
+curl -s 'localhost:26657/abci_query?data="name"'
 ```

 where the value is returned in hex.
@@ -134,10 +134,10 @@ This will install `go` and other dependencies, get the Tendermint source code, t
 Next, `cd` into `docs/examples`. Each command below should be run from each node, in sequence:

 ```
-tendermint node --home ./node0 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:46656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:46656"
-tendermint node --home ./node1 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:46656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:46656"
-tendermint node --home ./node2 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:46656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:46656"
-tendermint node --home ./node3 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:46656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:46656"
+tendermint node --home ./node0 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:26656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:26656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:26656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:26656"
+tendermint node --home ./node1 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:26656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:26656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:26656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:26656"
+tendermint node --home ./node2 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:26656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:26656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:26656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:26656"
+tendermint node --home ./node3 --proxy_app=kvstore --p2p.persistent_peers="167b80242c300bf0ccfb3ced3dec60dc2a81776e@IP1:26656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@IP2:26656,303a1a4312c30525c99ba66522dd81cca56a361a@IP3:26656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@IP4:26656"
 ```

 Note that after the third node is started, blocks will start to stream in
--- a/docs/examples/node0/config/config.toml
+++ b/docs/examples/node0/config/config.toml
@@ -5,7 +5,7 @@

 # TCP or UNIX socket address of the ABCI application,
 # or the name of an ABCI application compiled in with the Tendermint binary
-proxy_app = "tcp://127.0.0.1:46658"
+proxy_app = "tcp://127.0.0.1:26658"

 # A custom human readable name for this node
 moniker = "alpha"
@@ -51,7 +51,7 @@ filter_peers = false
 [rpc]

 # TCP or UNIX socket address for the RPC server to listen on
-laddr = "tcp://0.0.0.0:46657"
+laddr = "tcp://0.0.0.0:26657"

 # TCP or UNIX socket address for the gRPC server to listen on
 # NOTE: This server only supports /broadcast_tx_commit
@@ -64,7 +64,7 @@ unsafe = false
 [p2p]

 # Address to listen for incoming connections
-laddr = "tcp://0.0.0.0:46656"
+laddr = "tcp://0.0.0.0:26656"

 # Comma separated list of seed nodes to connect to
 seeds = ""
@@ -103,9 +103,6 @@ pex = true
 # Does not work if the peer-exchange reactor is disabled.
 seed_mode = false

-# Authenticated encryption
-auth_enc = true
-
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = ""

--- a/docs/examples/node1/config/config.toml
+++ b/docs/examples/node1/config/config.toml
@@ -5,7 +5,7 @@

 # TCP or UNIX socket address of the ABCI application,
 # or the name of an ABCI application compiled in with the Tendermint binary
-proxy_app = "tcp://127.0.0.1:46658"
+proxy_app = "tcp://127.0.0.1:26658"

 # A custom human readable name for this node
 moniker = "bravo"
@@ -51,7 +51,7 @@ filter_peers = false
 [rpc]

 # TCP or UNIX socket address for the RPC server to listen on
-laddr = "tcp://0.0.0.0:46657"
+laddr = "tcp://0.0.0.0:26657"

 # TCP or UNIX socket address for the gRPC server to listen on
 # NOTE: This server only supports /broadcast_tx_commit
@@ -64,7 +64,7 @@ unsafe = false
 [p2p]

 # Address to listen for incoming connections
-laddr = "tcp://0.0.0.0:46656"
+laddr = "tcp://0.0.0.0:26656"

 # Comma separated list of seed nodes to connect to
 seeds = ""
@@ -103,9 +103,6 @@ pex = true
 # Does not work if the peer-exchange reactor is disabled.
 seed_mode = false

-# Authenticated encryption
-auth_enc = true
-
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = ""

--- a/docs/examples/node2/config/config.toml
+++ b/docs/examples/node2/config/config.toml
@@ -5,7 +5,7 @@

 # TCP or UNIX socket address of the ABCI application,
 # or the name of an ABCI application compiled in with the Tendermint binary
-proxy_app = "tcp://127.0.0.1:46658"
+proxy_app = "tcp://127.0.0.1:26658"

 # A custom human readable name for this node
 moniker = "charlie"
@@ -51,7 +51,7 @@ filter_peers = false
 [rpc]

 # TCP or UNIX socket address for the RPC server to listen on
-laddr = "tcp://0.0.0.0:46657"
+laddr = "tcp://0.0.0.0:26657"

 # TCP or UNIX socket address for the gRPC server to listen on
 # NOTE: This server only supports /broadcast_tx_commit
@@ -64,7 +64,7 @@ unsafe = false
 [p2p]

 # Address to listen for incoming connections
-laddr = "tcp://0.0.0.0:46656"
+laddr = "tcp://0.0.0.0:26656"

 # Comma separated list of seed nodes to connect to
 seeds = ""
@@ -103,9 +103,6 @@ pex = true
 # Does not work if the peer-exchange reactor is disabled.
 seed_mode = false

-# Authenticated encryption
-auth_enc = true
-
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = ""

--- a/docs/examples/node3/config/config.toml
+++ b/docs/examples/node3/config/config.toml
@@ -5,7 +5,7 @@

 # TCP or UNIX socket address of the ABCI application,
 # or the name of an ABCI application compiled in with the Tendermint binary
-proxy_app = "tcp://127.0.0.1:46658"
+proxy_app = "tcp://127.0.0.1:26658"

 # A custom human readable name for this node
 moniker = "delta"
@@ -51,7 +51,7 @@ filter_peers = false
 [rpc]

 # TCP or UNIX socket address for the RPC server to listen on
-laddr = "tcp://0.0.0.0:46657"
+laddr = "tcp://0.0.0.0:26657"

 # TCP or UNIX socket address for the gRPC server to listen on
 # NOTE: This server only supports /broadcast_tx_commit
@@ -64,7 +64,7 @@ unsafe = false
 [p2p]

 # Address to listen for incoming connections
-laddr = "tcp://0.0.0.0:46656"
+laddr = "tcp://0.0.0.0:26656"

 # Comma separated list of seed nodes to connect to
 seeds = ""
@@ -103,9 +103,6 @@ pex = true
 # Does not work if the peer-exchange reactor is disabled.
 seed_mode = false

-# Authenticated encryption
-auth_enc = true
-
 # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
 private_peer_ids = ""

--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@@ -0,0 +1,265 @@
+# Getting Started
+
+## First Tendermint App
+
+As a general purpose blockchain engine, Tendermint is agnostic to the
+application you want to run. So, to run a complete blockchain that does
+something useful, you must start two programs: one is Tendermint Core,
+the other is your application, which can be written in any programming
+language. Recall from [the intro to
+ABCI](introduction.html#ABCI-Overview) that Tendermint Core handles all
+the p2p and consensus stuff, and just forwards transactions to the
+application when they need to be validated, or when they're ready to be
+committed to a block.
+
+In this guide, we show you some examples of how to run an application
+using Tendermint.
+
+### Install
+
+The first apps we will work with are written in Go. To install them, you
+need to [install Go](https://golang.org/doc/install) and put
+`$GOPATH/bin` in your `$PATH`; see
+[here](https://github.com/tendermint/tendermint/wiki/Setting-GOPATH) for
+more info.
+
+Then run
+
+    go get -u github.com/tendermint/abci/cmd/abci-cli
+
+If there is an error, install and run the
+[dep](https://github.com/golang/dep) tool to pin the dependencies:
+
+    cd $GOPATH/src/github.com/tendermint/abci
+    make get_tools
+    make get_vendor_deps
+    make install
+
+Now you should have the `abci-cli` installed; you'll see a couple of
+commands (`counter` and `kvstore`) that are example applications written
+in Go. See below for an application written in JavaScript.
+
+Now, let's run some apps!
+
+## KVStore - A First Example
+
+The kvstore app is a [Merkle
+tree](https://en.wikipedia.org/wiki/Merkle_tree) that just stores all
+transactions. If the transaction contains an `=`, e.g. `key=value`, then
+the `value` is stored under the `key` in the Merkle tree. Otherwise, the
+full transaction bytes are stored as the key and the value.
+
+Let's start a kvstore application.
+
+    abci-cli kvstore
+
+In another terminal, we can start Tendermint. If you have never run
+Tendermint before, use:
+
+    tendermint init
+    tendermint node
+
+If you have used Tendermint, you may want to reset the data for a new
+blockchain by running `tendermint unsafe_reset_all`. Then you can run
+`tendermint node` to start Tendermint, and connect to the app. For more
+details, see [the guide on using Tendermint](./using-tendermint.html).
+
+You should see Tendermint making blocks! We can get the status of our
+Tendermint node as follows:
+
+    curl -s localhost:26657/status
+
+The `-s` just silences `curl`. For nicer output, pipe the result into a
+tool like [jq](https://stedolan.github.io/jq/) or `json_pp`.
+
+Now let's send some transactions to the kvstore.
+
+    curl -s 'localhost:26657/broadcast_tx_commit?tx="abcd"'
+
+Note the single quote (`'`) around the url, which ensures that the
+double quotes (`"`) are not escaped by bash. This command sent a
+transaction with bytes `abcd`, so `abcd` will be stored as both the key
+and the value in the Merkle tree. The response should look something
+like:
+
+    {
+      "jsonrpc": "2.0",
+      "id": "",
+      "result": {
+        "check_tx": {
+          "fee": {}
+        },
+        "deliver_tx": {
+          "tags": [
+            {
+              "key": "YXBwLmNyZWF0b3I=",
+              "value": "amFl"
+            },
+            {
+              "key": "YXBwLmtleQ==",
+              "value": "YWJjZA=="
+            }
+          ],
+          "fee": {}
+        },
+        "hash": "9DF66553F98DE3C26E3C3317A3E4CED54F714E39",
+        "height": 14
+      }
+    }
+
+We can confirm that our transaction worked and the value got stored by
+querying the app:
+
+    curl -s 'localhost:26657/abci_query?data="abcd"'
+
+The result should look like:
+
+    {
+      "jsonrpc": "2.0",
+      "id": "",
+      "result": {
+        "response": {
+          "log": "exists",
+          "index": "-1",
+          "key": "YWJjZA==",
+          "value": "YWJjZA=="
+        }
+      }
+    }
+
+Note the `value` in the result (`YWJjZA==`); this is the base64-encoding
+of the ASCII of `abcd`. You can verify this in a python 2 shell by
+running `"61626364".decode('base64')` or in python 3 shell by running
+`import codecs; codecs.decode("61626364", 'base64').decode('ascii')`.
+Stay tuned for a future release that [makes this output more
+human-readable](https://github.com/tendermint/abci/issues/32).
+
+Now let's try setting a different key and value:
+
+    curl -s 'localhost:26657/broadcast_tx_commit?tx="name=satoshi"'
+
+Now if we query for `name`, we should get `satoshi`, or `c2F0b3NoaQ==`
+in base64:
+
+    curl -s 'localhost:26657/abci_query?data="name"'
+
+Try some other transactions and queries to make sure everything is
+working!
+
+## Counter - Another Example
+
+Now that we've got the hang of it, let's try another application, the
+`counter` app.
+
+The counter app doesn't use a Merkle tree, it just counts how many times
+we've sent a transaction, or committed the state.
+
+This application has two modes: `serial=off` and `serial=on`.
+
+When `serial=on`, transactions must be a big-endian encoded incrementing
+integer, starting at 0.
+
+If `serial=off`, there are no restrictions on transactions.
+
+In a live blockchain, transactions collect in memory before they are
+committed into blocks. To avoid wasting resources on invalid
+transactions, ABCI provides the `CheckTx` message, which application
+developers can use to accept or reject transactions, before they are
+stored in memory or gossipped to other peers.
+
+In this instance of the counter app, with `serial=on`, `CheckTx` only
+allows transactions whose integer is greater than the last committed
+one.
+
+Let's kill the previous instance of `tendermint` and the `kvstore`
+application, and start the counter app. We can enable `serial=on` with a
+flag:
+
+    abci-cli counter --serial
+
+In another window, reset then start Tendermint:
+
+    tendermint unsafe_reset_all
+    tendermint node
+
+Once again, you can see the blocks streaming by. Let's send some
+transactions. Since we have set `serial=on`, the first transaction must
+be the number `0`:
+
+    curl localhost:26657/broadcast_tx_commit?tx=0x00
+
+Note the empty (hence successful) response. The next transaction must be
+the number `1`. If instead, we try to send a `5`, we get an error:
+
+    > curl localhost:26657/broadcast_tx_commit?tx=0x05
+    {
+      "jsonrpc": "2.0",
+      "id": "",
+      "result": {
+        "check_tx": {
+          "fee": {}
+        },
+        "deliver_tx": {
+          "code": 2,
+          "log": "Invalid nonce. Expected 1, got 5",
+          "fee": {}
+        },
+        "hash": "33B93DFF98749B0D6996A70F64071347060DC19C",
+        "height": 34
+      }
+    }
+
+But if we send a `1`, it works again:
+
+    > curl localhost:26657/broadcast_tx_commit?tx=0x01
+    {
+      "jsonrpc": "2.0",
+      "id": "",
+      "result": {
+        "check_tx": {
+          "fee": {}
+        },
+        "deliver_tx": {
+          "fee": {}
+        },
+        "hash": "F17854A977F6FA7EEA1BD758E296710B86F72F3D",
+        "height": 60
+      }
+    }
+
+For more details on the `broadcast_tx` API, see [the guide on using
+Tendermint](./using-tendermint.html).
+
+## CounterJS - Example in Another Language
+
+We also want to run applications in another language - in this case,
+we'll run a Javascript version of the `counter`. To run it, you'll need
+to [install node](https://nodejs.org/en/download/).
+
+You'll also need to fetch the relevant repository, from
+[here](https://github.com/tendermint/js-abci) then install it. As go
+devs, we keep all our code under the `$GOPATH`, so run:
+
+    go get github.com/tendermint/js-abci &> /dev/null
+    cd $GOPATH/src/github.com/tendermint/js-abci/example
+    npm install
+    cd ..
+
+Kill the previous `counter` and `tendermint` processes. Now run the app:
+
+    node example/app.js
+
+In another window, reset and start `tendermint`:
+
+    tendermint unsafe_reset_all
+    tendermint node
+
+Once again, you should see blocks streaming by - but now, our
+application is written in javascript! Try sending some transactions, and
+like before - the results should be the same:
+
+    curl localhost:26657/broadcast_tx_commit?tx=0x00 # ok
+    curl localhost:26657/broadcast_tx_commit?tx=0x05 # invalid nonce
+    curl localhost:26657/broadcast_tx_commit?tx=0x01 # ok
+
+Neat, eh?
--- a/docs/getting-started.rst
+++ b/docs/getting-started.rst
@@ -1,325 +0,0 @@
-First Tendermint App
-====================
-
-As a general purpose blockchain engine, Tendermint is agnostic to the
-application you want to run. So, to run a complete blockchain that does
-something useful, you must start two programs: one is Tendermint Core,
-the other is your application, which can be written in any programming
-language. Recall from `the intro to ABCI <introduction.html#ABCI-Overview>`__ that
-Tendermint Core handles all the p2p and consensus stuff, and just
-forwards transactions to the application when they need to be validated,
-or when they're ready to be committed to a block.
-
-In this guide, we show you some examples of how to run an application
-using Tendermint.
-
-Install
-------
-
-The first apps we will work with are written in Go. To install them, you
-need to `install Go <https://golang.org/doc/install>`__ and put
-``$GOPATH/bin`` in your
-``$PATH``; see `here <https://github.com/tendermint/tendermint/wiki/Setting-GOPATH>`__ for more info.
-
-Then run
-
-::
-
-    go get -u github.com/tendermint/abci/cmd/abci-cli
-
-If there is an error, install and run the `dep <https://github.com/golang/dep>`__ tool to pin the
-dependencies:
-
-::
-
-    cd $GOPATH/src/github.com/tendermint/abci
-    make get_tools
-    make get_vendor_deps
-    make install
-
-Now you should have the ``abci-cli`` installed; you'll see
-a couple of commands (``counter`` and ``kvstore``) that are
-example applications written in Go. See below for an application
-written in JavaScript.
-
-Now, let's run some apps!
-
-KVStore - A First Example
-------------------------
-
-The kvstore app is a `Merkle
-tree <https://en.wikipedia.org/wiki/Merkle_tree>`__ that just stores all
-transactions. If the transaction contains an ``=``, e.g. ``key=value``,
-then the ``value`` is stored under the ``key`` in the Merkle tree.
-Otherwise, the full transaction bytes are stored as the key and the
-value.
-
-Let's start a kvstore application.
-
-::
-
-    abci-cli kvstore
-
-In another terminal, we can start Tendermint. If you have never run
-Tendermint before, use:
-
-::
-
-    tendermint init
-    tendermint node
-
-If you have used Tendermint, you may want to reset the data for a new
-blockchain by running ``tendermint unsafe_reset_all``. Then you can run
-``tendermint node`` to start Tendermint, and connect to the app. For
-more details, see `the guide on using
-Tendermint <./using-tendermint.html>`__.
-
-You should see Tendermint making blocks! We can get the status of our
-Tendermint node as follows:
-
-::
-
-    curl -s localhost:46657/status
-
-The ``-s`` just silences ``curl``. For nicer output, pipe the result into a
-tool like `jq <https://stedolan.github.io/jq/>`__ or ``json_pp``.
-
-Now let's send some transactions to the kvstore.
-
-::
-
-    curl -s 'localhost:46657/broadcast_tx_commit?tx="abcd"'
-
-Note the single quote (``'``) around the url, which ensures that the
-double quotes (``"``) are not escaped by bash. This command sent a
-transaction with bytes ``abcd``, so ``abcd`` will be stored as both the
-key and the value in the Merkle tree. The response should look something
-like:
-
-::
-
-    {
-      "jsonrpc": "2.0",
-      "id": "",
-      "result": {
-        "check_tx": {
-          "fee": {}
-        },
-        "deliver_tx": {
-          "tags": [
-            {
-              "key": "YXBwLmNyZWF0b3I=",
-              "value": "amFl"
-            },
-            {
-              "key": "YXBwLmtleQ==",
-              "value": "YWJjZA=="
-            }
-          ],
-          "fee": {}
-        },
-        "hash": "9DF66553F98DE3C26E3C3317A3E4CED54F714E39",
-        "height": 14
-      }
-    }
-
-We can confirm that our transaction worked and the value got stored by
-querying the app:
-
-::
-
-    curl -s 'localhost:46657/abci_query?data="abcd"'
-
-The result should look like:
-
-::
-
-    {
-      "jsonrpc": "2.0",
-      "id": "",
-      "result": {
-        "response": {
-          "log": "exists",
-          "index": "-1",
-          "key": "YWJjZA==",
-          "value": "YWJjZA=="
-        }
-      }
-    }
-
-Note the ``value`` in the result (``YWJjZA==``); this is the
-base64-encoding of the ASCII of ``abcd``. You can verify this in
-a python 2 shell by running ``"61626364".decode('base64')`` or in python 3 shell by running ``import codecs; codecs.decode("61626364", 'base64').decode('ascii')``. Stay
-tuned for a future release that `makes this output more human-readable <https://github.com/tendermint/abci/issues/32>`__.
-
-Now let's try setting a different key and value:
-
-::
-
-    curl -s 'localhost:46657/broadcast_tx_commit?tx="name=satoshi"'
-
-Now if we query for ``name``, we should get ``satoshi``, or
-``c2F0b3NoaQ==`` in base64:
-
-::
-
-    curl -s 'localhost:46657/abci_query?data="name"'
-
-Try some other transactions and queries to make sure everything is
-working!
-
-Counter - Another Example
-------------------------
-
-Now that we've got the hang of it, let's try another application, the
-**counter** app.
-
-The counter app doesn't use a Merkle tree, it just counts how many times
-we've sent a transaction, or committed the state.
-
-This application has two modes: ``serial=off`` and ``serial=on``.
-
-When ``serial=on``, transactions must be a big-endian encoded
-incrementing integer, starting at 0.
-
-If ``serial=off``, there are no restrictions on transactions.
-
-In a live blockchain, transactions collect in memory before they are
-committed into blocks. To avoid wasting resources on invalid
-transactions, ABCI provides the ``CheckTx`` message, which application
-developers can use to accept or reject transactions, before they are
-stored in memory or gossipped to other peers.
-
-In this instance of the counter app, with ``serial=on``, ``CheckTx``
-only allows transactions whose integer is greater than the last
-committed one.
-
-Let's kill the previous instance of ``tendermint`` and the ``kvstore``
-application, and start the counter app. We can enable ``serial=on`` with
-a flag:
-
-::
-
-    abci-cli counter --serial
-
-In another window, reset then start Tendermint:
-
-::
-
-    tendermint unsafe_reset_all
-    tendermint node
-
-Once again, you can see the blocks streaming by. Let's send some
-transactions. Since we have set ``serial=on``, the first transaction
-must be the number ``0``:
-
-::
-
-    curl localhost:46657/broadcast_tx_commit?tx=0x00
-
-Note the empty (hence successful) response. The next transaction must be
-the number ``1``. If instead, we try to send a ``5``, we get an error:
-
-::
-
-    > curl localhost:46657/broadcast_tx_commit?tx=0x05
-    {
-      "jsonrpc": "2.0",
-      "id": "",
-      "result": {
-        "check_tx": {
-          "fee": {}
-        },
-        "deliver_tx": {
-          "code": 2,
-          "log": "Invalid nonce. Expected 1, got 5",
-          "fee": {}
-        },
-        "hash": "33B93DFF98749B0D6996A70F64071347060DC19C",
-        "height": 34
-      }
-    }
-
-But if we send a ``1``, it works again:
-
-::
-
-    > curl localhost:46657/broadcast_tx_commit?tx=0x01
-    {
-      "jsonrpc": "2.0",
-      "id": "",
-      "result": {
-        "check_tx": {
-          "fee": {}
-        },
-        "deliver_tx": {
-          "fee": {}
-        },
-        "hash": "F17854A977F6FA7EEA1BD758E296710B86F72F3D",
-        "height": 60
-      }
-    }
-
-For more details on the ``broadcast_tx`` API, see `the guide on using
-Tendermint <./using-tendermint.html>`__.
-
-CounterJS - Example in Another Language
---------------------------------------
-
-We also want to run applications in another language - in this case,
-we'll run a Javascript version of the ``counter``. To run it, you'll
-need to `install node <https://nodejs.org/en/download/>`__.
-
-You'll also need to fetch the relevant repository, from `here <https://github.com/tendermint/js-abci>`__ then install it. As go devs, we
-keep all our code under the ``$GOPATH``, so run:
-
-::
-
-    go get github.com/tendermint/js-abci &> /dev/null
-    cd $GOPATH/src/github.com/tendermint/js-abci/example
-    npm install
-    cd ..
-
-Kill the previous ``counter`` and ``tendermint`` processes. Now run the
-app:
-
-::
-
-    node example/app.js
-
-In another window, reset and start ``tendermint``:
-
-::
-
-    tendermint unsafe_reset_all
-    tendermint node
-
-Once again, you should see blocks streaming by - but now, our
-application is written in javascript! Try sending some transactions, and
-like before - the results should be the same:
-
-::
-
-    curl localhost:46657/broadcast_tx_commit?tx=0x00 # ok
-    curl localhost:46657/broadcast_tx_commit?tx=0x05 # invalid nonce
-    curl localhost:46657/broadcast_tx_commit?tx=0x01 # ok
-
-Neat, eh?
-
-Basecoin - A More Interesting Example
-------------------------------------
-
-We saved the best for last; the `Cosmos SDK <https://github.com/cosmos/cosmos-sdk>`__ is a general purpose framework for building cryptocurrencies. Unlike the ``kvstore`` and ``counter``, which are strictly for example purposes. The reference implementation of Cosmos SDK is ``basecoin``, which demonstrates how to use the building blocks of the Cosmos SDK.
-
-The default ``basecoin`` application is a multi-asset cryptocurrency
-that supports inter-blockchain communication (IBC). For more details on how
-basecoin works and how to use it, see our `basecoin
-guide <http://cosmos-sdk.readthedocs.io/en/latest/basecoin-basics.html>`__
-
-In this tutorial you learned how to run applications using Tendermint
-on a single node. You saw how applications could be written in different
-languages, and how to send transactions and query for the latest state.
-But the true power of Tendermint comes from its ability to securely and
-efficiently run an application across a distributed network of nodes,
-while keeping them all in sync using its state-of-the-art consensus
-protocol. Next, we show you how to deploy Tendermint testnets.
--- a/docs/how-to-read-logs.rst
+++ b/docs/how-to-read-logs.rst
@@ -1,13 +1,9 @@
-How to read logs
-================
+# How to read logs

-Walk through example
--------------------
+## Walkabout example

 We first create three connections (mempool, consensus and query) to the
-application (running ``kvstore`` locally in this case).
-
-::
+application (running `kvstore` locally in this case).

    I[10-04|13:54:27.364] Starting multiAppConn                        module=proxy impl=multiAppConn
    I[10-04|13:54:27.366] Starting localClient                         module=abci-client connection=query impl=localClient
@@ -16,24 +12,20 @@ application (running ``kvstore`` locally in this case).

 Then Tendermint Core and the application perform a handshake.

-::
-
    I[10-04|13:54:27.367] ABCI Handshake                               module=consensus appHeight=90 appHash=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD
    I[10-04|13:54:27.368] ABCI Replay Blocks                           module=consensus appHeight=90 storeHeight=90 stateHeight=90
    I[10-04|13:54:27.368] Completed ABCI Handshake - Tendermint and App are synced module=consensus appHeight=90 appHash=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD

-After that, we start a few more things like the event switch, reactors, and
-perform UPNP discover in order to detect the IP address.
-
-::
+After that, we start a few more things like the event switch, reactors,
+and perform UPNP discover in order to detect the IP address.

    I[10-04|13:54:27.374] Starting EventSwitch                         module=types impl=EventSwitch
    I[10-04|13:54:27.375] This node is a validator                     module=consensus
    I[10-04|13:54:27.379] Starting Node                                module=main impl=Node
-    I[10-04|13:54:27.381] Local listener                               module=p2p ip=:: port=46656
+    I[10-04|13:54:27.381] Local listener                               module=p2p ip=:: port=26656
    I[10-04|13:54:27.382] Getting UPNP external address                module=p2p
    I[10-04|13:54:30.386] Could not perform UPNP discover              module=p2p err="write udp4 0.0.0.0:38238->239.255.255.250:1900: i/o timeout"
-    I[10-04|13:54:30.386] Starting DefaultListener                     module=p2p impl=Listener(@10.0.2.15:46656)
+    I[10-04|13:54:30.386] Starting DefaultListener                     module=p2p impl=Listener(@10.0.2.15:26656)
    I[10-04|13:54:30.387] Starting P2P Switch                          module=p2p impl="P2P Switch"
    I[10-04|13:54:30.387] Starting MempoolReactor                      module=mempool impl=MempoolReactor
    I[10-04|13:54:30.387] Starting BlockchainReactor                   module=blockchain impl=BlockchainReactor
@@ -48,28 +40,21 @@ validator". It also could be just an observer (regular node).

 Next we replay all the messages from the WAL.

-::
-
    I[10-04|13:54:30.390] Catchup by replaying consensus messages      module=consensus height=91
    I[10-04|13:54:30.390] Replay: New Step                             module=consensus height=91 round=0 step=RoundStepNewHeight
    I[10-04|13:54:30.390] Replay: Done                                 module=consensus

 "Started node" message signals that everything is ready for work.

-::
+    I[10-04|13:54:30.391] Starting RPC HTTP server on tcp socket 0.0.0.0:26657 module=rpc-server
+    I[10-04|13:54:30.392] Started node                                 module=main nodeInfo="NodeInfo{id: DF22D7C92C91082324A1312F092AA1DA197FA598DBBFB6526E, moniker: anonymous, network: test-chain-3MNw2N [remote , listen 10.0.2.15:26656], version: 0.11.0-10f361fc ([wire_version=0.6.2 p2p_version=0.5.0 consensus_version=v1/0.2.2 rpc_version=0.7.0/3 tx_index=on rpc_addr=tcp://0.0.0.0:26657])}"

-    I[10-04|13:54:30.391] Starting RPC HTTP server on tcp socket 0.0.0.0:46657 module=rpc-server
-    I[10-04|13:54:30.392] Started node                                 module=main nodeInfo="NodeInfo{id: DF22D7C92C91082324A1312F092AA1DA197FA598DBBFB6526E, moniker: anonymous, network: test-chain-3MNw2N [remote , listen 10.0.2.15:46656], version: 0.11.0-10f361fc ([wire_version=0.6.2 p2p_version=0.5.0 consensus_version=v1/0.2.2 rpc_version=0.7.0/3 tx_index=on rpc_addr=tcp://0.0.0.0:46657])}"
-
-Next follows a standard block creation cycle, where we enter a new round,
-propose a block, receive more than 2/3 of prevotes, then precommits and finally
-have a chance to commit a block. For details, please refer to `Consensus
-Overview
-<introduction.html#consensus-overview>`__
-or `Byzantine Consensus Algorithm
-<specification.html>`__.
-
-::
+Next follows a standard block creation cycle, where we enter a new
+round, propose a block, receive more than 2/3 of prevotes, then
+precommits and finally have a chance to commit a block. For details,
+please refer to [Consensus
+Overview](introduction.html#consensus-overview) or [Byzantine Consensus
+Algorithm](specification.html).

    I[10-04|13:54:30.393] enterNewRound(91/0). Current: 91/0/RoundStepNewHeight module=consensus
    I[10-04|13:54:30.393] enterPropose(91/0). Current: 91/0/RoundStepNewRound module=consensus
@@ -110,56 +95,36 @@ or `Byzantine Consensus Algorithm
    I[10-04|13:54:30.410] Committed state                              module=state height=91 txs=0 hash=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD
    I[10-04|13:54:30.410] Recheck txs                                  module=mempool numtxs=0 height=91

-List of modules
---------------
+## List of modules

-Here is the list of modules you may encounter in Tendermint's log and a little
-overview what they do.
+Here is the list of modules you may encounter in Tendermint's log and a
+little overview what they do.

- ``abci-client`` As mentioned in `Application Development Guide
-  <app-development.html#abci-design>`__,
-  Tendermint acts as an ABCI client with respect to the application and
-  maintains 3 connections: mempool, consensus and query. The code used by
-  Tendermint Core can be found `here
-  <https://github.com/tendermint/abci/tree/master/client>`__.
-
- ``blockchain``
-  Provides storage, pool (a group of peers), and reactor for both storing and
-  exchanging blocks between peers.
-
- ``consensus``
-  The heart of Tendermint core, which is the implementation of the consensus
-  algorithm. Includes two "submodules": ``wal`` (write-ahead logging) for
-  ensuring data integrity and ``replay`` to replay blocks and messages on
-  recovery from a crash.
-
- ``events``
-  Simple event notification system. The list of events can be found
-  `here
-  <https://github.com/tendermint/tendermint/blob/master/types/events.go>`__.
-  You can subscribe to them by calling ``subscribe`` RPC method.
-  Refer to `RPC docs
-  <specification/rpc.html>`__
-  for additional information.
-
- ``mempool``
-  Mempool module handles all incoming transactions, whenever they are
-  coming from peers or the application.
-
- ``p2p``
-  Provides an abstraction around peer-to-peer communication. For more details,
-  please check out the `README
-  <https://github.com/tendermint/tendermint/blob/56c60fba2381e4ac41d2ae38a1eb6569acfbc095/p2p/README.md>`__.
-
- ``rpc``
-  `Tendermint's RPC <specification/rpc.html>`__.
-
- ``rpc-server``
-  RPC server. For implementation details, please read the `README <https://github.com/tendermint/tendermint/blob/master/rpc/lib/README.md>`__.
-
- ``state``
-  Represents the latest state and execution submodule, which executes
-  blocks against the application.
-
- ``types``
-  A collection of the publicly exposed types and methods to work with them.
+-   `abci-client` As mentioned in [Application Development Guide](app-development.md#abci-design),    Tendermint acts as an ABCI
+    client with respect to the application and maintains 3 connections:
+    mempool, consensus and query. The code used by Tendermint Core can
+    be found [here](https://github.com/tendermint/abci/tree/master/client).
+-   `blockchain` Provides storage, pool (a group of peers), and reactor
+    for both storing and exchanging blocks between peers.
+-   `consensus` The heart of Tendermint core, which is the
+    implementation of the consensus algorithm. Includes two
+    "submodules": `wal` (write-ahead logging) for ensuring data
+    integrity and `replay` to replay blocks and messages on recovery
+    from a crash.
+-   `events` Simple event notification system. The list of events can be
+    found
+    [here](https://github.com/tendermint/tendermint/blob/master/types/events.go).
+    You can subscribe to them by calling `subscribe` RPC method. Refer
+    to [RPC docs](specification/rpc.html) for additional information.
+-   `mempool` Mempool module handles all incoming transactions, whenever
+    they are coming from peers or the application.
+-   `p2p` Provides an abstraction around peer-to-peer communication. For
+    more details, please check out the
+    [README](https://github.com/tendermint/tendermint/blob/master/p2p/README.md).
+-   `rpc` [Tendermint's RPC](specification/rpc.html).
+-   `rpc-server` RPC server. For implementation details, please read the
+    [README](https://github.com/tendermint/tendermint/blob/master/rpc/lib/README.md).
+-   `state` Represents the latest state and execution submodule, which
+    executes blocks against the application.
+-   `types` A collection of the publicly exposed types and methods to
+    work with them.
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -12,24 +12,18 @@ Welcome to Tendermint!
   :width: 200px
   :align: center

-Tendermint 101
--------------
+Introduction
+------------

 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1

-   introduction.rst
-   install.rst
-   getting-started.rst
-   using-tendermint.rst
-
-Tendermint Ecosystem
--------------------
-
-.. toctree::
-   :maxdepth: 2
-
-   ecosystem.rst
+   introduction.md
+   install.md
+   getting-started.md
+   using-tendermint.md
+   deploy-testnets.md
+   ecosystem.md

 Tendermint Tools
 ----------------
@@ -37,35 +31,38 @@ Tendermint Tools
 .. the tools/ files are pulled in from the tools repo
 .. see the bottom of conf.py
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1

-   deploy-testnets.rst
-   terraform-and-ansible.rst
-   tools/docker.rst
-   tools/benchmarking.rst
-   tools/monitoring.rst
+   tools/docker.md
+   terraform-and-ansible.md
+   tools/benchmarking.md
+   tools/monitoring.md

-Tendermint 102
--------------
+ABCI, Apps, Logging, Etc
+------------------------

 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1

-   abci-cli.rst
-   abci-spec.rst
-   app-architecture.rst
-   app-development.rst
-   how-to-read-logs.rst
+   abci-cli.md
+   abci-spec.md
+   app-architecture.md
+   app-development.md
+   subscribing-to-events-via-websocket.md
+   indexing-transactions.md
+   how-to-read-logs.md
+   running-in-production.md

-Tendermint 201
--------------
+Research & Specification
+------------------------

 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1

-   specification.rst
-   determinism.rst
-   transactional-semantics.rst
+   determinism.md
+   transactional-semantics.md
+
+.. specification.md ## keep this file for legacy purpose. needs to be fixed though

 * For a deeper dive, see `this thesis <https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769>`__.
 * There is also the `original whitepaper <https://tendermint.com/static/docs/tendermint.pdf>`__, though it is now quite outdated.
--- a/docs/indexing-transactions.md
+++ b/docs/indexing-transactions.md
@@ -0,0 +1,89 @@
+# Indexing Transactions
+
+Tendermint allows you to index transactions and later query or subscribe
+to their results.
+
+Let's take a look at the `[tx_index]` config section:
+
+    ##### transactions indexer configuration options #####
+    [tx_index]
+
+    # What indexer to use for transactions
+    #
+    # Options:
+    #   1) "null" (default)
+    #   2) "kv" - the simplest possible indexer, backed by key-value storage (defaults to levelDB; see DBBackend).
+    indexer = "kv"
+
+    # Comma-separated list of tags to index (by default the only tag is tx hash)
+    #
+    # It's recommended to index only a subset of tags due to possible memory
+    # bloat. This is, of course, depends on the indexer's DB and the volume of
+    # transactions.
+    index_tags = ""
+
+    # When set to true, tells indexer to index all tags. Note this may be not
+    # desirable (see the comment above). IndexTags has a precedence over
+    # IndexAllTags (i.e. when given both, IndexTags will be indexed).
+    index_all_tags = false
+
+By default, Tendermint will index all transactions by their respective
+hashes using an embedded simple indexer. Note, we are planning to add
+more options in the future (e.g., Postgresql indexer).
+
+## Adding tags
+
+In your application's `DeliverTx` method, add the `Tags` field with the
+pairs of UTF-8 encoded strings (e.g. "account.owner": "Bob", "balance":
+"100.0", "date": "2018-01-02").
+
+Example:
+
+    func (app *KVStoreApplication) DeliverTx(tx []byte) types.Result {
+        ...
+        tags := []cmn.KVPair{
+          {[]byte("account.name"), []byte("igor")},
+          {[]byte("account.address"), []byte("0xdeadbeef")},
+          {[]byte("tx.amount"), []byte("7")},
+        }
+        return types.ResponseDeliverTx{Code: code.CodeTypeOK, Tags: tags}
+    }
+
+If you want Tendermint to only index transactions by "account.name" tag,
+in the config set `tx_index.index_tags="account.name"`. If you to index
+all tags, set `index_all_tags=true`
+
+Note, there are a few predefined tags:
+
+-   `tm.event` (event type)
+-   `tx.hash` (transaction's hash)
+-   `tx.height` (height of the block transaction was committed in)
+
+Tendermint will throw a warning if you try to use any of the above keys.
+
+## Querying transactions
+
+You can query the transaction results by calling `/tx_search` RPC
+endpoint:
+
+    curl "localhost:26657/tx_search?query=\"account.name='igor'\"&prove=true"
+
+Check out [API docs](https://tendermint.github.io/slate/?shell#txsearch)
+for more information on query syntax and other options.
+
+## Subscribing to transactions
+
+Clients can subscribe to transactions with the given tags via Websocket
+by providing a query to `/subscribe` RPC endpoint.
+
+    {
+        "jsonrpc": "2.0",
+        "method": "subscribe",
+        "id": "0",
+        "params": {
+            "query": "account.name='igor'"
+        }
+    }
+
+Check out [API docs](https://tendermint.github.io/slate/#subscribe) for
+more information on query syntax and other options.
--- a/docs/install.rst
+++ b/docs/install.rst
@@ -1,6 +1,13 @@
 Install Tendermint
 ==================

+The fastest and easiest way to install the ``tendermint`` binary
+is to run `this script <https://github.com/tendermint/tendermint/blob/develop/scripts/install/install_tendermint_ubuntu.sh>`__ on
+a fresh Ubuntu instance,
+or `this script <https://github.com/tendermint/tendermint/blob/develop/scripts/install/install_tendermint_bsd.sh>`__
+on a fresh FreeBSD instance. Read the comments / instructions carefully (i.e., reset your terminal after running the script,
+make sure your okay with the network connections being made).
+
 From Binary
 -----------

--- a/docs/introduction.md
+++ b/docs/introduction.md
@@ -0,0 +1,331 @@
+# What is Tendermint?
+
+Tendermint is software for securely and consistently replicating an
+application on many machines. By securely, we mean that Tendermint works
+even if up to 1/3 of machines fail in arbitrary ways. By consistently,
+we mean that every non-faulty machine sees the same transaction log and
+computes the same state. Secure and consistent replication is a
+fundamental problem in distributed systems; it plays a critical role in
+the fault tolerance of a broad range of applications, from currencies,
+to elections, to infrastructure orchestration, and beyond.
+
+The ability to tolerate machines failing in arbitrary ways, including
+becoming malicious, is known as Byzantine fault tolerance (BFT). The
+theory of BFT is decades old, but software implementations have only
+became popular recently, due largely to the success of "blockchain
+technology" like Bitcoin and Ethereum. Blockchain technology is just a
+reformalization of BFT in a more modern setting, with emphasis on
+peer-to-peer networking and cryptographic authentication. The name
+derives from the way transactions are batched in blocks, where each
+block contains a cryptographic hash of the previous one, forming a
+chain. In practice, the blockchain data structure actually optimizes BFT
+design.
+
+Tendermint consists of two chief technical components: a blockchain
+consensus engine and a generic application interface. The consensus
+engine, called Tendermint Core, ensures that the same transactions are
+recorded on every machine in the same order. The application interface,
+called the Application BlockChain Interface (ABCI), enables the
+transactions to be processed in any programming language. Unlike other
+blockchain and consensus solutions, which come pre-packaged with built
+in state machines (like a fancy key-value store, or a quirky scripting
+language), developers can use Tendermint for BFT state machine
+replication of applications written in whatever programming language and
+development environment is right for them.
+
+Tendermint is designed to be easy-to-use, simple-to-understand, highly
+performant, and useful for a wide variety of distributed applications.
+
+## Tendermint vs. X
+
+Tendermint is broadly similar to two classes of software. The first
+class consists of distributed key-value stores, like Zookeeper, etcd,
+and consul, which use non-BFT consensus. The second class is known as
+"blockchain technology", and consists of both cryptocurrencies like
+Bitcoin and Ethereum, and alternative distributed ledger designs like
+Hyperledger's Burrow.
+
+### Zookeeper, etcd, consul
+
+Zookeeper, etcd, and consul are all implementations of a key-value store
+atop a classical, non-BFT consensus algorithm. Zookeeper uses a version
+of Paxos called Zookeeper Atomic Broadcast, while etcd and consul use
+the Raft consensus algorithm, which is much younger and simpler. A
+typical cluster contains 3-5 machines, and can tolerate crash failures
+in up to 1/2 of the machines, but even a single Byzantine fault can
+destroy the system.
+
+Each offering provides a slightly different implementation of a
+featureful key-value store, but all are generally focused around
+providing basic services to distributed systems, such as dynamic
+configuration, service discovery, locking, leader-election, and so on.
+
+Tendermint is in essence similar software, but with two key differences:
+- It is Byzantine Fault Tolerant, meaning it can only tolerate up to a
+1/3 of failures, but those failures can include arbitrary behaviour -
+including hacking and malicious attacks. - It does not specify a
+particular application, like a fancy key-value store. Instead, it
+focuses on arbitrary state machine replication, so developers can build
+the application logic that's right for them, from key-value store to
+cryptocurrency to e-voting platform and beyond.
+
+The layout of this Tendermint website content is also ripped directly
+and without shame from [consul.io](https://www.consul.io/) and the other
+[Hashicorp sites](https://www.hashicorp.com/#tools).
+
+### Bitcoin, Ethereum, etc.
+
+Tendermint emerged in the tradition of cryptocurrencies like Bitcoin,
+Ethereum, etc. with the goal of providing a more efficient and secure
+consensus algorithm than Bitcoin's Proof of Work. In the early days,
+Tendermint had a simple currency built in, and to participate in
+consensus, users had to "bond" units of the currency into a security
+deposit which could be revoked if they misbehaved -this is what made
+Tendermint a Proof-of-Stake algorithm.
+
+Since then, Tendermint has evolved to be a general purpose blockchain
+consensus engine that can host arbitrary application states. That means
+it can be used as a plug-and-play replacement for the consensus engines
+of other blockchain software. So one can take the current Ethereum code
+base, whether in Rust, or Go, or Haskell, and run it as a ABCI
+application using Tendermint consensus. Indeed, [we did that with
+Ethereum](https://github.com/tendermint/ethermint). And we plan to do
+the same for Bitcoin, ZCash, and various other deterministic
+applications as well.
+
+Another example of a cryptocurrency application built on Tendermint is
+[the Cosmos network](http://cosmos.network).
+
+### Other Blockchain Projects
+
+[Fabric](https://github.com/hyperledger/fabric) takes a similar approach
+to Tendermint, but is more opinionated about how the state is managed,
+and requires that all application behaviour runs in potentially many
+docker containers, modules it calls "chaincode". It uses an
+implementation of [PBFT](http://pmg.csail.mit.edu/papers/osdi99.pdf).
+from a team at IBM that is [augmented to handle potentially
+non-deterministic
+chaincode](https://www.zurich.ibm.com/~cca/papers/sieve.pdf) It is
+possible to implement this docker-based behaviour as a ABCI app in
+Tendermint, though extending Tendermint to handle non-determinism
+remains for future work.
+
+[Burrow](https://github.com/hyperledger/burrow) is an implementation of
+the Ethereum Virtual Machine and Ethereum transaction mechanics, with
+additional features for a name-registry, permissions, and native
+contracts, and an alternative blockchain API. It uses Tendermint as its
+consensus engine, and provides a particular application state.
+
+## ABCI Overview
+
+The [Application BlockChain Interface
+(ABCI)](https://github.com/tendermint/abci) allows for Byzantine Fault
+Tolerant replication of applications written in any programming
+language.
+
+### Motivation
+
+Thus far, all blockchains "stacks" (such as
+[Bitcoin](https://github.com/bitcoin/bitcoin)) have had a monolithic
+design. That is, each blockchain stack is a single program that handles
+all the concerns of a decentralized ledger; this includes P2P
+connectivity, the "mempool" broadcasting of transactions, consensus on
+the most recent block, account balances, Turing-complete contracts,
+user-level permissions, etc.
+
+Using a monolithic architecture is typically bad practice in computer
+science. It makes it difficult to reuse components of the code, and
+attempts to do so result in complex maintenance procedures for forks of
+the codebase. This is especially true when the codebase is not modular
+in design and suffers from "spaghetti code".
+
+Another problem with monolithic design is that it limits you to the
+language of the blockchain stack (or vice versa). In the case of
+Ethereum which supports a Turing-complete bytecode virtual-machine, it
+limits you to languages that compile down to that bytecode; today, those
+are Serpent and Solidity.
+
+In contrast, our approach is to decouple the consensus engine and P2P
+layers from the details of the application state of the particular
+blockchain application. We do this by abstracting away the details of
+the application to an interface, which is implemented as a socket
+protocol.
+
+Thus we have an interface, the Application BlockChain Interface (ABCI),
+and its primary implementation, the Tendermint Socket Protocol (TSP, or
+Teaspoon).
+
+### Intro to ABCI
+
+[Tendermint Core](https://github.com/tendermint/tendermint) (the
+"consensus engine") communicates with the application via a socket
+protocol that satisfies the [ABCI](https://github.com/tendermint/abci).
+
+To draw an analogy, lets talk about a well-known cryptocurrency,
+Bitcoin. Bitcoin is a cryptocurrency blockchain where each node
+maintains a fully audited Unspent Transaction Output (UTXO) database. If
+one wanted to create a Bitcoin-like system on top of ABCI, Tendermint
+Core would be responsible for
+
+-   Sharing blocks and transactions between nodes
+-   Establishing a canonical/immutable order of transactions
+    (the blockchain)
+
+The application will be responsible for
+
+-   Maintaining the UTXO database
+-   Validating cryptographic signatures of transactions
+-   Preventing transactions from spending non-existent transactions
+-   Allowing clients to query the UTXO database.
+
+Tendermint is able to decompose the blockchain design by offering a very
+simple API (ie. the ABCI) between the application process and consensus
+process.
+
+The ABCI consists of 3 primary message types that get delivered from the
+core to the application. The application replies with corresponding
+response messages.
+
+The messages are specified here: [ABCI Message
+Types](https://github.com/tendermint/abci#message-types).
+
+The **DeliverTx** message is the work horse of the application. Each
+transaction in the blockchain is delivered with this message. The
+application needs to validate each transaction received with the
+**DeliverTx** message against the current state, application protocol,
+and the cryptographic credentials of the transaction. A validated
+transaction then needs to update the application state — by binding a
+value into a key values store, or by updating the UTXO database, for
+instance.
+
+The **CheckTx** message is similar to **DeliverTx**, but it's only for
+validating transactions. Tendermint Core's mempool first checks the
+validity of a transaction with **CheckTx**, and only relays valid
+transactions to its peers. For instance, an application may check an
+incrementing sequence number in the transaction and return an error upon
+**CheckTx** if the sequence number is old. Alternatively, they might use
+a capabilities based system that requires capabilities to be renewed
+with every transaction.
+
+The **Commit** message is used to compute a cryptographic commitment to
+the current application state, to be placed into the next block header.
+This has some handy properties. Inconsistencies in updating that state
+will now appear as blockchain forks which catches a whole class of
+programming errors. This also simplifies the development of secure
+lightweight clients, as Merkle-hash proofs can be verified by checking
+against the block hash, and that the block hash is signed by a quorum.
+
+There can be multiple ABCI socket connections to an application.
+Tendermint Core creates three ABCI connections to the application; one
+for the validation of transactions when broadcasting in the mempool, one
+for the consensus engine to run block proposals, and one more for
+querying the application state.
+
+It's probably evident that applications designers need to very carefully
+design their message handlers to create a blockchain that does anything
+useful but this architecture provides a place to start. The diagram
+below illustrates the flow of messages via ABCI.
+
+![](assets/abci.png)
+
+## A Note on Determinism
+
+The logic for blockchain transaction processing must be deterministic.
+If the application logic weren't deterministic, consensus would not be
+reached among the Tendermint Core replica nodes.
+
+Solidity on Ethereum is a great language of choice for blockchain
+applications because, among other reasons, it is a completely
+deterministic programming language. However, it's also possible to
+create deterministic applications using existing popular languages like
+Java, C++, Python, or Go. Game programmers and blockchain developers are
+already familiar with creating deterministic programs by avoiding
+sources of non-determinism such as:
+
+-   random number generators (without deterministic seeding)
+-   race conditions on threads (or avoiding threads altogether)
+-   the system clock
+-   uninitialized memory (in unsafe programming languages like C
+    or C++)
+-   [floating point
+    arithmetic](http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/)
+-   language features that are random (e.g. map iteration in Go)
+
+While programmers can avoid non-determinism by being careful, it is also
+possible to create a special linter or static analyzer for each language
+to check for determinism. In the future we may work with partners to
+create such tools.
+
+## Consensus Overview
+
+Tendermint is an easy-to-understand, mostly asynchronous, BFT consensus
+protocol. The protocol follows a simple state machine that looks like
+this:
+
+![](assets/consensus_logic.png)
+
+Participants in the protocol are called **validators**; they take turns
+proposing blocks of transactions and voting on them. Blocks are
+committed in a chain, with one block at each **height**. A block may
+fail to be committed, in which case the protocol moves to the next
+**round**, and a new validator gets to propose a block for that height.
+Two stages of voting are required to successfully commit a block; we
+call them **pre-vote** and **pre-commit**. A block is committed when
+more than 2/3 of validators pre-commit for the same block in the same
+round.
+
+There is a picture of a couple doing the polka because validators are
+doing something like a polka dance. When more than two-thirds of the
+validators pre-vote for the same block, we call that a **polka**. Every
+pre-commit must be justified by a polka in the same round.
+
+Validators may fail to commit a block for a number of reasons; the
+current proposer may be offline, or the network may be slow. Tendermint
+allows them to establish that a validator should be skipped. Validators
+wait a small amount of time to receive a complete proposal block from
+the proposer before voting to move to the next round. This reliance on a
+timeout is what makes Tendermint a weakly synchronous protocol, rather
+than an asynchronous one. However, the rest of the protocol is
+asynchronous, and validators only make progress after hearing from more
+than two-thirds of the validator set. A simplifying element of
+Tendermint is that it uses the same mechanism to commit a block as it
+does to skip to the next round.
+
+Assuming less than one-third of the validators are Byzantine, Tendermint
+guarantees that safety will never be violated - that is, validators will
+never commit conflicting blocks at the same height. To do this it
+introduces a few **locking** rules which modulate which paths can be
+followed in the flow diagram. Once a validator precommits a block, it is
+locked on that block. Then,
+
+1)  it must prevote for the block it is locked on
+2)  it can only unlock, and precommit for a new block, if there is a
+    polka for that block in a later round
+
+## Stake
+
+In many systems, not all validators will have the same "weight" in the
+consensus protocol. Thus, we are not so much interested in one-third or
+two-thirds of the validators, but in those proportions of the total
+voting power, which may not be uniformly distributed across individual
+validators.
+
+Since Tendermint can replicate arbitrary applications, it is possible to
+define a currency, and denominate the voting power in that currency.
+When voting power is denominated in a native currency, the system is
+often referred to as Proof-of-Stake. Validators can be forced, by logic
+in the application, to "bond" their currency holdings in a security
+deposit that can be destroyed if they're found to misbehave in the
+consensus protocol. This adds an economic element to the security of the
+protocol, allowing one to quantify the cost of violating the assumption
+that less than one-third of voting power is Byzantine.
+
+The [Cosmos Network](http://cosmos.network) is designed to use this
+Proof-of-Stake mechanism across an array of cryptocurrencies implemented
+as ABCI applications.
+
+The following diagram is Tendermint in a (technical) nutshell. [See here
+for high resolution
+version](https://github.com/mobfoundry/hackatom/blob/master/tminfo.pdf).
+
+![](assets/tm-transaction-flow.png)
--- a/docs/introduction.rst
+++ b/docs/introduction.rst
@@ -1,231 +0,0 @@
-Introduction
-============
-
-Welcome to the Tendermint guide! This is the best place to start if you are new
-to Tendermint.
-
-What is Tendermint?
-------------------
-
-Tendermint is software for securely and consistently replicating an application on many machines.
-By securely, we mean that Tendermint works even if up to 1/3 of machines fail in arbitrary ways.
-By consistently, we mean that every non-faulty machine sees the same transaction log and computes the same state.
-Secure and consistent replication is a fundamental problem in distributed systems; 
-it plays a critical role in the fault tolerance of a broad range of applications, 
-from currencies, to elections, to infrastructure orchestration, and beyond.
-
-The ability to tolerate machines failing in arbitrary ways, including becoming malicious, is known as Byzantine fault tolerance (BFT).
-The theory of BFT is decades old, but software implementations have only became popular recently,
-due largely to the success of "blockchain technology" like Bitcoin and Ethereum. 
-Blockchain technology is just a reformalization of BFT in a more modern setting,
-with emphasis on peer-to-peer networking and cryptographic authentication.
-The name derives from the way transactions are batched in blocks,
-where each block contains a cryptographic hash of the previous one, forming a chain.
-In practice, the blockchain data structure actually optimizes BFT design.
-
-Tendermint consists of two chief technical components: a blockchain consensus engine and a generic application interface.
-The consensus engine, called Tendermint Core, ensures that the same transactions are recorded on every machine in the same order.
-The application interface, called the Application BlockChain Interface (ABCI), enables the transactions to be processed in any programming language.
-Unlike other blockchain and consensus solutions, which come pre-packaged with built in state machines (like a fancy key-value store,
-or a quirky scripting language), developers can use Tendermint for BFT state machine replication of applications written in 
-whatever programming language and development environment is right for them.
-
-Tendermint is designed to be easy-to-use, simple-to-understand, highly performant, and useful
-for a wide variety of distributed applications.
-
-Tendermint vs. X
----------------
-
-Tendermint vs. Other Software
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Tendermint is broadly similar to two classes of software.
-The first class consists of distributed key-value stores, 
-like Zookeeper, etcd, and consul, which use non-BFT consensus.
-The second class is known as "blockchain technology",
-and consists of both cryptocurrencies like Bitcoin and Ethereum, 
-and alternative distributed ledger designs like Hyperledger's Burrow.
-
-Zookeeper, etcd, consul
-~~~~~~~~~~~~~~~~~~~~~~~
-
-Zookeeper, etcd, and consul are all implementations of a key-value store atop a classical, 
-non-BFT consensus algorithm. Zookeeper uses a version of Paxos called Zookeeper Atomic Broadcast,
-while etcd and consul use the Raft consensus algorithm, which is much younger and simpler.
-A typical cluster contains 3-5 machines, and can tolerate crash failures in up to 1/2 of the machines,
-but even a single Byzantine fault can destroy the system.
-
-Each offering provides a slightly different implementation of a featureful key-value store,
-but all are generally focused around providing basic services to distributed systems,
-such as dynamic configuration, service discovery, locking, leader-election, and so on.
-
-Tendermint is in essence similar software, but with two key differences:
- It is Byzantine Fault Tolerant, meaning it can only tolerate up to a 1/3 of failures,
-but those failures can include arbitrary behaviour - including hacking and malicious attacks.
- It does not specify a particular application, like a fancy key-value store. Instead, 
-it focuses on arbitrary state machine replication, so developers can build the application logic
-that's right for them, from key-value store to cryptocurrency to e-voting platform and beyond.
-
-The layout of this Tendermint website content is also ripped directly and without shame from
-`consul.io <https://www.consul.io/>`__ and the other `Hashicorp sites <https://www.hashicorp.com/#tools>`__.
-
-Bitcoin, Ethereum, etc.
-~~~~~~~~~~~~~~~~~~~~~~~
-
-Tendermint emerged in the tradition of cryptocurrencies like Bitcoin, Ethereum, etc.
-with the goal of providing a more efficient and secure consensus algorithm than Bitcoin's Proof of Work.
-In the early days, Tendermint had a simple currency built in, and to participate in consensus,
-users had to "bond" units of the currency into a security deposit which could be revoked if they misbehaved -
-this is what made Tendermint a Proof-of-Stake algorithm.
-
-Since then, Tendermint has evolved to be a general purpose blockchain consensus engine that can host arbitrary application states.
-That means it can be used as a plug-and-play replacement for the consensus engines of other blockchain software.
-So one can take the current Ethereum code base, whether in Rust, or Go, or Haskell, and run it as a ABCI application
-using Tendermint consensus. Indeed, `we did that with Ethereum <https://github.com/tendermint/ethermint>`__.
-And we plan to do the same for Bitcoin, ZCash, and various other deterministic applications as well.
-
-Another example of a cryptocurrency application built on Tendermint is `the Cosmos network <http://cosmos.network>`__.
-
-Other Blockchain Projects
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-`Fabric <https://github.com/hyperledger/fabric>`__ takes a similar approach to Tendermint, but is more opinionated about how the state is managed,
-and requires that all application behaviour runs in potentially many docker containers, modules it calls "chaincode". 
-It uses an implementation of `PBFT <http://pmg.csail.mit.edu/papers/osdi99.pdf>`__.
-from a team at IBM that is 
-`augmented to handle potentially non-deterministic chaincode <https://www.zurich.ibm.com/~cca/papers/sieve.pdf>`__
-It is possible to implement this docker-based behaviour as a ABCI app in Tendermint, 
-though extending Tendermint to handle non-determinism remains for future work.
-
-`Burrow <https://github.com/hyperledger/burrow>`__ is an implementation of the Ethereum Virtual Machine and Ethereum transaction mechanics,
-with additional features for a name-registry, permissions, and native contracts, and an alternative blockchain API.
-It uses Tendermint as its consensus engine, and provides a particular application state.
-
-ABCI Overview
-------------
-
-The `Application BlockChain Interface (ABCI) <https://github.com/tendermint/abci>`__ allows for Byzantine Fault Tolerant replication of applications written in any programming language.
-
-Motivation
-~~~~~~~~~~
-
-Thus far, all blockchains "stacks" (such as `Bitcoin <https://github.com/bitcoin/bitcoin>`__) have had a monolithic design. That is, each blockchain stack is a single program that handles all the concerns of a decentralized ledger; this includes P2P connectivity, the "mempool" broadcasting of transactions, consensus on the most recent block, account balances, Turing-complete contracts, user-level permissions, etc.
-
-Using a monolithic architecture is typically bad practice in computer science.
-It makes it difficult to reuse components of the code, and attempts to do so result in complex maintenance procedures for forks of the codebase.
-This is especially true when the codebase is not modular in design and suffers from "spaghetti code".
-
-Another problem with monolithic design is that it limits you to the language of the blockchain stack (or vice versa).  In the case of Ethereum which supports a Turing-complete bytecode virtual-machine, it limits you to languages that compile down to that bytecode; today, those are Serpent and Solidity.
-
-In contrast, our approach is to decouple the consensus engine and P2P layers from the details of the application state of the particular blockchain application.
-We do this by abstracting away the details of the application to an interface, which is implemented as a socket protocol.
-
-Thus we have an interface, the Application BlockChain Interface (ABCI), and its primary implementation, the Tendermint Socket Protocol (TSP, or Teaspoon).
-
-Intro to ABCI
-~~~~~~~~~~~~~
-
-`Tendermint Core <https://github.com/tendermint/tendermint>`__ (the "consensus engine") communicates with the application via a socket protocol that 
-satisfies the `ABCI <https://github.com/tendermint/abci>`__.
-
-To draw an analogy, lets talk about a well-known cryptocurrency, Bitcoin. Bitcoin is a cryptocurrency blockchain where each node maintains a fully audited Unspent Transaction Output (UTXO) database. If one wanted to create a Bitcoin-like system on top of ABCI, Tendermint Core would be responsible for 
-
- Sharing blocks and transactions between nodes
- Establishing a canonical/immutable order of transactions (the blockchain)
-
-The application will be responsible for
-
- Maintaining the UTXO database
- Validating cryptographic signatures of transactions
- Preventing transactions from spending non-existent transactions
- Allowing clients to query the UTXO database.
-
-Tendermint is able to decompose the blockchain design by offering a very simple API (ie. the ABCI) between the application process and consensus process.
-
-The ABCI consists of 3 primary message types that get delivered from the core to the application. The application replies with corresponding response messages.
-
-The messages are specified here: `ABCI Message Types <https://github.com/tendermint/abci#message-types>`__.
-
-The **DeliverTx** message is the work horse of the application. Each transaction in the blockchain is delivered with this message. The application needs to validate each transaction received with the **DeliverTx** message against the current state, application protocol, and the cryptographic credentials of the transaction. A validated transaction then needs to update the application state — by binding a value into a key values store, or by updating the UTXO database, for instance.
-
-The **CheckTx** message is similar to **DeliverTx**, but it's only for validating transactions. Tendermint Core's mempool first checks the validity of a transaction with **CheckTx**, and only relays valid transactions to its peers. For instance, an application may check an incrementing sequence number in the transaction and return an error upon **CheckTx** if the sequence number is old. Alternatively, they might use a capabilities based system that requires capabilities to be renewed with every transaction.
-
-The **Commit** message is used to compute a cryptographic commitment to the current application state, to be placed into the next block header. This has some handy properties. Inconsistencies in updating that state will now appear as blockchain forks which catches a whole class of programming errors. This also simplifies the development of secure lightweight clients, as Merkle-hash proofs can be verified by checking against the block hash, and that the block hash is signed by a quorum.
-
-There can be multiple ABCI socket connections to an application. Tendermint Core creates three ABCI connections to the application; one for the validation of transactions when broadcasting in the mempool, one for the consensus engine to run block proposals, and one more for querying the application state.
-
-It's probably evident that applications designers need to very carefully design their message handlers to create a blockchain that does anything useful but this architecture provides a place to start. The diagram below illustrates the flow of messages via ABCI.
-
-.. figure:: assets/abci.png
-
-A Note on Determinism
-~~~~~~~~~~~~~~~~~~~~~
-
-The logic for blockchain transaction processing must be deterministic. If the application logic weren't deterministic, consensus would not be reached among the Tendermint Core replica nodes.
-
-Solidity on Ethereum is a great language of choice for blockchain applications because, among other reasons, it is a completely deterministic programming language. However, it's also possible to create deterministic applications using existing popular languages like Java, C++, Python, or Go.  Game programmers and blockchain developers are already familiar with creating deterministic programs by avoiding sources of non-determinism such as:
-
- * random number generators (without deterministic seeding)
- * race conditions on threads (or avoiding threads altogether)
- * the system clock
- * uninitialized memory (in unsafe programming languages like C or C++)
- * `floating point arithmetic <http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/>`__
- * language features that are random (e.g. map iteration in Go)
-
-While programmers can avoid non-determinism by being careful, it is also possible to create a special linter or static analyzer for each language to check for determinism.  In the future we may work with partners to create such tools.
-
-Consensus Overview
------------------
-
-Tendermint is an easy-to-understand, mostly asynchronous, BFT consensus protocol.
-The protocol follows a simple state machine that looks like this:
-
-.. figure:: assets/consensus_logic.png
-
-Participants in the protocol are called **validators**;
-they take turns proposing blocks of transactions and voting on them.
-Blocks are committed in a chain, with one block at each **height**.
-A block may fail to be committed, in which case the protocol moves to the next **round**,
-and a new validator gets to propose a block for that height.
-Two stages of voting are required to successfully commit a block;
-we call them **pre-vote** and **pre-commit**.
-A block is committed when more than 2/3 of validators pre-commit for the same block in the same round.
-
-There is a picture of a couple doing the polka because validators are doing something like a polka dance.
-When more than two-thirds of the validators pre-vote for the same block, we call that a **polka**.
-Every pre-commit must be justified by a polka in the same round.
-
-Validators may fail to commit a block for a number of reasons; 
-the current proposer may be offline, or the network may be slow.
-Tendermint allows them to establish that a validator should be skipped.
-Validators wait a small amount of time to receive a complete proposal block from the proposer before voting to move to the next round.
-This reliance on a timeout is what makes Tendermint a weakly synchronous protocol, rather than an asynchronous one.
-However, the rest of the protocol is asynchronous, and validators only make progress after hearing from more than two-thirds of the validator set.
-A simplifying element of Tendermint is that it uses the same mechanism to commit a block as it does to skip to the next round.
-
-Assuming less than one-third of the validators are Byzantine, Tendermint guarantees that safety will never be violated - that is, validators will never commit conflicting blocks at the same height.
-To do this it introduces a few **locking** rules which modulate which paths can be followed in the flow diagram.
-Once a validator precommits a block, it is locked on that block.
-Then, 
-
-1) it must prevote for the block it is locked on
-2) it can only unlock, and precommit for a new block, if there is a polka for that block in a later round
-
-Stake
-----
-
-In many systems, not all validators will have the same "weight" in the consensus protocol. 
-Thus, we are not so much interested in one-third or two-thirds of the validators, but in those proportions of the total voting power, 
-which may not be uniformly distributed across individual validators.
-
-Since Tendermint can replicate arbitrary applications, it is possible to define a currency, and denominate the voting power in that currency.
-When voting power is denominated in a native currency, the system is often referred to as Proof-of-Stake.
-Validators can be forced, by logic in the application, 
-to "bond" their currency holdings in a security deposit that can be destroyed if they're found to misbehave in the consensus protocol.
-This adds an economic element to the security of the protocol, allowing one to quantify the cost of violating the assumption that less than one-third of voting power is Byzantine. 
-
-The `Cosmos Network <http://cosmos.network>`__ is designed to use this Proof-of-Stake mechanism across an array of cryptocurrencies implemented as ABCI applications.
-
-The following diagram is Tendermint in a (technical) nutshell. `See here for high resolution version <https://github.com/mobfoundry/hackatom/blob/master/tminfo.pdf>`__.
-
-.. figure:: assets/tm-transaction-flow.png
--- a/docs/running-in-production.md
+++ b/docs/running-in-production.md
@@ -0,0 +1,206 @@
+# Running in production
+
+## Logging
+
+Default logging level (`main:info,state:info,*:`) should suffice for
+normal operation mode. Read [this
+post](https://blog.cosmos.network/one-of-the-exciting-new-features-in-0-10-0-release-is-smart-log-level-flag-e2506b4ab756)
+for details on how to configure `log_level` config variable. Some of the
+modules can be found [here](./how-to-read-logs.md#list-of-modules). If
+you're trying to debug Tendermint or asked to provide logs with debug
+logging level, you can do so by running tendermint with
+`--log_level="*:debug"`.
+
+## DOS Exposure and Mitigation
+
+Validators are supposed to setup [Sentry Node
+Architecture](https://blog.cosmos.network/tendermint-explained-bringing-bft-based-pos-to-the-public-blockchain-domain-f22e274a0fdb)
+to prevent Denial-of-service attacks. You can read more about it
+[here](https://github.com/tendermint/aib-data/blob/develop/medium/TendermintBFT.md).
+
+### P2P
+
+The core of the Tendermint peer-to-peer system is `MConnection`. Each
+connection has `MaxPacketMsgPayloadSize`, which is the maximum packet
+size and bounded send & receive queues. One can impose restrictions on
+send & receive rate per connection (`SendRate`, `RecvRate`).
+
+### RPC
+
+Endpoints returning multiple entries are limited by default to return 30
+elements (100 max).
+
+Rate-limiting and authentication are another key aspects to help protect
+against DOS attacks. While in the future we may implement these
+features, for now, validators are supposed to use external tools like
+[NGINX](https://www.nginx.com/blog/rate-limiting-nginx/) or
+[traefik](https://docs.traefik.io/configuration/commons/#rate-limiting)
+to achieve the same things.
+
+## Debugging Tendermint
+
+If you ever have to debug Tendermint, the first thing you should
+probably do is to check out the logs. See ["How to read
+logs"](./how-to-read-logs.md), where we explain what certain log
+statements mean.
+
+If, after skimming through the logs, things are not clear still, the
+second TODO is to query the /status RPC endpoint. It provides the
+necessary info: whenever the node is syncing or not, what height it is
+on, etc.
+
+    $ curl http(s)://{ip}:{rpcPort}/status
+
+`dump_consensus_state` will give you a detailed overview of the
+consensus state (proposer, lastest validators, peers states). From it,
+you should be able to figure out why, for example, the network had
+halted.
+
+    $ curl http(s)://{ip}:{rpcPort}/dump_consensus_state
+
+There is a reduced version of this endpoint - `consensus_state`, which
+returns just the votes seen at the current height.
+
+-   [Github Issues](https://github.com/tendermint/tendermint/issues)
+-   [StackOverflow
+    questions](https://stackoverflow.com/questions/tagged/tendermint)
+
+## Monitoring Tendermint
+
+Each Tendermint instance has a standard `/health` RPC endpoint, which
+responds with 200 (OK) if everything is fine and 500 (or no response) -
+if something is wrong.
+
+Other useful endpoints include mentioned earlier `/status`, `/net_info` and
+`/validators`.
+
+We have a small tool, called `tm-monitor`, which outputs information from
+the endpoints above plus some statistics. The tool can be found
+[here](https://github.com/tendermint/tools/tree/master/tm-monitor).
+
+## What happens when my app dies?
+
+You are supposed to run Tendermint under a [process
+supervisor](https://en.wikipedia.org/wiki/Process_supervision) (like
+systemd or runit). It will ensure Tendermint is always running (despite
+possible errors).
+
+Getting back to the original question, if your application dies,
+Tendermint will panic. After a process supervisor restarts your
+application, Tendermint should be able to reconnect successfully. The
+order of restart does not matter for it.
+
+## Signal handling
+
+We catch SIGINT and SIGTERM and try to clean up nicely. For other
+signals we use the default behaviour in Go: [Default behavior of signals
+in Go
+programs](https://golang.org/pkg/os/signal/#hdr-Default_behavior_of_signals_in_Go_programs).
+
+## Hardware
+
+### Processor and Memory
+
+While actual specs vary depending on the load and validators count,
+minimal requirements are:
+
+-   1GB RAM
+-   25GB of disk space
+-   1.4 GHz CPU
+
+SSD disks are preferable for applications with high transaction
+throughput.
+
+Recommended:
+
+-   2GB RAM
+-   100GB SSD
+-   x64 2.0 GHz 2v CPU
+
+While for now, Tendermint stores all the history and it may require
+significant disk space over time, we are planning to implement state
+syncing (See
+[this issue](https://github.com/tendermint/tendermint/issues/828)). So,
+storing all the past blocks will not be necessary.
+
+### Operating Systems
+
+Tendermint can be compiled for a wide range of operating systems thanks
+to Go language (the list of \$OS/\$ARCH pairs can be found
+[here](https://golang.org/doc/install/source#environment)).
+
+While we do not favor any operation system, more secure and stable Linux
+server distributions (like Centos) should be preferred over desktop
+operation systems (like Mac OS).
+
+### Miscellaneous
+
+NOTE: if you are going to use Tendermint in a public domain, make sure
+you read [hardware recommendations (see "4.
+Hardware")](https://cosmos.network/validators) for a validator in the
+Cosmos network.
+
+## Configuration parameters
+
+-   `p2p.flush_throttle_timeout` `p2p.max_packet_msg_payload_size`
+    `p2p.send_rate` `p2p.recv_rate`
+
+If you are going to use Tendermint in a private domain and you have a
+private high-speed network among your peers, it makes sense to lower
+flush throttle timeout and increase other params.
+
+    [p2p]
+
+    send_rate=20000000 # 2MB/s
+    recv_rate=20000000 # 2MB/s
+    flush_throttle_timeout=10
+    max_packet_msg_payload_size=10240 # 10KB
+
+-   `mempool.recheck`
+
+After every block, Tendermint rechecks every transaction left in the
+mempool to see if transactions committed in that block affected the
+application state, so some of the transactions left may become invalid.
+If that does not apply to your application, you can disable it by
+setting `mempool.recheck=false`.
+
+-   `mempool.broadcast`
+
+Setting this to false will stop the mempool from relaying transactions
+to other peers until they are included in a block. It means only the
+peer you send the tx to will see it until it is included in a block.
+
+-   `consensus.skip_timeout_commit`
+
+We want `skip_timeout_commit=false` when there is economics on the line
+because proposers should wait to hear for more votes. But if you don't
+care about that and want the fastest consensus, you can skip it. It will
+be kept false by default for public deployments (e.g. [Cosmos
+Hub](https://cosmos.network/intro/hub)) while for enterprise
+applications, setting it to true is not a problem.
+
+-   `consensus.peer_gossip_sleep_duration`
+
+You can try to reduce the time your node sleeps before checking if
+theres something to send its peers.
+
+-   `consensus.timeout_commit`
+
+You can also try lowering `timeout_commit` (time we sleep before
+proposing the next block).
+
+-   `consensus.max_block_size_txs`
+
+By default, the maximum number of transactions per a block is 10_000.
+Feel free to change it to suit your needs.
+
+-   `p2p.addr_book_strict`
+
+By default, Tendermint checks whenever a peer's address is routable before
+saving it to the address book. The address is considered as routable if the IP
+is [valid and within allowed
+ranges](https://github.com/tendermint/tendermint/blob/27bd1deabe4ba6a2d9b463b8f3e3f1e31b993e61/p2p/netaddress.go#L209).
+
+This may not be the case for private networks, where your IP range is usually
+strictly limited and private. If that case, you need to set `addr_book_strict`
+to `false` (turn off).
--- a/docs/spec/README.md
+++ b/docs/spec/README.md
@@ -14,26 +14,31 @@ please submit them to our [bug bounty](https://tendermint.com/security)!

 ### Data Structures

- [Encoding and Digests](./blockchain/encoding.md)
- [Blockchain](./blockchain/blockchain.md)
- [State](./blockchain/state.md)
+- [Encoding and Digests](https://github.com/tendermint/tendermint/blob/master/docs/spec/blockchain/encoding.md)
+- [Blockchain](https://github.com/tendermint/tendermint/blob/master/docs/spec/blockchain/blockchain.md)
+- [State](https://github.com/tendermint/tendermint/blob/master/docs/spec/blockchain/state.md)

 ### Consensus Protocol

- TODO
+- [Consensus Algorithm](/docs/spec/consensus/consensus.md)
+- [Time](/docs/spec/consensus/bft-time.md)
+- [Light-Client](/docs/spec/consensus/light-client.md)

 ### P2P and Network Protocols

- [The Base P2P Layer](p2p): multiplex the protocols ("reactors") on authenticated and encrypted TCP connections
- [Peer Exchange (PEX)](reactors/pex): gossip known peer addresses so peers can find each other
- [Block Sync](reactors/block_sync): gossip blocks so peers can catch up quickly
- [Consensus](reactors/consensus): gossip votes and block parts so new blocks can be committed
- [Mempool](reactors/mempool): gossip transactions so they get included in blocks
+- [The Base P2P Layer](https://github.com/tendermint/tendermint/tree/master/docs/spec/p2p): multiplex the protocols ("reactors") on authenticated and encrypted TCP connections
+- [Peer Exchange (PEX)](https://github.com/tendermint/tendermint/tree/master/docs/spec/reactors/pex): gossip known peer addresses so peers can find each other
+- [Block Sync](https://github.com/tendermint/tendermint/tree/master/docs/spec/reactors/block_sync): gossip blocks so peers can catch up quickly
+- [Consensus](https://github.com/tendermint/tendermint/tree/master/docs/spec/reactors/consensus): gossip votes and block parts so new blocks can be committed
+- [Mempool](https://github.com/tendermint/tendermint/tree/master/docs/spec/reactors/mempool): gossip transactions so they get included in blocks
 - Evidence: TODO

-### More
- Light Client: TODO
- Persistence: TODO
+### Software
+
+- [ABCI](/docs/spec/software/abci.md): Details about interactions between the
+  application and consensus engine over ABCI
+- [Write-Ahead Log](/docs/spec/software/wal.md): Details about how the consensus
+  engine preserves data and recovers from crash failures

 ## Overview

@@ -42,10 +47,9 @@ hash-linked batches of transactions. Such transaction batches are called "blocks
 Hence, Tendermint defines a "blockchain".

 Each block in Tendermint has a unique index - its Height.
-A block at `Height == H` can only be committed *after* the
-block at `Height == H-1`.
+Height's in the blockchain are monotonic.
 Each block is committed by a known set of weighted Validators.
-Membership and weighting within this set may change over time.
+Membership and weighting within this validator set may change over time.
 Tendermint guarantees the safety and liveness of the blockchain
 so long as less than 1/3 of the total weight of the Validator set
 is malicious or faulty.
--- a/docs/spec/blockchain/blockchain.md
+++ b/docs/spec/blockchain/blockchain.md
@@ -162,7 +162,7 @@ We refer to certain globally available objects:
 and `state` keeps track of the validator set, the consensus parameters
 and other results from the application.
 Elements of an object are accessed as expected,
-ie. `block.Header`. See [here](state.md) for the definition of `state`.
+ie. `block.Header`. See [here](https://github.com/tendermint/tendermint/blob/master/docs/spec/blockchain/state.md) for the definition of `state`.

 ### Header

@@ -372,16 +372,23 @@ against the given signature and message bytes.

 ## Evidence

-TODO
+There is currently only one kind of evidence:

 ```
-TODO
+// amino: "tendermint/DuplicateVoteEvidence"
+type DuplicateVoteEvidence struct {
+	PubKey crypto.PubKey
+	VoteA  *Vote
+	VoteB  *Vote
+}
 ```

-Every piece of evidence contains two conflicting votes from a single validator that
-was active at the height indicated in the votes.
-The votes must not be too old.
+DuplicateVoteEvidence `ev` is valid if

+- `ev.VoteA` and `ev.VoteB` can be verified with `ev.PubKey`
+- `ev.VoteA` and `ev.VoteB` have the same `Height, Round, Address, Index, Type`
+- `ev.VoteA.BlockID != ev.VoteB.BlockID`
+- `(block.Height - ev.VoteA.Height) < MAX_EVIDENCE_AGE`

 # Execution

--- a/docs/spec/blockchain/encoding.md
+++ b/docs/spec/blockchain/encoding.md
@@ -2,7 +2,7 @@

 ## Amino

-Tendermint uses the Protobuf3 derrivative [Amino]() for all data structures.
+Tendermint uses the Protobuf3 derivative [Amino](https://github.com/tendermint/go-amino) for all data structures.
 Think of Amino as an object-oriented Protobuf3 with native JSON support.
 The goal of the Amino encoding protocol is to bring parity between application
 logic objects and persistence objects.
@@ -39,7 +39,7 @@ place of the public key. Here we list the concrete types, their names,
 and prefix bytes for public keys and signatures, as well as the address schemes
 for each PubKey. Note for brevity we don't
 include details of the private keys beyond their type and name, as they can be
-derrived the same way as the others using Amino.
+derived the same way as the others using Amino.

 All registered objects are encoded by Amino using a 4-byte PrefixBytes that
 uniquely identifies the object and includes information about its underlying
@@ -49,107 +49,60 @@ spec](https://github.com/tendermint/go-amino#computing-the-prefix-and-disambigua
 In what follows, we provide the type names and prefix bytes directly.
 Notice that when encoding byte-arrays, the length of the byte-array is appended
 to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes>
-<Length> <ByteArray>`
+<Length> <ByteArray>`. In other words, to encode any type listed below you do not need to be
+familiar with amino encoding.
+You can simply use below table and concatenate Prefix || Length (of raw bytes) || raw bytes
+( while || stands for byte concatenation here).

-(NOTE: the remainder of this section on Public Key Cryptography can be generated
-from [this script](./scripts/crypto.go))
+| Type | Name | Prefix | Length |
+| ---- | ---- | ------ | ----- |
+| PubKeyEd25519 | tendermint/PubKeyEd25519 | 0x1624DE62 | 0x20 |
+| PubKeyLedgerEd25519 | tendermint/PubKeyLedgerEd25519 | 0x5C3453B2 | 0x20 |
+| PubKeySecp256k1 | tendermint/PubKeySecp256k1 | 0xEB5AE982 | 0x21 |
+| PrivKeyEd25519 | tendermint/PrivKeyEd25519 | 0xA3288912 | 0x40 |
+| PrivKeySecp256k1 | tendermint/PrivKeySecp256k1 | 0xE1B0F79A | 0x20 |
+| PrivKeyLedgerSecp256k1 | tendermint/PrivKeyLedgerSecp256k1 | 0x10CAB393 | variable |
+| PrivKeyLedgerEd25519 | tendermint/PrivKeyLedgerEd25519 | 0x0CFEEF9B | variable |
+| SignatureEd25519 | tendermint/SignatureKeyEd25519 | 0x3DA1DB2A | 0x40 |
+| SignatureSecp256k1 | tendermint/SignatureKeySecp256k1 | 0x16E1FEEA | variable |

-### PubKeyEd25519
+### Examples

-```
-// Name: tendermint/PubKeyEd25519
-// PrefixBytes: 0x1624DE62
-// Length: 0x20
-// Notes: raw 32-byte Ed25519 pubkey
-type PubKeyEd25519 [32]byte
-
-func (pubkey PubKeyEd25519) Address() []byte {
-      // NOTE: hash of the Amino encoded bytes!
-        return RIPEMD160(AminoEncode(pubkey))
-}
-```
-
-For example, the 32-byte Ed25519 pubkey
-`CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E` would be
-encoded as
-`1624DE6220CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E`.
-
-The address would then be
-`RIPEMD160(0x1624DE6220CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E)`
-or `430FF75BAF1EC4B0D51BB3EEC2955479D0071605`
-
-### SignatureEd25519
-
-```
-// Name: tendermint/SignatureKeyEd25519
-// PrefixBytes: 0x3DA1DB2A
-// Length: 0x40
-// Notes: raw 64-byte Ed25519 signature
-type SignatureEd25519 [64]byte
-```
-
-For example, the 64-byte Ed25519 signature
-`1B6034A8ED149D3C94FDA13EC03B26CC0FB264D9B0E47D3FA3DEF9FCDE658E49C80B35F9BE74949356401B15B18FB817D6E54495AD1C4A8401B248466CB0DB0B`
+1. For example, the 33-byte (or 0x21-byte in hex) Secp256k1 pubkey
+`020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
 would be encoded as
-`3DA1DB2A401B6034A8ED149D3C94FDA13EC03B26CC0FB264D9B0E47D3FA3DEF9FCDE658E49C80B35F9BE74949356401B15B18FB817D6E54495AD1C4A8401B248466CB0DB0B`
-
-### PrivKeyEd25519
-
-```
-// Name: tendermint/PrivKeyEd25519
-// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key
-type PrivKeyEd25519 [64]byte
-```
-
-### PubKeySecp256k1
-
-```
-// Name: tendermint/PubKeySecp256k1
-// PrefixBytes: 0xEB5AE982
-// Length: 0x21
-// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03
-type PubKeySecp256k1 [33]byte
-
-func (pubkey PubKeySecp256k1) Address() []byte {
-      // NOTE: hash of the raw pubkey bytes (not Amino encoded!).
-        // Compatible with Bitcoin addresses.
-          return RIPEMD160(SHA256(pubkey[:]))
-}
-```
-
-For example, the 33-byte Secp256k1 pubkey
-`020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9` would be
-encoded as
 `EB5AE98221020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`

-The address would then be
-`RIPEMD160(SHA256(0x020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9))`
-or `0AE5BEE929ABE51BAD345DB925EEA652680783FC`
-
-### SignatureSecp256k1
-
-```
-// Name: tendermint/SignatureKeySecp256k1
-// PrefixBytes: 0x16E1FEEA
-// Length: Variable
-// Encoding prefix: Variable
-// Notes: raw bytes of the Secp256k1 signature
-type SignatureSecp256k1 []byte
-```
-
-For example, the Secp256k1 signature
+2. For example, the variable size Secp256k1 signature (in this particular example 70 or 0x46 bytes)
 `304402201CD4B8C764D2FD8AF23ECFE6666CA8A53886D47754D951295D2D311E1FEA33BF02201E0F906BB1CF2C30EAACFFB032A7129358AFF96B9F79B06ACFFB18AC90C2ADD7`
 would be encoded as
 `16E1FEEA46304402201CD4B8C764D2FD8AF23ECFE6666CA8A53886D47754D951295D2D311E1FEA33BF02201E0F906BB1CF2C30EAACFFB032A7129358AFF96B9F79B06ACFFB18AC90C2ADD7`

-### PrivKeySecp256k1
+### Addresses
+
+Addresses for each public key types are computed as follows:
+
+#### Ed25519
+
+RIPEMD160 hash of the Amino encoded public key:

 ```
-// Name: tendermint/PrivKeySecp256k1
-// Notes: raw 32-byte priv key
-type PrivKeySecp256k1 [32]byte
+address = RIPEMD160(AMINO(pubkey))
 ```

+NOTE: this will soon change to the truncated 20-bytes of the SHA256 of the raw
+public key
+
+#### Secp256k1
+
+RIPEMD160 hash of the SHA256 hash of the OpenSSL compressed public key:
+
+```
+address = RIPEMD160(SHA256(pubkey))
+```
+
+This is the same as Bitcoin.
+
 ## Other Common Types

 ### BitArray
@@ -290,6 +243,7 @@ Amino also supports JSON encoding - registered types are simply encoded as:
  "type": "<DisfixBytes>",
  "value": <JSON>
 }
+```

 For instance, an ED25519 PubKey would look like:

--- a/docs/spec/blockchain/pre-amino.md
+++ b/docs/spec/blockchain/pre-amino.md
@@ -1,246 +0,0 @@
-# Tendermint Encoding (Pre-Amino)
-
-## PubKeys and Addresses
-
-PubKeys are prefixed with a type-byte, followed by the raw bytes of the public
-key.
-
-Two keys are supported with the following type bytes:
-
-```
-TypeByteEd25519 = 0x1
-TypeByteSecp256k1 = 0x2
-```
-
-```
-// TypeByte: 0x1
-type PubKeyEd25519 [32]byte
-
-func (pub PubKeyEd25519) Encode() []byte {
-    return 0x1 | pub
-}
-
-func (pub PubKeyEd25519) Address() []byte {
-    // NOTE: the length (0x0120) is also included
-    return RIPEMD160(0x1 | 0x0120 | pub)
-}
-
-// TypeByte: 0x2
-// NOTE: OpenSSL compressed pubkey (x-cord with 0x2 or 0x3)
-type PubKeySecp256k1 [33]byte
-
-func (pub PubKeySecp256k1) Encode() []byte {
-    return 0x2 | pub
-}
-
-func (pub PubKeySecp256k1) Address() []byte {
-    return RIPEMD160(SHA256(pub))
-}
-```
-
-See https://github.com/tendermint/go-crypto/blob/v0.5.0/pub_key.go for more.
-
-## Binary Serialization (go-wire)
-
-Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs
-are laid out in memory.
-Variable length items are length-prefixed.
-While the encoding was inspired by Go, it is easily implemented in other languages as well, given its intuitive design.
-
-XXX: This is changing to use real varints and 4-byte-prefixes.
-See https://github.com/tendermint/go-wire/tree/sdk2.
-
-### Fixed Length Integers
-
-Fixed length integers are encoded in Big-Endian using the specified number of bytes.
-So `uint8` and `int8` use one byte, `uint16` and `int16` use two bytes,
-`uint32` and `int32` use 3 bytes, and `uint64` and `int64` use 4 bytes.
-
-Negative integers are encoded via twos-complement.
-
-Examples:
-
-```go
-encode(uint8(6))    == [0x06]
-encode(uint32(6))   == [0x00, 0x00, 0x00, 0x06]
-
-encode(int8(-6))    == [0xFA]
-encode(int32(-6))   == [0xFF, 0xFF, 0xFF, 0xFA]
-```
-
-### Variable Length Integers
-
-Variable length integers are encoded as length-prefixed Big-Endian integers.
-The length-prefix consists of a single byte and corresponds to the length of the encoded integer.
-
-Negative integers are encoded by flipping the leading bit of the length-prefix to a `1`.
-
-Zero is encoded as `0x00`. It is not length-prefixed.
-
-Examples:
-
-```go
-encode(uint(6))     == [0x01, 0x06]
-encode(uint(70000)) == [0x03, 0x01, 0x11, 0x70]
-
-encode(int(-6))     == [0xF1, 0x06]
-encode(int(-70000)) == [0xF3, 0x01, 0x11, 0x70]
-
-encode(int(0))      == [0x00]
-```
-
-### Strings
-
-An encoded string is length-prefixed followed by the underlying bytes of the string.
-The length-prefix is itself encoded as an `int`.
-
-The empty string is encoded as `0x00`. It is not length-prefixed.
-
-Examples:
-
-```go
-encode("")      == [0x00]
-encode("a")     == [0x01, 0x01, 0x61]
-encode("hello") == [0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F]
-encode("¥")     == [0x01, 0x02, 0xC2, 0xA5]
-```
-
-### Arrays (fixed length)
-
-An encoded fix-lengthed array is the concatenation of the encoding of its elements.
-There is no length-prefix.
-
-Examples:
-
-```go
-encode([4]int8{1, 2, 3, 4})     == [0x01, 0x02, 0x03, 0x04]
-encode([4]int16{1, 2, 3, 4})    == [0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
-encode([4]int{1, 2, 3, 4})      == [0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x04]
-encode([2]string{"abc", "efg"}) == [0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
-```
-
-### Slices (variable length)
-
-An encoded variable-length array is length-prefixed followed by the concatenation of the encoding of
-its elements.
-The length-prefix is itself encoded as an `int`.
-
-An empty slice is encoded as `0x00`. It is not length-prefixed.
-
-Examples:
-
-```go
-encode([]int8{})                == [0x00]
-encode([]int8{1, 2, 3, 4})      == [0x01, 0x04, 0x01, 0x02, 0x03, 0x04]
-encode([]int16{1, 2, 3, 4})     == [0x01, 0x04, 0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
-encode([]int{1, 2, 3, 4})       == [0x01, 0x04, 0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x4]
-encode([]string{"abc", "efg"})  == [0x01, 0x02, 0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
-```
-
-### BitArray
-
-BitArray is encoded as an `int` of the number of bits, and with an array of `uint64` to encode
-value of each array element.
-
-```go
-type BitArray struct {
-    Bits  int
-    Elems []uint64
-}
-```
-
-### Time
-
-Time is encoded as an `int64` of the number of nanoseconds since January 1, 1970,
-rounded to the nearest millisecond.
-
-Times before then are invalid.
-
-Examples:
-
-```go
-encode(time.Time("Jan 1 00:00:00 UTC 1970"))            == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
-encode(time.Time("Jan 1 00:00:01 UTC 1970"))            == [0x00, 0x00, 0x00, 0x00, 0x3B, 0x9A, 0xCA, 0x00] // 1,000,000,000 ns
-encode(time.Time("Mon Jan 2 15:04:05 -0700 MST 2006"))  == [0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
-```
-
-### Structs
-
-An encoded struct is the concatenation of the encoding of its elements.
-There is no length-prefix.
-
-Examples:
-
-```go
-type MyStruct struct{
-    A int
-    B string
-    C time.Time
-}
-encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) ==
-    [0x01, 0x04, 0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F, 0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
-```
-
-## Merkle Trees
-
-Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
-
-RIPEMD160 is always used as the hashing function.
-
-The function `SimpleMerkleRoot` is a simple recursive function defined as follows:
-
-```go
-func SimpleMerkleRoot(hashes [][]byte) []byte{
-    switch len(hashes) {
-    case 0:
-        return nil
-    case 1:
-        return hashes[0]
-    default:
-        left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2])
-        right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:])
-        return RIPEMD160(append(left, right))
-    }
-}
-```
-
-Note: we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
-For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to
-field name and then hashing them.
-For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements.
-
-## JSON (TMJSON)
-
-Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN.
-TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64.
-
-When signing, the elements of a message are sorted by key and the sorted message is embedded in an
-outer JSON that includes a `chain_id` field.
-We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look
-like:
-
-```json
-{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2}
-```
-
-Note how the fields within each level are sorted.
-
-## Other
-
-### MakeParts
-
-Encode an object using TMBIN and slice it into parts.
-
-```go
-MakeParts(object, partSize)
-```
-
-### Part
-
-```go
-type Part struct {
-    Index int
-    Bytes byte[]
-    Proof byte[]
-}
-```
--- a/docs/spec/blockchain/state.md
+++ b/docs/spec/blockchain/state.md
@@ -77,5 +77,4 @@ func TotalVotingPower(vals []Validators) int64{

 ### ConsensusParams

-TODO:
-
+TODO
--- a/docs/spec/consensus/abci.md
+++ b/docs/spec/consensus/abci.md
@@ -1,168 +1 @@
-# Application Blockchain Interface (ABCI)
-
-ABCI is the interface between Tendermint (a state-machine replication engine)
-and an application (the actual state machine).
-
-The ABCI message types are defined in a [protobuf
-file](https://github.com/tendermint/abci/blob/master/types/types.proto).
-
-For full details on the ABCI message types and protocol, see the [ABCI
-specificaiton](https://github.com/tendermint/abci/blob/master/specification.rst).
-Be sure to read the specification if you're trying to build an ABCI app!
-
-For additional details on server implementation, see the [ABCI
-readme](https://github.com/tendermint/abci#implementation).
-
-Here we provide some more details around the use of ABCI by Tendermint and
-clarify common "gotchas".
-
-## ABCI connections
-
-Tendermint opens 3 ABCI connections to the app: one for Consensus, one for
-Mempool, one for Queries.
-
-## Async vs Sync
-
-The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages.
-This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward
-transactions to the app before it's finished processing previous ones.
-
-Thus, DeliverTx and CheckTx messages are sent asycnhronously, while all other
-messages are sent synchronously.
-
-## CheckTx and Commit
-
-It is typical to hold three distinct states in an ABCI app: CheckTxState, DeliverTxState,
-QueryState. The QueryState contains the latest committed state for a block.
-The CheckTxState and DeliverTxState may be updated concurrently with one another.
-Before Commit is called, Tendermint locks and flushes the mempool so that no new changes will happen
-to CheckTxState. When Commit completes, it unlocks the mempool.
-
-Thus, during Commit, it is safe to reset the QueryState and the CheckTxState to the latest DeliverTxState
-(ie. the new state from executing all the txs in the block).
-
-Note, however, that it is not possible to send transactions to Tendermint during Commit - if your app
-tries to send a `/broadcast_tx` to Tendermint during Commit, it will deadlock.
-
-
-## EndBlock Validator Updates
-
-Updates to the Tendermint validator set can be made by returning `Validator`
-objects in the `ResponseBeginBlock`:
-
-```
-message Validator {
-    bytes pub_key = 1;
-    int64 power = 2;
-}
-```
-
-The `pub_key` is the Amino encoded public key for the validator. For details on
-Amino encoded public keys, see the [section of the encoding spec](./encoding.md#public-key-cryptography).
-
-For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 32-byte Ed25519 pubkey
-`76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
-Amino encoded as
-`1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`
-
-(Note: in old versions of Tendermint (pre-v0.19.0), the pubkey is just prefixed with a
-single type byte, so for ED25519 we'd have `pub_key = 0x1 | pub`)
-
-The `power` is the new voting power for the validator, with the
-following rules:
-
- power must be non-negative
- if power is 0, the validator must already exist, and will be removed from the
-  validator set
- if power is non-0:
-    - if the validator does not already exist, it will be added to the validator
-      set with the given power
-    - if the validator does already exist, its power will be adjusted to the given power
-
-## Query
-
-Query is a generic message type with lots of flexibility to enable diverse sets
-of queries from applications. Tendermint has no requirements from the Query
-message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too.
-That said, Tendermint makes a number of queries to support some optional
-features. These are:
-
-### Peer Filtering
-
-When Tendermint connects to a peer, it sends two queries to the ABCI application
-using the following paths, with no additional data:
-
- - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and
-   the port of the connection
- - `p2p/filter/pubkey/<ID>`, where `<ID>` is the peer node ID (ie. the
-   pubkey.Address() for the peer's PubKey)
-
-If either of these queries return a non-zero ABCI code, Tendermint will refuse
-to connect to the peer.
-
-## Info and the Handshake/Replay
-
-On startup, Tendermint calls Info on the Query connection to get the latest
-committed state of the app. The app MUST return information consistent with the
-last block it succesfully completed Commit for.
-
-If the app succesfully committed block H but not H+1, then `last_block_height =
-H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app
-failed during the Commit of block H, then `last_block_height = H-1` and
-`last_block_app_hash = <hash returned by Commit for block H-1, which is the hash
-in the header of block H>`.
-
-We now distinguish three heights, and describe how Tendermint syncs itself with
-the app.
-
-```
-storeBlockHeight = height of the last block Tendermint saw a commit for
-stateBlockHeight = height of the last block for which Tendermint completed all
-    block processing and saved all ABCI results to disk
-appBlockHeight = height of the last block for which ABCI app succesfully
-    completely Commit
-
-```
-
-Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight`
-Note also we never call Commit on an ABCI app twice for the same height.
-
-The procedure is as follows.
-
-First, some simeple start conditions:
-
-If `appBlockHeight == 0`, then call InitChain.
-
-If `storeBlockHeight == 0`, we're done.
-
-Now, some sanity checks:
-
-If `storeBlockHeight < appBlockHeight`, error
-If `storeBlockHeight < stateBlockHeight`, panic
-If `storeBlockHeight > stateBlockHeight+1`, panic
-
-Now, the meat:
-
-If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`,
-	replay all blocks in full from `appBlockHeight` to `storeBlockHeight`.
-	This happens if we completed processing the block, but the app forgot its height.
-
-If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done
-	This happens if we crashed at an opportune spot.
-
-If `storeBlockHeight == stateBlockHeight+1`
-	This happens if we started processing the block but didn't finish.
-
-	If `appBlockHeight < stateBlockHeight`
-		replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`,
-		and replay the block at `storeBlockHeight` using the WAL.
-	This happens if the app forgot the last block it committed.
-
-	If `appBlockHeight == stateBlockHeight`,
-		replay the last block (storeBlockHeight) in full.
-	This happens if we crashed before the app finished Commit
-
-	If appBlockHeight == storeBlockHeight {
-		update the state using the saved ABCI responses but dont run the block against the real app.
-	This happens if we crashed after the app finished Commit but before Tendermint saved the state.
-
+[Moved](/docs/spec/software/abci.md)
--- a/docs/spec/consensus/consensus.md
+++ b/docs/spec/consensus/consensus.md
@@ -0,0 +1,9 @@
+We are working to finalize an updated Tendermint specification with formal
+proofs of safety and liveness.
+
+In the meantime, see the [description in the
+docs](http://tendermint.readthedocs.io/en/master/specification/byzantine-consensus-algorithm.html).
+
+There are also relevant but somewhat outdated descriptions in Jae Kwon's [original
+whitepaper](https://tendermint.com/static/docs/tendermint.pdf) and Ethan Buchman's [master's
+thesis](https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769).
--- a/docs/spec/blockchain/light-client.md
+++ b/docs/spec/blockchain/light-client.md
--- a/docs/spec/consensus/wal.md
+++ b/docs/spec/consensus/wal.md
@@ -0,0 +1 @@
+[Moved](/docs/spec/software/wal.md)
--- a/docs/spec/p2p/config.md
+++ b/docs/spec/p2p/config.md
@@ -12,14 +12,14 @@ and upon incoming connection shares some peers and disconnects.

 ## Seeds

-`--p2p.seeds “1.2.3.4:466656,2.3.4.5:4444”`
+`--p2p.seeds “1.2.3.4:26656,2.3.4.5:4444”`

 Dials these seeds when we need more peers. They should return a list of peers and then disconnect.
 If we already have enough peers in the address book, we may never need to dial them.

 ## Persistent Peers

-`--p2p.persistent_peers “1.2.3.4:46656,2.3.4.5:466656”`
+`--p2p.persistent_peers “1.2.3.4:26656,2.3.4.5:26656”`

 Dial these peers and auto-redial them if the connection fails.
 These are intended to be trusted persistent peers that can help
@@ -32,7 +32,7 @@ and that the node may not be able to keep the connection persistent.

 ## Private Persistent Peers

-`--p2p.private_persistent_peers “1.2.3.4:46656,2.3.4.5:466656”`
+`--p2p.private_persistent_peers “1.2.3.4:26656,2.3.4.5:26656”`

 These are persistent peers that we do not add to the address book or
 gossip to other peers. They stay private to us.
--- a/docs/spec/p2p/node.md
+++ b/docs/spec/p2p/node.md
@@ -12,7 +12,7 @@ Seeds should operate full nodes with the PEX reactor in a "crawler" mode
 that continuously explores to validate the availability of peers.

 Seeds should only respond with some top percentile of the best peers it knows about.
-See [the peer-exchange docs](/docs/specification/new-spec/reactors/pex/pex.md)for details on peer quality.
+See [the peer-exchange docs](https://github.com/tendermint/tendermint/blob/master/docs/spec/reactors/pex/pex.md)for details on peer quality.

 ## New Full Node

--- a/docs/spec/p2p/peer.md
+++ b/docs/spec/p2p/peer.md
@@ -2,7 +2,7 @@

 This document explains how Tendermint Peers are identified and how they connect to one another.

-For details on peer discovery, see the [peer exchange (PEX) reactor doc](/docs/specification/new-spec/reactors/pex/pex.md).
+For details on peer discovery, see the [peer exchange (PEX) reactor doc](https://github.com/tendermint/tendermint/blob/master/docs/spec/reactors/pex/pex.md).

 ## Peer Identity

@@ -17,9 +17,6 @@ We will attempt to connect to the peer at IP:PORT, and verify,
 via authenticated encryption, that it is in possession of the private key
 corresponding to `<ID>`. This prevents man-in-the-middle attacks on the peer layer.

-If `auth_enc = false`, peers can use an arbitrary ID, but they must always use
-one. Authentication can then happen out-of-band of Tendermint, for instance via VPN.
-
 ## Connections

 All p2p connections use TCP.
--- a/docs/spec/reactors/block_sync/img/bc-reactor.png
+++ b/docs/spec/reactors/block_sync/img/bc-reactor.png
--- a/docs/spec/reactors/block_sync/reactor.md
+++ b/docs/spec/reactors/block_sync/reactor.md
@@ -44,6 +44,264 @@ type bcStatusResponseMessage struct {
 }
 ```

-## Protocol
+## Architecture and algorithm

-TODO
+The Blockchain reactor is organised as a set of concurrent tasks: 
+    - Receive routine of Blockchain Reactor 
+    - Task for creating Requesters
+    - Set of Requesters tasks and 
+    - Controller task.        
+
+![Blockchain Reactor Architecture Diagram](img/bc-reactor.png)
+
+### Data structures
+
+These are the core data structures necessarily to provide the Blockchain Reactor logic.
+
+Requester data structure is used to track assignment of request for `block` at position `height` to a 
+peer with id equals to `peerID`. 
+
+```go
+type Requester {
+  mtx          Mutex		
+  block        Block
+  height       int64  
+   peerID       p2p.ID
+  redoChannel  chan struct{}
+}
+```
+Pool is core data structure that stores last executed block (`height`), assignment of requests to peers (`requesters`), 
+current height for each peer and number of pending requests for each peer (`peers`), maximum peer height, etc. 
+
+```go
+type Pool {
+  mtx Mutex	
+  requesters       map[int64]*Requester
+   height           int64  
+  peers            map[p2p.ID]*Peer
+   maxPeerHeight    int64    
+   numPending       int32   
+  store            BlockStore
+   requestsChannel  chan<- BlockRequest
+   errorsChannel    chan<- peerError
+}
+```
+
+Peer data structure stores for each peer current `height` and number of pending requests sent to 
+the peer (`numPending`), etc.
+
+```go
+type Peer struct {
+  id         p2p.ID	
+  height     int64
+  numPending int32
+  timeout    *time.Timer
+  didTimeout bool
+}
+```
+
+BlockRequest is internal data structure used to denote current mapping of request for a block at some `height` to 
+a peer (`PeerID`).
+	
+```go
+type BlockRequest {
+  Height int64
+  PeerID p2p.ID  
+}
+```
+
+### Receive routine of Blockchain Reactor
+
+It is executed upon message reception on the BlockchainChannel inside p2p receive routine. There is a separate p2p 
+receive routine (and therefore receive routine of the Blockchain Reactor) executed for each peer. Note that
+try to send will not block (returns immediately) if outgoing buffer is full.    
+
+```go
+handleMsg(pool, m):
+    upon receiving bcBlockRequestMessage m from peer p:
+      block = load block for height m.Height from pool.store		  
+      if block != nil then       
+        try to send BlockResponseMessage(block) to p 	      
+      else	     
+        try to send bcNoBlockResponseMessage(m.Height) to p	    
+
+    upon receiving bcBlockResponseMessage m from peer p:    
+      pool.mtx.Lock()	  
+      requester = pool.requesters[m.Height]	  
+      if requester == nil then	  
+        error("peer sent us a block we didn't expect")	    
+        continue        		
+
+      if requester.block == nil and requester.peerID == p then	  
+        requester.block = m
+        pool.numPending -= 1  // atomic decrement         
+        peer = pool.peers[p]                        
+        if peer != nil then                		
+          peer.numPending--                		  
+          if peer.numPending == 0 then                		  
+            peer.timeout.Stop()          
+            // NOTE: we don't send Quit signal to the corresponding requester task!                  
+        else                 		  
+          trigger peer timeout to expire after peerTimeout          
+      pool.mtx.Unlock()          	  
+		
+		
+    upon receiving bcStatusRequestMessage m from peer p:
+      try to send bcStatusResponseMessage(pool.store.Height)    	
+
+    upon receiving bcStatusResponseMessage m from peer p:
+      pool.mtx.Lock()		
+      peer = pool.peers[p]	  
+      if peer != nil then	  
+        peer.height = m.height	    
+      else	  
+        peer = create new Peer data structure with id = p and height = m.Height	    
+        pool.peers[p] = peer  
+
+      if m.Height > pool.maxPeerHeight then       
+        pool.maxPeerHeight = m.Height		    
+      pool.mtx.Unlock()    
+	    
+onTimeout(p):
+  send error message to pool error channel  
+  peer = pool.peers[p]  
+  peer.didTimeout = true  
+```
+
+### Requester tasks
+
+Requester task is responsible for fetching a single block at position `height`.   
+
+```go
+fetchBlock(height, pool):
+  while true do    
+    peerID = nil	
+    block = nil
+    peer = pickAvailablePeer(height)    
+    peerId = peer.id	
+
+    enqueue BlockRequest(height, peerID) to pool.requestsChannel
+    redo = false    
+    while !redo do	
+      select { 	  
+        upon receiving Quit message do
+          return        
+        upon receiving message on redoChannel do        
+          mtx.Lock()  
+          pool.numPending++
+          redo = true          
+          mtx.UnLock()          
+      }          
+
+pickAvailablePeer(height):
+  selectedPeer = nil	
+  while selectedPeer = nil do	
+    pool.mtx.Lock()	  
+    for each peer in pool.peers do 		
+      if !peer.didTimeout and peer.numPending < maxPendingRequestsPerPeer and peer.height >= height then		  
+        peer.numPending++		    
+        selectedPeer = peer		    
+        break 
+    pool.mtx.Unlock()                	
+    
+    if selectedPeer = nil then          
+      sleep requestIntervalMS	        		
+
+  return selectedPeer 	
+```
+sleep for requestIntervalMS
+### Task for creating Requesters
+
+This task is responsible for continuously creating and starting Requester tasks.
+```go
+createRequesters(pool):
+  while true do	
+    if !pool.isRunning then break  
+    if pool.numPending < maxPendingRequests or size(pool.requesters) < maxTotalRequesters then  
+      pool.mtx.Lock()
+      nextHeight = pool.height + size(pool.requesters)
+      requester = create new requester for height nextHeight                     
+      pool.requesters[nextHeight] = requester                    
+      pool.numPending += 1 // atomic increment                    
+      start requester task                              
+      pool.mtx.Unlock()                           
+    else     
+      sleep requestIntervalMS
+      pool.mtx.Lock()           
+      for each peer in pool.peers do      
+        if !peer.didTimeout && peer.numPending > 0 && peer.curRate < minRecvRate then              
+          send error on pool error channel          
+          peer.didTimeout = true
+        if peer.didTimeout then  
+          for each requester in pool.requesters do                  
+            if requester.getPeerID() == peer then                     
+              enqueue msg on requestor's redoChannel
+          delete(pool.peers, peerID)                  
+      pool.mtx.Unlock()        
+```
+  
+
+### Main blockchain reactor controller task 
+```go
+main(pool):
+  create trySyncTicker with interval trySyncIntervalMS 
+  create statusUpdateTicker with interval statusUpdateIntervalSeconds   	
+  create switchToConsensusTicker with interbal switchToConsensusIntervalSeconds   	
+  
+  while true do	
+    select {  		
+	  upon receiving BlockRequest(Height, Peer) on pool.requestsChannel:
+	    try to send bcBlockRequestMessage(Height) to Peer	
+
+	  upon receiving error(peer) on errorsChannel:
+	    stop peer for error	
+
+	  upon receiving message on statusUpdateTickerChannel:
+	    broadcast bcStatusRequestMessage(bcR.store.Height) // message sent in a separate routine	
+
+	  upon receiving message on switchToConsensusTickerChannel:
+	    pool.mtx.Lock()	
+	    receivedBlockOrTimedOut = pool.height > 0 || (time.Now() - pool.startTime) > 5 Seconds	
+	    ourChainIsLongestAmongPeers = pool.maxPeerHeight == 0 || pool.height >= pool.maxPeerHeight	
+	    haveSomePeers = size of pool.peers > 0	
+	    pool.mtx.Unlock()
+	    if haveSomePeers && receivedBlockOrTimedOut && ourChainIsLongestAmongPeers then
+	      switch to consensus mode	    
+
+          upon receiving message on trySyncTickerChannel:
+            for i = 0; i < 10; i++ do      	
+              pool.mtx.Lock()         	
+              firstBlock = pool.requesters[pool.height].block
+              secondBlock = pool.requesters[pool.height].block
+              if firstBlock == nil or secondBlock == nil then continue
+              pool.mtx.Unlock()
+              verify firstBlock using LastCommit from secondBlock           		    
+              if verification failed           		     
+                pool.mtx.Lock()		     
+                peerID = pool.requesters[pool.height].peerID
+                redoRequestsForPeer(peerId)
+                delete(pool.peers, peerID)
+                stop peer peerID for error            		              
+                pool.mtx.Unlock()                           		        
+              else                      
+                delete(pool.requesters, pool.height)
+                save firstBlock to store
+                pool.height++                        
+                execute firstBlock            
+    }
+    
+redoRequestsForPeer(pool, peerId):
+  for each requester in pool.requesters do        
+    if requester.getPeerID() == peerID 		         
+  	  enqueue msg on redoChannel for requester 	             
+```
+                
+## Channels
+
+Defines `maxMsgSize` for the maximum size of incoming messages,
+`SendQueueCapacity` and `RecvBufferCapacity` for maximum sending and
+receiving buffers respectively. These are supposed to prevent amplification
+attacks by setting up the upper limit on how much data we can receive & send to
+a peer.
+
+Sending incorrectly encoded data will result in stopping the peer.
--- a/docs/spec/reactors/consensus/consensus-reactor.md
+++ b/docs/spec/reactors/consensus/consensus-reactor.md
@@ -342,3 +342,11 @@ It broadcasts `NewRoundStepMessage` or `CommitStepMessage` upon new round state
 broadcasting these messages does not depend on the PeerRoundState; it is sent on the StateChannel. 
 Upon receiving VoteMessage it broadcasts `HasVoteMessage` message to its peers on the StateChannel. 
 `ProposalHeartbeatMessage` is sent the same way on the StateChannel.
+
+## Channels
+
+Defines 4 channels: state, data, vote and vote_set_bits. Each channel
+has  `SendQueueCapacity` and `RecvBufferCapacity` and
+`RecvMessageCapacity` set to `maxMsgSize`.
+
+Sending incorrectly encoded data will result in stopping the peer.
--- a/docs/spec/reactors/consensus/consensus.md
+++ b/docs/spec/reactors/consensus/consensus.md
@@ -16,7 +16,7 @@ explained in a forthcoming document.
 For efficiency reasons, validators in Tendermint consensus protocol do not agree directly on the
 block as the block size is big, i.e., they don't embed the block inside `Proposal` and
 `VoteMessage`. Instead, they reach agreement on the `BlockID` (see `BlockID` definition in
-[Blockchain](blockchain.md) section) that uniquely identifies each block. The block itself is
+[Blockchain](https://github.com/tendermint/tendermint/blob/master/docs/spec/blockchain/blockchain.md#blockid) section) that uniquely identifies each block. The block itself is
 disseminated to validator processes using peer-to-peer gossiping protocol. It starts by having a
 proposer first splitting a block into a number of block parts, that are then gossiped between
 processes using `BlockPartMessage`.
@@ -69,7 +69,7 @@ BlockID contains PartSetHeader.
 ## VoteMessage

 VoteMessage is sent to vote for some block (or to inform others that a process does not vote in the
-current round). Vote is defined in [Blockchain](blockchain.md) section and contains validator's
+current round). Vote is defined in the [Blockchain](https://github.com/tendermint/tendermint/blob/master/docs/spec/blockchain/blockchain.md#blockid) section and contains validator's
 information (validator address and index), height and round for which the vote is sent, vote type,
 blockID if process vote for some block (`nil` otherwise) and a timestamp when the vote is sent. The
 message is signed by the validator private key.
--- a/docs/spec/reactors/consensus/proposer-selection.md
+++ b/docs/spec/reactors/consensus/proposer-selection.md
@@ -44,4 +44,3 @@ p0, p0, p0, p0, p0, p0, p0, p0, p0, p0, p0, p0, p0, p1, p0, p0, p0, p0, p0, etc

 This basically means that almost all rounds have the same proposer. But in this case, the process p0 has anyway enough 
 voting power to decide whatever he wants, so the fact that he coordinates almost all rounds seems correct.
-
--- a/docs/spec/reactors/evidence/reactor.md
+++ b/docs/spec/reactors/evidence/reactor.md
@@ -0,0 +1,10 @@
+# Evidence Reactor
+
+## Channels
+
+[#1503](https://github.com/tendermint/tendermint/issues/1503)
+
+Sending invalid evidence will result in stopping the peer.
+
+Sending incorrectly encoded data or data exceeding `maxMsgSize` will result
+in stopping the peer.
--- a/docs/spec/reactors/mempool/README.md
+++ b/docs/spec/reactors/mempool/README.md
@@ -1,11 +0,0 @@
-# Mempool Specification
-
-This package contains documents specifying the functionality
-of the mempool module.
-
-Components:
-
-* [Config](./config.md) - how to configure it
-* [External Messages](./messages.md) - The messages we accept over p2p and rpc interfaces
-* [Functionality](./functionality.md) - high-level description of the functionality it provides
-* [Concurrency Model](./concurrency.md) - What guarantees we provide, what locks we require.
--- a/docs/spec/reactors/mempool/messages.md
+++ b/docs/spec/reactors/mempool/messages.md
@@ -31,7 +31,8 @@ It can be posted via `broadcast_commit`, `broadcast_sync` or
 wait before returning (sync makes sure CheckTx passes, commit
 makes sure it was included in a signed block).

-Request (`POST http://gaia.zone:46657/`):
+Request (`POST http://gaia.zone:26657/`):
+
 ```json
 {
 "id": "",
@@ -43,8 +44,8 @@ Request (`POST http://gaia.zone:46657/`):
 }
 ```

-
 Response:
+
 ```json
 {
 "error": "",
--- a/docs/spec/reactors/mempool/reactor.md
+++ b/docs/spec/reactors/mempool/reactor.md
@@ -0,0 +1,14 @@
+# Mempool Reactor
+
+## Channels
+
+[#1503](https://github.com/tendermint/tendermint/issues/1503)
+
+Mempool maintains a cache of the last 10000 transactions to prevent
+replaying old transactions (plus transactions coming from other
+validators, who are continually exchanging transactions). Read [Replay
+Protection](https://tendermint.readthedocs.io/projects/tools/en/master/app-development.html?#replay-protection)
+for details.
+
+Sending incorrectly encoded data or data exceeding `maxMsgSize` will result
+in stopping the peer.
--- a/docs/spec/reactors/pex/pex.md
+++ b/docs/spec/reactors/pex/pex.md
@@ -117,7 +117,7 @@ current, past, and rate-of-change data to inform peer quality.
 While a PID trust metric has been implemented, it remains for future work
 to use it in the PEX.

-See the [trustmetric](../../../architecture/adr-006-trust-metric.md )
-and [trustmetric useage](../../../architecture/adr-007-trust-metric-usage.md )
+See the [trustmetric](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-006-trust-metric.md)
+and [trustmetric useage](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-007-trust-metric-usage.md)
 architecture docs for more details.

--- a/docs/spec/reactors/pex/reactor.md
+++ b/docs/spec/reactors/pex/reactor.md
@@ -0,0 +1,12 @@
+# PEX Reactor
+
+## Channels
+
+Defines only `SendQueueCapacity`. [#1503](https://github.com/tendermint/tendermint/issues/1503)
+
+Implements rate-limiting by enforcing minimal time between two consecutive
+`pexRequestMessage` requests. If the peer sends us addresses we did not ask,
+it is stopped.
+
+Sending incorrectly encoded data or data exceeding `maxMsgSize` will result
+in stopping the peer.
--- a/docs/spec/software/abci.md
+++ b/docs/spec/software/abci.md
@@ -0,0 +1,192 @@
+# Application Blockchain Interface (ABCI)
+
+ABCI is the interface between Tendermint (a state-machine replication engine)
+and an application (the actual state machine).
+
+The ABCI message types are defined in a [protobuf
+file](https://github.com/tendermint/abci/blob/master/types/types.proto).
+
+For full details on the ABCI message types and protocol, see the [ABCI
+specificaiton](https://github.com/tendermint/abci/blob/master/specification.rst).
+Be sure to read the specification if you're trying to build an ABCI app!
+
+For additional details on server implementation, see the [ABCI
+readme](https://github.com/tendermint/abci#implementation).
+
+Here we provide some more details around the use of ABCI by Tendermint and
+clarify common "gotchas".
+
+## ABCI connections
+
+Tendermint opens 3 ABCI connections to the app: one for Consensus, one for
+Mempool, one for Queries.
+
+## Async vs Sync
+
+The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages.
+This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward
+transactions to the app before it's finished processing previous ones.
+
+Thus, DeliverTx and CheckTx messages are sent asycnhronously, while all other
+messages are sent synchronously.
+
+## CheckTx and Commit
+
+It is typical to hold three distinct states in an ABCI app: CheckTxState, DeliverTxState,
+QueryState. The QueryState contains the latest committed state for a block.
+The CheckTxState and DeliverTxState may be updated concurrently with one another.
+Before Commit is called, Tendermint locks and flushes the mempool so that no new changes will happen
+to CheckTxState. When Commit completes, it unlocks the mempool.
+
+Thus, during Commit, it is safe to reset the QueryState and the CheckTxState to the latest DeliverTxState
+(ie. the new state from executing all the txs in the block).
+
+Note, however, that it is not possible to send transactions to Tendermint during Commit - if your app
+tries to send a `/broadcast_tx` to Tendermint during Commit, it will deadlock.
+
+
+## EndBlock Validator Updates
+
+Updates to the Tendermint validator set can be made by returning `Validator`
+objects in the `ResponseBeginBlock`:
+
+```
+message Validator {
+  bytes address = 1;
+  PubKey pub_key = 2;
+  int64 power = 3;
+}
+
+message PubKey {
+  string type = 1;
+  bytes  data = 2;
+}
+
+```
+
+The `pub_key` currently supports two types:
+    - `type = "ed25519" and `data = <raw 32-byte public key>`
+    - `type = "secp256k1" and `data = <33-byte OpenSSL compressed public key>`
+
+If the address is provided, it must match the address of the pubkey, as
+specified [here](/docs/spec/blockchain/encoding.md#Addresses)
+
+(Note: In the v0.19 series, the `pub_key` is the [Amino encoded public
+key](/docs/spec/blockchain/encoding.md#public-key-cryptography).
+For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 32-byte Ed25519 pubkey
+`76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
+Amino encoded as
+`1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`)
+
+(Note: In old versions of Tendermint (pre-v0.19.0), the pubkey is just prefixed with a
+single type byte, so for ED25519 we'd have `pub_key = 0x1 | pub`)
+
+The `power` is the new voting power for the validator, with the
+following rules:
+
+- power must be non-negative
+- if power is 0, the validator must already exist, and will be removed from the
+  validator set
+- if power is non-0:
+    - if the validator does not already exist, it will be added to the validator
+      set with the given power
+    - if the validator does already exist, its power will be adjusted to the given power
+
+## InitChain Validator Updates
+
+ResponseInitChain has the option to return a list of validators.
+If the list is not empty, Tendermint will adopt it for the validator set.
+This way the application can determine the initial validator set for the
+blockchain.
+
+Note that if addressses are included in the returned validators, they must match
+the address of the public key.
+
+ResponseInitChain also includes ConsensusParams, but these are presently
+ignored.
+
+## Query
+
+Query is a generic message type with lots of flexibility to enable diverse sets
+of queries from applications. Tendermint has no requirements from the Query
+message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too.
+That said, Tendermint makes a number of queries to support some optional
+features. These are:
+
+### Peer Filtering
+
+When Tendermint connects to a peer, it sends two queries to the ABCI application
+using the following paths, with no additional data:
+
+ - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and
+   the port of the connection
+ - `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the
+   pubkey.Address() for the peer's PubKey)
+
+If either of these queries return a non-zero ABCI code, Tendermint will refuse
+to connect to the peer.
+
+## Info and the Handshake/Replay
+
+On startup, Tendermint calls Info on the Query connection to get the latest
+committed state of the app. The app MUST return information consistent with the
+last block it succesfully completed Commit for.
+
+If the app succesfully committed block H but not H+1, then `last_block_height =
+H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app
+failed during the Commit of block H, then `last_block_height = H-1` and
+`last_block_app_hash = <hash returned by Commit for block H-1, which is the hash
+in the header of block H>`.
+
+We now distinguish three heights, and describe how Tendermint syncs itself with
+the app.
+
+```
+storeBlockHeight = height of the last block Tendermint saw a commit for
+stateBlockHeight = height of the last block for which Tendermint completed all
+    block processing and saved all ABCI results to disk
+appBlockHeight = height of the last block for which ABCI app succesfully
+    completely Commit
+```
+
+Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight`
+Note also we never call Commit on an ABCI app twice for the same height.
+
+The procedure is as follows.
+
+First, some simeple start conditions:
+
+If `appBlockHeight == 0`, then call InitChain.
+
+If `storeBlockHeight == 0`, we're done.
+
+Now, some sanity checks:
+
+If `storeBlockHeight < appBlockHeight`, error
+If `storeBlockHeight < stateBlockHeight`, panic
+If `storeBlockHeight > stateBlockHeight+1`, panic
+
+Now, the meat:
+
+If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`,
+	replay all blocks in full from `appBlockHeight` to `storeBlockHeight`.
+	This happens if we completed processing the block, but the app forgot its height.
+
+If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done
+	This happens if we crashed at an opportune spot.
+
+If `storeBlockHeight == stateBlockHeight+1`
+	This happens if we started processing the block but didn't finish.
+
+	If `appBlockHeight < stateBlockHeight`
+		replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`,
+		and replay the block at `storeBlockHeight` using the WAL.
+	This happens if the app forgot the last block it committed.
+
+	If `appBlockHeight == stateBlockHeight`,
+		replay the last block (storeBlockHeight) in full.
+	This happens if we crashed before the app finished Commit
+
+	If appBlockHeight == storeBlockHeight {
+		update the state using the saved ABCI responses but dont run the block against the real app.
+	This happens if we crashed after the app finished Commit but before Tendermint saved the state.
--- a/docs/spec/software/wal.md
+++ b/docs/spec/software/wal.md
@@ -0,0 +1,33 @@
+# WAL
+
+Consensus module writes every message to the WAL (write-ahead log).
+
+It also issues fsync syscall through
+[File#Sync](https://golang.org/pkg/os/#File.Sync) for messages signed by this
+node (to prevent double signing).
+
+Under the hood, it uses
+[autofile.Group](https://godoc.org/github.com/tendermint/tmlibs/autofile#Group),
+which rotates files when those get too big (> 10MB).
+
+The total maximum size is 1GB. We only need the latest block and the block before it,
+but if the former is dragging on across many rounds, we want all those rounds.
+
+## Replay
+
+Consensus module will replay all the messages of the last height written to WAL
+before a crash (if such occurs).
+
+The private validator may try to sign messages during replay because it runs
+somewhat autonomously and does not know about replay process.
+
+For example, if we got all the way to precommit in the WAL and then crash,
+after we replay the proposal message, the private validator will try to sign a
+prevote. But it will fail. That's ok because we’ll see the prevote later in the
+WAL. Then it will go to precommit, and that time it will work because the
+private validator contains the `LastSignBytes` and then we’ll replay the
+precommit from the WAL.
+
+Make sure to read about [WAL
+corruption](https://tendermint.readthedocs.io/projects/tools/en/master/specification/corruption.html#wal-corruption)
+and recovery strategies.
--- a/docs/specification/block-structure.rst
+++ b/docs/specification/block-structure.rst
@@ -62,7 +62,7 @@ that come from the `ABCI
 application <https://github.com/tendermint/abci>`__. It represents the
 state of the actual application, rather that the state of the blockchain
 itself. This means it's necessary in order to perform any business
-logic, such as verifying and account balance.
+logic, such as verifying an account balance.

 **Note** After the transactions are committed to a block, they still
 need to be processed in a separate step, which happens between the
--- a/docs/specification/configuration.rst
+++ b/docs/specification/configuration.rst
@@ -24,7 +24,7 @@ like the file below, however, double check by inspecting the

    # TCP or UNIX socket address of the ABCI application,
    # or the name of an ABCI application compiled in with the Tendermint binary
-    proxy_app = "tcp://127.0.0.1:46658"
+    proxy_app = "tcp://127.0.0.1:26658"

    # A custom human readable name for this node
    moniker = "anonymous"
@@ -70,7 +70,7 @@ like the file below, however, double check by inspecting the
    [rpc]

    # TCP or UNIX socket address for the RPC server to listen on
-    laddr = "tcp://0.0.0.0:46657"
+    laddr = "tcp://0.0.0.0:26657"

    # TCP or UNIX socket address for the gRPC server to listen on
    # NOTE: This server only supports /broadcast_tx_commit
@@ -83,7 +83,7 @@ like the file below, however, double check by inspecting the
    [p2p]

    # Address to listen for incoming connections
-    laddr = "tcp://0.0.0.0:46656"
+    laddr = "tcp://0.0.0.0:26656"

    # Comma separated list of seed nodes to connect to
    seeds = ""
@@ -122,9 +122,6 @@ like the file below, however, double check by inspecting the
    # Does not work if the peer-exchange reactor is disabled.
    seed_mode = false

-    # Authenticated encryption
-    auth_enc = true
-
    # Comma separated list of peer IDs to keep private (will not be gossiped to other peers)
    private_peer_ids = ""

--- a/docs/specification/new-spec/README.md
+++ b/docs/specification/new-spec/README.md
@@ -1 +1 @@
-Spec moved to [docs/spec](/docs/spec).
+Spec moved to [docs/spec](https://github.com/tendermint/tendermint/tree/master/docs/spec).
--- a/docs/specification/rpc.rst
+++ b/docs/specification/rpc.rst
@@ -1,190 +1,4 @@
 RPC
 ===

-Coming soon: RPC docs powered by `slate <https://github.com/lord/slate>`__. Until then, read on.
-
-Tendermint supports the following RPC protocols:
-
-  URI over HTTP
-  JSONRPC over HTTP
-  JSONRPC over websockets
-
-Tendermint RPC is build using `our own RPC
-library <https://github.com/tendermint/tendermint/tree/master/rpc/lib>`__.
-Documentation and tests for that library could be found at
-``tendermint/rpc/lib`` directory.
-
-Configuration
-~~~~~~~~~~~~~
-
-Set the ``laddr`` config parameter under ``[rpc]`` table in the
-$TMHOME/config/config.toml file or the ``--rpc.laddr`` command-line flag to the
-desired protocol://host:port setting. Default: ``tcp://0.0.0.0:46657``.
-
-Arguments
-~~~~~~~~~
-
-Arguments which expect strings or byte arrays may be passed as quoted
-strings, like ``"abc"`` or as ``0x``-prefixed strings, like
-``0x616263``.
-
-URI/HTTP
-~~~~~~~~
-
-Example request:
-
-.. code:: bash
-
-    curl -s 'http://localhost:46657/broadcast_tx_sync?tx="abc"' | jq .
-
-Response:
-
-.. code:: json
-
-    {
-      "error": "",
-      "result": {
-        "hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
-        "log": "",
-        "data": "",
-        "code": 0
-      },
-      "id": "",
-      "jsonrpc": "2.0"
-    }
-
-The first entry in the result-array (``96``) is the method this response
-correlates with. ``96`` refers to "ResultTypeBroadcastTx", see
-`responses.go <https://github.com/tendermint/tendermint/blob/master/rpc/core/types/responses.go>`__
-for a complete overview.
-
-JSONRPC/HTTP
-~~~~~~~~~~~~
-
-JSONRPC requests can be POST'd to the root RPC endpoint via HTTP (e.g.
-``http://localhost:46657/``).
-
-Example request:
-
-.. code:: json
-
-    {
-      "method": "broadcast_tx_sync",
-      "jsonrpc": "2.0",
-      "params": [ "abc" ],
-      "id": "dontcare"
-    }
-
-JSONRPC/websockets
-~~~~~~~~~~~~~~~~~~
-
-JSONRPC requests can be made via websocket. The websocket endpoint is at
-``/websocket``, e.g. ``http://localhost:46657/websocket``. Asynchronous
-RPC functions like event ``subscribe`` and ``unsubscribe`` are only
-available via websockets.
-
-Endpoints
-~~~~~~~~~
-
-An HTTP Get request to the root RPC endpoint (e.g.
-``http://localhost:46657``) shows a list of available endpoints.
-
-::
-
-    Available endpoints:
-    http://localhost:46657/abci_info
-    http://localhost:46657/dump_consensus_state
-    http://localhost:46657/genesis
-    http://localhost:46657/net_info
-    http://localhost:46657/num_unconfirmed_txs
-    http://localhost:46657/health
-    http://localhost:46657/status
-    http://localhost:46657/unconfirmed_txs
-    http://localhost:46657/unsafe_flush_mempool
-    http://localhost:46657/unsafe_stop_cpu_profiler
-    http://localhost:46657/validators
-
-    Endpoints that require arguments:
-    http://localhost:46657/abci_query?path=_&data=_&prove=_
-    http://localhost:46657/block?height=_
-    http://localhost:46657/blockchain?minHeight=_&maxHeight=_
-    http://localhost:46657/broadcast_tx_async?tx=_
-    http://localhost:46657/broadcast_tx_commit?tx=_
-    http://localhost:46657/broadcast_tx_sync?tx=_
-    http://localhost:46657/commit?height=_
-    http://localhost:46657/dial_seeds?seeds=_
-    http://localhost:46657/dial_peers?peers=_&persistent=_
-    http://localhost:46657/subscribe?event=_
-    http://localhost:46657/tx?hash=_&prove=_
-    http://localhost:46657/unsafe_start_cpu_profiler?filename=_
-    http://localhost:46657/unsafe_write_heap_profile?filename=_
-    http://localhost:46657/unsubscribe?event=_
-
-tx
-~~
-
-Returns a transaction matching the given transaction hash.
-
-**Parameters**
-
-1. hash - the transaction hash
-2. prove - include a proof of the transaction inclusion in the block in
-   the result (optional, default: false)
-
-**Returns**
-
-  ``proof``: the ``types.TxProof`` object
-  ``tx``: ``[]byte`` - the transaction
-  ``tx_result``: the ``abci.Result`` object
-  ``index``: ``int`` - index of the transaction
-  ``height``: ``int`` - height of the block where this transaction was
-   in
-
-**Example**
-
-.. code:: bash
-
-    curl -s 'http://localhost:46657/broadcast_tx_commit?tx="abc"' | jq .
-    # {
-    #   "error": "",
-    #   "result": {
-    #     "hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
-    #     "log": "",
-    #     "data": "",
-    #     "code": 0
-    #   },
-    #   "id": "",
-    #   "jsonrpc": "2.0"
-    # }
-
-    curl -s 'http://localhost:46657/tx?hash=0x2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF' | jq .
-    # {
-    #   "error": "",
-    #   "result": {
-    #     "proof": {
-    #       "Proof": {
-    #         "aunts": []
-    #       },
-    #       "Data": "YWJjZA==",
-    #       "RootHash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
-    #       "Total": 1,
-    #       "Index": 0
-    #     },
-    #     "tx": "YWJjZA==",
-    #     "tx_result": {
-    #       "log": "",
-    #       "data": "",
-    #       "code": 0
-    #     },
-    #     "index": 0,
-    #     "height": 52
-    #   },
-    #   "id": "",
-    #   "jsonrpc": "2.0"
-    # }
-
-More Examples
-~~~~~~~~~~~~~
-
-See the various bash tests using curl in ``test/``, and examples using
-the ``Go`` API in ``rpc/client/``.
+The RPC documentation is hosted `here <https://tendermint.github.io/slate>`__ and is generated by the CI from our `Slate repo <https://github.com/tendermint/slate>`__. To update the documentation, edit the relevant ``godoc`` comments in the `rpc/core directory <https://github.com/tendermint/tendermint/tree/develop/rpc/core>`__.
--- a/docs/specification/secure-p2p.rst
+++ b/docs/specification/secure-p2p.rst
@@ -65,9 +65,7 @@ are connected to at least one validator.
 Config
 ------

-Authenticated encryption is enabled by default. If you wish to use another
-authentication scheme or your peers are connected via VPN, you can turn it off
-by setting ``auth_enc`` to ``false`` in the config file.
+Authenticated encryption is enabled by default.

 Additional Reading
 ------------------
--- a/docs/subscribing-to-events-via-websocket.md
+++ b/docs/subscribing-to-events-via-websocket.md
@@ -0,0 +1,26 @@
+# Subscribing to events via Websocket
+
+Tendermint emits different events, to which you can subscribe via
+[Websocket](https://en.wikipedia.org/wiki/WebSocket). This can be useful
+for third-party applications (for analysys) or inspecting state.
+
+[List of events](https://godoc.org/github.com/tendermint/tendermint/types#pkg-constants)
+
+You can subscribe to any of the events above by calling `subscribe` RPC
+method via Websocket.
+
+    {
+        "jsonrpc": "2.0",
+        "method": "subscribe",
+        "id": "0",
+        "params": {
+            "query": "tm.event='NewBlock'"
+        }
+    }
+
+Check out [API docs](https://tendermint.github.io/slate/#subscribe) for
+more information on query syntax and other options.
+
+You can also use tags, given you had included them into DeliverTx
+response, to query transaction results. See [Indexing
+transactions](./indexing-transactions.md) for details.
--- a/docs/terraform-and-ansible.md
+++ b/docs/terraform-and-ansible.md
@@ -0,0 +1,147 @@
+# Terraform & Ansible
+
+Automated deployments are done using
+[Terraform](https://www.terraform.io/) to create servers on Digital
+Ocean then [Ansible](http://www.ansible.com/) to create and manage
+testnets on those servers.
+
+## Install
+
+NOTE: see the [integration bash
+script](https://github.com/tendermint/tendermint/blob/develop/networks/remote/integration.sh)
+that can be run on a fresh DO droplet and will automatically spin up a 4
+node testnet. The script more or less does everything described below.
+
+-   Install [Terraform](https://www.terraform.io/downloads.html) and
+    [Ansible](http://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
+    on a Linux machine.
+-   Create a [DigitalOcean API
+    token](https://cloud.digitalocean.com/settings/api/tokens) with read
+    and write capability.
+-   Install the python dopy package (`pip install dopy`)
+-   Create SSH keys (`ssh-keygen`)
+-   Set environment variables:
+
+```    
+export DO_API_TOKEN="abcdef01234567890abcdef01234567890"
+export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"
+```
+
+These will be used by both `terraform` and `ansible`.
+
+### Terraform
+
+This step will create four Digital Ocean droplets. First, go to the
+correct directory:
+
+    cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/terraform
+
+then:
+
+    terraform init
+    terraform apply -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
+
+and you will get a list of IP addresses that belong to your droplets.
+
+With the droplets created and running, let's setup Ansible.
+
+### Ansible
+
+The playbooks in [the ansible
+directory](https://github.com/tendermint/tendermint/tree/master/networks/remote/ansible)
+run ansible roles to configure the sentry node architecture. You must
+switch to this directory to run ansible
+(`cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/ansible`).
+
+There are several roles that are self-explanatory:
+
+First, we configure our droplets by specifying the paths for tendermint
+(`BINARY`) and the node files (`CONFIGDIR`). The latter expects any
+number of directories named `node0, node1, ...` and so on (equal to the
+number of droplets created). For this example, we use pre-created files
+from [this
+directory](https://github.com/tendermint/tendermint/tree/master/docs/examples).
+To create your own files, use either the `tendermint testnet` command or
+review [manual deployments](./deploy-testnets.md).
+
+Here's the command to run:
+
+    ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$GOPATH/src/github.com/tendermint/tendermint/build/tendermint -e CONFIGDIR=$GOPATH/src/github.com/tendermint/tendermint/docs/examples
+
+Voila! All your droplets now have the `tendermint` binary and required
+configuration files to run a testnet.
+
+Next, we run the install role:
+
+    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
+
+which as you'll see below, executes
+`tendermint node --proxy_app=kvstore` on all droplets. Although we'll
+soon be modifying this role and running it again, this first execution
+allows us to get each `node_info.id` that corresponds to each
+`node_info.listen_addr`. (This part will be automated in the future). In
+your browser (or using `curl`), for every droplet, go to IP:26657/status
+and note the two just mentioned `node_info` fields. Notice that blocks
+aren't being created (`latest_block_height` should be zero and not
+increasing).
+
+Next, open `roles/install/templates/systemd.service.j2` and look for the
+line `ExecStart` which should look something like:
+
+    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore
+
+and add the `--p2p.persistent_peers` flag with the relevant information
+for each node. The resulting file should look something like:
+
+    [Unit]
+    Description={{service}}
+    Requires=network-online.target
+    After=network-online.target
+
+    [Service]
+    Restart=on-failure
+    User={{service}}
+    Group={{service}}
+    PermissionsStartOnly=true
+    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore --p2p.persistent_peers=167b80242c300bf0ccfb3ced3dec60dc2a81776e@165.227.41.206:26656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@165.227.43.146:26656,303a1a4312c30525c99ba66522dd81cca56a361a@159.89.115.32:26656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@159.89.119.125:26656
+    ExecReload=/bin/kill -HUP $MAINPID
+    KillSignal=SIGTERM
+
+    [Install]
+    WantedBy=multi-user.target
+
+Then, stop the nodes:
+
+    ansible-playbook -i inventory/digital_ocean.py -l sentrynet stop.yml
+
+Finally, we run the install role again:
+
+    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
+
+to re-run `tendermint node` with the new flag, on all droplets. The
+`latest_block_hash` should now be changing and `latest_block_height`
+increasing. Your testnet is now up and running :)
+
+Peek at the logs with the status role:
+
+    ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml
+
+### Logging
+
+The crudest way is the status role described above. You can also ship
+logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana)
+service provider. You can set up your nodes to log there automatically.
+Create an account and get your API key from the notes on [this
+page](https://app.logz.io/#/dashboard/data-sources/Filebeat), then:
+
+    yum install systemd-devel || echo "This will only work on RHEL-based systems."
+    apt-get install libsystemd-dev || echo "This will only work on Debian-based systems."
+
+    go get github.com/mheese/journalbeat
+    ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345
+
+### Cleanup
+
+To remove your droplets, run:
+
+    terraform destroy -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
--- a/docs/terraform-and-ansible.rst
+++ b/docs/terraform-and-ansible.rst
@@ -1,138 +0,0 @@
-Terraform & Ansible
-===================
-
-Automated deployments are done using `Terraform <https://www.terraform.io/>`__ to create servers on Digital Ocean then
-`Ansible <http://www.ansible.com/>`__ to create and manage testnets on those servers.
-
-Install
-------
-
-NOTE: see the `integration bash script <https://github.com/tendermint/tendermint/blob/develop/networks/remote/integration.sh>`__ that can be run on a fresh DO droplet and will automatically spin up a 4 node testnet. The script more or less does everything described below.
-
- Install `Terraform <https://www.terraform.io/downloads.html>`__ and `Ansible <http://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html>`__ on a Linux machine.
- Create a `DigitalOcean API token <https://cloud.digitalocean.com/settings/api/tokens>`__ with read and write capability.
- Install the python dopy package (``pip install dopy``)
- Create SSH keys (``ssh-keygen``)
- Set environment variables:
-
-::
-
-    export DO_API_TOKEN="abcdef01234567890abcdef01234567890"
-    export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"
-
-These will be used by both ``terraform`` and ``ansible``.
-
-Terraform
---------
-
-This step will create four Digital Ocean droplets. First, go to the correct directory:
-
-::
-
-    cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/terraform
-
-then:
-
-::
-
-    terraform init
-    terraform apply -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
-
-and you will get a list of IP addresses that belong to your droplets.
-
-With the droplets created and running, let's setup Ansible.
-
-Using Ansible
-------------
-
-The playbooks in `the ansible directory <https://github.com/tendermint/tendermint/tree/master/networks/remote/ansible>`__
-run ansible roles to configure the sentry node architecture. You must switch to this directory to run ansible (``cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/ansible``).
-
-There are several roles that are self-explanatory:
-
-First, we configure our droplets by specifying the paths for tendermint (``BINARY``) and the node files (``CONFIGDIR``). The latter expects any number of directories named ``node0, node1, ...`` and so on (equal to the number of droplets created). For this example, we use pre-created files from `this directory <https://github.com/tendermint/tendermint/tree/master/docs/examples>`__. To create your own files, use either the ``tendermint testnet`` command or review `manual deployments <./deploy-testnets.html>`__.
-
-Here's the command to run:
-
-::
-
-    ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$GOPATH/src/github.com/tendermint/tendermint/build/tendermint -e CONFIGDIR=$GOPATH/src/github.com/tendermint/tendermint/docs/examples
-
-Voila! All your droplets now have the ``tendermint`` binary and required configuration files to run a testnet.
-
-Next, we run the install role:
-
-::
-
-    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
-
-which as you'll see below, executes ``tendermint node --proxy_app=kvstore`` on all droplets. Although we'll soon be modifying this role and running it again, this first execution allows us to get each ``node_info.id`` that corresponds to each ``node_info.listen_addr``. (This part will be automated in the future). In your browser (or using ``curl``), for every droplet, go to IP:46657/status and note the two just mentioned ``node_info`` fields. Notice that blocks aren't being created (``latest_block_height`` should be zero and not increasing).
-
-Next, open ``roles/install/templates/systemd.service.j2`` and look for the line ``ExecStart`` which should look something like:
-
-::
-
-    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore
-
-and add the ``--p2p.persistent_peers`` flag with the relevant information for each node. The resulting file should look something like:
-
-::
-
-    [Unit]
-    Description={{service}}
-    Requires=network-online.target
-    After=network-online.target
-    
-    [Service]
-    Restart=on-failure
-    User={{service}}
-    Group={{service}}
-    PermissionsStartOnly=true
-    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore --p2p.persistent_peers=167b80242c300bf0ccfb3ced3dec60dc2a81776e@165.227.41.206:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@165.227.43.146:46656,303a1a4312c30525c99ba66522dd81cca56a361a@159.89.115.32:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@159.89.119.125:46656
-    ExecReload=/bin/kill -HUP $MAINPID
-    KillSignal=SIGTERM
-
-    [Install]
-    WantedBy=multi-user.target
-
-Then, stop the nodes:
-
-::
-
-    ansible-playbook -i inventory/digital_ocean.py -l sentrynet stop.yml
-
-Finally, we run the install role again:
-
-::
-
-    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
-
-to re-run ``tendermint node`` with the new flag, on all droplets. The ``latest_block_hash`` should now be changing and ``latest_block_height`` increasing. Your testnet is now up and running :)
-
-Peek at the logs with the status role:
-
-::
-
-    ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml
-
-Logging
-------
-
-The crudest way is the status role described above. You can also ship logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana) service provider. You can set up your nodes to log there automatically. Create an account and get your API key from the notes on `this page <https://app.logz.io/#/dashboard/data-sources/Filebeat>`__, then:
-
-::
-
-   yum install systemd-devel || echo "This will only work on RHEL-based systems."
-   apt-get install libsystemd-dev || echo "This will only work on Debian-based systems."
-
-   go get github.com/mheese/journalbeat
-   ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345
-
-Cleanup
-------
-
-To remove your droplets, run:
-
-::
-
-    terraform destroy -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
--- a/docs/transactional-semantics.md
+++ b/docs/transactional-semantics.md
@@ -0,0 +1,25 @@
+# Transactional Semantics
+
+In [Using Tendermint](./using-tendermint.md#broadcast-api) we
+discussed different API endpoints for sending transactions and
+differences between them.
+
+What we have not yet covered is transactional semantics.
+
+When you send a transaction using one of the available methods, it first
+goes to the mempool. Currently, it does not provide strong guarantees
+like "if the transaction were accepted, it would be eventually included
+in a block (given CheckTx passes)."
+
+For instance a tx could enter the mempool, but before it can be sent to
+peers the node crashes.
+
+We are planning to provide such guarantees by using a WAL and replaying
+transactions (See
+[this issue](https://github.com/tendermint/tendermint/issues/248)), but
+it's non-trivial to do this all efficiently.
+
+The temporary solution is for clients to monitor the node and resubmit
+transaction(s) and/or send them to more nodes at once, so the
+probability of all of them crashing at the same time and losing the msg
+decreases substantially.
--- a/docs/transactional-semantics.rst
+++ b/docs/transactional-semantics.rst
@@ -1,27 +0,0 @@
-Transactional Semantics
-=======================
-
-In `Using
-Tendermint <./using-tendermint.html#broadcast-api>`__ we
-discussed different API endpoints for sending transactions and
-differences between them.
-
-What we have not yet covered is transactional semantics.
-
-When you send a transaction using one of the available methods, it
-first goes to the mempool. Currently, it does not provide strong
-guarantees like "if the transaction were accepted, it would be
-eventually included in a block (given CheckTx passes)."
-
-For instance a tx could enter the mempool, but before it can be sent
-to peers the node crashes.
-
-We are planning to provide such guarantees by using a WAL and
-replaying transactions (See
-`GH#248 <https://github.com/tendermint/tendermint/issues/248>`__), but
-it's non-trivial to do this all efficiently.
-
-The temporary solution is for clients to monitor the node and resubmit
-transaction(s) or/and send them to more nodes at once, so the
-probability of all of them crashing at the same time and losing the
-msg decreases substantially.
--- a/docs/using-tendermint.md
+++ b/docs/using-tendermint.md
@@ -0,0 +1,420 @@
+# Using Tendermint
+
+This is a guide to using the `tendermint` program from the command line.
+It assumes only that you have the `tendermint` binary installed and have
+some rudimentary idea of what Tendermint and ABCI are.
+
+You can see the help menu with `tendermint --help`, and the version
+number with `tendermint version`.
+
+## Directory Root
+
+The default directory for blockchain data is `~/.tendermint`. Override
+this by setting the `TMHOME` environment variable.
+
+## Initialize
+
+Initialize the root directory by running:
+
+    tendermint init
+
+This will create a new private key (`priv_validator.json`), and a
+genesis file (`genesis.json`) containing the associated public key, in
+`$TMHOME/config`. This is all that's necessary to run a local testnet
+with one validator.
+
+For more elaborate initialization, see the tesnet command:
+
+    tendermint testnet --help
+
+## Run
+
+To run a Tendermint node, use
+
+    tendermint node
+
+By default, Tendermint will try to connect to an ABCI application on
+[127.0.0.1:26658](127.0.0.1:26658). If you have the `kvstore` ABCI app
+installed, run it in another window. If you don't, kill Tendermint and
+run an in-process version of the `kvstore` app:
+
+    tendermint node --proxy_app=kvstore
+
+After a few seconds you should see blocks start streaming in. Note that
+blocks are produced regularly, even if there are no transactions. See
+*No Empty Blocks*, below, to modify this setting.
+
+Tendermint supports in-process versions of the `counter`, `kvstore` and
+`nil` apps that ship as examples in the [ABCI
+repository](https://github.com/tendermint/abci). It's easy to compile
+your own app in-process with Tendermint if it's written in Go. If your
+app is not written in Go, simply run it in another process, and use the
+`--proxy_app` flag to specify the address of the socket it is listening
+on, for instance:
+
+    tendermint node --proxy_app=/var/run/abci.sock
+
+## Transactions
+
+To send a transaction, use `curl` to make requests to the Tendermint RPC
+server, for example:
+
+    curl http://localhost:26657/broadcast_tx_commit?tx=\"abcd\"
+
+We can see the chain's status at the `/status` end-point:
+
+    curl http://localhost:26657/status | json_pp
+
+and the `latest_app_hash` in particular:
+
+    curl http://localhost:26657/status | json_pp | grep latest_app_hash
+
+Visit http://localhost:26657> in your browser to see the list of other
+endpoints. Some take no arguments (like `/status`), while others specify
+the argument name and use `_` as a placeholder.
+
+### Formatting
+
+The following nuances when sending/formatting transactions should be
+taken into account:
+
+With `GET`:
+
+To send a UTF8 string byte array, quote the value of the tx pramater:
+
+    curl 'http://localhost:26657/broadcast_tx_commit?tx="hello"'
+
+which sends a 5 byte transaction: "h e l l o" \[68 65 6c 6c 6f\].
+
+Note the URL must be wrapped with single quoes, else bash will ignore
+the double quotes. To avoid the single quotes, escape the double quotes:
+
+    curl http://localhost:26657/broadcast_tx_commit?tx=\"hello\"
+
+Using a special character:
+
+    curl 'http://localhost:26657/broadcast_tx_commit?tx="€5"'
+
+sends a 4 byte transaction: "€5" (UTF8) \[e2 82 ac 35\].
+
+To send as raw hex, omit quotes AND prefix the hex string with `0x`:
+
+    curl http://localhost:26657/broadcast_tx_commit?tx=0x01020304
+
+which sends a 4 byte transaction: \[01 02 03 04\].
+
+With `POST` (using `json`), the raw hex must be `base64` encoded:
+
+    curl --data-binary '{"jsonrpc":"2.0","id":"anything","method":"broadcast_tx_commit","params": {"tx": "AQIDBA=="}}' -H 'content-type:text/plain;' http://localhost:26657
+
+which sends the same 4 byte transaction: \[01 02 03 04\].
+
+Note that raw hex cannot be used in `POST` transactions.
+
+## Reset
+
+**WARNING: UNSAFE** Only do this in development and only if you can
+afford to lose all blockchain data!
+
+To reset a blockchain, stop the node, remove the `~/.tendermint/data`
+directory and run
+
+    tendermint unsafe_reset_priv_validator
+
+This final step is necessary to reset the `priv_validator.json`, which
+otherwise prevents you from making conflicting votes in the consensus
+(something that could get you in trouble if you do it on a real
+blockchain). If you don't reset the `priv_validator.json`, your fresh
+new blockchain will not make any blocks.
+
+## Configuration
+
+Tendermint uses a `config.toml` for configuration. For details, see [the
+config specification](./specification/configuration.html).
+
+Notable options include the socket address of the application
+(`proxy_app`), the listening address of the Tendermint peer
+(`p2p.laddr`), and the listening address of the RPC server
+(`rpc.laddr`).
+
+Some fields from the config file can be overwritten with flags.
+
+## No Empty Blocks
+
+This much requested feature was implemented in version 0.10.3. While the
+default behaviour of `tendermint` is still to create blocks
+approximately once per second, it is possible to disable empty blocks or
+set a block creation interval. In the former case, blocks will be
+created when there are new transactions or when the AppHash changes.
+
+To configure Tendermint to not produce empty blocks unless there are
+transactions or the app hash changes, run Tendermint with this
+additional flag:
+
+    tendermint node --consensus.create_empty_blocks=false
+
+or set the configuration via the `config.toml` file:
+
+    [consensus]
+    create_empty_blocks = false
+
+Remember: because the default is to *create empty blocks*, avoiding
+empty blocks requires the config option to be set to `false`.
+
+The block interval setting allows for a delay (in seconds) between the
+creation of each new empty block. It is set via the `config.toml`:
+
+    [consensus]
+    create_empty_blocks_interval = 5
+
+With this setting, empty blocks will be produced every 5s if no block
+has been produced otherwise, regardless of the value of
+`create_empty_blocks`.
+
+## Broadcast API
+
+Earlier, we used the `broadcast_tx_commit` endpoint to send a
+transaction. When a transaction is sent to a Tendermint node, it will
+run via `CheckTx` against the application. If it passes `CheckTx`, it
+will be included in the mempool, broadcasted to other peers, and
+eventually included in a block.
+
+Since there are multiple phases to processing a transaction, we offer
+multiple endpoints to broadcast a transaction:
+
+    /broadcast_tx_async
+    /broadcast_tx_sync
+    /broadcast_tx_commit
+
+These correspond to no-processing, processing through the mempool, and
+processing through a block, respectively. That is, `broadcast_tx_async`,
+will return right away without waiting to hear if the transaction is
+even valid, while `broadcast_tx_sync` will return with the result of
+running the transaction through `CheckTx`. Using `broadcast_tx_commit`
+will wait until the transaction is committed in a block or until some
+timeout is reached, but will return right away if the transaction does
+not pass `CheckTx`. The return value for `broadcast_tx_commit` includes
+two fields, `check_tx` and `deliver_tx`, pertaining to the result of
+running the transaction through those ABCI messages.
+
+The benefit of using `broadcast_tx_commit` is that the request returns
+after the transaction is committed (i.e. included in a block), but that
+can take on the order of a second. For a quick result, use
+`broadcast_tx_sync`, but the transaction will not be committed until
+later, and by that point its effect on the state may change.
+
+## Tendermint Networks
+
+When `tendermint init` is run, both a `genesis.json` and
+`priv_validator.json` are created in `~/.tendermint/config`. The
+`genesis.json` might look like:
+
+    {
+      "validators" : [
+        {
+          "pub_key" : {
+            "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
+            "type" : "AC26791624DE60"
+          },
+          "power" : 10,
+          "name" : ""
+        }
+      ],
+      "app_hash" : "",
+      "chain_id" : "test-chain-rDlYSN",
+      "genesis_time" : "0001-01-01T00:00:00Z"
+    }
+
+And the `priv_validator.json`:
+
+    {
+      "last_step" : 0,
+      "last_round" : 0,
+      "address" : "B788DEDE4F50AD8BC9462DE76741CCAFF87D51E2",
+      "pub_key" : {
+        "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
+        "type" : "AC26791624DE60"
+      },
+      "last_height" : 0,
+      "priv_key" : {
+        "value" : "JPivl82x+LfVkp8i3ztoTjY6c6GJ4pBxQexErOCyhwqHeGT5ATxzpAtPJKnxNx/NyUnD8Ebv3OIYH+kgD4N88Q==",
+        "type" : "954568A3288910"
+      }
+    }
+
+The `priv_validator.json` actually contains a private key, and should
+thus be kept absolutely secret; for now we work with the plain text.
+Note the `last_` fields, which are used to prevent us from signing
+conflicting messages.
+
+Note also that the `pub_key` (the public key) in the
+`priv_validator.json` is also present in the `genesis.json`.
+
+The genesis file contains the list of public keys which may participate
+in the consensus, and their corresponding voting power. Greater than 2/3
+of the voting power must be active (i.e. the corresponding private keys
+must be producing signatures) for the consensus to make progress. In our
+case, the genesis file contains the public key of our
+`priv_validator.json`, so a Tendermint node started with the default
+root directory will be able to make progress. Voting power uses an int64
+but must be positive, thus the range is: 0 through 9223372036854775807.
+Because of how the current proposer selection algorithm works, we do not
+recommend having voting powers greater than 10\^12 (ie. 1 trillion) (see
+[Proposals section of Byzantine Consensus
+Algorithm](./specification/byzantine-consensus-algorithm.html#proposals)
+for details).
+
+If we want to add more nodes to the network, we have two choices: we can
+add a new validator node, who will also participate in the consensus by
+proposing blocks and voting on them, or we can add a new non-validator
+node, who will not participate directly, but will verify and keep up
+with the consensus protocol.
+
+### Peers
+
+#### Seed
+A seed node is a node who relays the addresses of other peers which they know
+of. These nodes constantly crawl the network to try to get more peers. The
+addresses which the seed node relays get saved into a local address book. Once
+these are in the address book, you will connect to those addresses directly.
+Basically the seed nodes job is just to relay everyones addresses. You won't
+connect to seed nodes once you have received enough addresses, so typically you
+only need them on the first start. The seed node will immediately disconnect
+from you after sending you some addresses.
+
+#### Persistent Peer
+Persistent peers are people you want to be constantly connected with. If you
+disconnect you will try to connect directly back to them as opposed to using
+another address from the address book. On restarts you will always try to
+connect to these peers regardless of the size of your address book.
+
+All peers relay peers they know of by default. This is called the peer exchange
+protocol (PeX). With PeX, peers will be gossipping about known peers and forming
+a network, storing peer addresses in the addrbook. Because of this, you don't
+have to use a seed node if you have a live persistent peer.
+
+#### Connecting to Peers
+
+To connect to peers on start-up, specify them in the
+`$TMHOME/config/config.toml` or on the command line. Use `seeds` to
+specify seed nodes, and
+`persistent_peers` to specify peers that your node will maintain
+persistent connections with.
+
+For example,
+
+    tendermint node --p2p.seeds "f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656,0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:26656"
+
+Alternatively, you can use the `/dial_seeds` endpoint of the RPC to
+specify seeds for a running node to connect to:
+
+    curl 'localhost:26657/dial_seeds?seeds=\["f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656","0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:26656"\]'
+
+Note, with PeX enabled, you
+should not need seeds after the first start.
+
+If you want Tendermint to connect to specific set of addresses and
+maintain a persistent connection with each, you can use the
+`--p2p.persistent_peers` flag or the corresponding setting in the
+`config.toml` or the `/dial_peers` RPC endpoint to do it without
+stopping Tendermint core instance.
+
+    tendermint node --p2p.persistent_peers "429fcf25974313b95673f58d77eacdd434402665@10.11.12.13:26656,96663a3dd0d7b9d17d4c8211b191af259621c693@10.11.12.14:26656"
+    curl 'localhost:26657/dial_peers?persistent=true&peers=\["429fcf25974313b95673f58d77eacdd434402665@10.11.12.13:26656","96663a3dd0d7b9d17d4c8211b191af259621c693@10.11.12.14:26656"\]'
+
+### Adding a Non-Validator
+
+Adding a non-validator is simple. Just copy the original `genesis.json`
+to `~/.tendermint/config` on the new machine and start the node,
+specifying seeds or persistent peers as necessary. If no seeds or
+persistent peers are specified, the node won't make any blocks, because
+it's not a validator, and it won't hear about any blocks, because it's
+not connected to the other peer.
+
+### Adding a Validator
+
+The easiest way to add new validators is to do it in the `genesis.json`,
+before starting the network. For instance, we could make a new
+`priv_validator.json`, and copy it's `pub_key` into the above genesis.
+
+We can generate a new `priv_validator.json` with the command:
+
+    tendermint gen_validator
+
+Now we can update our genesis file. For instance, if the new
+`priv_validator.json` looks like:
+
+    {
+      "address" : "5AF49D2A2D4F5AD4C7C8C4CC2FB020131E9C4902",
+      "pub_key" : {
+        "value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=",
+        "type" : "AC26791624DE60"
+      },
+      "priv_key" : {
+        "value" : "EDJY9W6zlAw+su6ITgTKg2nTZcHAH1NMTW5iwlgmNDuX1f35+OR4HMN88ZtQzsAwhETq4k3vzM3n6WTk5ii16Q==",
+        "type" : "954568A3288910"
+      },
+      "last_step" : 0,
+      "last_round" : 0,
+      "last_height" : 0
+    }
+
+then the new `genesis.json` will be:
+
+    {
+      "validators" : [
+        {
+          "pub_key" : {
+            "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
+            "type" : "AC26791624DE60"
+          },
+          "power" : 10,
+          "name" : ""
+        },
+        {
+          "pub_key" : {
+            "value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=",
+            "type" : "AC26791624DE60"
+          },
+          "power" : 10,
+          "name" : ""
+        }
+      ],
+      "app_hash" : "",
+      "chain_id" : "test-chain-rDlYSN",
+      "genesis_time" : "0001-01-01T00:00:00Z"
+    }
+
+Update the `genesis.json` in `~/.tendermint/config`. Copy the genesis
+file and the new `priv_validator.json` to the `~/.tendermint/config` on
+a new machine.
+
+Now run `tendermint node` on both machines, and use either
+`--p2p.persistent_peers` or the `/dial_peers` to get them to peer up.
+They should start making blocks, and will only continue to do so as long
+as both of them are online.
+
+To make a Tendermint network that can tolerate one of the validators
+failing, you need at least four validator nodes (e.g., 2/3).
+
+Updating validators in a live network is supported but must be
+explicitly programmed by the application developer. See the [application
+developers guide](./app-development.html) for more details.
+
+### Local Network
+
+To run a network locally, say on a single machine, you must change the
+`_laddr` fields in the `config.toml` (or using the flags) so that the
+listening addresses of the various sockets don't conflict. Additionally,
+you must set `addrbook_strict=false` in the `config.toml`, otherwise
+Tendermint's p2p library will deny making connections to peers with the
+same IP address.
+
+### Upgrading
+
+The Tendermint development cycle currently includes a lot of breaking changes.
+Upgrading from an old version to a new version usually means throwing
+away the chain data. Try out the
+[tm-migrate](https://github.com/hxzqlh/tm-tools) tool written by
+[@hxzqlh](https://github.com/hxzqlh) if you are keen to preserve the
+state of your chain when upgrading to newer versions.
--- a/docs/using-tendermint.rst
+++ b/docs/using-tendermint.rst
@@ -1,463 +0,0 @@
-Using Tendermint
-================
-
-This is a guide to using the ``tendermint`` program from the command
-line. It assumes only that you have the ``tendermint`` binary installed
-and have some rudimentary idea of what Tendermint and ABCI are.
-
-You can see the help menu with ``tendermint --help``, and the version
-number with ``tendermint version``.
-
-Directory Root
--------------
-
-The default directory for blockchain data is ``~/.tendermint``. Override
-this by setting the ``TMHOME`` environment variable.
-
-Initialize
----------
-
-Initialize the root directory by running:
-
-::
-
-    tendermint init
-
-This will create a new private key (``priv_validator.json``), and a
-genesis file (``genesis.json``) containing the associated public key,
-in ``$TMHOME/config``.
-This is all that's necessary to run a local testnet with one validator.
-
-For more elaborate initialization, see the `tesnet` command:
-
-::
-
-    tendermint testnet --help
-
-Run
---
-
-To run a Tendermint node, use
-
-::
-
-    tendermint node
-
-By default, Tendermint will try to connect to an ABCI application on
-`127.0.0.1:46658 <127.0.0.1:46658>`__. If you have the ``kvstore`` ABCI
-app installed, run it in another window. If you don't, kill Tendermint
-and run an in-process version of the ``kvstore`` app:
-
-::
-
-    tendermint node --proxy_app=kvstore
-
-After a few seconds you should see blocks start streaming in. Note that
-blocks are produced regularly, even if there are no transactions. See *No Empty Blocks*, below, to modify this setting.
-
-Tendermint supports in-process versions of the ``counter``, ``kvstore`` and ``nil``
-apps that ship as examples in the `ABCI
-repository <https://github.com/tendermint/abci>`__. It's easy to compile
-your own app in-process with Tendermint if it's written in Go. If your
-app is not written in Go, simply run it in another process, and use the
-``--proxy_app`` flag to specify the address of the socket it is
-listening on, for instance:
-
-::
-
-    tendermint node --proxy_app=/var/run/abci.sock
-
-Transactions
------------
-
-To send a transaction, use ``curl`` to make requests to the Tendermint
-RPC server, for example:
-
-::
-
-    curl http://localhost:46657/broadcast_tx_commit?tx=\"abcd\"
-
-We can see the chain's status at the ``/status`` end-point:
-
-::
-
-    curl http://localhost:46657/status | json_pp
-
-and the ``latest_app_hash`` in particular:
-
-::
-
-    curl http://localhost:46657/status | json_pp | grep latest_app_hash
-
-Visit http://localhost:46657 in your browser to see the list of other
-endpoints. Some take no arguments (like ``/status``), while others
-specify the argument name and use ``_`` as a placeholder.
-
-Formatting
-~~~~~~~~~~
-
-The following nuances when sending/formatting transactions should
-be taken into account:
-
-With ``GET``:
-
-To send a UTF8 string byte array, quote the value of the tx pramater:
-
-::
-
-    curl 'http://localhost:46657/broadcast_tx_commit?tx="hello"'
-
-which sends a 5 byte transaction: "h e l l o" [68 65 6c 6c 6f].
-
-Note the URL must be wrapped with single quoes, else bash will ignore the double quotes.
-To avoid the single quotes, escape the double quotes:
-
-::
-
-    curl http://localhost:46657/broadcast_tx_commit?tx=\"hello\"
-
-
-
-Using a special character:
-
-::
-
-    curl 'http://localhost:46657/broadcast_tx_commit?tx="€5"'
-
-sends a 4 byte transaction: "€5" (UTF8) [e2 82 ac 35].
-
-To send as raw hex, omit quotes AND prefix the hex string with ``0x``:
-
-::
-
-    curl http://localhost:46657/broadcast_tx_commit?tx=0x01020304
-
-which sends a 4 byte transaction: [01 02 03 04].
-
-With ``POST`` (using ``json``), the raw hex must be ``base64`` encoded:
-
-::
-
-    curl --data-binary '{"jsonrpc":"2.0","id":"anything","method":"broadcast_tx_commit","params": {"tx": "AQIDBA=="}}' -H 'content-type:text/plain;' http://localhost:46657
-
-which sends the same 4 byte transaction: [01 02 03 04].
-
-Note that raw hex cannot be used in ``POST`` transactions.
-
-Reset
-----
-
-**WARNING: UNSAFE** Only do this in development and only if you can
-afford to lose all blockchain data!
-
-To reset a blockchain, stop the node, remove the ``~/.tendermint/data``
-directory and run
-
-::
-
-    tendermint unsafe_reset_priv_validator
-
-This final step is necessary to reset the ``priv_validator.json``, which
-otherwise prevents you from making conflicting votes in the consensus
-(something that could get you in trouble if you do it on a real
-blockchain). If you don't reset the ``priv_validator.json``, your fresh
-new blockchain will not make any blocks.
-
-Configuration
-------------
-
-Tendermint uses a ``config.toml`` for configuration. For details, see
-`the config specification <./specification/configuration.html>`__.
-
-Notable options include the socket address of the application
-(``proxy_app``), the listening address of the Tendermint peer
-(``p2p.laddr``), and the listening address of the RPC server
-(``rpc.laddr``).
-
-Some fields from the config file can be overwritten with flags.
-
-No Empty Blocks
---------------
-
-This much requested feature was implemented in version 0.10.3. While the
-default behaviour of ``tendermint`` is still to create blocks approximately
-once per second, it is possible to disable empty blocks or set a block creation
-interval. In the former case, blocks will be created when there are new
-transactions or when the AppHash changes.
-
-To configure Tendermint to not produce empty blocks unless there are
-transactions or the app hash changes, run Tendermint with this additional flag:
-
-::
-
-    tendermint node --consensus.create_empty_blocks=false
-
-or set the configuration via the ``config.toml`` file:
-
-::
-
-    [consensus]
-    create_empty_blocks = false
-
-Remember: because the default is to *create empty blocks*, avoiding empty blocks requires the config option to be set to ``false``.
-
-The block interval setting allows for a delay (in seconds) between the creation of each new empty block. It is set via the ``config.toml``:
-
-::
-
-    [consensus]
-    create_empty_blocks_interval = 5
-
-With this setting, empty blocks will be produced every 5s if no block has been produced otherwise,
-regardless of the value of ``create_empty_blocks``.
-
-Broadcast API
-------------
-
-Earlier, we used the ``broadcast_tx_commit`` endpoint to send a
-transaction. When a transaction is sent to a Tendermint node, it will
-run via ``CheckTx`` against the application. If it passes ``CheckTx``,
-it will be included in the mempool, broadcasted to other peers, and
-eventually included in a block.
-
-Since there are multiple phases to processing a transaction, we offer
-multiple endpoints to broadcast a transaction:
-
-::
-
-    /broadcast_tx_async
-    /broadcast_tx_sync
-    /broadcast_tx_commit
-
-These correspond to no-processing, processing through the mempool, and
-processing through a block, respectively. That is,
-``broadcast_tx_async``, will return right away without waiting to hear
-if the transaction is even valid, while ``broadcast_tx_sync`` will
-return with the result of running the transaction through ``CheckTx``.
-Using ``broadcast_tx_commit`` will wait until the transaction is
-committed in a block or until some timeout is reached, but will return
-right away if the transaction does not pass ``CheckTx``. The return
-value for ``broadcast_tx_commit`` includes two fields, ``check_tx`` and
-``deliver_tx``, pertaining to the result of running the transaction
-through those ABCI messages.
-
-The benefit of using ``broadcast_tx_commit`` is that the request returns
-after the transaction is committed (i.e. included in a block), but that
-can take on the order of a second. For a quick result, use
-``broadcast_tx_sync``, but the transaction will not be committed until
-later, and by that point its effect on the state may change.
-
-Note: see the Transactions => Formatting section for details about
-transaction formating.
-
-Tendermint Networks
-------------------
-
-When ``tendermint init`` is run, both a ``genesis.json`` and
-``priv_validator.json`` are created in ``~/.tendermint/config``. The
-``genesis.json`` might look like:
-
-::
-
-    {
-      "validators" : [
-        {
-          "pub_key" : {
-            "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
-            "type" : "AC26791624DE60"
-          },
-          "power" : 10,
-          "name" : ""
-        }
-      ],
-      "app_hash" : "",
-      "chain_id" : "test-chain-rDlYSN",
-      "genesis_time" : "0001-01-01T00:00:00Z"
-    }
-
-And the ``priv_validator.json``:
-
-::
-
-    {
-      "last_step" : 0,
-      "last_round" : 0,
-      "address" : "B788DEDE4F50AD8BC9462DE76741CCAFF87D51E2",
-      "pub_key" : {
-        "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
-        "type" : "AC26791624DE60"
-      },
-      "last_height" : 0,
-      "priv_key" : {
-        "value" : "JPivl82x+LfVkp8i3ztoTjY6c6GJ4pBxQexErOCyhwqHeGT5ATxzpAtPJKnxNx/NyUnD8Ebv3OIYH+kgD4N88Q==",
-        "type" : "954568A3288910"
-      }
-    }
-
-The ``priv_validator.json`` actually contains a private key, and should
-thus be kept absolutely secret; for now we work with the plain text.
-Note the ``last_`` fields, which are used to prevent us from signing
-conflicting messages.
-
-Note also that the ``pub_key`` (the public key) in the
-``priv_validator.json`` is also present in the ``genesis.json``.
-
-The genesis file contains the list of public keys which may participate in the
-consensus, and their corresponding voting power. Greater than 2/3 of the voting
-power must be active (i.e. the corresponding private keys must be producing
-signatures) for the consensus to make progress. In our case, the genesis file
-contains the public key of our ``priv_validator.json``, so a Tendermint node
-started with the default root directory will be able to make progress. Voting
-power uses an `int64` but must be positive, thus the range is: 0 through
-9223372036854775807. Because of how the current proposer selection algorithm works,
-we do not recommend having voting powers greater than 10^12 (ie. 1 trillion)
-(see `Proposals section of Byzantine Consensus Algorithm
-<./specification/byzantine-consensus-algorithm.html#proposals>`__ for details).
-
-If we want to add more nodes to the network, we have two choices: we can
-add a new validator node, who will also participate in the consensus by
-proposing blocks and voting on them, or we can add a new non-validator
-node, who will not participate directly, but will verify and keep up
-with the consensus protocol.
-
-Peers
-~~~~~
-
-To connect to peers on start-up, specify them in the ``$TMHOME/config/config.toml`` or
-on the command line. Use `seeds` to specify seed nodes from which you can get many other
-peer addresses, and ``persistent_peers`` to specify peers that your node will maintain
-persistent connections with.
-
-For instance,
-
-::
-
-    tendermint node --p2p.seeds "f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:46656,0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:46656"
-
-Alternatively, you can use the ``/dial_seeds`` endpoint of the RPC to
-specify seeds for a running node to connect to:
-
-::
-
-    curl 'localhost:46657/dial_seeds?seeds=\["f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:46656","0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:46656"\]'
-
-Note, if the peer-exchange protocol (PEX) is enabled (default), you should not
-normally need seeds after the first start. Peers will be gossipping about known
-peers and forming a network, storing peer addresses in the addrbook.
-
-If you want Tendermint to connect to specific set of addresses and maintain a
-persistent connection with each, you can use the ``--p2p.persistent_peers``
-flag or the corresponding setting in the ``config.toml`` or the
-``/dial_peers`` RPC endpoint to do it without stopping Tendermint
-core instance.
-
-::
-
-    tendermint node --p2p.persistent_peers "429fcf25974313b95673f58d77eacdd434402665@10.11.12.13:46656,96663a3dd0d7b9d17d4c8211b191af259621c693@10.11.12.14:46656"
-    curl 'localhost:46657/dial_peers?persistent=true&peers=\["429fcf25974313b95673f58d77eacdd434402665@10.11.12.13:46656","96663a3dd0d7b9d17d4c8211b191af259621c693@10.11.12.14:46656"\]'
-
-Adding a Non-Validator
-~~~~~~~~~~~~~~~~~~~~~~
-
-Adding a non-validator is simple. Just copy the original
-``genesis.json`` to ``~/.tendermint/config`` on the new machine and start the
-node, specifying seeds or persistent peers as necessary. If no seeds or persistent
-peers are specified, the node won't make any blocks, because it's not a validator,
-and it won't hear about any blocks, because it's not connected to the other peer.
-
-Adding a Validator
-~~~~~~~~~~~~~~~~~~
-
-The easiest way to add new validators is to do it in the
-``genesis.json``, before starting the network. For instance, we could
-make a new ``priv_validator.json``, and copy it's ``pub_key`` into the
-above genesis.
-
-We can generate a new ``priv_validator.json`` with the command:
-
-::
-
-    tendermint gen_validator
-
-Now we can update our genesis file. For instance, if the new
-``priv_validator.json`` looks like:
-
-::
-
-    {
-      "address" : "5AF49D2A2D4F5AD4C7C8C4CC2FB020131E9C4902",
-      "pub_key" : {
-        "value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=",
-        "type" : "AC26791624DE60"
-      },
-      "priv_key" : {
-        "value" : "EDJY9W6zlAw+su6ITgTKg2nTZcHAH1NMTW5iwlgmNDuX1f35+OR4HMN88ZtQzsAwhETq4k3vzM3n6WTk5ii16Q==",
-        "type" : "954568A3288910"
-      },
-      "last_step" : 0,
-      "last_round" : 0,
-      "last_height" : 0
-    }
-
-then the new ``genesis.json`` will be:
-
-::
-
-    {
-      "validators" : [
-        {
-          "pub_key" : {
-            "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=",
-            "type" : "AC26791624DE60"
-          },
-          "power" : 10,
-          "name" : ""
-        },
-        {
-          "pub_key" : {
-            "value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=",
-            "type" : "AC26791624DE60"
-          },
-          "power" : 10,
-          "name" : ""
-        }
-      ],
-      "app_hash" : "",
-      "chain_id" : "test-chain-rDlYSN",
-      "genesis_time" : "0001-01-01T00:00:00Z"
-    }
-
-Update the ``genesis.json`` in ``~/.tendermint/config``. Copy the genesis file
-and the new ``priv_validator.json`` to the ``~/.tendermint/config`` on a new
-machine.
-
-Now run ``tendermint node`` on both machines, and use either
-``--p2p.persistent_peers`` or the ``/dial_peers`` to get them to peer up. They
-should start making blocks, and will only continue to do so as long as
-both of them are online.
-
-To make a Tendermint network that can tolerate one of the validators
-failing, you need at least four validator nodes (> 2/3).
-
-Updating validators in a live network is supported but must be
-explicitly programmed by the application developer. See the `application
-developers guide <./app-development.html>`__ for more
-details.
-
-Local Network
-~~~~~~~~~~~~~
-
-To run a network locally, say on a single machine, you must change the
-``_laddr`` fields in the ``config.toml`` (or using the flags) so that
-the listening addresses of the various sockets don't conflict.
-Additionally, you must set ``addrbook_strict=false`` in the
-``config.toml``, otherwise Tendermint's p2p library will deny making
-connections to peers with the same IP address.
-
-Upgrading
-~~~~~~~~~
-
-The Tendermint development cycle includes a lot of breaking changes. Upgrading from
-an old version to a new version usually means throwing away the chain data. Try out
-the `tm-migrate <https://github.com/hxzqlh/tm-tools>`__ tool written by `@hxzqlh <https://github.com/hxzqlh>`__  if
-you are keen to preserve the state of your chain when upgrading to newer versions.
--- a/Show More
+++ b/Show More