From e2f0778c14ada5fd89c17ab9c40359961ae50554 Mon Sep 17 00:00:00 2001
From: Ethan Buchman <ethan@coinculture.info>
Date: Thu, 26 Apr 2018 09:09:56 -0400
Subject: [PATCH 1/6] spec: update encoding.md

---
 docs/specification/new-spec/encoding.md | 212 +++++++++++-------------
 docs/specification/new-spec/wire.go     |  80 ---------
 2 files changed, 99 insertions(+), 193 deletions(-)
 delete mode 100644 docs/specification/new-spec/wire.go

diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md
index e0317b7e..b7dd47e8 100644
--- a/docs/specification/new-spec/encoding.md
+++ b/docs/specification/new-spec/encoding.md
@@ -1,101 +1,119 @@
 # Tendermint Encoding
 
-## Binary Serialization (TMBIN)
+## Amino
 
-Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs
-are laid out in memory.
-Variable length items are length-prefixed.
-While the encoding was inspired by Go, it is easily implemented in other languages as well, given its intuitive design.
+Tendermint uses the Protobuf3 derrivative [Amino]() for all data structures.
+Thik of Amino as an object-oriented Protobuf3 with native JSON support.
+The goal of the Amino encoding protocol is to bring parity between application
+logic objects and persistence objects.
 
-XXX: This is changing to use real varints and 4-byte-prefixes.
-See https://github.com/tendermint/go-wire/tree/sdk2.
+Please see the [Amino
+specification](https://github.com/tendermint/go-amino#amino-encoding-for-go) for
+more details.
 
-### Fixed Length Integers
+Notably, every object that satisfies an interface (eg. a particular kind of p2p message,
+or a particular kind of pubkey) is registered with a global name, the hash of
+which is included in the object's encoding as the so-called "prefix bytes".
 
-Fixed length integers are encoded in Big-Endian using the specified number of bytes.
-So `uint8` and `int8` use one byte, `uint16` and `int16` use two bytes,
-`uint32` and `int32` use 3 bytes, and `uint64` and `int64` use 4 bytes.
+## Public Key Cryptography
 
-Negative integers are encoded via twos-complement.
+Tendermint uses Amino to distinguish between different types of private keys,
+public keys, and signatures. Additionally, for each public key, Tendermint
+defines an Address function that can be used as a more compact identifier in
+place of the public key. Here we list the concrete types, their names,
+and prefix bytes for public keys and signatures. Note for brevity we don't
+include details of the private keys beyond their type and name, as they can be
+derrived the same way as the others using Amino.
 
-Examples:
+All registered objects are encoded by Amino using a 4-byte PrefixBytes that
+uniquely identifies the object and includes information about its underlying
+type. For details on how PrefixBytes are computed, see the [Amino
+spec](https://github.com/tendermint/go-amino#computing-the-prefix-and-disambiguation-bytes).
 
-```go
-encode(uint8(6))    == [0x06]
-encode(uint32(6))   == [0x00, 0x00, 0x00, 0x06]
+In what follows, we provide the type names and prefix bytes directly.
+Notice that when encoding byte-arrays, the length of the byte-array is appended
+to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes>
+<Length> <ByteArray>`
 
-encode(int8(-6))    == [0xFA]
-encode(int32(-6))   == [0xFF, 0xFF, 0xFF, 0xFA]
+### PubKeyEd25519
+
+```
+// Name: tendermint/PubKeyEd25519
+// PrefixBytes: 0x1624DE62
+// Length: 0x20
+// Notes: raw 32-byte Ed25519 pubkey
+type PubKeyEd25519 [32]byte
 ```
 
-### Variable Length Integers
+For example, the 32-byte Ed25519 pubkey
+`76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
+encoded as
+`1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`
 
-Variable length integers are encoded as length-prefixed Big-Endian integers.
-The length-prefix consists of a single byte and corresponds to the length of the encoded integer.
+### SignatureEd25519
 
-Negative integers are encoded by flipping the leading bit of the length-prefix to a `1`.
-
-Zero is encoded as `0x00`. It is not length-prefixed.
-
-Examples:
-
-```go
-encode(uint(6))     == [0x01, 0x06]
-encode(uint(70000)) == [0x03, 0x01, 0x11, 0x70]
-
-encode(int(-6))     == [0xF1, 0x06]
-encode(int(-70000)) == [0xF3, 0x01, 0x11, 0x70]
-
-encode(int(0))      == [0x00]
+```
+// Name: tendermint/SignatureKeyEd25519
+// PrefixBytes: 0x3DA1DB2A
+// Length: 0x40
+// Notes: raw 64-byte Ed25519 signature
+type SignatureEd25519 [64]byte
 ```
 
-### Strings
+For example, the 64-byte Ed25519 signature
+`005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F`
+would be encoded as
+`3DA1DB2A40005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F`
 
-An encoded string is length-prefixed followed by the underlying bytes of the string.
-The length-prefix is itself encoded as an `int`.
+### PrivKeyEd25519
 
-The empty string is encoded as `0x00`. It is not length-prefixed.
+```
+// Name: tendermint/PrivKeyEd25519
+// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key
+type PrivKeyEd25519 [64]byte
+```
+### PubKeySecp256k1
 
-Examples:
-
-```go
-encode("")      == [0x00]
-encode("a")     == [0x01, 0x01, 0x61]
-encode("hello") == [0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F]
-encode("¥")     == [0x01, 0x02, 0xC2, 0xA5]
+```
+// Name: tendermint/PubKeySecp256k1
+// PrefixBytes: 0xEB5AE982
+// Length: 0x21
+// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03
+type PubKeySecp256k1 [33]byte
 ```
 
-### Arrays (fixed length)
+For example, the 33-byte Secp256k1 pubkey
+`03573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380` would be
+encoded as
+`EB5AE9822103573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380`
 
-An encoded fix-lengthed array is the concatenation of the encoding of its elements.
-There is no length-prefix.
+### SignatureSecp256k1
 
-Examples:
-
-```go
-encode([4]int8{1, 2, 3, 4})     == [0x01, 0x02, 0x03, 0x04]
-encode([4]int16{1, 2, 3, 4})    == [0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
-encode([4]int{1, 2, 3, 4})      == [0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x04]
-encode([2]string{"abc", "efg"}) == [0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
+```
+// Name: tendermint/SignatureKeySecp256k1
+// PrefixBytes: 0x16E1FEEA
+// Length: Variable
+// Encoding prefix: Variable
+// Notes: raw bytes of the Secp256k1 signature
+type SignatureSecp256k1 []byte
 ```
 
-### Slices (variable length)
+For example, the Secp256k1 signature
+`304402207447640A5C12A72BAA052D110B666FB6DF717A7B863361C092E751D016C6C08802205C20F9DEBF8915DED310B98BFA890105F43925FDB2B67B78510FE18EDA2B30DA` would
+be encoded as
+`16E1FEEA46304402202C10C874E413AF538D97EBEF2B01024719F8B7CC559CEEBDC7C380F9DCC4A6E002200EDE9B62F8531933F88DB2A62E73BA3D43ACEB1CBD23070C2F792AAA18717A4A`
 
-An encoded variable-length array is length-prefixed followed by the concatenation of the encoding of
-its elements.
-The length-prefix is itself encoded as an `int`.
+### PrivKeySecp256k1
 
-An empty slice is encoded as `0x00`. It is not length-prefixed.
-
-Examples:
-
-```go
-encode([]int8{})                == [0x00]
-encode([]int8{1, 2, 3, 4})      == [0x01, 0x04, 0x01, 0x02, 0x03, 0x04]
-encode([]int16{1, 2, 3, 4})     == [0x01, 0x04, 0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04]
-encode([]int{1, 2, 3, 4})       == [0x01, 0x04, 0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x4]
-encode([]string{"abc", "efg"})  == [0x01, 0x02, 0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
 ```
+// Name: tendermint/PrivKeySecp256k1
+// Notes: raw 32-byte priv key
+type PrivKeySecp256k1 [32]byte
+```
+
+
+
+## Other Common Types
 
 ### BitArray
 
@@ -109,36 +127,22 @@ type BitArray struct {
 }
 ```
 
-### Time
-
-Time is encoded as an `int64` of the number of nanoseconds since January 1, 1970,
-rounded to the nearest millisecond.
-
-Times before then are invalid.
-
-Examples:
+### Part
 
 ```go
-encode(time.Time("Jan 1 00:00:00 UTC 1970"))            == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
-encode(time.Time("Jan 1 00:00:01 UTC 1970"))            == [0x00, 0x00, 0x00, 0x00, 0x3B, 0x9A, 0xCA, 0x00] // 1,000,000,000 ns
-encode(time.Time("Mon Jan 2 15:04:05 -0700 MST 2006"))  == [0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
+type Part struct {
+    Index int
+    Bytes byte[]
+    Proof byte[]
+}
 ```
 
-### Structs
+### MakeParts
 
-An encoded struct is the concatenation of the encoding of its elements.
-There is no length-prefix.
-
-Examples:
+Encode an object using Amino and slice it into parts.
 
 ```go
-type MyStruct struct{
-    A int
-    B string
-    C time.Time
-}
-encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) ==
-    [0x01, 0x04, 0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F, 0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00]
+MakeParts(object, partSize)
 ```
 
 ## Merkle Trees
@@ -169,10 +173,9 @@ For `struct` arguments, we compute a `[][]byte` by sorting elements of the `stru
 field name and then hashing them.
 For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements.
 
-## JSON (TMJSON)
+## AminoJSON
 
-Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN.
-TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64.
+Signed messages (eg. votes, proposals) in the consensus are encoded in AminoJSON, rather than binary Amino.
 
 When signing, the elements of a message are sorted by key and the sorted message is embedded in an
 outer JSON that includes a `chain_id` field.
@@ -185,22 +188,5 @@ like:
 
 Note how the fields within each level are sorted.
 
-## Other
 
-### MakeParts
 
-Encode an object using TMBIN and slice it into parts.
-
-```go
-MakeParts(object, partSize)
-```
-
-### Part
-
-```go
-type Part struct {
-    Index int
-    Bytes byte[]
-    Proof byte[]
-}
-```
diff --git a/docs/specification/new-spec/wire.go b/docs/specification/new-spec/wire.go
deleted file mode 100644
index af76f366..00000000
--- a/docs/specification/new-spec/wire.go
+++ /dev/null
@@ -1,80 +0,0 @@
-package main
-
-import (
-	"fmt"
-	"time"
-
-	wire "github.com/tendermint/go-wire"
-)
-
-func main() {
-
-	encode(uint8(6))
-	encode(uint32(6))
-	encode(int8(-6))
-	encode(int32(-6))
-	Break()
-	encode(uint(6))
-	encode(uint(70000))
-	encode(int(0))
-	encode(int(-6))
-	encode(int(-70000))
-	Break()
-	encode("")
-	encode("a")
-	encode("hello")
-	encode("¥")
-	Break()
-	encode([4]int8{1, 2, 3, 4})
-	encode([4]int16{1, 2, 3, 4})
-	encode([4]int{1, 2, 3, 4})
-	encode([2]string{"abc", "efg"})
-	Break()
-	encode([]int8{})
-	encode([]int8{1, 2, 3, 4})
-	encode([]int16{1, 2, 3, 4})
-	encode([]int{1, 2, 3, 4})
-	encode([]string{"abc", "efg"})
-	Break()
-
-	timeFmt := "Mon Jan 2 15:04:05 -0700 MST 2006"
-	t1, _ := time.Parse(timeFmt, timeFmt)
-	n := (t1.UnixNano() / 1000000.) * 1000000
-	encode(n)
-	encode(t1)
-
-	t2, _ := time.Parse(timeFmt, "Thu Jan 1 00:00:00 -0000 UTC 1970")
-	encode(t2)
-
-	t2, _ = time.Parse(timeFmt, "Thu Jan 1 00:00:01 -0000 UTC 1970")
-	fmt.Println("N", t2.UnixNano())
-	encode(t2)
-	Break()
-	encode(struct {
-		A int
-		B string
-		C time.Time
-	}{
-		4,
-		"hello",
-		t1,
-	})
-}
-
-func encode(i interface{}) {
-	Println(wire.BinaryBytes(i))
-
-}
-
-func Println(b []byte) {
-	s := "["
-	for _, x := range b {
-		s += fmt.Sprintf("0x%.2X, ", x)
-	}
-	s = s[:len(s)-2] + "]"
-	fmt.Println(s)
-}
-
-func Break() {
-	fmt.Println("------")
-}

From 91c81ef9a1139eef5affa750e48027bb78207bc3 Mon Sep 17 00:00:00 2001
From: Ethan Buchman <ethan@coinculture.info>
Date: Thu, 26 Apr 2018 10:42:58 -0400
Subject: [PATCH 2/6] spec: note on byte arrays, clean up bitarrays and more,
 add merkle proof, add crypto.go script

---
 docs/specification/new-spec/encoding.md       |  91 ++++++++++++++-
 docs/specification/new-spec/scripts/crypto.go | 108 ++++++++++++++++++
 2 files changed, 193 insertions(+), 6 deletions(-)
 create mode 100644 docs/specification/new-spec/scripts/crypto.go

diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md
index b7dd47e8..de747892 100644
--- a/docs/specification/new-spec/encoding.md
+++ b/docs/specification/new-spec/encoding.md
@@ -15,6 +15,18 @@ Notably, every object that satisfies an interface (eg. a particular kind of p2p
 or a particular kind of pubkey) is registered with a global name, the hash of
 which is included in the object's encoding as the so-called "prefix bytes".
 
+## Byte Arrays
+
+The encoding of a byte array is simply the raw-bytes prefixed with the length of
+the array as a `UVarint` (what Protobuf calls a `Varint`).
+
+For details on varints, see the [protobuf
+spec](https://developers.google.com/protocol-buffers/docs/encoding#varints).
+
+For example, the byte-array `[0xA, 0xB]` would be encoded as `0x020A0B`,
+while a byte-array containing 300 entires beginning with `[0xA, 0xB, ...]` would
+be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300.
+
 ## Public Key Cryptography
 
 Tendermint uses Amino to distinguish between different types of private keys,
@@ -117,8 +129,10 @@ type PrivKeySecp256k1 [32]byte
 
 ### BitArray
 
-BitArray is encoded as an `int` of the number of bits, and with an array of `uint64` to encode
-value of each array element.
+The BitArray is used in block headers and some consensus messages to signal
+whether or not something was done by each validator. BitArray is represented
+with a struct containing the number of bits (`Bits`) and the bit-array itself
+encoded in base64 (`Elems`).
 
 ```go
 type BitArray struct {
@@ -127,8 +141,21 @@ type BitArray struct {
 }
 ```
 
+This type is easily encoded directly by Amino.
+
+Note BitArray receives a special JSON encoding in the form of `x` and `_`
+representing `1` and `0`. Ie. the BitArray `10110` would be JSON encoded as
+`"x_xx_"`
+
 ### Part
 
+Part is used to break up blocks into pieces that can be gossiped in parallel
+and securely verified using a Merkle tree of the parts.
+
+Part contains the index of the part in the larger set (`Index`), the actual
+underlying data of the part (`Bytes`), and a simple Merkle proof that the part is contained in
+the larger set (`Proof`).
+
 ```go
 type Part struct {
     Index int
@@ -142,14 +169,16 @@ type Part struct {
 Encode an object using Amino and slice it into parts.
 
 ```go
-MakeParts(object, partSize)
+func MakeParts(obj interface{}, partSize int) []Part
 ```
 
 ## Merkle Trees
 
 Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
 
-RIPEMD160 is always used as the hashing function.
+SHA256 is always used as the hashing function.
+
+### Simple Merkle Root
 
 The function `SimpleMerkleRoot` is a simple recursive function defined as follows:
 
@@ -163,16 +192,66 @@ func SimpleMerkleRoot(hashes [][]byte) []byte{
     default:
         left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2])
         right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:])
-        return RIPEMD160(append(left, right))
+        return SimpleConcatHash(left, right)
     }
 }
+
+func SimpleConcatHash(left, right []byte) []byte{
+    left = encodeByteSlice(left)
+    right = encodeByteSlice(right)
+    return SHA256(append(left, right))
+}
 ```
 
-Note: we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
+Note that the leaves are Amino encoded as byte-arrays (ie. simple Uvarint length
+prefix) before being concatenated together and hashed.
+
+Note: we will abuse notion and invoke `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
 For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to
 field name and then hashing them.
 For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements.
 
+### Simple Merkle Proof
+
+Proof that a leaf is in a Merkle tree consists of a simple structure:
+
+
+```
+type SimpleProof struct {
+        Aunts [][]byte
+}
+```
+
+Which is verified using the following:
+
+```
+func (proof SimpleProof) Verify(index, total int, leafHash, rootHash []byte) bool {
+	computedHash := computeHashFromAunts(index, total, leafHash, proof.Aunts)
+    return computedHash == rootHash
+}
+
+func computeHashFromAunts(index, total int, leafHash []byte, innerHashes [][]byte) []byte{
+	assert(index < total && index >= 0 && total > 0)
+
+	if total == 1{
+		assert(len(proof.Aunts) == 0)
+		return leafHash
+	}
+
+	assert(len(innerHashes) > 0)
+
+	numLeft := (total + 1) / 2
+	if index < numLeft {
+		leftHash := computeHashFromAunts(index, numLeft, leafHash, innerHashes[:len(innerHashes)-1])
+		assert(leftHash != nil)
+		return SimpleHashFromTwoHashes(leftHash, innerHashes[len(innerHashes)-1])
+	}
+	rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1])
+	assert(rightHash != nil)
+	return SimpleHashFromTwoHashes(innerHashes[len(innerHashes)-1], rightHash)
+}
+```
+
 ## AminoJSON
 
 Signed messages (eg. votes, proposals) in the consensus are encoded in AminoJSON, rather than binary Amino.
diff --git a/docs/specification/new-spec/scripts/crypto.go b/docs/specification/new-spec/scripts/crypto.go
new file mode 100644
index 00000000..704a21c3
--- /dev/null
+++ b/docs/specification/new-spec/scripts/crypto.go
@@ -0,0 +1,108 @@
+package main
+
+import (
+	"fmt"
+
+	crypto "github.com/tendermint/go-crypto"
+)
+
+func printEd() {
+	priv := crypto.GenPrivKeyEd25519()
+	pub := priv.PubKey().(crypto.PubKeyEd25519)
+	sig := priv.Sign([]byte("hello")).(crypto.SignatureEd25519)
+
+	name := "tendermint/PubKeyEd25519"
+	length := len(pub[:])
+
+	fmt.Println("### PubKeyEd25519")
+	fmt.Println("")
+	fmt.Println("```")
+	fmt.Printf("// Name: %s\n", name)
+	fmt.Printf("// PrefixBytes: 0x%X \n", pub.Bytes()[:4])
+	fmt.Printf("// Length: 0x%X \n", length)
+	fmt.Println("// Notes: raw 32-byte Ed25519 pubkey")
+	fmt.Println("type PubKeyEd25519 [32]byte")
+	fmt.Println("```")
+	fmt.Println("")
+	fmt.Printf("For example, the 32-byte Ed25519 pubkey `%X` would be encoded as `%X`\n", pub[:], pub.Bytes())
+	fmt.Println("")
+
+	name = "tendermint/SignatureKeyEd25519"
+	length = len(sig[:])
+
+	fmt.Println("### SignatureEd25519")
+	fmt.Println("")
+	fmt.Println("```")
+	fmt.Printf("// Name: %s\n", name)
+	fmt.Printf("// PrefixBytes: 0x%X \n", sig.Bytes()[:4])
+	fmt.Printf("// Length: 0x%X \n", length)
+	fmt.Println("// Notes: raw 64-byte Ed25519 signature")
+	fmt.Println("type SignatureEd25519 [64]byte")
+	fmt.Println("```")
+	fmt.Println("")
+	fmt.Printf("For example, the 64-byte Ed25519 signature `%X` would be encoded as `%X`\n", sig[:], sig.Bytes())
+	fmt.Println("")
+
+	name = "tendermint/PrivKeyEd25519"
+
+	fmt.Println("### PrivKeyEd25519")
+	fmt.Println("")
+	fmt.Println("```")
+	fmt.Println("// Name:", name)
+	fmt.Println("// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key")
+	fmt.Println("type PrivKeyEd25519 [64]byte")
+	fmt.Println("```")
+}
+
+func printSecp() {
+	priv := crypto.GenPrivKeySecp256k1()
+	pub := priv.PubKey().(crypto.PubKeySecp256k1)
+	sig := priv.Sign([]byte("hello")).(crypto.SignatureSecp256k1)
+
+	name := "tendermint/PubKeySecp256k1"
+	length := len(pub[:])
+
+	fmt.Println("### PubKeySecp256k1")
+	fmt.Println("")
+	fmt.Println("```")
+	fmt.Printf("// Name: %s\n", name)
+	fmt.Printf("// PrefixBytes: 0x%X \n", pub.Bytes()[:4])
+	fmt.Printf("// Length: 0x%X \n", length)
+	fmt.Println("// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03")
+	fmt.Println("type PubKeySecp256k1 [33]byte")
+	fmt.Println("```")
+	fmt.Println("")
+	fmt.Printf("For example, the 33-byte Secp256k1 pubkey `%X` would be encoded as `%X`\n", pub[:], pub.Bytes())
+	fmt.Println("")
+
+	name = "tendermint/SignatureKeySecp256k1"
+
+	fmt.Println("### SignatureSecp256k1")
+	fmt.Println("")
+	fmt.Println("```")
+	fmt.Printf("// Name: %s\n", name)
+	fmt.Printf("// PrefixBytes: 0x%X \n", sig.Bytes()[:4])
+	fmt.Printf("// Length: Variable\n")
+	fmt.Printf("// Encoding prefix: Variable\n")
+	fmt.Println("// Notes: raw bytes of the Secp256k1 signature")
+	fmt.Println("type SignatureSecp256k1 []byte")
+	fmt.Println("```")
+	fmt.Println("")
+	fmt.Printf("For example, the Secp256k1 signature `%X` would be encoded as `%X`\n", []byte(sig[:]), sig.Bytes())
+	fmt.Println("")
+
+	name = "tendermint/PrivKeySecp256k1"
+
+	fmt.Println("### PrivKeySecp256k1")
+	fmt.Println("")
+	fmt.Println("```")
+	fmt.Println("// Name:", name)
+	fmt.Println("// Notes: raw 32-byte priv key")
+	fmt.Println("type PrivKeySecp256k1 [32]byte")
+	fmt.Println("```")
+}
+
+func main() {
+	printEd()
+	printSecp()
+}

From 97be1eef870786f256fb322c7eee19e89ef2bca4 Mon Sep 17 00:00:00 2001
From: Ethan Buchman <ethan@coinculture.info>
Date: Thu, 26 Apr 2018 11:08:34 -0400
Subject: [PATCH 3/6] add abci notes

---
 docs/specification/new-spec/abci.md     | 48 +++++++++++++++++++++++++
 docs/specification/new-spec/encoding.md |  3 +-
 2 files changed, 50 insertions(+), 1 deletion(-)
 create mode 100644 docs/specification/new-spec/abci.md

diff --git a/docs/specification/new-spec/abci.md b/docs/specification/new-spec/abci.md
new file mode 100644
index 00000000..d942c800
--- /dev/null
+++ b/docs/specification/new-spec/abci.md
@@ -0,0 +1,48 @@
+# Application Blockchain Interface (ABCI)
+
+ABCI is the interface between Tendermint (a state-machine replication engine)
+and an application (the actual state machine).
+
+The ABCI message types are defined in a [protobuf
+file](https://github.com/tendermint/abci/blob/master/types/types.proto).
+For full details on the ABCI message types and protocol, see the [ABCI
+specificaiton](https://github.com/tendermint/abci/blob/master/specification.rst).
+For additional details on server implementation, see the [ABCI
+readme](https://github.com/tendermint/abci#implementation).
+
+Here we provide some more details around the use of ABCI by Tendermint and
+clarify common "gotchas".
+
+## Validator Updates
+
+Updates to the Tendermint validator set can be made by returning `Validator`
+objects in the `ResponseBeginBlock`:
+
+```
+message Validator {
+    bytes pub_key = 1;
+    int64 power = 2;
+}
+```
+
+The `pub_key` is the Amino encoded public key for the validator. For details on
+Amino encoded public keys, see the [section of the encoding spec](./encoding.md#public-key-cryptography).
+
+For example, the 32-byte Ed25519 pubkey
+`76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
+Amino encoded as
+`1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`
+
+The `power` is the new voting power for the validator, with the
+following rules:
+
+- power must be non-negative
+- if power is 0, the validator must already exist, and will be removed from the
+  validator set
+- if power is non-0:
+    - if the validator does not already exist, it will be added to the validator
+      set with the given power
+    - if the validator does already exist, its power will be adjusted to the given power
+
+
+## Query
diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md
index de747892..83154c85 100644
--- a/docs/specification/new-spec/encoding.md
+++ b/docs/specification/new-spec/encoding.md
@@ -33,7 +33,8 @@ Tendermint uses Amino to distinguish between different types of private keys,
 public keys, and signatures. Additionally, for each public key, Tendermint
 defines an Address function that can be used as a more compact identifier in
 place of the public key. Here we list the concrete types, their names,
-and prefix bytes for public keys and signatures. Note for brevity we don't
+and prefix bytes for public keys and signatures, as well as the address schemes
+for each PubKey. Note for brevity we don't
 include details of the private keys beyond their type and name, as they can be
 derrived the same way as the others using Amino.
 

From 0e1414ef9def6ae352aed64a5b3aa3e1110388cc Mon Sep 17 00:00:00 2001
From: Ethan Buchman <ethan@coinculture.info>
Date: Thu, 26 Apr 2018 11:46:20 -0400
Subject: [PATCH 4/6] spec: add Address spec. notes about Query

---
 docs/specification/new-spec/abci.md           | 22 ++++++++-
 docs/specification/new-spec/encoding.md       | 46 ++++++++++++++-----
 docs/specification/new-spec/scripts/crypto.go | 25 ++++++++--
 3 files changed, 76 insertions(+), 17 deletions(-)

diff --git a/docs/specification/new-spec/abci.md b/docs/specification/new-spec/abci.md
index d942c800..75eed6b5 100644
--- a/docs/specification/new-spec/abci.md
+++ b/docs/specification/new-spec/abci.md
@@ -28,7 +28,7 @@ message Validator {
 The `pub_key` is the Amino encoded public key for the validator. For details on
 Amino encoded public keys, see the [section of the encoding spec](./encoding.md#public-key-cryptography).
 
-For example, the 32-byte Ed25519 pubkey
+For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 32-byte Ed25519 pubkey
 `76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
 Amino encoded as
 `1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`
@@ -44,5 +44,23 @@ following rules:
       set with the given power
     - if the validator does already exist, its power will be adjusted to the given power
 
-
 ## Query
+
+Query is a generic message type with lots of flexibility to enable diverse sets
+of queries from applications. Tendermint has no requirements from the Query
+message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too.
+That said, Tendermint makes a number of queries to support some optional
+features. These are:
+
+### Peer Filtering
+
+When Tendermint connects to a peer, it sends two queries to the ABCI application
+using the following paths, with no additional data:
+
+ - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and
+   the port of the connection
+ - `p2p/filter/pubkey/<ID>`, where `<ID>` is the peer node ID (ie. the
+   pubkey.Address() for the peer's PubKey)
+
+If either of these queries return a non-zero ABCI code, Tendermint will refuse
+to connect to the peer.
diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md
index 83154c85..e15c370b 100644
--- a/docs/specification/new-spec/encoding.md
+++ b/docs/specification/new-spec/encoding.md
@@ -15,6 +15,9 @@ Notably, every object that satisfies an interface (eg. a particular kind of p2p
 or a particular kind of pubkey) is registered with a global name, the hash of
 which is included in the object's encoding as the so-called "prefix bytes".
 
+We define the `func AminoEncode(obj interface{}) []byte` function to take an
+arbitrary object and return the Amino encoded bytes.
+
 ## Byte Arrays
 
 The encoding of a byte array is simply the raw-bytes prefixed with the length of
@@ -48,6 +51,9 @@ Notice that when encoding byte-arrays, the length of the byte-array is appended
 to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes>
 <Length> <ByteArray>`
 
+(NOTE: the remainder of this section on Public Key Cryptography can be generated
+from [this script](./scripts/crypto.go))
+
 ### PubKeyEd25519
 
 ```
@@ -56,12 +62,21 @@ to the PrefixBytes. Thus the encoding of a byte array becomes `<PrefixBytes>
 // Length: 0x20
 // Notes: raw 32-byte Ed25519 pubkey
 type PubKeyEd25519 [32]byte
+
+func (pubkey PubKeyEd25519) Address() []byte {
+      // NOTE: hash of the Amino encoded bytes!
+        return RIPEMD160(AminoEncode(pubkey))
+}
 ```
 
 For example, the 32-byte Ed25519 pubkey
-`76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be
+`CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E` would be
 encoded as
-`1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`
+`1624DE6220CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E`.
+
+The address would then be
+`RIPEMD160(0x1624DE6220CCACD52F9B29D04393F01CD9AF6535455668115641F3D8BAEFD2295F24BAF60E)`
+or `430FF75BAF1EC4B0D51BB3EEC2955479D0071605`
 
 ### SignatureEd25519
 
@@ -74,9 +89,9 @@ type SignatureEd25519 [64]byte
 ```
 
 For example, the 64-byte Ed25519 signature
-`005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F`
+`1B6034A8ED149D3C94FDA13EC03B26CC0FB264D9B0E47D3FA3DEF9FCDE658E49C80B35F9BE74949356401B15B18FB817D6E54495AD1C4A8401B248466CB0DB0B`
 would be encoded as
-`3DA1DB2A40005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F`
+`3DA1DB2A401B6034A8ED149D3C94FDA13EC03B26CC0FB264D9B0E47D3FA3DEF9FCDE658E49C80B35F9BE74949356401B15B18FB817D6E54495AD1C4A8401B248466CB0DB0B`
 
 ### PrivKeyEd25519
 
@@ -85,6 +100,7 @@ would be encoded as
 // Notes: raw 32-byte priv key concatenated to raw 32-byte pub key
 type PrivKeyEd25519 [64]byte
 ```
+
 ### PubKeySecp256k1
 
 ```
@@ -93,12 +109,22 @@ type PrivKeyEd25519 [64]byte
 // Length: 0x21
 // Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03
 type PubKeySecp256k1 [33]byte
+
+func (pubkey PubKeySecp256k1) Address() []byte {
+      // NOTE: hash of the raw pubkey bytes (not Amino encoded!).
+        // Compatible with Bitcoin addresses.
+          return RIPEMD160(SHA256(pubkey[:]))
+}
 ```
 
 For example, the 33-byte Secp256k1 pubkey
-`03573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380` would be
+`020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9` would be
 encoded as
-`EB5AE9822103573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380`
+`EB5AE98221020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9`
+
+The address would then be
+`RIPEMD160(SHA256(0x020BD40F225A57ED383B440CF073BC5539D0341F5767D2BF2D78406D00475A2EE9))`
+or `0AE5BEE929ABE51BAD345DB925EEA652680783FC`
 
 ### SignatureSecp256k1
 
@@ -112,9 +138,9 @@ type SignatureSecp256k1 []byte
 ```
 
 For example, the Secp256k1 signature
-`304402207447640A5C12A72BAA052D110B666FB6DF717A7B863361C092E751D016C6C08802205C20F9DEBF8915DED310B98BFA890105F43925FDB2B67B78510FE18EDA2B30DA` would
-be encoded as
-`16E1FEEA46304402202C10C874E413AF538D97EBEF2B01024719F8B7CC559CEEBDC7C380F9DCC4A6E002200EDE9B62F8531933F88DB2A62E73BA3D43ACEB1CBD23070C2F792AAA18717A4A`
+`304402201CD4B8C764D2FD8AF23ECFE6666CA8A53886D47754D951295D2D311E1FEA33BF02201E0F906BB1CF2C30EAACFFB032A7129358AFF96B9F79B06ACFFB18AC90C2ADD7`
+would be encoded as
+`16E1FEEA46304402201CD4B8C764D2FD8AF23ECFE6666CA8A53886D47754D951295D2D311E1FEA33BF02201E0F906BB1CF2C30EAACFFB032A7129358AFF96B9F79B06ACFFB18AC90C2ADD7`
 
 ### PrivKeySecp256k1
 
@@ -124,8 +150,6 @@ be encoded as
 type PrivKeySecp256k1 [32]byte
 ```
 
-
-
 ## Other Common Types
 
 ### BitArray
diff --git a/docs/specification/new-spec/scripts/crypto.go b/docs/specification/new-spec/scripts/crypto.go
index 704a21c3..e4dbd8a2 100644
--- a/docs/specification/new-spec/scripts/crypto.go
+++ b/docs/specification/new-spec/scripts/crypto.go
@@ -6,8 +6,11 @@ import (
 	crypto "github.com/tendermint/go-crypto"
 )
 
+// SECRET
+var SECRET = []byte("some secret")
+
 func printEd() {
-	priv := crypto.GenPrivKeyEd25519()
+	priv := crypto.GenPrivKeyEd25519FromSecret(SECRET)
 	pub := priv.PubKey().(crypto.PubKeyEd25519)
 	sig := priv.Sign([]byte("hello")).(crypto.SignatureEd25519)
 
@@ -22,9 +25,15 @@ func printEd() {
 	fmt.Printf("// Length: 0x%X \n", length)
 	fmt.Println("// Notes: raw 32-byte Ed25519 pubkey")
 	fmt.Println("type PubKeyEd25519 [32]byte")
+	fmt.Println("")
+	fmt.Println(`func (pubkey PubKeyEd25519) Address() []byte {
+  // NOTE: hash of the Amino encoded bytes!
+  return RIPEMD160(AminoEncode(pubkey))
+}`)
 	fmt.Println("```")
 	fmt.Println("")
-	fmt.Printf("For example, the 32-byte Ed25519 pubkey `%X` would be encoded as `%X`\n", pub[:], pub.Bytes())
+	fmt.Printf("For example, the 32-byte Ed25519 pubkey `%X` would be encoded as `%X`.\n\n", pub[:], pub.Bytes())
+	fmt.Printf("The address would then be `RIPEMD160(0x%X)` or `%X`\n", pub.Bytes(), pub.Address())
 	fmt.Println("")
 
 	name = "tendermint/SignatureKeyEd25519"
@@ -55,7 +64,7 @@ func printEd() {
 }
 
 func printSecp() {
-	priv := crypto.GenPrivKeySecp256k1()
+	priv := crypto.GenPrivKeySecp256k1FromSecret(SECRET)
 	pub := priv.PubKey().(crypto.PubKeySecp256k1)
 	sig := priv.Sign([]byte("hello")).(crypto.SignatureSecp256k1)
 
@@ -70,9 +79,16 @@ func printSecp() {
 	fmt.Printf("// Length: 0x%X \n", length)
 	fmt.Println("// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03")
 	fmt.Println("type PubKeySecp256k1 [33]byte")
+	fmt.Println("")
+	fmt.Println(`func (pubkey PubKeySecp256k1) Address() []byte {
+  // NOTE: hash of the raw pubkey bytes (not Amino encoded!).
+  // Compatible with Bitcoin addresses.
+  return RIPEMD160(SHA256(pubkey[:]))
+}`)
 	fmt.Println("```")
 	fmt.Println("")
-	fmt.Printf("For example, the 33-byte Secp256k1 pubkey `%X` would be encoded as `%X`\n", pub[:], pub.Bytes())
+	fmt.Printf("For example, the 33-byte Secp256k1 pubkey `%X` would be encoded as `%X`\n\n", pub[:], pub.Bytes())
+	fmt.Printf("The address would then be `RIPEMD160(SHA256(0x%X))` or `%X`\n", pub[:], pub.Address())
 	fmt.Println("")
 
 	name = "tendermint/SignatureKeySecp256k1"
@@ -104,5 +120,6 @@ func printSecp() {
 
 func main() {
 	printEd()
+	fmt.Println("")
 	printSecp()
 }

From e5951acfb47fa55f3e932170854e518e2ad5b5f7 Mon Sep 17 00:00:00 2001
From: Ethan Buchman <ethan@coinculture.info>
Date: Thu, 26 Apr 2018 11:46:50 -0400
Subject: [PATCH 5/6] SHA256 -> RIPEMD160

---
 docs/specification/new-spec/encoding.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md
index e15c370b..abecf592 100644
--- a/docs/specification/new-spec/encoding.md
+++ b/docs/specification/new-spec/encoding.md
@@ -201,7 +201,7 @@ func MakeParts(obj interface{}, partSize int) []Part
 
 Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
 
-SHA256 is always used as the hashing function.
+RIPEMD160 is always used as the hashing function.
 
 ### Simple Merkle Root
 
@@ -224,7 +224,7 @@ func SimpleMerkleRoot(hashes [][]byte) []byte{
 func SimpleConcatHash(left, right []byte) []byte{
     left = encodeByteSlice(left)
     right = encodeByteSlice(right)
-    return SHA256(append(left, right))
+    return RIPEMD160 (append(left, right))
 }
 ```
 

From f1ead2df70f5e17848219393ae34a4fab4eaaf22 Mon Sep 17 00:00:00 2001
From: Ethan Buchman <ethan@coinculture.info>
Date: Thu, 26 Apr 2018 11:50:45 -0400
Subject: [PATCH 6/6] typo

---
 docs/specification/new-spec/encoding.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md
index abecf592..ae6e003d 100644
--- a/docs/specification/new-spec/encoding.md
+++ b/docs/specification/new-spec/encoding.md
@@ -3,7 +3,7 @@
 ## Amino
 
 Tendermint uses the Protobuf3 derrivative [Amino]() for all data structures.
-Thik of Amino as an object-oriented Protobuf3 with native JSON support.
+Think of Amino as an object-oriented Protobuf3 with native JSON support.
 The goal of the Amino encoding protocol is to bring parity between application
 logic objects and persistence objects.