8.4 KiB
Tendermint Encoding
Amino
Tendermint uses the Protobuf3 derrivative Amino for all data structures. Thik of Amino as an object-oriented Protobuf3 with native JSON support. The goal of the Amino encoding protocol is to bring parity between application logic objects and persistence objects.
Please see the Amino specification for more details.
Notably, every object that satisfies an interface (eg. a particular kind of p2p message, or a particular kind of pubkey) is registered with a global name, the hash of which is included in the object's encoding as the so-called "prefix bytes".
Byte Arrays
The encoding of a byte array is simply the raw-bytes prefixed with the length of
the array as a UVarint (what Protobuf calls a Varint).
For details on varints, see the protobuf spec.
For example, the byte-array [0xA, 0xB] would be encoded as 0x020A0B,
while a byte-array containing 300 entires beginning with [0xA, 0xB, ...] would
be encoded as 0xAC020A0B... where 0xAC02 is the UVarint encoding of 300.
Public Key Cryptography
Tendermint uses Amino to distinguish between different types of private keys, public keys, and signatures. Additionally, for each public key, Tendermint defines an Address function that can be used as a more compact identifier in place of the public key. Here we list the concrete types, their names, and prefix bytes for public keys and signatures. Note for brevity we don't include details of the private keys beyond their type and name, as they can be derrived the same way as the others using Amino.
All registered objects are encoded by Amino using a 4-byte PrefixBytes that uniquely identifies the object and includes information about its underlying type. For details on how PrefixBytes are computed, see the Amino spec.
In what follows, we provide the type names and prefix bytes directly.
Notice that when encoding byte-arrays, the length of the byte-array is appended
to the PrefixBytes. Thus the encoding of a byte array becomes <PrefixBytes> <Length> <ByteArray>
PubKeyEd25519
// Name: tendermint/PubKeyEd25519
// PrefixBytes: 0x1624DE62
// Length: 0x20
// Notes: raw 32-byte Ed25519 pubkey
type PubKeyEd25519 [32]byte
For example, the 32-byte Ed25519 pubkey
76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85 would be
encoded as
1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85
SignatureEd25519
// Name: tendermint/SignatureKeyEd25519
// PrefixBytes: 0x3DA1DB2A
// Length: 0x40
// Notes: raw 64-byte Ed25519 signature
type SignatureEd25519 [64]byte
For example, the 64-byte Ed25519 signature
005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F
would be encoded as
3DA1DB2A40005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F
PrivKeyEd25519
// Name: tendermint/PrivKeyEd25519
// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key
type PrivKeyEd25519 [64]byte
PubKeySecp256k1
// Name: tendermint/PubKeySecp256k1
// PrefixBytes: 0xEB5AE982
// Length: 0x21
// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03
type PubKeySecp256k1 [33]byte
For example, the 33-byte Secp256k1 pubkey
03573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380 would be
encoded as
EB5AE9822103573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380
SignatureSecp256k1
// Name: tendermint/SignatureKeySecp256k1
// PrefixBytes: 0x16E1FEEA
// Length: Variable
// Encoding prefix: Variable
// Notes: raw bytes of the Secp256k1 signature
type SignatureSecp256k1 []byte
For example, the Secp256k1 signature
304402207447640A5C12A72BAA052D110B666FB6DF717A7B863361C092E751D016C6C08802205C20F9DEBF8915DED310B98BFA890105F43925FDB2B67B78510FE18EDA2B30DA would
be encoded as
16E1FEEA46304402202C10C874E413AF538D97EBEF2B01024719F8B7CC559CEEBDC7C380F9DCC4A6E002200EDE9B62F8531933F88DB2A62E73BA3D43ACEB1CBD23070C2F792AAA18717A4A
PrivKeySecp256k1
// Name: tendermint/PrivKeySecp256k1
// Notes: raw 32-byte priv key
type PrivKeySecp256k1 [32]byte
Other Common Types
BitArray
The BitArray is used in block headers and some consensus messages to signal
whether or not something was done by each validator. BitArray is represented
with a struct containing the number of bits (Bits) and the bit-array itself
encoded in base64 (Elems).
type BitArray struct {
Bits int
Elems []uint64
}
This type is easily encoded directly by Amino.
Note BitArray receives a special JSON encoding in the form of x and _
representing 1 and 0. Ie. the BitArray 10110 would be JSON encoded as
"x_xx_"
Part
Part is used to break up blocks into pieces that can be gossiped in parallel and securely verified using a Merkle tree of the parts.
Part contains the index of the part in the larger set (Index), the actual
underlying data of the part (Bytes), and a simple Merkle proof that the part is contained in
the larger set (Proof).
type Part struct {
Index int
Bytes byte[]
Proof byte[]
}
MakeParts
Encode an object using Amino and slice it into parts.
func MakeParts(obj interface{}, partSize int) []Part
Merkle Trees
Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
SHA256 is always used as the hashing function.
Simple Merkle Root
The function SimpleMerkleRoot is a simple recursive function defined as follows:
func SimpleMerkleRoot(hashes [][]byte) []byte{
switch len(hashes) {
case 0:
return nil
case 1:
return hashes[0]
default:
left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2])
right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:])
return SimpleConcatHash(left, right)
}
}
func SimpleConcatHash(left, right []byte) []byte{
left = encodeByteSlice(left)
right = encodeByteSlice(right)
return SHA256(append(left, right))
}
Note that the leaves are Amino encoded as byte-arrays (ie. simple Uvarint length prefix) before being concatenated together and hashed.
Note: we will abuse notion and invoke SimpleMerkleRoot with arguments of type struct or type []struct.
For struct arguments, we compute a [][]byte by sorting elements of the struct according to
field name and then hashing them.
For []struct arguments, we compute a [][]byte by hashing the individual struct elements.
Simple Merkle Proof
Proof that a leaf is in a Merkle tree consists of a simple structure:
type SimpleProof struct {
Aunts [][]byte
}
Which is verified using the following:
func (proof SimpleProof) Verify(index, total int, leafHash, rootHash []byte) bool {
computedHash := computeHashFromAunts(index, total, leafHash, proof.Aunts)
return computedHash == rootHash
}
func computeHashFromAunts(index, total int, leafHash []byte, innerHashes [][]byte) []byte{
assert(index < total && index >= 0 && total > 0)
if total == 1{
assert(len(proof.Aunts) == 0)
return leafHash
}
assert(len(innerHashes) > 0)
numLeft := (total + 1) / 2
if index < numLeft {
leftHash := computeHashFromAunts(index, numLeft, leafHash, innerHashes[:len(innerHashes)-1])
assert(leftHash != nil)
return SimpleHashFromTwoHashes(leftHash, innerHashes[len(innerHashes)-1])
}
rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1])
assert(rightHash != nil)
return SimpleHashFromTwoHashes(innerHashes[len(innerHashes)-1], rightHash)
}
AminoJSON
Signed messages (eg. votes, proposals) in the consensus are encoded in AminoJSON, rather than binary Amino.
When signing, the elements of a message are sorted by key and the sorted message is embedded in an
outer JSON that includes a chain_id field.
We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look
like:
{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2}
Note how the fields within each level are sorted.