tendermint/docs/specification/new-spec/encoding.md

Tendermint Encoding

Amino

Tendermint uses the Protobuf3 derrivative Amino for all data structures. Thik of Amino as an object-oriented Protobuf3 with native JSON support. The goal of the Amino encoding protocol is to bring parity between application logic objects and persistence objects.

Please see the Amino specification for more details.

Notably, every object that satisfies an interface (eg. a particular kind of p2p message, or a particular kind of pubkey) is registered with a global name, the hash of which is included in the object's encoding as the so-called "prefix bytes".

Public Key Cryptography

Tendermint uses Amino to distinguish between different types of private keys, public keys, and signatures. Additionally, for each public key, Tendermint defines an Address function that can be used as a more compact identifier in place of the public key. Here we list the concrete types, their names, and prefix bytes for public keys and signatures. Note for brevity we don't include details of the private keys beyond their type and name, as they can be derrived the same way as the others using Amino.

All registered objects are encoded by Amino using a 4-byte PrefixBytes that uniquely identifies the object and includes information about its underlying type. For details on how PrefixBytes are computed, see the Amino spec.

In what follows, we provide the type names and prefix bytes directly. Notice that when encoding byte-arrays, the length of the byte-array is appended to the PrefixBytes. Thus the encoding of a byte array becomes <PrefixBytes> <Length> <ByteArray>

PubKeyEd25519

// Name: tendermint/PubKeyEd25519
// PrefixBytes: 0x1624DE62
// Length: 0x20
// Notes: raw 32-byte Ed25519 pubkey
type PubKeyEd25519 [32]byte

For example, the 32-byte Ed25519 pubkey 76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85 would be encoded as 1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85

SignatureEd25519

// Name: tendermint/SignatureKeyEd25519
// PrefixBytes: 0x3DA1DB2A
// Length: 0x40
// Notes: raw 64-byte Ed25519 signature
type SignatureEd25519 [64]byte

For example, the 64-byte Ed25519 signature 005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F would be encoded as 3DA1DB2A40005E76B3B0D790959B03F862A9EF8F6236457032B5F522C4CAB5AAD7C44A00A12669E1A2761798E70A0A923DA0CF981839558123CF6466553BCBFF25DADD630F

PrivKeyEd25519

// Name: tendermint/PrivKeyEd25519
// Notes: raw 32-byte priv key concatenated to raw 32-byte pub key
type PrivKeyEd25519 [64]byte

PubKeySecp256k1

// Name: tendermint/PubKeySecp256k1
// PrefixBytes: 0xEB5AE982
// Length: 0x21
// Notes: OpenSSL compressed pubkey prefixed with 0x02 or 0x03
type PubKeySecp256k1 [33]byte

For example, the 33-byte Secp256k1 pubkey 03573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380 would be encoded as EB5AE9822103573E0EC1F989DECC3913AC7D44D0509C1A992ECE700845594A1078DAF19A3380

SignatureSecp256k1

// Name: tendermint/SignatureKeySecp256k1
// PrefixBytes: 0x16E1FEEA
// Length: Variable
// Encoding prefix: Variable
// Notes: raw bytes of the Secp256k1 signature
type SignatureSecp256k1 []byte

For example, the Secp256k1 signature 304402207447640A5C12A72BAA052D110B666FB6DF717A7B863361C092E751D016C6C08802205C20F9DEBF8915DED310B98BFA890105F43925FDB2B67B78510FE18EDA2B30DA would be encoded as 16E1FEEA46304402202C10C874E413AF538D97EBEF2B01024719F8B7CC559CEEBDC7C380F9DCC4A6E002200EDE9B62F8531933F88DB2A62E73BA3D43ACEB1CBD23070C2F792AAA18717A4A

PrivKeySecp256k1

// Name: tendermint/PrivKeySecp256k1
// Notes: raw 32-byte priv key
type PrivKeySecp256k1 [32]byte

Other Common Types

BitArray

BitArray is encoded as an int of the number of bits, and with an array of uint64 to encode value of each array element.

type BitArray struct {
    Bits  int
    Elems []uint64
}

Part

type Part struct {
    Index int
    Bytes byte[]
    Proof byte[]
}

MakeParts

Encode an object using Amino and slice it into parts.

MakeParts(object, partSize)

Merkle Trees

Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.

RIPEMD160 is always used as the hashing function.

The function SimpleMerkleRoot is a simple recursive function defined as follows:

func SimpleMerkleRoot(hashes [][]byte) []byte{
    switch len(hashes) {
    case 0:
        return nil
    case 1:
        return hashes[0]
    default:
        left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2])
        right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:])
        return RIPEMD160(append(left, right))
    }
}

Note: we abuse notion and call SimpleMerkleRoot with arguments of type struct or type []struct. For struct arguments, we compute a [][]byte by sorting elements of the struct according to field name and then hashing them. For []struct arguments, we compute a [][]byte by hashing the individual struct elements.

AminoJSON

Signed messages (eg. votes, proposals) in the consensus are encoded in AminoJSON, rather than binary Amino.

When signing, the elements of a message are sorted by key and the sorted message is embedded in an outer JSON that includes a chain_id field. We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look like:

{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2}

Note how the fields within each level are sorted.