Refactored Ansible, added tendermint and basecoin configuration and multiple playbooks

ansible/README.md

@@ -1,4 +1,4 @@
-# Ansible playbook for Tendermint on DigitalOcean
+# Ansible playbook for Tendermint
 
 ![Ansible plus Tendermint](img/a_plus_t.png)
 
@@ -7,42 +7,39 @@
 * [Running the playbook](#Running the playbook)
 * [Example playbook that configures a Tendermint on Ubuntu](#example-playbook-that-configures-a-tendermint-on-ubuntu)
 
-The playbook in this folder contains [ansible](http://www.ansible.com/) roles which:
+The playbooks in this folder run [ansible](http://www.ansible.com/) roles which:
 
-* installs tendermint
-* configures tendermint
-* configures tendermint service
-* installs basecoin
-* configures basecoin
+* install and configure tendermint
+* install and configure  basecoin
+* start/stop tendermint and basecoin
 
 ## Prerequisites
 
 * Ansible 2.0 or higher
-* DigitalOcean API Token
 * SSH key to the servers
+
+Optional for DigitalOcean droplets:
+* DigitalOcean API Token
 * python dopy package
 
 Head over to the [Terraform folder](https://github.com/tendermint/tools) for a description on how to get a DigitalOcean API Token.
 
-The DigitalOcean inventory script comes from the ansible team at https://github.com/ansible/ansible. You can get the latest version from the contrib/inventory folder.
+Optional for Amazon AWS instances:
+* Amazon AWS API access key ID and secret access key.
+
+The cloud inventory scripts come from the ansible team at https://github.com/ansible/ansible. You can get the latest version from the contrib/inventory folder.
 
 ## Ansible setup
 
-Ansible requires a "command machine" or "local machine" or "orchestrator machine" to run on. This can be your laptop or any machine that runs linux. (It does not have to be part of the DigitalOcean network.)
+Ansible requires a "command machine" or "local machine" or "orchestrator machine" to run on. This can be your laptop or any machine that runs linux. (It does not have to be part of the cloud network that hosts your servers.)
+
+Note: The below commands use the Ubuntu/Debian `apt-get` command. To make it compatible with RedHat/CentOS, replace it with `yum`.
 
-Example on RedHat/CentOS:
 ```
-sudo yum install ansible python-pip
-sudo pip install dopy
+sudo apt-get install ansible
 ```
 
-Example on Ubuntu/Debian:
-```
-sudo apt-get install ansible python-pip
-sudo pip install dopy
-```
-
-To make life easier, you can start an SSH Agent and load your SSH key(s) into it. This way ansible will have an uninterrupted way of connecting to the droplets.
+To make life easier, you can start an SSH Agent and load your SSH key(s). This way ansible will have an uninterrupted way of connecting to tour servers.
 
 ```
 ssh-agent > ~/.ssh/ssh.env
@@ -53,6 +50,21 @@ ssh-add private.key
 
 Subsequently, as long as the agent is running, you can use `source ~/.ssh/ssh.env` to load the keys to the current session.
 
+### Optional cloud dependencies
+
+If you are using a cloud provider to host your servers, you need the below dependencies installed on your local machine.
+
+DigitalOcean inventory dependencies:
+```
+sudo apt-get install python-pip
+sudo pip install dopy
+```
+
+Amazon AWS inventory dependencies:
+```
+sudo apt-get install python-boto
+```
+
 ## Refreshing the DigitalOcean inventory
 
 If you just finished creating droplets, the local DigitalOcean inventory cache is not up-to-date. To refresh it, run:
@@ -62,23 +74,61 @@ DO_API_TOKEN="<The API token received from DigitalOcean>"
 python -u inventory/digital_ocean.py --refresh-cache 1> /dev/null
 ```
 
-## Running the playbook
+## Refreshing the Amazon AWS inventory
 
-The playbook is locked down to only run if the environment variable `TF_VAR_TESTNET_NAME` is populated. This is a precaution so you don't accidentally run the playbook on all your DigitalOcean droplets.
-
-The variable `TF_VAR_TESTNET_NAME` contains the testnet name defined when the droplets were created using Terraform.
+If you just finished creating Amazon AWS EC2 instances, the local AWS inventory cache is not up-to-date. To refresh it, run:
 
 ```
+AWS_ACCESS_KEY_ID='<The API access key ID received from Amazon>'
+AWS_SECRET_ACCESS_KEY='<The API secret access key received from Amazon>'
+python -u inventory/ec2.py --refresh-cache 1> /dev/null
+```
+
+Note you don't need the access key and secret key set, if you are running ansible on an Amazon AMI instance with the proper IAM permissions set.
+
+## Running the playbook
+
+The playbook is locked down to only run if the environment variable `TF_VAR_TESTNET_NAME` is populated. This is a precaution so you don't accidentally run the playbook on all your servers.
+
+The variable `TF_VAR_TESTNET_NAME` contains the testnet name which ansible translates into an ansible group. If you used Terraform to create the servers, it was the testnet name used there.
+
+If the playbook cannot connect to the servers because of public key denial, your SSH Agent is not set up properly. Alternatively you can add the SSH key to ansible using the `--private-key` option.
+
+### DigitalOcean
+```
+DO_API_TOKEN="<The API token received from DigitalOcean>"
 TF_VAR_TESTNET_NAME="testnet-servers"
 ansible-playbook -i inventory/digital_ocean.py install.yml
 ```
 
-If the playbook cannot connect to the servers because of public key denial, your SSH Agent is not set up properly. Alternatively you can add the SSH key to ansible using the `--private-key` option.
+### Amazon AWS
+```
+AWS_ACCESS_KEY_ID='<The API access key ID received from Amazon>'
+AWS_SECRET_ACCESS_KEY='<The API secret access key received from Amazon>'
+TF_VAR_TESTNET_NAME="testnet-servers"
+ansible-playbook -i inventory/ec2.py install.yml
+```
 
-## Starting the cluster
+### Installing custom versions
+
+By default ansible installs the tendermint and basecoin binary versions defined in its [default variables](#Default variables). If you built your own version of the binaries, you can tell ansible to install that instead.
+
+```
+GOPATH="<your go path>"
+go get -u github.com/tendermint/tendermint/cmd/tendermint
+go get -u github.com/tendermint/basecoin/cmd/basecoin
+
+DO_API_TOKEN="<The API token received from DigitalOcean>"
+TF_VAR_TESTNET_NAME="testnet-servers"
+ansible-playbook -i inventory/digital_ocean.py install.yml -e tendermint_release_install=false -e basecoin_release_install=false
+```
+
+## Starting the servers 
 
 To be continued...
 
-## Role details
+## Default variables
+
 
 To be continued...
+

ansible/config.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  user: root
+  roles:
+    - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
+

ansible/group_vars/all

@@ -0,0 +1,45 @@
+---
+#tendermint_release_install: true
+#tendermint_version: 0.10.0-rc1
+#tendermint_download: "https://s3-us-west-2.amazonaws.com/tendermint/binaries/tendermint/v{{tendermint_version}}/tendermint_{{tendermint_version}}_linux_amd64.zip"
+
+##If tendermint_release_install == false, copy the binary from here
+#tendermint_binary: "{{ lookup('env','GOPATH') | default('') }}/bin/tendermint"
+
+#tendermint_user: tendermint
+#tendermint_group: tendermint
+
+## Upstart start/stop conditions can vary by distribution and environment
+#tendermint_upstart_start_on: start on runlevel [345]
+#tendermint_upstart_stop_on: stop on runlevel [!345]
+
+#tendermint_home: /var/lib/tendermint
+#tendermint_log_file: /var/log/tendermint.log
+
+#basecoin_release_install: true
+#basecoin_version: 0.4.0
+#basecoin_download: "https://s3-us-west-2.amazonaws.com/tendermint/binaries/basecoin/v{{basecoin_version}}/basecoin_{{basecoin_version}}_linux_amd64.zip"
+
+##If basecoin_release_install == false, copy the binary from here
+#basecoin_binary: "{{ lookup('env','GOPATH') | default('') }}/bin/basecoin"
+
+#basecoin_user: basecoin
+#basecoin_group: basecoin
+
+## Upstart start/stop conditions can vary by distribution and environment
+#basecoin_upstart_start_on: start on runlevel [345]
+#basecoin_upstart_stop_on: stop on runlevel [!345]
+
+#basecoin_home: /var/lib/basecoin
+#basecoin_log_file: /var/log/basecoin.log
+
+#basecoin_inprocess: false
+
+#Used by the config role
+#testnet_name: test-chain
+
+
+basecoin_inprocess: false
+tendermint_release_install: false
+basecoin_release_install: false
+

ansible/install.yml

@@ -1,9 +1,11 @@
 ---
 
 - hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
-  vars:
-    testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
   user: root
   roles:
-    - tendermint
-    - basecoin
+    - install-tendermint
+    - install-basecoin
+    - cleanupconfig
+    - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
+    - start
+

ansible/inventory/ec2.ini

@@ -0,0 +1,209 @@
+# Ansible EC2 external inventory script settings
+#
+
+[ec2]
+
+# to talk to a private eucalyptus instance uncomment these lines
+# and edit edit eucalyptus_host to be the host name of your cloud controller
+#eucalyptus = True
+#eucalyptus_host = clc.cloud.domain.org
+
+# AWS regions to make calls to. Set this to 'all' to make request to all regions
+# in AWS and merge the results together. Alternatively, set this to a comma
+# separated list of regions. E.g. 'us-east-1,us-west-1,us-west-2' and do not
+# provide the 'regions_exclude' option. If this is set to 'auto', AWS_REGION or
+# AWS_DEFAULT_REGION environment variable will be read to determine the region.
+regions = all
+regions_exclude = us-gov-west-1, cn-north-1
+
+# When generating inventory, Ansible needs to know how to address a server.
+# Each EC2 instance has a lot of variables associated with it. Here is the list:
+#   http://docs.pythonboto.org/en/latest/ref/ec2.html#module-boto.ec2.instance
+# Below are 2 variables that are used as the address of a server:
+#   - destination_variable
+#   - vpc_destination_variable
+
+# This is the normal destination variable to use. If you are running Ansible
+# from outside EC2, then 'public_dns_name' makes the most sense. If you are
+# running Ansible from within EC2, then perhaps you want to use the internal
+# address, and should set this to 'private_dns_name'. The key of an EC2 tag
+# may optionally be used; however the boto instance variables hold precedence
+# in the event of a collision.
+destination_variable = public_dns_name
+
+# This allows you to override the inventory_name with an ec2 variable, instead
+# of using the destination_variable above. Addressing (aka ansible_ssh_host)
+# will still use destination_variable. Tags should be written as 'tag_TAGNAME'.
+#hostname_variable = tag_Name
+
+# For server inside a VPC, using DNS names may not make sense. When an instance
+# has 'subnet_id' set, this variable is used. If the subnet is public, setting
+# this to 'ip_address' will return the public IP address. For instances in a
+# private subnet, this should be set to 'private_ip_address', and Ansible must
+# be run from within EC2. The key of an EC2 tag may optionally be used; however
+# the boto instance variables hold precedence in the event of a collision.
+# WARNING: - instances that are in the private vpc, _without_ public ip address
+# will not be listed in the inventory until You set:
+# vpc_destination_variable = private_ip_address
+vpc_destination_variable = ip_address
+
+# The following two settings allow flexible ansible host naming based on a
+# python format string and a comma-separated list of ec2 tags.  Note that:
+#
+# 1) If the tags referenced are not present for some instances, empty strings
+#    will be substituted in the format string.
+# 2) This overrides both destination_variable and vpc_destination_variable.
+#
+#destination_format = {0}.{1}.example.com
+#destination_format_tags = Name,environment
+
+# To tag instances on EC2 with the resource records that point to them from
+# Route53, set 'route53' to True.
+route53 = False
+
+# To use Route53 records as the inventory hostnames, uncomment and set
+# to equal the domain name you wish to use. You must also have 'route53' (above)
+# set to True.
+# route53_hostnames = .example.com
+
+# To exclude RDS instances from the inventory, uncomment and set to False.
+#rds = False
+
+# To exclude ElastiCache instances from the inventory, uncomment and set to False.
+#elasticache = False
+
+# Additionally, you can specify the list of zones to exclude looking up in
+# 'route53_excluded_zones' as a comma-separated list.
+# route53_excluded_zones = samplezone1.com, samplezone2.com
+
+# By default, only EC2 instances in the 'running' state are returned. Set
+# 'all_instances' to True to return all instances regardless of state.
+all_instances = False
+
+# By default, only EC2 instances in the 'running' state are returned. Specify
+# EC2 instance states to return as a comma-separated list. This
+# option is overridden when 'all_instances' is True.
+# instance_states = pending, running, shutting-down, terminated, stopping, stopped
+
+# By default, only RDS instances in the 'available' state are returned.  Set
+# 'all_rds_instances' to True return all RDS instances regardless of state.
+all_rds_instances = False
+
+# Include RDS cluster information (Aurora etc.)
+include_rds_clusters = False
+
+# By default, only ElastiCache clusters and nodes in the 'available' state
+# are returned. Set 'all_elasticache_clusters' and/or 'all_elastic_nodes'
+# to True return all ElastiCache clusters and nodes, regardless of state.
+#
+# Note that all_elasticache_nodes only applies to listed clusters. That means
+# if you set all_elastic_clusters to false, no node will be return from
+# unavailable clusters, regardless of the state and to what you set for
+# all_elasticache_nodes.
+all_elasticache_replication_groups = False
+all_elasticache_clusters = False
+all_elasticache_nodes = False
+
+# API calls to EC2 are slow. For this reason, we cache the results of an API
+# call. Set this to the path you want cache files to be written to. Two files
+# will be written to this directory:
+#   - ansible-ec2.cache
+#   - ansible-ec2.index
+cache_path = ~/.ansible/tmp
+
+# The number of seconds a cache file is considered valid. After this many
+# seconds, a new API call will be made, and the cache file will be updated.
+# To disable the cache, set this value to 0
+cache_max_age = 300
+
+# Organize groups into a nested/hierarchy instead of a flat namespace.
+nested_groups = False
+
+# Replace - tags when creating groups to avoid issues with ansible
+replace_dash_in_groups = True
+
+# If set to true, any tag of the form "a,b,c" is expanded into a list
+# and the results are used to create additional tag_* inventory groups.
+expand_csv_tags = False
+
+# The EC2 inventory output can become very large. To manage its size,
+# configure which groups should be created.
+group_by_instance_id = True
+group_by_region = True
+group_by_availability_zone = True
+group_by_aws_account = False
+group_by_ami_id = True
+group_by_instance_type = True
+group_by_instance_state = False
+group_by_key_pair = True
+group_by_vpc_id = True
+group_by_security_group = True
+group_by_tag_keys = True
+group_by_tag_none = True
+group_by_route53_names = True
+group_by_rds_engine = True
+group_by_rds_parameter_group = True
+group_by_elasticache_engine = True
+group_by_elasticache_cluster = True
+group_by_elasticache_parameter_group = True
+group_by_elasticache_replication_group = True
+
+# If you only want to include hosts that match a certain regular expression
+# pattern_include = staging-*
+
+# If you want to exclude any hosts that match a certain regular expression
+# pattern_exclude = staging-*
+
+# Instance filters can be used to control which instances are retrieved for
+# inventory. For the full list of possible filters, please read the EC2 API
+# docs: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html#query-DescribeInstances-filters
+# Filters are key/value pairs separated by '=', to list multiple filters use
+# a list separated by commas. See examples below.
+
+# If you want to apply multiple filters simultaneously, set stack_filters to
+# True. Default behaviour is to combine the results of all filters. Stacking
+# allows the use of multiple conditions to filter down, for example by
+# environment and type of host.
+stack_filters = False
+
+# Retrieve only instances with (key=value) env=staging tag
+# instance_filters = tag:env=staging
+
+# Retrieve only instances with role=webservers OR role=dbservers tag
+# instance_filters = tag:role=webservers,tag:role=dbservers
+
+# Retrieve only t1.micro instances OR instances with tag env=staging
+# instance_filters = instance-type=t1.micro,tag:env=staging
+
+# You can use wildcards in filter values also. Below will list instances which
+# tag Name value matches webservers1*
+# (ex. webservers15, webservers1a, webservers123 etc)
+# instance_filters = tag:Name=webservers1*
+
+# An IAM role can be assumed, so all requests are run as that role.
+# This can be useful for connecting across different accounts, or to limit user
+# access
+# iam_role = role-arn
+
+# A boto configuration profile may be used to separate out credentials
+# see http://boto.readthedocs.org/en/latest/boto_config_tut.html
+# boto_profile = some-boto-profile-name
+
+
+[credentials]
+
+# The AWS credentials can optionally be specified here. Credentials specified
+# here are ignored if the environment variable AWS_ACCESS_KEY_ID or
+# AWS_PROFILE is set, or if the boto_profile property above is set.
+#
+# Supplying AWS credentials here is not recommended, as it introduces
+# non-trivial security concerns. When going down this route, please make sure
+# to set access permissions for this file correctly, e.g. handle it the same
+# way as you would a private SSH key.
+#
+# Unlike the boto and AWS configure files, this section does not support
+# profiles.
+#
+# aws_access_key_id = AXXXXXXXXXXXXXX
+# aws_secret_access_key = XXXXXXXXXXXXXXXXXXX
+# aws_security_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXX

ansible/reinit.yml

@@ -0,0 +1,8 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  user: root
+  roles:
+    - cleanupconfig
+    - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
+

ansible/reset.yml

@@ -0,0 +1,9 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  user: root
+  roles:
+    - stop
+    - cleanupconfig
+    - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
+    - start

ansible/restart.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  user: root
+  roles:
+    - stop
+    - start

ansible/roles/basecoin/tasks/start.yml

@@ -1,26 +0,0 @@
----
-
-- name: copy genesis file
-  template: >
-    src=genesis.json.j2
-    dest={{tendermint_home}}/genesis.json
-    owner={{tendermint_user}}
-    group={{tendermint_group}}
-    mode=0644
-  register: genesis_file_copied
-
-- name: get validators number from genesis file
-  shell: cat {{tendermint_home}}/genesis.json | jq ".validators | length"
-  register: num_validators
-  when: genesis_file_copied|changed
-
-- name: restart tendermint if number of validators greater than 0
-  shell: echo "restarting ..."
-  when: genesis_file_copied|changed and num_validators.stdout|int > 0
-  notify: restart tendermint
-
-- service: >
-    name=tendermint
-    state=started
-    enabled=yes
-  when: tendermint_manage_service

ansible/roles/cleanupconfig/defaults/main.yml

@@ -0,0 +1,6 @@
+---
+tendermint_home: /var/lib/tendermint
+
+basecoin_home: /var/lib/basecoin
+basecoin_inprocess: false
+

ansible/roles/cleanupconfig/tasks/basecoin.yml

@@ -0,0 +1,5 @@
+---
+
+- name: clear basecoin configuration folder
+  file: "path={{basecoin_home}}/.basecoin state=absent"
+

ansible/roles/cleanupconfig/tasks/main.yml

@@ -0,0 +1,9 @@
+---
+
+- include: tendermint.yml
+  when: not basecoin_inprocess|bool
+  tags: config-tendermint
+
+- include: basecoin.yml
+  tags: config-basecoin
+

ansible/roles/cleanupconfig/tasks/tendermint.yml

@@ -0,0 +1,5 @@
+---
+
+- name: clear tendermint configuration folder
+  file: "path={{tendermint_home}}/.tendermint state=absent"
+

ansible/roles/config/defaults/main.yml

@@ -0,0 +1,12 @@
+---
+tendermint_user: tendermint
+tendermint_group: tendermint
+tendermint_home: /var/lib/tendermint
+
+basecoin_user: basecoin
+basecoin_group: basecoin
+basecoin_home: /var/lib/basecoin
+basecoin_inprocess: false
+
+testnet_name: test-chain
+

ansible/roles/config/tasks/basecoin.yml

@@ -0,0 +1,43 @@
+---
+
+- name: set basecoin configuration folder
+  file: "path={{basecoin_home}}/.basecoin state=directory mode=0700 owner={{basecoin_user}} group={{basecoin_group}}"
+
+- name: generate basecoin keys
+  when: basecoin_inprocess|bool
+  shell: "tendermint gen_validator > {{basecoin_home}}/.basecoin/priv_validator.json && chmod 0400 {{basecoin_home}}/.basecoin/priv_validator.json"
+  args:
+    warn: no
+    creates: "{{basecoin_home}}/.basecoin/priv_validator.json"
+  become: yes
+  become_user: "{{basecoin_user}}"
+
+- name: gather basecoin public keys
+  when: basecoin_inprocess|bool
+  command: "jq '.pub_key | .data' {{basecoin_home}}/.basecoin/priv_validator.json"
+  become: yes
+  become_user: "{{basecoin_user}}"
+  register: basecoinpubkeys
+  changed_when: false
+
+- name: register basecoin public keys as host facts
+  when: basecoin_inprocess|bool
+  set_fact: "basecoin_pubkey={{basecoinpubkeys.stdout_lines[0]}}"
+  connection: local
+
+- name: copy basecoin genesis.json - genesis_time will be updated
+  when: basecoin_inprocess|bool
+  template:
+    src: genesis-basecoin.json.j2
+    dest: "{{basecoin_home}}/.basecoin/genesis.json"
+  become: yes
+  become_user: "{{basecoin_user}}"
+
+- name: copy basecoin config.toml
+  when: basecoin_inprocess|bool
+  template:
+    src: config.toml.j2
+    dest: "{{basecoin_home}}/.basecoin/config.toml"
+  become: yes
+  become_user: "{{basecoin_user}}"
+

ansible/roles/config/tasks/main.yml

@@ -0,0 +1,9 @@
+---
+
+- include: tendermint.yml
+  when: not basecoin_inprocess|bool
+  tags: config-tendermint
+
+- include: basecoin.yml
+  tags: config-basecoin
+

ansible/roles/config/tasks/tendermint.yml

@@ -0,0 +1,38 @@
+---
+
+- name: set tendermint configuration folder
+  file: "path={{tendermint_home}}/.tendermint state=directory mode=0700 owner={{tendermint_user}} group={{tendermint_group}}"
+
+- name: generate tendermint keys
+  shell: "tendermint gen_validator > {{tendermint_home}}/.tendermint/priv_validator.json && chmod 0400 {{tendermint_home}}/.tendermint/priv_validator.json"
+  args:
+    warn: no
+    creates: "{{tendermint_home}}/.tendermint/priv_validator.json"
+  become: yes
+  become_user: "{{tendermint_user}}"
+
+- name: gather tendermint public keys
+  command: "jq '.pub_key | .data' {{tendermint_home}}/.tendermint/priv_validator.json"
+  become: yes
+  become_user: "{{tendermint_user}}"
+  register: tendermintpubkeys
+  changed_when: false
+
+- name: register tendermint public keys as host facts
+  set_fact: "tendermint_pubkey={{tendermintpubkeys.stdout_lines[0]}}"
+  connection: local
+
+- name: copy tendermint genesis.json - genesis_time will be updated
+  template:
+    src: genesis-tendermint.json.j2
+    dest: "{{tendermint_home}}/.tendermint/genesis.json"
+  become: yes
+  become_user: "{{tendermint_user}}"
+
+- name: copy tendermint config.toml
+  template:
+    src: config.toml.j2
+    dest: "{{tendermint_home}}/.tendermint/config.toml"
+  become: yes
+  become_user: "{{tendermint_user}}"
+

ansible/roles/config/templates/config.toml.j2

@@ -0,0 +1,16 @@
+# This is a TOML config file.
+# For more information, see https://github.com/toml-lang/toml
+
+proxy_app = "tcp://127.0.0.1:46658"
+moniker = "{{inventory_hostname}}"
+fast_sync = true
+db_backend = "leveldb"
+log_level = "debug"
+
+[rpc]
+laddr = "tcp://{{ do_private_ip | default('0.0.0.0') }}:46657"
+
+[p2p]
+laddr = "tcp://{{inventory_hostname}}:46656"
+{% set comma = joiner(",") %}seeds = "{% for host in groups[testnet_name]|difference(inventory_hostname) %}{{ comma() }}{{hostvars[host]["inventory_hostname"]}}:46656{% endfor %}"
+

ansible/roles/config/templates/genesis-basecoin.json.j2

@@ -0,0 +1,32 @@
+{
+  "app_options": {
+    "accounts": [{
+      "pub_key": {
+        "type": "ed25519",
+        "data": "619D3678599971ED29C7529DDD4DA537B97129893598A17C82E3AC9A8BA95279"
+      },
+      "coins": [
+        {
+          "denom": "mycoin",
+          "amount": 9007199254740992
+        }
+      ]
+    }]
+  },
+  "genesis_time":"{{ansible_date_time.iso8601_micro}}",
+  "chain_id":"{{testnet_name}}",
+  "validators":
+  [
+{% set comma = joiner(",") %}
+{% for host in groups[testnet_name] %}
+    {{ comma() }}
+    {
+      "pub_key":{"type":"ed25519","data":{{hostvars[host]["basecoin_pubkey"]}}},
+      "amount":10,
+      "name":"{{hostvars[host]["ansible_hostname"]}}"
+    }
+{% endfor %}
+  ],
+  "app_hash":""
+}
+

ansible/roles/config/templates/genesis-tendermint.json.j2

@@ -0,0 +1,32 @@
+{
+  "app_options": {
+    "accounts": [{
+      "pub_key": {
+        "type": "ed25519",
+        "data": "619D3678599971ED29C7529DDD4DA537B97129893598A17C82E3AC9A8BA95279"
+      },
+      "coins": [
+        {
+          "denom": "mycoin",
+          "amount": 9007199254740992
+        }
+      ]
+    }]
+  },
+  "genesis_time":"{{ansible_date_time.iso8601}}",
+  "chain_id":"{{testnet_name}}",
+  "validators":
+  [
+{% set comma = joiner(",") %}
+{% for host in groups[testnet_name] %}
+    {{ comma() }}
+    {
+      "pub_key":{"type":"ed25519","data":{{hostvars[host]["tendermint_pubkey"]}}},
+      "amount":10,
+      "name":"{{hostvars[host]["ansible_hostname"]}}"
+    }
+{% endfor %}
+  ],
+  "app_hash":""
+}
+

ansible/roles/install-basecoin/defaults/main.yml

@@ -1,5 +1,5 @@
 ---
-basecoin_release_install: false
+basecoin_release_install: true
 basecoin_version: 0.4.0
 basecoin_download: "https://s3-us-west-2.amazonaws.com/tendermint/binaries/basecoin/v{{basecoin_version}}/basecoin_{{basecoin_version}}_linux_amd64.zip"
 
@@ -12,12 +12,9 @@ basecoin_group: basecoin
 # Upstart start/stop conditions can vary by distribution and environment
 basecoin_upstart_start_on: start on runlevel [345]
 basecoin_upstart_stop_on: stop on runlevel [!345]
-basecoin_manage_service: true
-basecoin_use_upstart: true
-basecoin_use_systemd: false
-basecoin_upstart_template: "basecoin.conf.j2"
-basecoin_systemd_template: "basecoin.systemd.j2"
 
 basecoin_home: /var/lib/basecoin
 basecoin_log_file: /var/log/basecoin.log
 
+basecoin_inprocess: false
+

ansible/roles/install-basecoin/files/key.json

@@ -0,0 +1,11 @@
+{
+  "address": "1B1BE55F969F54064628A63B9559E7C21C925165",
+  "priv_key": {
+    "type": "ed25519",
+    "data": "C70D6934B4F55F1B7BC33B56B9CA8A2061384AFC19E91E44B40C4BBA182953D1619D3678599971ED29C7529DDD4DA537B97129893598A17C82E3AC9A8BA95279"
+  },
+  "pub_key": {
+    "type": "ed25519",
+    "data": "619D3678599971ED29C7529DDD4DA537B97129893598A17C82E3AC9A8BA95279"
+  }
+}

ansible/roles/install-basecoin/files/key2.json

@@ -0,0 +1,11 @@
+{
+  "address": "1DA7C74F9C219229FD54CC9F7386D5A3839F0090",
+  "priv_key": {
+    "type": "ed25519",
+    "data": "34BAE9E65CE8245FAD035A0E3EED9401BDE8785FFB3199ACCF8F5B5DDF7486A8352195DA90CB0B90C24295B90AEBA25A5A71BC61BAB2FE2387241D439698B7B8"
+  },
+  "pub_key": {
+    "type": "ed25519",
+    "data": "352195DA90CB0B90C24295B90AEBA25A5A71BC61BAB2FE2387241D439698B7B8"
+  }
+}

ansible/roles/install-basecoin/tasks/install.yml

@@ -69,7 +69,7 @@
   changed_when: false
 
 - name: copy and unpack release binary
-  when: basecoin_release_install
+  when: basecoin_release_install|bool
   unarchive: >
     src={{basecoin_download}}
     dest=/usr/local/bin
@@ -77,33 +77,38 @@
     mode=0755
 
 - name: copy compiled binary
-  when: not basecoin_release_install
+  when: not basecoin_release_install|bool
   copy: >
     src={{basecoin_binary}}
     dest=/usr/local/bin
     mode=0755
 
-- name: initialize basecoin
-  command: "basecoin init"
-  become: yes
-  become_user: "{{basecoin_user}}"
-
 - name: copy upstart script
   template: >
-    src={{basecoin_upstart_template}}
+    src=basecoin.conf.j2
     dest=/etc/init/basecoin.conf
     owner=root
     group=root
     mode=0644
-  when: basecoin_use_upstart
+  when: "ansible_service_mgr == 'upstart'"
 
 - name: copy systemd script
   template: >
-    src={{basecoin_systemd_template}}
+    src=basecoin.systemd.j2
     dest=/etc/systemd/system/basecoin.service
     owner=root
     group=root
     mode=0644
-  when: basecoin_use_systemd
+  when: "ansible_service_mgr == 'systemd'"
   notify:
     - reload systemd
+
+- name: Create example folder
+  file: path=/usr/share/basecoin/example-keys state=directory
+
+- name: Copy example keys
+  copy: "src={{item}} dest=/usr/share/basecoin/example-keys/{{item}}"
+  with_items:
+  - key.json
+  - key2.json
+

ansible/roles/install-basecoin/tasks/main.yml

@@ -5,4 +5,4 @@
     - "{{ ansible_os_family }}-{{ ansible_distribution_major_version}}.yml"
     - "{{ ansible_os_family }}.yml"
 - include: install.yml
-#- include: start.yml
+

ansible/roles/install-basecoin/templates/basecoin.conf.j2

@@ -6,7 +6,7 @@ stop on {{ basecoin_upstart_stop_on }}
 
 script
   # Use su to become tendermint user non-interactively on old Upstart versions (see http://superuser.com/a/234541/76168)
-  exec su -s /bin/sh -c 'TMROOT={{basecoin_home}} exec "$0" "$@" >> {{ basecoin_log_file }} 2>&1' {{basecoin_user}} -- /usr/local/bin/basecoin start --without-tendermint
+  exec su -s /bin/sh -c 'BCHOME={{basecoin_home}}/.basecoin exec "$0" "$@" >> {{ basecoin_log_file }} 2>&1' {{basecoin_user}} -- /usr/local/bin/basecoin start{{basecoin_inprocess|bool | ternary('',' --without-tendermint')}}
 end script
 
 respawn

ansible/roles/install-basecoin/templates/basecoin.systemd.j2

@@ -4,12 +4,12 @@ Requires=network-online.target
 After=network-online.target
 
 [Service]
-Environment="TMROOT={{tendermint_home}}"
+Environment="BCHOME={{basecoin_home}}/.basecoin"
 Restart=on-failure
-User={{ tendermint_user }}
-Group={{ tendermint_group }}
+User={{ basecoin_user }}
+Group={{ basecoin_group }}
 PermissionsStartOnly=true
-ExecStart=/usr/local/bin/basecoin start --without-tendermint
+ExecStart=/usr/local/bin/basecoin start{{basecoin_inprocess|bool | ternary('',' --without-tendermint')}}
 ExecReload=/bin/kill -HUP $MAINPID
 KillSignal=SIGINT
 

ansible/roles/install-tendermint/defaults/main.yml

@@ -1,5 +1,5 @@
 ---
-tendermint_release_install: false
+tendermint_release_install: true
 tendermint_version: 0.10.0-rc1
 tendermint_download: "https://s3-us-west-2.amazonaws.com/tendermint/binaries/tendermint/v{{tendermint_version}}/tendermint_{{tendermint_version}}_linux_amd64.zip"
 
@@ -12,11 +12,6 @@ tendermint_group: tendermint
 # Upstart start/stop conditions can vary by distribution and environment
 tendermint_upstart_start_on: start on runlevel [345]
 tendermint_upstart_stop_on: stop on runlevel [!345]
-tendermint_manage_service: true
-tendermint_use_upstart: true
-tendermint_use_systemd: false
-tendermint_upstart_template: "tendermint.conf.j2"
-tendermint_systemd_template: "tendermint.systemd.j2"
 
 tendermint_home: /var/lib/tendermint
 tendermint_log_file: /var/log/tendermint.log

ansible/roles/install-tendermint/tasks/install.yml

@@ -69,7 +69,7 @@
   changed_when: false
 
 - name: copy and unpack release binary
-  when: tendermint_release_install
+  when: tendermint_release_install|bool
   unarchive: >
     src={{tendermint_download}}
     dest=/usr/local/bin
@@ -77,33 +77,28 @@
     mode=0755
 
 - name: copy compiled binary
-  when: not tendermint_release_install
+  when: not tendermint_release_install|bool
   copy: >
     src={{tendermint_binary}}
     dest=/usr/local/bin
     mode=0755
 
-- name: initialize tendermint
-  command: "tendermint init"
-  become: yes
-  become_user: "{{tendermint_user}}"
-
 - name: copy upstart script
   template: >
-    src={{tendermint_upstart_template}}
+    src=tendermint.conf.j2
     dest=/etc/init/tendermint.conf
     owner=root
     group=root
     mode=0644
-  when: tendermint_use_upstart
+  when: "ansible_service_mgr == 'upstart'"
 
 - name: copy systemd script
   template: >
-    src={{tendermint_systemd_template}}
+    src=tendermint.systemd.j2
     dest=/etc/systemd/system/tendermint.service
     owner=root
     group=root
     mode=0644
-  when: tendermint_use_systemd
+  when: "ansible_service_mgr == 'systemd'"
   notify:
     - reload systemd

ansible/roles/install-tendermint/tasks/main.yml

@@ -5,5 +5,4 @@
     - "{{ ansible_os_family }}-{{ ansible_distribution_major_version}}.yml"
     - "{{ ansible_os_family }}.yml"
 - include: install.yml
-- include: genesis.yml
-#- include: start.yml
+

ansible/roles/install-tendermint/templates/tendermint.conf.j2

@@ -6,7 +6,7 @@ stop on {{ tendermint_upstart_stop_on }}
 
 script
   # Use su to become tendermint user non-interactively on old Upstart versions (see http://superuser.com/a/234541/76168)
-  exec su -s /bin/sh -c 'TMROOT={{tendermint_home}} exec "$0" "$@" >> {{ tendermint_log_file }} 2>&1' {{ tendermint_user }} -- /usr/local/bin/tendermint node --moniker={{inventory_hostname}}
+  exec su -s /bin/sh -c 'TMHOME={{tendermint_home}}/.tendermint exec "$0" "$@" >> {{ tendermint_log_file }} 2>&1' {{ tendermint_user }} -- /usr/local/bin/tendermint node
 end script
 
 respawn

ansible/roles/install-tendermint/templates/tendermint.systemd.j2

@@ -4,12 +4,12 @@ Requires=network-online.target
 After=network-online.target
 
 [Service]
-Environment="TMROOT={{tendermint_home}}"
+Environment="TMHOME={{tendermint_home}}/.tendermint"
 Restart=on-failure
 User={{ tendermint_user }}
 Group={{ tendermint_group }}
 PermissionsStartOnly=true
-ExecStart=/usr/local/bin/tendermint node --moniker={{inventory_hostname}}
+ExecStart=/usr/local/bin/tendermint node
 ExecReload=/bin/kill -HUP $MAINPID
 KillSignal=SIGINT
 

ansible/roles/start/defaults/main.yml

@@ -0,0 +1,3 @@
+---
+basecoin_inprocess: false
+

ansible/roles/start/tasks/main.yml

@@ -0,0 +1,9 @@
+---
+
+- name: start basecoin
+  service: name=basecoin state=started
+
+- name: start tendermint
+  when: not basecoin_inprocess|bool
+  service: name=tendermint state=started
+

ansible/roles/stop/defaults/main.yml

@@ -0,0 +1,3 @@
+---
+basecoin_inprocess: false
+

ansible/roles/stop/tasks/main.yml

@@ -0,0 +1,9 @@
+---
+
+- name: stop tendermint
+  when: not basecoin_inprocess|bool
+  service: name=tendermint state=stopped
+
+- name: stop basecoin
+  service: name=basecoin state=stopped
+

ansible/roles/tendermint/tasks/genesis.yml

@@ -1,19 +0,0 @@
----
-- name: gather tendermint public keys
-  command: "jq '.pub_key | .data' {{tendermint_home}}/.tendermint/priv_validator.json"
-  become: yes
-  become_user: "{{tendermint_user}}"
-  register: tendermintpubkeys
-
-- name: register public keys as host facts
-  set_fact: "pubkey={{tendermintpubkeys.stdout_lines[0]}}"
-  connection: local
-
-- name: copy genesis.json
-  template:
-    src: genesis.json.j2
-    dest: "{{tendermint_home}}/.tendermint/genesis.json"
-  become: yes
-  become_user: "{{tendermint_user}}"
-
-

ansible/roles/tendermint/tasks/start.yml

@@ -1,26 +0,0 @@
----
-
-- name: copy genesis file
-  template: >
-    src=genesis.json.j2
-    dest={{tendermint_home}}/genesis.json
-    owner={{tendermint_user}}
-    group={{tendermint_group}}
-    mode=0644
-  register: genesis_file_copied
-
-- name: get validators number from genesis file
-  shell: cat {{tendermint_home}}/genesis.json | jq ".validators | length"
-  register: num_validators
-  when: genesis_file_copied|changed
-
-- name: restart tendermint if number of validators greater than 0
-  shell: echo "restarting ..."
-  when: genesis_file_copied|changed and num_validators.stdout|int > 0
-  notify: restart tendermint
-
-- service: >
-    name=tendermint
-    state=started
-    enabled=yes
-  when: tendermint_manage_service

ansible/roles/tendermint/templates/genesis.json.j2

@@ -1,16 +0,0 @@
-{
-  "genesis_time":"{{ansible_date_time.iso8601}}",
-  "chain_id":"{{testnet_name}}",
-  "validators":
-  [
-{% for host in groups[testnet_name] %}
-    {
-      "pub_key":{"type":"ed25519","data":{{hostvars[host]["pubkey"]}}},
-      "amount":10,
-      "name":"{{hostvars[host]["ansible_hostname"]}}"
-    }
-{% endfor %}
-  ],
-  "app_hash":""
-}
-

ansible/start.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  user: root
+  roles:
+    - start
+

ansible/stop.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  user: root
+  roles:
+    - stop
+

create-digitalocean-testnet.sh

@@ -2,10 +2,16 @@
 # This is an example set of commands that uses Terraform and Ansible to create a testnet on Digital Ocean.
 
 # Prerequisites: terraform, ansible, DigitalOcean API token, ssh-agent running with the same SSH keys added that are set up during terraform
-
+# Optional: GOPATH if you build the app yourself
 #export DO_API_TOKEN="<This contains the DigitalOcean API token>"
+#export GOPATH="<your go path>"
 
-TF_VAR_TESTNET_NAME="$1"
+###
+# Find out TF_VAR_TESTNET_NAME (testnet name)
+###
+if [ $# -gt 0 ]; then
+  TF_VAR_TESTNET_NAME="$1"
+fi
 
 if [ -z "$TF_VAR_TESTNET_NAME" ]; then
   echo "Usage: $0 <TF_VAR_TESTNET_NAME>"
@@ -14,15 +20,38 @@ if [ -z "$TF_VAR_TESTNET_NAME" ]; then
   exit
 fi
 
-cd terraforce
+###
+# Build Digital Ocean infrastructure
+###
+SERVERS=2
+cd terraform-digitalocean
 terraform init
 terraform env new "$TF_VAR_TESTNET_NAME"
-terraform apply -var servers=4 -var DO_API_TOKEN="$DO_API_TOKEN"
+terraform apply -var servers=$SERVERS -var DO_API_TOKEN="$DO_API_TOKEN"
 cd ..
 
+###
+# Build applications (optional)
+###
+if [ -n "$GOPATH" ]; then
+  go get -u github.com/tendermint/tendermint/cmd/tendermint
+  go get -u github.com/tendermint/basecoin/cmd/basecoin
+  ANSIBLE_ADDITIONAL_VARS="-e tendermint_release_install=false -e basecoin_release_intall=false"
+fi
+
+###
+# Deploy application
+###
 #Note that SSH Agent needs to be running with SSH keys added or ansible-playbook requires the --private-key option.
 cd ansible
 python -u inventory/digital_ocean.py --refresh-cache 1> /dev/null
-ansible-playbook -i inventory/digital_ocean.py install.yml
+ansible-playbook -i inventory/digital_ocean.py install.yml $ANSIBLE_ADDITIONAL_VARS
+cd ..
+
+###
+# Start application
+###
+cd ansible
+ansible-playbook -i inventory/digital_ocean.py start.yml
 cd ..
 

terraforce/scripts/copy_run.sh

@@ -1,10 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-RUN=$2 # path to run script
-
-N_=$((N-1))
-
-# stop all tendermint
-terraforce scp --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" $RUN run.sh

terraforce/scripts/init.sh

@@ -1,43 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-TESTNET=$2 # path to folder containing testnet info
-CONFIG=$3 # path to folder containing `bins` and `run.sh` files
-
-if [[ ! -f $CONFIG/bins ]]; then
-	echo "config folder ($CONFIG) must contain bins file"
-	exit 1
-fi
-if [[ ! -f $CONFIG/run.sh ]]; then
-	echo "config folder ($CONFIG) must contain run.sh file"
-	exit 1
-fi
-
-KEY=$HOME/.ssh/id_rsa
-
-FLAGS="-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
-
-N_=$((N-1)) # 0-based index
-
-MACH_ROOT="$TESTNET/mach?"
-
-
-# mkdir
-terraforce ssh --user root --ssh-key $KEY  --machines "[0-$N_]" mkdir .tendermint
-
-# copy over  genesis/priv_val
-terraforce scp --user root --ssh-key $KEY  --iterative --machines "[0-$N_]" "$MACH_ROOT/priv_validator.json" .tendermint/priv_validator.json
-terraforce scp --user root --ssh-key $KEY  --iterative --machines "[0-$N_]" "$MACH_ROOT/genesis.json" .tendermint/genesis.json
-
-# copy the run script
-terraforce scp --user root --ssh-key $KEY --machines "[0-$N_]" $CONFIG/run.sh run.sh 
-
-# copy the binaries
-while read line; do
-	local_bin=$(eval echo $line)
-  	remote_bin=$(basename $local_bin)
-  	echo $local_bin
- 	terraforce scp --user root --ssh-key $KEY --machines "[0-$N_]" $local_bin $remote_bin
- 	terraforce ssh --user root --ssh-key $KEY --machines "[0-$N_]" chmod +x $remote_bin
-done <$CONFIG/bins

terraforce/scripts/query.sh

@@ -1,11 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-QUERY=$2
-
-N_=$((N-1))
-
-# start all tendermint nodes
-terraforce ssh --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" curl -s localhost:46657/$QUERY
-

terraforce/scripts/reset.sh

@@ -1,10 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-
-N_=$((N-1))
-
-# stop all tendermint
-terraforce ssh --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" rm -rf .tendermint/data
-terraforce ssh --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" ./tendermint unsafe_reset_priv_validator

terraforce/scripts/restart.sh

@@ -1,9 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-
-N_=$((N-1))
-
-# start
-terraforce ssh --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" SEEDS=$(terraform output seeds) bash run.sh

terraforce/scripts/start.sh

@@ -1,10 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-
-N_=$((N-1))
-
-# start all tendermint nodes
-terraforce ssh --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" SEEDS=$(terraform output seeds) bash run.sh
-

terraforce/scripts/stop.sh

@@ -1,9 +0,0 @@
-#! /bin/bash
-set -u
-
-N=$1 # number of nodes
-
-N_=$((N-1))
-
-# stop all tendermint
-terraforce ssh --user root --ssh-key $HOME/.ssh/id_rsa --machines "[0-$N_]" killall tendermint

terraform-aws/README.md

@@ -0,0 +1,4 @@
+# Terraform for Amazon AWS
+
+To be done...
+

terraform-digitalocean/main.tf

@@ -1,4 +1,4 @@
-#DigitalOcean Terraform Configuration
+#Terraform Configuration
 
 variable "DO_API_TOKEN" {
   description = "DigitalOcean Access Token"