The Challenge and Solution

Starting a new project from scratch can often be a daunting task, especially when it involves setting up infrastructure and provisioning hosts. I have used Python Cookiecutter in the past to generate project templates for python command line applications. In this project, I use it to generate a template for Ansible provisioning.

Usage

  1. Install the latest version of Cookiecutter. You can find other options here.
pip install -U cookiecutter
  1. Use Cookiecutter to generate an Ansible project using the template here
cookiecutter https://github.com/zer0ttl/cookiecutter-ansible-setup.git

A new directory will be created. By default the directory is ansible. You will be prompted to change the directory when setting up the project.

The file_name is the name of the primary playbook that will be run. By default it is main.yml. You can change it during the setup.

$ cookiecutter https://github.com/zer0ttl/cookiecutter-ansible-setup.git
  [1/2] directory_name (ansible): my-provisioning-project
  [2/2] file_name (main):
  1. cd into the project directory and start using the project.
cd my-provisioning-project/
ansible -i inventories/production main.yml

By default, the ansible setup will run against the localhost.

$ ansible-playbook -i inventories/production main.yml

PLAY [Read data files] ***************************************************************************************************************************

TASK [Save the Json data to a Variable as a Fact] ************************************************************************************************
ok: [host1]
ok: [host2]

PLAY [Running play on group1] ********************************************************************************************************************

TASK [my-role : Access and use hostvars on host1] ************************************************************************************************
ok: [host1] => {
    "msg": [
        "host_var1 is of type: AnsibleUnicode",
        "host_var1 is        : host1_var1_value1"
    ]
}

# SOME OUTPUT SNIPPED

PLAY RECAP ***************************************************************************************************************************************
host1                      : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
host2                      : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Ansible Template

Following is the Ansible project layout and the description for each file. You can skip over the next sections if you can read through this.

my-provisioning-project/
├── ansible.cfg             # ansible.cfg file
├── data                    # data folder for extra config
│   └── config.json         # extra config
├── data.yml                # load extra config from data folder
├── inventories             # inventory files
│   ├── production          # inventory and var files for production hosts
│   │   ├── group_vars      # group vars
│   │   │   ├── all.yml     # here we assign variables to all production hosts
│   │   │   ├── group1.yml  # here we assign variables to group1
│   │   │   └── group2.yml  # here we assign variables to group1
│   │   ├── hosts.yml       # inventory file for production hosts
│   │   └── host_vars       # host vars
│   │       ├── host1.yml   # here we assign variables specific to host1
│   │       └── host2.yml   # here we assign variables specific to host1
│   ├── staging             # inventory and var files for staging hosts
│   │   ├── group_vars      # group vars
│   │   │   ├── all.yml     # here we assign variables to all staging hosts
│   │   │   ├── group1.yml  # here we assign variables to group1
│   │   │   └── group2.yml  # here we assign variables to group1
│   │   ├── hosts.yml       # inventory file for staging hosts
│   │   └── host_vars       # host vars
│   │       ├── host1.yml   # here we assign variables specific to host1
│   │       └── host2.yml   # here we assign variables specific to host1
├── main.yml                # main playbook
├── playbook_group1.yml     # playbook for group1
├── playbook_group2.yml     # playbook for group1
├── README.md               # 
├── requirements.yml        # here we list required roles and modules
└── roles                   # roles
    └── my-role             # role: my-role
        ├── tasks           #
        │   └── main.yml    # main task file for role
        └── vars            # 
            └── main.yml    # variables associated with this role

The files ansible.cfg and requirements.yml are self explanatory. The README.md file has some examples of running the playbooks.

Playbooks

main.yml is the mail playbook. It imports the plays from playbook_group1.yml and playbook_group2.yml. The two playbooks have some sample plays

Inventories

my-provisioning-project/
# SOME OUTPUT SNIPPED
├── inventories
│   ├── production
│   │   ├── group_vars
│   │   │   ├── all.yml
│   │   │   ├── group1.yml
│   │   │   └── group2.yml
│   │   ├── hosts.yml
│   │   └── host_vars
│   │       ├── host1.yml
│   │       └── host2.yml
│   └── staging
│       ├── group_vars
│       │   ├── all.yml
│       │   ├── group1.yml
│       │   └── group2.yml
│       ├── hosts.yml
│       └── host_vars
│           ├── host1.yml
│           └── host2.yml
# SOME OUTPUT SNIPPED

There are two environments: production and staging. You can add more. Each environment has its own group_vars and host_vars directories. Group specific variables are defined under group_vars folders. Host specific variables are defined under host_vars folders.

hosts.yml file is used to group the hosts.

All the hosts in the inventories are localhost. They are grouped as follows:

@all:
  |--@ungrouped:
  |--@group1:
  |  |--host1
  |--@group2:
  |  |--host2

Roles

my-provisioning-project/
# SOME OUTPUT SNIPPED
└── roles
    └── my-role
        ├── tasks
        │   └── main.yml
        └── vars
            └── main.yml
# SOME OUTPUT SNIPPED

Roles go into the roles directory. my-role is a sample role that has examples of that reference and use variables defined at different levels like host vars, group vars, role vars, and so on.

Data

my-provisioning-project/
# SOME OUTPUT SNIPPED
├── data
│   └── config.json
├── data.yml
# SOME OUTPUT SNIPPED

Sometimes you might have a requirement to include additional configuration in your playbooks. For such cases, there is provision to use the data folder. There is an example configuration in data/config.json. This file is then referenced by the data.yml playbook and sets all the read information as ansible facts.

These facts can then be used in your playbooks or roles.

Playbook Patterns

The Ansible tempalte enables you to run playbooks by environment, group, host, or tags.

Environment Specific Plays

ansible-playbook main.yml -i inventories/production
ansible-playbook main.yml -i inventories/staging

Group Specific Plays

ansible-playbook main.yml -i inventories/production --limit group1
ansible-playbook main.yml -i inventories/staging --limit group2

Host Specific Plays

ansible-playbook main.yml -i inventories/production --limit host1
ansible-playbook main.yml -i inventories/staging --limit host2

Tag Specific Plays

ansible-playbook main.yml -i inventories/production -t my-tag1
ansible-playbook main.yml -i inventories/production -t tag-group1

Challenges

As Cookiecutter and Ansible both use jinja2 templating engine, I had an issue with Ansible variables. Cookiecutter would try to interpret the Ansible variables. The following cookiecutter.json config helped resolve the issue.

{
  "_copy_without_render": [
    "*.yml"
  ]
}

References