Git Cycle

Git Cycle is a process that I created to help in small-ish environments where Linux servers are often unique little snowflakes and the admins are accustomed to making changes on the fly rather than through config management. By using the Git Cycle, I only had to convince the admins to git add and git commit the config files they were changing on those systems.

I developed this methodology using SaltStack as my config management system, but there’s no reason why it can’t be used in other systems. However, everything here is based on using SaltStack.

Overview Example:
1. An Admin makes a change in the nrpe config file, /etc/nagios/nrpe.cfg
2. The Admin then git adds and commits /etc/nagios/nrpe.cfg
3. From the Salt Master, the git_cycle.sh script is kicked off via crontab.
4. It connects to this server and pulls the changes from a git repo
5. The next time I execute a state run in the config management system, the changes that were found and pulled back will be used, thus avoiding overwriting the changes that the admin made.

How it works:

On the Salt Master we need to do some setup work:

1. Create a git user and group.
2. Generate a SSH public/private key pair for the git user and put the public key into the SaltStack pillar data for git_cycle.
3. Copy the git_cycle.sh script into the git user’s home directory.
4. Create an entry in the git user’s crontab to execute the git_cycle.sh script. Bonus points for redirecting the output to a log file.
5. Create a folder to store the files from your hosts in. For me, I put those files in alongside the state files that will be using them. So, /srv/salt/hosts.
6. For each Minion I want to retrieve files for, just create a sub directory.

/srv/salt/hosts/
 └── Minion1
 ├── Minion2

Running the git_cycle/init.sls SaltStack state file on a Minion does several things:

1. It creates a gituser and group.
2. Adds a public key to the git user’s authorized_keys file.
3. Adds specified users to the git group
4. Copies a .gitconfig file to each user’s home directory.
5. It creates a git repository in the root of the minion’s filesystem. Putting it there allows files from anywhere in the filesystem to be added.

With all that done, go through and git add and commit some files on your minion.

Then, back on the Salt Master as the git user:

1. Execute the bash script git_cycle.sh. Accept when prompted about the remote hosts keys.
2. You should now see the Minion files populating in the /srv/salt/hosts/ directory.
3. Keep in mind that you have these files now when crafting your states. In the below example, salt first tries to use the host specific nrpe.cfg file that we git pull from the server. If there isn’t one, it will fall back to the default file which I keep stored with the state file.

# Drop in the nrpe.cfg file specific for this host. Or if its a new host, just use the default
{% set hostname = salt.grains.get('id') %}
nrpe_config_file:
  file.managed:
    - name: /etc/nagios/nrpe.cfg
    - source: 
      - salt://hosts/{{ hostname }}/etc/nagios/nrpe.cfg
      - salt://nrpe/nrpe.cfg
    - makedirs: True
    - user: root
    - group: root
    - mode: 644

Originally published at https://smartaleksolutions.com on December 15, 2015.