Batch execution
I’ve used a few batch execution tools. Some more deeply than others. The two I’ve spent the most time with are Ansible and Fabric. At the moment we make regular use of a collection of fabric scripts that we’ve written over the years. I’m spending some time writing a (toy) replacement for Fabric that will hopefully make some of the things we want to do with fabric possible.
Ansible
It’s been a very long time since I last used Ansible. My gripes with it at the time (having being the person that initially advocated for it), were largely around the buy-in required to properly use it. Playbooks and Roles aren’t just logical abstractions like an “Inventory” is; they have technical meaning. For example, a playbook is not simply a list of commands to run. It is a specific format utilising Jinja templating and it’s own flavour of YAML to provide basic programatic functionality like conditions. Similarly, roles aren’t just a collection of playbooks that run together - they come with files, variables, templates and metdata represented in a hierarchical directory structure. No you don’t need to use these things. But you’ll suffer less in the long run by buying into it.
Honestly - Ansible is probably a good fit for us these days. But owing to my views, I don’t intend to advocate for its reintroduction. It’s too much work at the moment, and doesn’t have discernable benefits over simpler solutions unless we were to go all-in.
Fabric
Fabric replaced Ansible for ourselves. It offered the ability to execute a simple series of commands on a remote host in the context of a real language (Python, as opposed to YAML), and had no meaningful server-side dependencies aside from SSH. Furthermore, Fabric allowed us to buy into it gradually.
While entirely possible to take the stdout of a command and parse it with Python (and I would recommend doing this), there’s nothing preventing a user utilising pipes in the command itself. The people I work with know Bash, they don’t all know Python. But anyone comfortable with any procedural language can figure out how to edit a fabric script to have it run a different command.
Mechanical Turk
Mechanical Turk is a thin wrapper around the Ruby net/ssh SSH implementation that will eventually provide (among other things):
- Loading groups of hosts from a TOML file
- Multiple sources of auth (prompt, authinfo.gpg)
- Executing one or more commands on those hosts
- Collating the results of commands into a report
- Allow printing that report in either plain text or json
This project is half playing around with Ruby, which I haven’t touched in years. Half trying to build the small modules I’ve wrapped around Fabric over time into something cohesive.
As for the name - I was reading “The Mechanical Turk: The True Story of the Chess-Playing Machine That Fooled the World” (ISBN 0713995254). It was just interesting to draw comparison between an automota that was really just a puppet, and a remote execution system that’s really just some bash commands over a SSH tunnel.