"If you disable any direct shell access to the server and force all configuration changes to be applied by running the recipe from version control, you have an excellent audit mechanism that ensures every change to the environment is logged."

Um. Yeah. You're also creating a situation where, should anything possibly go wrong (now, how could that happen), you cannot hop on the server, diagnose, and possibly do some quick-try fixes. I can see limiting shell to a small trusted subset of colleagues (sorry, devs), but not eliminating it altogether.

Automated management, "system-as-code", devops, and all that has its place. The problem with virtually all existing systems for automated configuration management (cfengine, puppet, chef, Oopsware), is that they are highly non-transitive, and effectively bin years or decades of administration experience. They're fine once you've worked out the kinks, but as anyone who's worked with these tools can tell you, the one thing they're best at is screwing up all of your systems simultaneously.

"Etch" (http://sourceforge.net/projects/etch/) is one tool I've looked at briefly that appears to take a different tack, and is amenable to taking on-host changes and incorporating them into the configuration management system itself. One of the criticisms I've seen of it is that it's rather Linux and/or specifically Debian-centric, which may well be as Debian offers some very strong tools for managing, assessing, and maintaining system state (policy, APT, debconf). While dependencies can be painful when they keep you from doing what you want to do, as with most good safety systems, it's generally because you really don't want to go there (and if you do, there are means, within the framework provided by Debian, to get you there).

One way to avoid snowflake systems is to use tools that manage dependencies, stick within them to the greatest extent possible, and where that's not an option, to put your own modificiations within that same framework.

I just use fabric.

I just start writing code to provision/configure the machine for the role in question and don't stop until I have something that can reliably take a blank-slate server and have it rolling by the end of the function.

Easiest way to do devops I've seen yet. I didn't care for chef/puppet/cfengine.

Particularly since I can cherry-pick servers for testing pretty easily with fabric before I run the code against the rest of the machines.

URL?

Not a particularly searchable term.

http://fabfile.org/

Seems to be down right now though.

According to the Internet Archive, it hasn't been online since roughly 2009.

They are up right now.

http://pypi.python.org/pypi/Fabric/0.9.0

Edit:

The primary weakness of my approach is that for convenience's sake, I rely heavily on apt-get. So hypothetically I'm tied to the debian family. (we use Ubuntu LTS of a couple different versions)

Strictly speaking this disadvantage is unnecessary, you could do all from-source builds using fabric if you wanted. Didn't seem like a constructive use of my time though.

Sample task in my fabfile:

https://gist.github.com/3086836

I'll let you guess from the function name what it does.

The magic word of devops is "idempotence".

I had a modest epiphany some weeks back thinking about server configuration management: if you want to manage configurations on a given node (to use Chef lingo), it's best to work within a system that facilitates and/or encourages this.

For a number of reasons, Debian (or faithful derivatives) comes closest to this of any platform I've encountered.

That said: fabric appears to be little more than a multi-system aware task runner. Somewhat like dsh (distributed ssh / dancer's shell): http://www.netfort.gr.jp/~dancer/software/dsh.html

... or like Capistrano, as the Debian package description suggests (modulo installing Rails).

There's a bad pimp joke around "idempotence" waiting to be made.

Your task here would seem to install Oracle's Java 7 HostSpot JVM.

How do you get around the licensing question(s) (or is that what the "-y" is for)?

That's what the -y is for.

fabric appears to be little more than a multi-system aware task runner.

And C is little more than a portable assembler ;)

There's a new project called Ansible that may be of interest to you:

http://ansible.github.com/

While that page has a long list of things they do, the important bits relevant to your comment are

1. a tighter focus on idempotence than Fabric 2. an easy-ish way to integrate package management so you could potentially use the same script to kick off either yum or apt depending on the box

The original reason I abandoned chef and puppet is that I didn't like something getting between me and the shell.

Why would I make the same mistake twice? My only real dissatisfaction with Flask is that it cannot handle dispatching its work in parallel. Not a big deal though.

I like knowing exactly how things get done.

Doesn't Fabric support parallel instructions now? I think I heard something about that, although I'm not entirely sure...

I was thinking more along the lines of parallel connections.

I did the same thing. What's nice about the Phoenix approach is that you start with a known state (base Ubuntu install or whatever) and end with a known state (running your setup) and completely ignore all other possible states. A tool like chef or puppet has to have a lot of complexity around examine a node, determine what state it's in and figure out how to get it to the desired state.

I also use boto in my fabfile, so that I can run a single command to launch an EC2 instance, install & configure software on it, save an AMI and terminate the instance. It's a like having a Makefile that compiles machine images:

    > fab build:<role> <region>

I wouldn't go as far as Fowler's "disable the shell" idea, but it doesn't take much discipline to do all significant changes by building new AMIs.

I don't believe in AMIs.

You mean you don't believe in making your own AMIs? I don't see how you could run an instance without an AMI.

I mean I don't believe in AMIs. You seem to think I'm an Amazon customer.

I'm curious, would acquiring/hacking your fabrick "admin" machine give me access to all your servers?

Do you have a large number of servers or just smallish deployments? Fabric is cool, but at some point the pull model starts to be better than push. You probably don't want to actually wait at the console and try to spot errors when deploying 100+ nodes - and this is where chef/puppet is actually nice.