The Frugal Networker

Automation is a perennial technical buzzword among System Administrators (SAs) and in management circles alike. Business owners and managers demand automation with the thought that it will save “man hours” and possibly decrease the need for a full technical staff. System Administrators realize that this is not the case nor is staff reduction the inevitable result of automation. The bad news is that the purpose of automation isn’t to reduce staff numbers. The good news is that there are several reasons for automation that make it a worthwhile pursuit.

• Reduce human error – Computers execute the same task repeatedly with no typos.
• Decrease repetitive tasks – Computers aren’t susceptible to boredom.
• Increase speed – Computers perform tasks very quickly—much faster than typing.
• Decrease low-level tasks – Free up System Administrator time for higher-level tasks.
• Present human-readable data – Automated tasks can output data into HTML.

Using Linux for an automation host makes sense for those who’ve embraced Linux for their businesses but it also makes sense for those who’ve traditionally relied on commercial UNIX. The skills your UNIX SAs spent years acquiring on those platforms translate quite well to Linux. Linux looks and reacts like commercial UNIX “flavors.” In fact, many of the same open source tools that Linux uses are available on it commercial counterparts.

The point is that using Linux for automation won’t require a huge financial outlay for training. Your UNIX SAs have the skills they need to make it work.

Distribution Selection

There’s no single correct answer to the, “Which distribution should I use?” question for every company. The correct answer is to use the distribution that you’re comfortable with or that you currently use. For example, if your company uses Red Hat Enterprise Linux (RHEL) for other tasks, then RHEL or one of its free derivatives (Fedora, CentOS, Scientific Linux) is an appropriate choice.

If you haven’t yet selected a distribution, you should start with one of the so-called “top-level” or parent distributions such as Debian, SUSE, Red Hat, Gentoo or Slackware. Ubuntu Linux, based on Debian, is also very popular, easy to use and well supported.

Components/Tools

If you’re going for the minimalist approach to automation, then you have everything you need with a basic Linux installation: The shell. The shell, commonly the Bourne-Again Shell (BASH or bash), contains all of the necessary components that you need to successfully automate just about any procedure performed manually at the command line. But, often minimal isn’t optimal and plenty of tools exist that are both powerful and easy to use. For example, if an SA wanted to expand a bit on the lowly shell for automation, Expect is an excellent choice for that first expansion.

Expect is an automation tool and it’s one of the tools considered as “essential” among thousands of SAs. From the official Expect site at http://expect.sourceforge.net, their apt description:

“Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc. Expect really makes this stuff trivial. Expect is also useful for testing these same applications. And by adding Tk, you can also wrap interactive applications in X11 GUIs. Expect can make easy all sorts of tasks that are prohibitively difficult with anything else. You will find that Expect is an absolutely invaluable tool – using it, you will be able to automate tasks that you’ve never even thought of before – and you’ll be able to do this automation quickly and easily.”

Expect is a scripting language that allows you to automate keystrokes including anticipating prompts, entering passwords and automating “interactive” sessions in the shell. In addition to Expect, I suggest that anyone who doesn’t enjoy the tedium of watching and entering username, password and other shell prompts also install Autoexpect.

Autoexpect “watches” your interactive shell session as you login, respond to various system prompts and logout. You have to clean up your automated Autoexpect scripts but it beats the alternative of manual entry.

For the non-minimalist SAs, several other valuable automation tools to add to your toolbox include the following. The lists show equivalent options under each category but isn’t meant to imply that you need ALL of the listed items in each category.

Scripting Languages
• PHP
• Ruby
• Python
• PERL

Web Services
• Apache
• NGINX
• Lighthttpd
• Thttpd

Database (RDBMS)
• MySQL
• PostgreSQL
• SQLite
• NoSQL

PERL is a common scripting language that is part of the base installation on most Linux distributions. It’s a powerful, mature and well-supported language. The selection of tools, however, is often a matter of personal preference and not a matter of superiority of one technology over another.

Whatever choices you make for your automation host tools, remember to document and comment your scripts so that when you return to debug or upgrade them, you’ll understand their functions and purposes.

User Interface

How you display your data gathered by your scripts is another concern for SAs. Some scripts run and do their jobs without the need for output to the screen, to a logfile or to a text file of any kind. However, if your automation includes gathering performance data, executing commands that produce interesting or valuable output or information that requires review, then you need to think seriously about how you’ll display that data.

Plain text files can be ugly to look at, logfiles can be tedious to pore through and screen-captured text can be impossible to realign into a meaningful document. The great equalizer is HTML. Viewing output in the comfort of a web browser has its advantages. Formatting, searchability and customizability are a few HTML advantages that come to mind.

Screen output can be bound by the HTML tags to preserve formatting so that the SA can read information exactly as the system designers originally intended it. You can also present data in tables, in CSV files or in specific file formats for use with proprietary applications. A very good method for gathering and presenting data is to load the data into a database for extraction by a scripting language and presented via HTML.

But, the user interface required for your data depends on your audience. System Administrators have different preferences and requirements than those who’d like to view data in a more formal way.

Automation Strategies

Strategies can run the gamut from automating a few tedious processes to a full-blown, 90%+ automated system. But, there’s one catch: You still have to have someone that knows what to do when things go wrong. A few years ago, when the big automation push first reared its head, SAs worried that they would eventually automate themselves out of a job. Automation hasn’t taken away but a few of the lowest level job duties. Notice the term, “job duties” not jobs.

Automation can offer cost savings to a company in that it saves labor on lower-level tasks. Senior-level SAs shouldn’t have to locate files that haven’t been touched in six months and flag them for long-term storage. An automated process should take care of that and the results reported to a text file, database or web page for casual review.

The automated gathering of performance data can save companies money by flagging systems that are overloaded or those that are underutilized. Automated processes can track file and filesystem changes to provide a pre-emptive notification of a system breach. Tracking user activity to measure productivity is an excellent feedback method during your annual personnel evaluation time.

All of these strategies save money. They save money because they save time and time is money. Using Linux is a part of the overall money-saving strategy. Automating processes with Linux is a step in the right direction for leveraging automation for your business.

You have to decide how much automation is correct for your business or business unit. The rule of thumb is that if a human regularly repeats a particular task, then that task should fall under serious scrutiny for automation. Tasks that require decisions, very complex tasks or tasks that require random timing to perform should remain manual.

Summary

Automation can offer significant cost-savings and advantages over “old school” manual processes. The range of possibilities for automated processes is wide. From simple archive scripts to very elaborate content management systems to fully automated performance data gathering and reporting scenarios. It’s wise to assess each process individually and strategically instead of announcing a blanket mandate to “automate everything” in the data center. Automation has its drawbacks. As long as everything works as designed, automation is an excellent way to streamline processes. However, when something goes wrong, you’ll need the human element to intercede and you can’t automate that.

Linux systems offer businesses a cost-effective and perfectly-suited-to-automation set of tools to suit almost any need. And, generally speaking, the automation tools listed in this article are cross-platform, which means that they work on a variety of operating systems. This cross-platform feature also means that scripts created on one system are portable to others.

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet.