Your Linux Data Center Experts
Linux System/Network Administrator
Title: Seasoned Linux System/Network Administrator
Location: Work from home, Fort Collins, Colorado (physical presence desired)
Pay: ~$46,000 salary plus profit sharing, Health Insurance,
Android phone, Laptop with cellular data and other benefits
Type: Full time
Skills: Strong communication skills, self driven and
motivated, love of Linux.
tummy.com, ltd., founded in 1995, is looking for a greater than intermediate level Linux system and network administrator. We provide fun, challenging,
Linux-focused work with heavy community and open source ethics.
Our primary business is providing Linux and networking expertise to other
companies located throughout the US, primarily via e-mail, phone, and IM.
While candidate's presence in Northern Colorado is required for training
and regular meetings, working from home is the norm.
Ideal candidates will have strong Linux and IP networking skills and
experience, with the motivation and drive to be able to work successfully
in a company that has no central office. Strong communication skills are a
must, with the ability to communicate with a variety of people via both
written and spoken English.
tummy.com, ltd. has been in business for around 20 years, providing our
clients with experienced Linux System Administration resources. Roughly
60% of our revenue comes from Linux System and Network Administration
consulting, and 40% comes from dedicated "managed hosting". This provides
us with a broad diversity of work coming in the door at any time.
Primarily, we provide outsourced system administration. For some of
our clients, we are their primary sysadmins, for others they have their own
in-house sysadmins who call us when they need additional help or expertise
which they may not have. We are regularly called upon to do simple tasks,
such as adding another virtual host to a system, as well as complex tasks
such as designing and implementing a High Availability cluster of machines.
tummy.com, ltd. has a strong reputation and is nationally known.
Because of this, the vast majority of our clients are not in the
local area. We primarily work remotely via SSH on servers in remote
parts of the country. This allows us to have an extremely flexible work
environment, securely working from home or the coffee shop or the like.
We don't have a central office and would like to keep it that way because
of the flexibility that provides.
We are heavily involved in the Linux and open source communities,
and very much believe in these ideals. We have a staff of 6, and have
contributions including Python core, Co-Author of the Linux Security
HOWTO, donating hosting for area LUGs and a public 3TB mirror of software,
Fedora packaging and Fedora Steering Committee Board Member. While there
is a lot of work to be done for our clients, we do encourage our staff
to be involved in the Open Source community.
In short, tummy.com, ltd. provides (we have been told) a "dream job"
for someone who is dedicated and enthusiastic about Linux and the open
The following are some of the skills that the successful candidate
will bring to this position.
Better than Intermediate skill-level with Linux system and Network
Administration. We do plan to spend significant time working
with new employees getting them up to speed on our exact set of
values and policies and procedures. However, we are currently
looking for candidates who are experienced professional Linux system
and network administrators.
Enthusiasm and Passion for Linux. We all use Linux on our
desktop and are quite happy with that. Most of our employees truly
enjoy the work. While we realize that for some people and uses other
solutions may be a better fit, we are typically fairly passionate
about Linux and Open Source, though not to the extent of being
Self motivation. Because we don't have a central office,
much of the time we spend working is at our own discretion and
direction. The successful candidate will have the space and ability
to be able to work on items without excessive distraction or
diversion. We do have extensive communication methods including an
IRC channel, mailing lists, the ability to call other team members,
and in person get-togethers at coffee shops and similar. We have a
very strong team that works together when beneficial. While we do
have "reinforcements", and help with scheduling and achieving
progress, the most successful candidates will have strong self drive
Strong written and verbal communication skills. Much of the
time you will be communicating with other staff members and
customers using the English language. Much of our communication
is via e-mail and IRC or other Instant messaging and phone.
Enjoy flexibility. We are, primarily, results-oriented.
While this is a full-time position, we have the ability to support
non-9-to-5 schedules. While 9-to-5 is fine, we have quite a bit of
flexibility in our work-day. However, some tasks may need to be
scheduled outside of the work-day, particularly on production systems.
Take initiative on tasks assigned to you. You will have
the authority to work tasks through to completion. We can provide
input when needed, but hopefully you'll be able to apply your
professional judgment to meet clients needs.
Ability to plan your day and follow through. Our most
useful tool in being productive is a set of procedures and tools we
use to plan what will be worked on at the beginning of the day, and
then follow through with that plan. While there are many urgent
items that come in and need to be worked over the day, in general you
should be interested in and able to put together a plan and work
through it, calling on additional resources as required.
To get a feel for the work, here are some examples of tasks we have
performed over the last week.
Cloud Deployments. Work on setting up and migrating
to Linux cloud services (Amazon Web Services AWS, or Rackspace Cloud) for DNS, web services and streaming media
DevOps Automation. Use Puppet, Ansible or Chef to create versioned, repeatable builds of services.
Backups and Disaster Recovery Tracking down a timeout failure of a backup for a server which had a similar issue a year ago. An annual burst of activity increasing the size of a log file to greater than 2GB was identified and resolved.
Updates and Log Reviews A patched system is a reliable system. A fingertip knowledge of logs helps us spot unusual events. We try to architect systems which can be gracefully updated during regular hours, ensuring uninterrupted service and the highest quality of attention. We support CentOS, Ubuntu, Red Hat and Fedora servers.
Mail servers including Postfix Setting up
several new mail servers using Postfix, various custom and off the
shelf anti-spam solutions, and IMAP/POP services via Dovecot.
Testing and migrating to these services when migration is complete.
VPNs using OpenVPN. We have dozens of systems running
various virtual networks including our own for our VoIP and private
network, and clients for whom we set up and manage (primarily OpenVPN)
Using networking tools to track down and identify networking
anomalies. Reports of packet loss or inability to reach remote
Using workflows for new system setup. We regularly install
new systems for our clients using both standardized and customized
sets of workflows. This includes everything from initial hardware
burn-in and testing, to OS installation and post-install
Consulting with clients on off-site backup options. We
have a client who has had trouble historically with using traditional
off-site backup mechanisms, so we're spending some time working with
them to come up with a solution that more closely meets their needs
Migrating network syslog services to a new machine. A
client is consolidating two machines down into one, with this server
primarily providing syslog services for a cluster of other machines.
Logs must be moved, regular jobs checked, and services migrated.
System monitoring design, setup, and tuning with nagios, Munin, Opsview, or Icinga.
Many of our clients use nagios to monitor their production
environments. We regularly need to tune and adjust these systems to
ensure that we get alerts about important events, while not getting
Responding to alerts. Alerts such as disc space
consumption, Apache web servers not responding, etc. Reviewing these
and taking action to resolve the problem. Identifying large disc
space use and restarting Apache after identifying cause of issue.
Version control repository testing and backend migration (git or subversion).
Converting subversion repository from bsddb to fsfs backend, coming
up with a migration and testing plan.
MySQL replication errors and restarting. Identifying
problems with MySQL master/slave replication and restarting after
PostgreSQL database setup and testing. Setting up a new HA
cluster for PostgreSQL and migrating data from existing machines.
Working with on-site person to identify storage errors.
One system was failing hard drive burn-in with timeouts and errors.
Walked them through some things to try which identified a bad cable
connection and resolved.
Moving name-based virtual web-site from one server to another.
Client was moving from a single machine to a load-balanced cluster
and needed a name-based Virtual in Apache moved to the cluster.
Using /etc/hosts to test the new server before rolling out DNS
Does the above sound like you?
If so, please provide a resume and cover letter in either plain text
format, on the Internet (HTML), or PDF via e-mail to email@example.com
with the subject "System Administrator".
In your cover letter be sure to explain why you are a good candidate
for this job. In particular, cover all items above in whatever detail you
feel comfortable. More detail is probably better.
Thanks for your consideration.