Linux System/Network Administrator

Overview

Title: Seasoned Linux System/Network Administrator
Location: Work from home, Fort Collins, Colorado (physical presence desired)
Pay: ~$46,000 salary plus profit sharing, Health Insurance, Android phone, Laptop with cellular data and other benefits
Type: Full time
Skills: Strong communication skills, self driven and motivated, love of Linux.

tummy.com, ltd., founded in 1995, is looking for a greater than intermediate level Linux system and network administrator. We provide fun, challenging, Linux-focused work with heavy community and open source ethics. Our primary business is providing Linux and networking expertise to other companies located throughout the US, primarily via e-mail, phone, and IM. While candidate's presence in Northern Colorado is required for training and regular meetings, working from home is the norm.

Ideal candidates will have strong Linux and IP networking skills and experience, with the motivation and drive to be able to work successfully in a company that has no central office. Strong communication skills are a must, with the ability to communicate with a variety of people via both written and spoken English.

Description

tummy.com, ltd. has been in business for around 20 years, providing our clients with experienced Linux System Administration resources. Roughly 60% of our revenue comes from Linux System and Network Administration consulting, and 40% comes from dedicated "managed hosting". This provides us with a broad diversity of work coming in the door at any time.

Primarily, we provide outsourced system administration. For some of our clients, we are their primary sysadmins, for others they have their own in-house sysadmins who call us when they need additional help or expertise which they may not have. We are regularly called upon to do simple tasks, such as adding another virtual host to a system, as well as complex tasks such as designing and implementing a High Availability cluster of machines.

tummy.com, ltd. has a strong reputation and is nationally known. Because of this, the vast majority of our clients are not in the local area. We primarily work remotely via SSH on servers in remote parts of the country. This allows us to have an extremely flexible work environment, securely working from home or the coffee shop or the like. We don't have a central office and would like to keep it that way because of the flexibility that provides.

We are heavily involved in the Linux and open source communities, and very much believe in these ideals. We have a staff of 6, and have contributions including Python core, Co-Author of the Linux Security HOWTO, donating hosting for area LUGs and a public 3TB mirror of software, Fedora packaging and Fedora Steering Committee Board Member. While there is a lot of work to be done for our clients, we do encourage our staff to be involved in the Open Source community.

In short, tummy.com, ltd. provides (we have been told) a "dream job" for someone who is dedicated and enthusiastic about Linux and the open source community.

Desired Skills

The following are some of the skills that the successful candidate will bring to this position.

Better than Intermediate skill-level with Linux system and Network Administration. We do plan to spend significant time working with new employees getting them up to speed on our exact set of values and policies and procedures. However, we are currently looking for candidates who are experienced professional Linux system and network administrators.
Enthusiasm and Passion for Linux. We all use Linux on our desktop and are quite happy with that. Most of our employees truly enjoy the work. While we realize that for some people and uses other solutions may be a better fit, we are typically fairly passionate about Linux and Open Source, though not to the extent of being zealots.
Self motivation. Because we don't have a central office, much of the time we spend working is at our own discretion and direction. The successful candidate will have the space and ability to be able to work on items without excessive distraction or diversion. We do have extensive communication methods including an IRC channel, mailing lists, the ability to call other team members, and in person get-togethers at coffee shops and similar. We have a very strong team that works together when beneficial. While we do have "reinforcements", and help with scheduling and achieving progress, the most successful candidates will have strong self drive and motivation.
Strong written and verbal communication skills. Much of the time you will be communicating with other staff members and customers using the English language. Much of our communication is via e-mail and IRC or other Instant messaging and phone.
Enjoy flexibility. We are, primarily, results-oriented. While this is a full-time position, we have the ability to support non-9-to-5 schedules. While 9-to-5 is fine, we have quite a bit of flexibility in our work-day. However, some tasks may need to be scheduled outside of the work-day, particularly on production systems.
Take initiative on tasks assigned to you. You will have the authority to work tasks through to completion. We can provide input when needed, but hopefully you'll be able to apply your professional judgment to meet clients needs.
Ability to plan your day and follow through. Our most useful tool in being productive is a set of procedures and tools we use to plan what will be worked on at the beginning of the day, and then follow through with that plan. While there are many urgent items that come in and need to be worked over the day, in general you should be interested in and able to put together a plan and work through it, calling on additional resources as required.

Example Tasks

To get a feel for the work, here are some examples of tasks we have performed over the last week.

Cloud Deployments. Work on setting up and migrating to Linux cloud services (Amazon Web Services AWS, or Rackspace Cloud) for DNS, web services and streaming media delivery.
DevOps Automation. Use Puppet, Ansible or Chef to create versioned, repeatable builds of services.
Backups and Disaster Recovery Tracking down a timeout failure of a backup for a server which had a similar issue a year ago. An annual burst of activity increasing the size of a log file to greater than 2GB was identified and resolved.
Updates and Log Reviews A patched system is a reliable system. A fingertip knowledge of logs helps us spot unusual events. We try to architect systems which can be gracefully updated during regular hours, ensuring uninterrupted service and the highest quality of attention. We support CentOS, Ubuntu, Red Hat and Fedora servers.
Mail servers including Postfix Setting up several new mail servers using Postfix, various custom and off the shelf anti-spam solutions, and IMAP/POP services via Dovecot. Testing and migrating to these services when migration is complete.
VPNs using OpenVPN. We have dozens of systems running various virtual networks including our own for our VoIP and private network, and clients for whom we set up and manage (primarily OpenVPN) VPN networks.
Using networking tools to track down and identify networking anomalies. Reports of packet loss or inability to reach remote sites.
Using workflows for new system setup. We regularly install new systems for our clients using both standardized and customized sets of workflows. This includes everything from initial hardware burn-in and testing, to OS installation and post-install configuration.
Consulting with clients on off-site backup options. We have a client who has had trouble historically with using traditional off-site backup mechanisms, so we're spending some time working with them to come up with a solution that more closely meets their needs and budget.
Migrating network syslog services to a new machine. A client is consolidating two machines down into one, with this server primarily providing syslog services for a cluster of other machines. Logs must be moved, regular jobs checked, and services migrated.
System monitoring design, setup, and tuning with nagios, Munin, Opsview, or Icinga. Many of our clients use nagios to monitor their production environments. We regularly need to tune and adjust these systems to ensure that we get alerts about important events, while not getting false-alarms.
Responding to alerts. Alerts such as disc space consumption, Apache web servers not responding, etc. Reviewing these and taking action to resolve the problem. Identifying large disc space use and restarting Apache after identifying cause of issue.
Version control repository testing and backend migration (git or subversion). Converting subversion repository from bsddb to fsfs backend, coming up with a migration and testing plan.
MySQL replication errors and restarting. Identifying problems with MySQL master/slave replication and restarting after resolving.
PostgreSQL database setup and testing. Setting up a new HA cluster for PostgreSQL and migrating data from existing machines.
Working with on-site person to identify storage errors. One system was failing hard drive burn-in with timeouts and errors. Walked them through some things to try which identified a bad cable connection and resolved.
Moving name-based virtual web-site from one server to another. Client was moving from a single machine to a load-balanced cluster and needed a name-based Virtual in Apache moved to the cluster. Using /etc/hosts to test the new server before rolling out DNS changes.

To Apply

Does the above sound like you?

If so, please provide a resume and cover letter in either plain text format, on the Internet (HTML), or PDF via e-mail to jobs-2017@tummy.com with the subject "System Administrator".

In your cover letter be sure to explain why you are a good candidate for this job. In particular, cover all items above in whatever detail you feel comfortable. More detail is probably better.

Thanks for your consideration.