This blog post was originally published on the Siyelo blog in July 2012.

We shared our Rails Deployment experience with Red Comet Labs, and in this article we’ll discuss a checklist of some devops practices and tools we are familiar with.

Extract sensitive data

All sensitive data like passwords, API keys & tokens that are in the application are extracted as environment variables outside of the source code repository. We are using rbevn for managing rubies, there is a nice plugin for setting up environment variablesrbenv-vars.

Continuous Integration & Deployment

There are many great open source and commercial tools out there for Continuous Integration. We are using Jenkins CI to run our test suite and handle Continuous Deployment (if the tests are green) for our staging branch. For production releases, we suggest doing releases manually at a scheduled time with the whole team ready and available.

Deployment tools

If you need more advanced tool for doing the deploy, Capistrano is the definitely the tool to go with. However, if you want something more light-weight you can try gitploy (bare minimum, git-based tool for deployment). Alternatively, check out another tool from this list.

Deployment Documentation

Document server setup and everything about the deployment process. It may seem tedious now but in the long-run it will help because setups are easily forgetten and the person who initially setup the server is not always on hand to assist. By using an automation tool like Chef or Puppet you are kind of documenting things on the fly.

Server security

Don’t use your root user when you access your server - it’s the simplest way to prevent accidentally breaking things. Create another user that can execute superuser commands using sudo. Use this non-root user all the time. You should use SSH key authentication to protect your server against brute-force password cracking attacks. Furthermore, you should disable SSH password authentication and the ability to login as root. Read more on this in Linode’s guide on how to secure your server.

Start clean on boot

All the services that are being used by the application need to start cleanly when the server boots up - there should be no need for manual intervention. If you are deploying to Ubuntu, update-rc.d can be used for init.d processes. And, for the application Procfile-based processes you can use the foreman gem and there is an easy export to upstart for monitoring and boot start up setup.

Log files

Prevent log files from growing to the point where you don’t have any space left on disc device by using logrotate. Analyze request log of your Rails application to produce a performance report with request-log-analyzer.

Cron jobs

Whether it’s a backup script or background task that needs to be run as a job, it’s a good idea to have documentation on it within the application. Cron’s scheduling format is often difficult to read so we suggest using the whenever gem to make it more readable.

Backups

For both database & uploaded content (like images and documents) we need to have regular backups distributed in different physical or cloud locations, backup gem is very handy tool for that.

Failover

If server crashes, can you fail over to another server? How much time do you need for that operation and have you tested the procedure before it actually happens? Hosting services offer daily/weekly/monthly node backups but the procedure need to be tested before there is a need for doing that.

External monitoring tools

Service like New Relic, Pingdom or Alerta are interesting for monitoring server uptime and performance and they can send email, text and alerts when the server is not working.

Notifications for Exceptions

Setup exception notifications to receive email notifications when application crashes. You can use exception_notification gem or some paid service like Airbrake, Exceptional.

Process monitoring tools

Monitor your DB server, web server & background processes and restart them if they consume too much memory. Tools like Upstart, Bluepill, God and Monit are useful.

Server performance

Monitor server performance with vmstat or some 3rd party tool like scoutapp.

Application performance

Monitor your application performance, uptime, slow DB queries, background tasks with New Relic. They also have Availability monitoring if you don’t want to use other service for checking that your server is up and running.

The list is long, but it takes lots of patience in understanding the software business and delivering great software.