Please help us welcome the first OtherInbox baby, Charlotte Olivia Wexler Subelsky. She was born last Friday and the rest of the OtherInbox family is looking forward to meeting her soon! We have every expectation that she’ll be up to speed on Rails 2.1 by the end of the year.

I haven’t received an email from the Washington Post in at least a year, although I did give them my email address a long time ago. Then out of nowhere, I get an email for their DC Scout Weddings. Now, I have no idea why they thought I would be interested in weddings in Washington, DC. I am already married and I don’t live in DC. But even if I was about to get married in DC, I still would have considered this spam because I did not expect to get messages like this as a result of creating an account to read the news on the WashingtonPost.com website.

I was wondering why they sent me this and so I took a closer look. Then I noticed that they have an invalid From address of simply, “washingtonpost.com” — it is not a valid email address. So any replies to this message will bounce.

Finally, I scroll to the bottom and click on the unsubscribe link, which takes me to the following page which is not CAN-SPAM compliant under the recent FTC CAN-SPAM regulations because it requires a password in order to unsubscribe:

I will be speaking at the Lone Star Ruby Conference in September about how we use Ruby to deploy, monitor, and manage a cluster of servers running in the Amazon Web Services virtual cloud.

In OtherInbox, almost every system administration task imaginable is carried out using Ruby, meaning we as developers can enjoy all of Ruby’s expressive benefits and spend less time scripting the shell, writing cron tasks, or using other languages. Because we make fewer context switches from thinking in Ruby to thinking in other languages, we also reap a big productivity benefit.

Using Ruby throughout our cloud also means that porting the application to run in different production environments is a trivial task, because Ruby is the glue connecting the Ruby components together, thus all we require is a Ruby interpreter to deploy.

Two key Ruby technologies have matured in the previous 18 months which make it ideal for almost every layer of managing a cluster of servers:

• god.rb allows fine-grained process monitoring and daemon control (a la monit)
• rufus-scheduler enables Ruby-based scheduling (replacing cron, and providing a great facility for running daemons that must be executed on a recurring basis)

When combined with these Ruby workhorses, developers today can spend much more of their time writing Ruby code, and less time struggling with the vagaries of their production environment:

• Ruby standard library utilities (File, FileUtils, etc.)
• Rake
• Capistrano (for any kind of remote application, not just Rails)

The talk will also include a discussion of using several different AWS gems to make cloud computing simple, by illustrating the use of Amazon’s S3 and SQS services to distribute asychnronous work and handle communication between servers.

It seems like there are really 2 different kinds of RSS feeds for me. With many feeds I want to see every post as soon as it comes out and consciously decide to read it or delete it. It’s not surprising that so many people choose to read their RSS in email by using a service like Feedburner or RSSFwd to get each individual blog posting delivered as an email message.

For example, I like enjoy reading the blogs of Marc Andreessen and David Byrne. I try to read every post. So it makes sense to have the arrive in my email where they will sit in my Inbox (actually I automatically file them into a folder with OtherInbox) until I read them or delete them.

Yet there are other types of feeds that are more like reading the newspaper. They are full of many different topics and I only want to read the ones that catch my eye or are the most popular. This is the kind of thing that people read in an RSS reader such as Google Reader or Bloglines. This is how I read the blogs of all of my competitors products – I just want to scan the recent headlines I don’t have time to look at every single one and if they pile up then I just want to see the most recent ones.

I wonder which is more popular? Do most people read RSS feeds and blogs from their web browser, from a specialized news reader, or by subscribing to a Feedburner email? In the past I’ve always used the email option if it was available. Which do you prefer?

The sessions at RailsConf 2008 Day 2 were more technical and relevant to the work we’re doing at OtherInbox. Jeremy Kemper introduced Rails 2.1′s new features; I’m most excited about partial updates, because there are a few places in our code where we update a single attribute, but end up having to do a lot of extra writes to the database to update all of the columns of a saved object. The 2.1 timezone code should also make our lives as developers much easier.

Michael Koziarski gave an interesting overview of Rails optimization:

• It’s impossible to to find a common benchmark that would fairly represent Rails performance across a wide range of applications. His ultimate answer was that many developers should benchmark new optimizations so that only those that benefit the majority of apps become part of Rails core.
• Many optimization suggestions seem useful at first, but turn out to be useless in most real-world apps.
• He advised us of ways to take advantage of recent optimizations; the best recommendations was to always use named routes. Named routes are now generated in constant time, but url_for is much more expensive.
• Most performance problems in his experience are problems with rendering the view.
• Folk wisdom about performance problems is never right. Usually the thing that slows your app down is what you least expect, not the obvious thing, so it pays to collect good data.

In Scaling Rails, a bunch of heavyweights, including the original developer of Twitter, talked about scaling. They didn’t have a lot of practical advice besides:

• Not worth it to do much pre-optimization, because you often can’t predict what the slow parts of your app will be until you have heavy usage.
• Twitter’s scaling problems are not related to Rails, despite all the rumor-mongering. Scaling is fundamentally about architecture, not the language.
• Scale vertically as much as possible (add more CPU cores and more memory, etc) to buy time while you make your software better — and there may be more return on investment for hardware optimization in the long run.

The ThoughtWorks crew gave an awesome talk on testing called “Fast, Sexy, and Svelte”. I’ve been following their writings for a long time and it was great to see their entire testing scheme laid out in one session:

• Unit tests should be fine-grained and fast, testing only one object and one method at a time. These tests should not hit the database for maximum speed (and because that’s better handled by functional tests) and reduced brittleness. They demonstrated how to use the unit_record gem to achieve this for a huge test speedup. They showed an example where 4920 unit tests running on a Mac Mini took 19 seconds!
• You should unit test every object, including controllers. Use mocks and stubs to isolate controllers from models. We already do this because we mostly use RSpec, which encourages this kind of testing. So I can speak from firsthand experience that this is a great tip and makes tests less brittle (meaning I can more easily change a model or controller without having a whole bunch of unrelated tests fail).
• Don’t unit test views, because there shouldn’t be any interesting logic in the views anyway.
• Functional tests are more coarse, and are meant to show failures where objects interact.
• Functional tests require real data, but fixtures make them too brittle (and cause problems on teams with more than a few developers). Use factory methods instead, which provide reasonable, overwritable defaults.
• Don’t test the contents of views in functional tests; just make sure they render properly (what you are testing is whether the controller assigns instance variables properly)
• Functional tests are much slower, because they involve multiple objects and database access. They illustrated uses of the deeptest gem to run functional tests in parallel on one server, or distributed across many servers. This was my favorite tip of the conference and I can’t wait to start using this in the OtherInbox tests.
• Integration tests aren’t as useful, in their opinion, because most behavior will have been well tested at the unit and functional levels. They felt that integration tests were good for special, high-level cases such as performance testing, smoke testing, and testing the effect of simultaneous users. Since we use several integration tests as smoke tests, this also seemed right on. These tests are very high level and exercise the critical behaviors of the entire system (for us, things like signing in, signing up, and mail delivery)
• Acceptance testing is the highest level testing possible, and takes place in the browser. They use Selenium to mimic exactly what a real user would be doing.
• Acceptance tests should use a helper class to encapsulate implementation details of each view in the app. This makes the tests much less brittle, because if you change something minor like the ID of a <div>, you only have to change the helper class one time to update all of the affected acceptance tests.
• Acceptance tests should still be somewhat focused; you can “cheat” by putting some data into the session or database beforehand. For example, if you are testing actions that require the user to be signed-in, you don’t need to have the user sign in for every single test. It’s more reasonable to set up the session automatically and get on with testing the relevant behavior.
• Use the same factory for acceptance tests as for functional tests. You just have to make sure you app server is pointed at the test database where the factory objects are being created.
• It’s possible to run Selenium tests in parallel with Selenium grid.

In Small Things, Written Fast, Joined Loosely, Justin Gehtland of Relevance demo’d an app that appeared to the user as one unified whole, but was actually three different applications running on different servers. The key to making this work was use of single-sign-on, via the RubyCAS-Server.

I led two Birds of a Feather sessions at RailsConf 2008. Here’s my recap:

Rails Presenter Classes and the Power of Abstraction

We discussed how to use presenter classes to more elegantly solve three common rails problems:

1. Designing web views that manipulate multiple models simultaneously (where signup involves creating multiple different objects which must both be validated before either can be created).
2. Building complex reports that can accept a wide variety of options to control the scope of the report.
3. Writing complex SQL statements to speed up database access, by marrying the strengths of Ruby with the strengths of SQL.

The audience was super cool, asked a lot of questions, and helped sharpen my own understanding of this technique. They also encouraged me to publish the code I’ve written to make Presenters easier to implement and use in views, so stay tuned for that.

Also check out Jay Fields’ blog article about Presenters.

Rails and Amazon Web Services

I talked about my experiences deploying OtherInbox to Amazon’s EC2 environment, with our app completely running on a virtual cloud of servers. Matt Conway from Mobicious showed off his new Capistrano plugin, rubber, which has the potential to make my hand-written EC2 deployment script more systematic, maintainable, and repeatable.

Everyone seems to agree that email is great. There are many types of communication that I prefer to receive via email over other channels such as postal mail, the telephone, fax, carrier pigeon, etc. I love how at the Apple retail stores they will email my receipt to me instead of printing it out (save a tree!). I get much less postal mail now that all of my financial institutions and utilities have “paperless billing” options that send monthly statements by email. Judging by the popularity of these services, I’m not the only one who feels this way.

Yet everyone knows that email is broken. The problem is bigger than just spam. Michael Arrington recently declared email bankruptcy and deleted his inbox. There are two fundamental problems, both of which are only getting worse:

1. Our actions generate more incoming email than we have time to read
2. We want to respond to more emails than we have time

Spam is part of the first problem. But another part of the first problem is that its a lot easier to sign up for new emails than to decide which ones to unsubscribe from. Over time, you get more and more things that you’ve signed up for as well as more and more spam that you don’t want. You start off with a clean slate and then just add more and more baggage over time until eventually the only solution is to get a new email address and start over.

The second problem stems from the fact that email lets you easily communicate with more people than you could before. It’s very easy to write a single email, copy and paste it for 5 different recipients, and end up with 5 replies coming back at you. Email multiplies quickly. Yet we’re still typing each reply one at a time.

Both of these problems are just getting worse. As we more offline things go online we will only get more newsletters, receipts, statements, warnings, notifications, etc. Soon my car will be sending me emails and my house and my dishwasher. There will be too much to go into a single “inbox”.

And as more things go online we connect with more other people. The more people we connect with, the more we want to communicate, the more emails we send and the more replies that come back at us. RSS, Blogs, social networks and Twitter all help to alleviate this problem. Rather than blasting an email out to 100 people, you can just post it to your blog or on Twitter.

As the number of online communications grows, not all of it needs to happen in email and not all of it will be manageable within email. I don’t really want to have receipts in my email — that’s just the most convenient option available to me. In the end, that receipt needs to end up in a filing cabinet, database or expense report. I want to be able to find the receipt later if I need it, but it doesn’t really belong in my Inbox.

This doesn’t mean email is dead or that email shouldn’t evolve. Quite the opposite, email needs to get better and helping us find the messages we want and ignore the ones we don’t. We need to be able to reliably unsubscribe from emails we don’t want and trust that the ones we do want are legitimate.

We need to start thinking about a more sophisticated Inbox than just a list of messages sorted by date.

If you’ll be attending in RailsConf 2008 in Portland next week, please join our very own Mike Subelsky at the BOF Session: Rails and Amazon Web Services. It’s on Friday May 30 at 9:00pm in room D134. Hope to see you there!

Mike’s topic will focus on:

The example cluster is operated by OtherInbox.com, a consumer-driven Rails app that automates email tasks and spam blocking. Every system administration task imaginable is carried out using Ruby, which means the developers can enjoy all of Ruby’s benefits without having to delve into shell scripting, writing cron tasks, or use of other languages.

It also means that porting the entire app to run on different operating systems is a trivial task, because Ruby is the glue connecting the Ruby components together; all that is required is a Ruby interpreter.

Two key Ruby technologies have matured in the previous 18 months which make it ideal for almost every layer of managing a cluster of Rails servers:

• god.rb allows fine-grained process monitoring and daemon control (replacing monit)
• rufus-scheduler enables ruby-based scheduling (replacing cron, and providing a great facility for running daemons that must be executed on a recurring basis)

When combined with these Ruby workhorses, developers today can spend much more of their time writing Rails code, and less time struggling with the vagaries of their production environment:

• Ruby standard library utilities (File, FileUtils, etc.)
• Rake
• Capistrano

The talk would also include a discussion of several different AWS gems that make cloud computing simple, by illustrating how Amazon’s S3 and SQS services can handle asynchronous work and communication between servers. I would illustrate real-world code that enables servers to communicate with one another reliably via SQS.

If you’ll be attending in RailsConf 2008 in Portland next week, please join our very own Mike Subelsky at the BOF Session: Presenter Classes and the Power of Abstraction. It’s on Thursday May 29 at 7:30pm in room D134. Hope to see you there!

Mike will focus on three examples from OtherInbox.com:

1. # Writing complex SQL statements to speed up database access, by marrying the strengths of Ruby with the strengths of SQL. Developers can use Presenters to generate SQL statements in a way that is DRY, readable, maintainable, and easily tested, while keeping their models free of clutter.
2. Designing web views that manipulate multiple models simultaneously (where signup involves creating multiple different objects which must both be validated before either can be created).
3. Building complex reports that can accept a wide variety of options to control the scope of the report.

The talk would conclude with a discussion of the details of Presenter implementation, using the Forwardable library and some custom techniques he’s developed to make Presenters easier to write.