Introduction to Backbone.js for Rails developers

These slides are for the presentation I gave at WellRailed.

Download PDF

Preliminary thoughts on Cucumber and behaviour driven development

I've been experimenting with behaviour driven development (BDD) using Cucumber lately. The idea is that you write (formalised) specs which describe the behaviour of your system, and they can also be executed as tests. E.g. I have this in my tests:

  Scenario: Transcriber tries to start a transcription session when there are no calls
    Given I log in as a normal user
    And there is no work waiting
    When I follow "Start processing"
    Then I should see "There is nothing to process"

Each line of text in the scenario has a matching test method. The way I have things set up, when this is executed as a test, it starts up the browser, performs the login, presses the button and then checks the content of the response. The user simulation is done using Capybara which in turn uses Selenium to drive Firefox. It's capable of executing JavaScript too.

You could write the same thing in Ruby, e.g. using RSpec (a test library), and with appropriate helper methods written, the same test would be something like:

it "should inform the user when there is no work available" do
  login()
  delete_waiting_work()
  click_link "Start processing"
  page.should have_content "There is no work available"
end

It looks quite similar for this example, however I find that using Cucumber (i.e. the original example) has advantages:

• the main thing I think, is that it forces you to think at a high level, because you want to write steps as succinctly as possible, and you'll have to write a lot if you don't keep up the level of abstraction
• it's easy to start writing tests and describing the behaviour of the system before starting the implementation, because it's decoupled from the code
• it makes it easier to think in terms of user actions (e.g. I login, I click on a link, I see a table with these rows)
• you get a readable spec, which is useful e.g. when exploring what somebody else's application
• making the spec and the tests one and the same is the easiest way to have an up-to-date spec.
• you can even expect non-technical people to be able to read it, which may be useful, e.g. I could email this test to my customer to confirm that it's the behaviour he wants, and he would understand what's happening.

I'm sure some of this could be achieved with RSpec or another test library but it would require more discipline. With Cucumber, you can't really slip into lower-level code in your test. And by the way, I still write some unit tests using RSpec, but they are limited to testing out particular functions.

The key thing, I think, is not so much the syntax but the fact that this approach results in tests which are focused on the behaviour of the system as seen by the user, at a higher level than classes or even individual components.

Using CoffeeScript in RubyMine 3

The other day I was looking for a way to see the output of the CoffeeScript compiler from within RubyMine, and I saw a mention of CoffeeBrew, which is an alternative syntax highlighting plugin to coffeescript-idea.

As an aside, when I moved to RubyMine 3, coffeescript-idea wouldn't install for me, so I had to manually copy the plugin .jar file from RubyMine 2 to RubyMine 3.

CoffeeBrew has a couple of advantages over coffeescript-idea:

• block comments!
• matching brace highlighting

At the moment it has a problem with highlighting interpolated expressions within strings, but the author is looking into a fix. The plugin can be installed from RubyMine.

Back to showing the CoffeeScript compiler output in RubyMine:

• Open Settings and go to External Tools
• Add a new tool with these parameters:
• Program: coffee
• Parameters: -c -o public/javascripts --watch app/scripts (I store *.coffee files in app/scripts and compiled JavaScript goes into public/javascripts)
• Working directory: $ProjectFileDir$
• Start the tool from the Tools menu, and you will get a window that shows the output of the compiler. The compiler will run every time you save changes to a *.coffee file in the watched directory.

Next on my wishlist: support for CoffeeScript navigation (such as going to class declaration). 

The benefits of the web as a platform

I'll be the first to admit that from the technical standpoint the web is very, very far from being a coherent stack of technologies. It's more of a patchwork of languages of varying levels of unpleasantness and protocols that may only vaguely fit what they are used for. The simplest things can require arcane tricks to make them work. Case in point: page layouts. Or file uploading. Or rich text editors. The list goes on.

However, the web also has a large number of advantages compared to other platforms and I thought it would be interesting to list them.

For users:

• There is nothing to install, just open a web page
• You never need to install updates either, you're always on the latest version
• For the most part, you don't need to worry about viruses as you aren't downloading anything
• Storage, backups and security are taken care of (at least in theory); in practice local backups may still make sense, storage isn't unlimited and may get expensive, and you can't be certain about the level of security; however, all these things are probably improving
• If we're talking about business users, the need for “IT guys” is greatly reduced because of the above
• Reduced learning curve because web apps typically have simpler UI
• The apps can easily be accessed from anywhere (possibly even from your phone!), again without requiring setup from an IT guy
• The apps will work on any platform; it doesn't matter if you're on a PC or a Mac, or even use a bizarre Linux OS (unless Flash or Silverlight or a similar plugin is involved; at least the days of ActiveX seem to be behind us)
• No need to pay a large amount of money upfront because web apps are subscription based

In short, web apps save time, are more convenient, and you can save quite a bit of money on IT help.

For vendors:

• No distribution issues as people don't need to download installers or updates
• Less need for training as the UI is generally simpler and there isn't much of a learning curve
• A lot less need for support because for the most part the apps just work and you don't need to deal with a million different configurations (some issues can be caused by browser extensions like AdBlock for example, but generally speaking the problem space is much smaller)
• No need to worry about different platforms (other than smartphones)
• Piracy is impossible or at least significantly reduced with web apps
• Subscription based access provides recurring revenue and thus a steadier income stream
• It can be easier to sell web apps because they can provide instant gratification to the user
• You have an unprecedented ability to track users' activity; you know how many people are actively using your app, what they do with it, you can make usability improvements by tracking their interaction with each page and you can even correlate account cancellations with usage patterns
• The web allows the availability of various services that can be integrated with as little effort as pasting one line of HTML (examples include billing, live chat, support and knowledge base functionality, surveys and so on).
• As the platform becomes more and more popular, the pool of available developers, designers etc. is growing
• It's probably easier to obtain investment/funding for web apps because of all these benefits

In short, the web allows you to make products faster, sell them easier, and have lower running costs.

CoffeeScript: an awesome alternative to JavaScript

I'm working on a project with a lot of JavaScript, and after a while I started getting lost in the sea of parentheses, curly braces, semicolons and anonymous functions which seem unavoidable in any non-trivial script. Even using jQuery and Underscore.js didn't help matters much, my scripts were becoming hard to read.

By a lucky coincidence, I stumbled on a post about CoffeeScript around this time. CoffeeScript is a language with a much nicer syntax than JavaScript and some very handy features like array comprehensions, so I decided to give it a go.

Short summary of the results 

Even though CoffeeScript isn't quite mature yet, and I encountered a glitch or two along the way, I converted the JavaScript in my project to it and hope to never go back.

Longer version

CoffeeScript is about 30% shorter in terms of lines than the JavaScript it produces. Not only that, the lines themselves are a lot shorter and clearer, and thus everything becomes a lot easier to read and understand. Perhaps handwritten JavaScript would be a bit shorter, but I'm not sure about that.

Overall, it's so nice that it even made me change my opinion about significant whitespace which I never liked before.

Compilation

For the sake of simplicity, I just run up this script in a shell:

coffee -wc -o <output dir> <CoffeeScript dir>

It compiles the files in the CoffeeScript dir as soon as they are saved, and outputs errors, if any. Most of the time the errors are very easy to track down and fix.

For Rails apps, another option is to use the bistro_car gem to serve CoffeeScript. For Rails 3, there is also Barista.

I also chose to use JavaScript in my views, first because there should be as little JavaScript in the views as possible, and second, to reduce the number of moving parts. However, there are things like a HAML CoffeeScript filter available.

Debugging

The JavaScript output of CoffeeScript is readable and preserves the function names, so it can be debugged in Firebug.

Deployment

I opted for simply including the JavaScript output in the deployment. This way there aren't any extra steps required other than adding the output files to git.

Testing

Since you're really testing the JavaScript output, any JavaScript testing approach is valid. I'm using QUnit at the moment.

Other considerations

Naturally, my IDE doesn't have any support for CoffeeScript beyond syntax highlighting. It would be nice to have built-in navigation, auto-complete and syntax checking, but so far my CoffeeScripts are simple enough that I can do just fine without these features.

Why enums in the database are a good idea

I've recently had a bit of a discussion about database enums on StackOverflow. My opinion is that enums are useful, while Richard thinks they are quite evil.

There are times when your schema is fairly simple and you only work with the database through your application and have a nice GUI, so you don't mind that your tables are filled with endless columns of opaque numbers.

However, I had a job where I spent heaps of time in psql running queries to track down issues or find patterns in data or produce one-off reports. I was using Postgres 8.3 at the time, and I found myself wishing for enum fields as they would give me

• simpler, shorter, more readable queries 
• easier to read query results in psql
• fewer joins to type out and make mistakes in
• fewer table names to remember.

Database enums produce similar benefits in your application code, regardless of whether you are using an ORM library, so I was pleased that they were introduced in Postgres 8.4.

I'd like to address Richard's concerns about enums:

1. Enums are case sensitive, spaces are significant

This is no different than a lot of programming languages, so developers are well equipped to handle this issue.

2. Enums are sometimes not comparable

True, and that's why you choose the right tool for the right job - don't use an enum when you need to compare values, or write your own comparison function. I would say that there are heaps of cases where you have 2-3 possible values of an attribute and you don't need to compare them - enums are great for that.

3. Enums do not allow translation into other languages

Absolutely, so you should treat enum values same as you would treat integers - they are internal values, and you look up the translation for the value as needed. Enums are there for the developer's benefit (i.e. better readability and simpler queries), and their values probably shouldn't go straight to the UI.

4. Enums are not ANSI standard and are therefore implemented differently (if at all) in different database engines

Sure, and this is where a layer of abstraction like an ORM library can hide those details.

5. When using a text string to compare within code it is possible to misspell it and the language will not warn you

People say similar things about dynamic languages but it turns out that better unit testing and the increase in productivity more than offset this issue.

6. Enums are ordered in sorts in the order in which they were created. 

If you've got integer constants, your sorts are only going to work until requirements change and you need to insert a new value right in the middle of your carefully arranged range to get the right sort order. In practice, you'll probably end up calculating an extra sort field in queries regardless of whether you have an enum or an int column, unless you are happy to update all the values. Otherwise it's the same as point 2 - don't use an enum if it's not going to do the job.

7. Enums are not easily maintained, and lock the database into a version of the world when the database was designed

It is possible to alter tables, and database migrations (e.g. in Ruby on Rails) are not a difficult task to handle. The previous point shows that integers have their share of problems anyway. As a side note, everyone is using automated migrations, right? I have run into issues with altering the schema when working with large tables (tens and hundreds of millions of rows) as it was simply taking unacceptable amounts of time. However, this is hardly an enum specific issue, and I maintain that at that scale you need to architect your database and applications with these issues in mind (e.g. by including a switchover node to handle schema changes).


To summarise, I am not advocating mixing identity and description with enums, what I am advocating is ease of use and enums provide that in a lot of cases. However, like any other tool they have to be used appropriately.

How to use specs efficiently: exploration and time estimates

I think we had a successful approach to specifications in my previous job as we delivered the project on time and with only a couple of issues found in production.

The purpose of writing specs was twofold:

• we used them to make time estimates
• they were an exploratory tool to help us understand the scope of the project.

We wrote specs upfront for the whole product but without going into too much detail, and with an emphasis on the UI. This was a means for us to get a feel for what had to be done and for the scope of the project. After that we went through the specs and listed all the tasks that had to be carried out at the level of "create the options dialog layout", "implement getting and setting of options" and so on. This unearthed a lot of things that would otherwise have turned into surprises and potential delays during implementation.

The next step was to assign time to each task. There was two of us performing estimates, which is important because when we disagreed it often highlighted things we'd forgotten about and allowed us to improve estimates, or otherwise we just took the average of our numbers for a better guess. We used a granularity of 0.5 days for our estimates, and kept the "multiply your estimate by 2" rule firmly in mind.

We found that this works out to be fairly accurate. Although the averaged times for particular tasks can still end up being wrong, you finish some of them later and some earlier because of 0.5 day granularity.

When we started implementing things, we had to work everything out in a lot more detail and inevitably had to do some things differently from the spec. To that end, we got together and worked out the best approach to implementing each feature, sometimes with prototypes. We didn't update the original spec but we did make notes after the meetings as it's very easy to forget the details afterwards.

Depending on the project, it may also be a good idea to keep the original spec up to date as the application evolves, but we didn't need to do it this time.

In spite of the Agile bent of the project, we found that writing a complete spec in the beginning was a great investment of our time, even though we fully expected it to get outdated quickly.

Setup email notifications for mdadm (with a splash of Ruby)

The instructions here are adapted from this post.

Mdadm expects a local mail server (e.g. postfix) to be configured to send emails. My preference, however, was to use GMail's SMTP server, as I didn't want to figure out how to configure postfix.

I wrote a simple Ruby script to send out emails:

#!/usr/bin/ruby
require 'net/smtp'

to = "******@gmail.com"
from = '******@gmail.com'
password = '******'
msg = "Net/SMTP email from Ruby"

smtp = Net::SMTP.new "smtp.gmail.com", 587
smtp.enable_starttls
smtp.start(Socket.gethostname, from, password, 'plain' ) { |smtp|  
  smtp.open_message_stream(from, [to]) do |f|
    f.puts "From: #{from}"
    f.puts "To: #{to}"
    f.puts "Subject: #{ARGV[1]}"
    f.puts    
    f.puts ARGV[0]  
  end
}

It accepts the text of the message and the subject as command line arguments.

Here is how I configured mdadm to use it:

• put the script into /usr/sbin/email.rb and make sure it's executable
• add this line to /etc/mdadm/mdadm.conf:
  PROGRAM /usr/sbin/email.rb

This will send you a message with the RAID device name in the subject.

To test that it's working:

• edit /etc/default/mdadm and add --test to the end of the DAEMON_OPTIONS string
• restart mdadm with sudo service mdadm restart and you should receive an email
• remove --test from /etc/default/mdadm and restart mdadm again.

One more thing to note that the first real notification you are likely to receive will be about a rebuild, which can be confusing. This is caused by a monthly (read-only) redundancy check of the array. More details here.

Works with: Ubuntu 9.10, mdadm 2.6.7.1

Using ActiveSupport::Dependencies outside a Rails project

I wanted to have the ability to auto-load dependencies based on the symbol name the way Rails does in my own non-Rails project, as it would save me writing a lot of require statements.

This can be achieved using ActiveSupport.

I added a file called boot.rb which is included in the other files of the project.

First of all, the file should require the active_support gem. I use Bundler, so in my case I specified a dependency on activesupport in Gemfile and then loaded the Bundler environment in boot.rb:

require "#{File.dirname(__FILE__)}/vendor/bundler_gems/environment"
Bundler.require_env  

After that, I added the following lines:

include ActiveSupport
Dependencies.load_paths << File.join(File.dirname(__FILE__), 'config')
Dependencies.load_paths << File.join(File.dirname(__FILE__), 'lib')

This code means that unknown symbols will be resolved by searching the files inside the config and lib folders. Also, to force the inclusion of a particular file in case the load can't be triggered by an unknown symbol name, you can do:

Dependencies.depend_on("activerecord_extensions") 

After setting up boot.rb, you just need to require it in the files that do actual real work to get automatic dependency loading:

require File.expand_path("#{File.dirname(__FILE__)}/boot")

Works using: activesupport 2.3.5

Create long integer or double precision columns

Rails doesn't provide types to create long integer or double precision columns, however it can be done using the :limit parameter:

create_table :my_table do |t|
  t.integer :long_int_column, :limit => 8
  t.float :double_column, :limit => 53
end

8 and 53 are magic numbers. I normally generate the migration first and then modify the migration file to add the limit parameter.

This works for PostgreSQL and MySQL databases, but I haven't tried any others.

Works on: Rails 2.3.5

Delete expired ActiveRecord based sessions regularly

Rails doesn't do anything about expired sessions by default. Here is how I take care of that in the case of ActiveRecord based sessions.

I have a file called delete_expired_sessions.rb in my_app/lib folder containing the call to delete old sessions:

ActiveRecord::SessionStore::Session.delete_all(["updated_at < ?", 12.hours.ago])

I also have an entry in the crontab which executes this script at 4am every day:

0 4 * * * cd /data/my_app/current && RAILS_ENV=production ./script/runner lib/delete_expired_sessions.rb

The only other thing is to make sure that you regularly write something into active sessions so they don't go stale and get deleted. And that's it.

Works on: Rails 2.3.5

Add case insensitive finders by extending ActiveRecord::Base

I wanted to have an easy way of performing case insensitive searches using dynamic finders without manually constructing the query, so if I called a finder with the _ci suffix, e.g. Customer.find_by_name_ci('Big Kahuna Burgers'), it would result in a query with lower(name) = lower('Big Kahuna Burgers') in the where clause.

I could have created a new class that inherited from ActiveRecord::Base and overridden method_missing there, but then I and other developers would have to remember to derive all my models from this new class instead of ActiveRecord::Base. Instead I chose to extend ActiveRecord::Base itself.

Overriding in ActiveRecord::Base

This is the most obvious way of doing it - just reopen the class and add the functionality in. It's ok if you're adding something small but can get unwieldy as you add more.



In theory, it's also possible to put the new functionality into a separate module to make things a bit cleaner. You can do it using either extend or include. However, I couldn't make this work in production mode (but I'm including the code for the sake of completeness below).

Overriding using extend

The only trick is that you need to add your new method to ActiveRecord::Base as a class method, so you have to use extend instead of include:

Overriding using include

It's also possible to move alias_method_chain into your module, in which case you add just one line to ActiveRecord::Base:



To make this work, in your module you have to override self.included which gets called when the module is included. You also need to put your method in a sub-module, and call base.extend(ClassMethods). Also note the use of base.class_eval.



I put this code into activerecord_extensions.rb in the models folder, but it can go anywhere so long as it gets automatically loaded by Rails before you access your models.

Works on: Rails 2.3.5