2nd Gen Accelerators, Rails and attachment_fu

File this one under WTF.

I’ve been working on an updated version of the Soil Quality website for a little while now, and recently needed to deploy the site to a staging server for testing. I ordered up a 1/4Gb Accelerator from Joyent, configured it, and deployed the app, just like I’ve done with 20+ other sites, and BOOM, straight into a brick wall.

I opened up the log files and saw this error:

** Daemonized, any open files are closed.  Look at /tmp/soilquality-mongrel.8200.pid and log/mongrel.8200.log for info.
** Starting Mongrel listening at 127.0.0.1:8200
** Define INLINEDIR or HOME in your environment and try again

Wonderfully descriptive I know, but something I’d never run into before. So, fire the the google and it turns out it’s a common error, with a common fix: just put

ENV['INLINEDIR'] = RAILS_ROOT + "/tmp"

into your config/environment.rb file, make sure the directory exists and that it’s writeable by the mongrel, restart, and away you go.

But of course, that didn’t fix it, did it.

Much hair pulling ensued. I finally enlisted the help of Darcy Laycock and together we managed to track the problem down to the attachment_fu plugin, which was causing the problem as the mongrel process booted. OK, so now we knew where the problem lay, but what was causing it.

It turned out to be the ImageScience image processor, or more specifically the way the attachment_fu plugin and ImageScience work together ON A 2ND GENERATION JOYENT ACCELERATOR – eg the ones that use pkgsrc. Didn’t seem to cause the problem on an ubuntu machine, nor on one of the older BlastWave based Accelerators. I’m not sure why as of yet, as I was more worried about getting some sleep last night when we managed to fix the problem.

And the fix? Basically, strip ImageScience out of attachment_fu. Remove

vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/processors/image_science_processor.rb

and remove “ImageScience” from this line

@@default_processors = %w(ImageScience Rmagick MiniMagick Gd2 CoreImage)

in this file

vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/attachment_fu.rb

Like I said, I have no real idea why this is happening at the moment, but I’ll try and work it out and update this post if I get anywhere.

Once again, thanks to Darcy for A) being a sounding board, B) helping fix the problem and C) being up and available when I needed him :)

Freezing Rails with Git

Now that the Ruby On Rails team has moved the codebase over to github, some of the standard rake tasks aren’t working the way that they used to. When it was on SVN, it was possible to type

rake rails:freeze:edge TAG=rel_2-0-1

and the appropriate version would be copied into your vendor/rails directory.

Now if you do that, rake downloads a zip of the edge release. Which is fine and all, but sometimes you don’t want to be on edge … like in any production site.

So, I found a screencast that goes through the process, but I thought I’d actually put the text into a post, mainly for my own reference more than anything else.

$ rails path_to_app

$ cd path_to_app

$ git init

$ git submodule add git://github.com/rails/rails.git vendor/rails

At this point, git will effectively clone the repository, so that you can then choose one of the branches to “freeze” to. Type “git tag” to get a list of all the available tagged branches. Choose the one you want and type

$ git checkout v2.1.0

And that’s it. Slightly more involved than the old way, but still none too shabby.

Getting Rails, Git and Capistrano to work on a Joyent Accelerator

After several days of repeatedly smashing my head into both a metaphorical and an all too real brick wall, I seem to have managed to get git and capistrano working happily together on my Joyent Accelerator. I’m also using github for my git hosting, which threw up it’s own little challenge mid-way through the entire process.

Now, I should probably also say that I already had my site up and running using subversion, capistrano and my accelerator, so this article isn’t necessarily going to help with getting everything setup the first time. For that, you probably need to read this wiki entry.

Things You Will Need

• A github.org account
• A dedicated Joyent Accelerator (I have no idea how to do all of this on a shared accelerator. Sorry.)
• Also, I’m really only talking about RubyOnRails apps here … not too sure how applicable a lot of this is to other frameworks (it will probably help at least.)

Getting started

Assuming that you have your project in a git repository, and have a github account (and obviously an Accelerator) we can start.

Compiling git on your accelerator

Unfortunately, the first step in the process was, for me at least, a total nightmare. I’m not the biggest unix-head by any stretch, but I can do some basic tasks with a degree of proficiency. Unfortunately, I went into a dark place trying to get git compiled. One thing to note is that I’m talking about setting up the git client here, not a git server. Because capistrano executes scripts on your remote server, you need to have a copy of the client software setup for capistrano to call.

So what did I do? There are a couple of helpful threads on the Joyent Forums:

• Got git?
• Git and Cap 2.0
• Installing git

Hopefully those threads will put you onto the path of successfully compiling and installing git onto your Accelerator.

Setting up SSH keys with github and your accelerator

When you setup your account on github, you need to setup an SSH key for authentication. github has a really good tutorial on how to do this. I have a user defined on my accelerator that my website “runs” under, so what I did was to create a key for that user which gets stored into the ~/.ssh directory. I then added the contents of the id_rsa.pub key to my github account, which allows that user to access the repository.

Another tip: don’t forget your passphrase. It’s needed in the next step.

Configuring capistrano

Assuming that you have your capistrano deploy.rb file setup as outlined here there are a few changes that you will need to make to get things working with git.

I’m using Capistrano 2.2 at the moment. I don’t think it will work with earlier versions because of the relatively new git support.

Here’s my deploy.rb file:

  require 'erb'
  require 'config/accelerator/accelerator_tasks'

  set :application, "website" 
  set :repository, "git@github.com:your_username/website.git" 

  default_run_options[:pty] = true
  set :domain, 'XX.XX.XX.XX' #Your Accelerators public IP address
  set :deploy_to, "/var/www/apps/#{application}" 
  set :user, 'website_account_username'
  set :scm, :git
  set :scm_username, "github_username" 
  set :scm_passphrase, "your passphrase here" 

  role :app, domain
  role :web, domain
  role :db,  domain, :primary => true

  set :server_name, "url.for.website" 
  set :server_alias, "*.url.for.website" 

  # Example dependancies
  depend :remote, :command, :gem
  depend :remote, :gem, :money, '>=1.7.1'
  depend :remote, :gem, :mongrel, '>=1.0.1'
  depend :remote, :gem, :image_science, '>=1.1.3'
  depend :remote, :gem, :rake, '>=0.7'
  depend :remote, :gem, :BlueCloth, '>=1.0.0'
  depend :remote, :gem, :RubyInline, '>=3.6.3'

  ################################
  # Some tasks for the old server
  ################################

  task :after_deploy do
    # tasks to run after deploy
  end

  ################################
  # End tasks for the old server
  ################################

  deploy.task :restart do
    accelerator.smf_restart
    accelerator.restart_apache
  end

  deploy.task :start do
    accelerator.smf_start
    accelerator.restart_apache
  end

  deploy.task :stop do
    accelerator.smf_stop
    accelerator.restart_apache
  end

  after :deploy, 'deploy:cleanup'

It appears that the important line here is ’ default_run_options[:pty] = true ’. This means that capistrano can respond automatically for the request for the SSH Key passphrase that github replies with when you try to clone the repository.

If everything is working, you can type ‘cap deploy’ and it should all deploy nicely. If you get this error:

[err] Permission denied (publickey).

then there’s a problem with your SSH key and your settings on github. Make sure the key you copied into your github account is the public key for the SSH in your .ssh directory.

Hopefully, you’ll be up and running. If you have any tips, recommendations or corrections, leave a comment.

Connecting to Joyent Accelerator with CocoaMySQL

First things first: What’s an accelerator? And why would you care about connecting to it with CocoaMySQL?

Well, basically, an accelerator is kind of like a virtual server, offered by Joyent. Calling it a virtual server is a bit of a misnomer, because it conjures up images of a linux slice, but it’s a bit more than that. Running on OpenSolaris, it’s built from the ground up to offer scalable hosting. You get root access, and the ability to do pretty much whatever you want with it. Out of the box, they are setup to be rather special RubyOnRails/PHP/Python servers, with mySQL all setup and running like a champ.

OK. So where does CocoaMySQL come in? Your accelerator comes with PHPmyAdmin configured by default, but sometimes you want a little more than that you know? And with an app like CocoaMySQL you get a sweet GUI to do all your admin tasks, and nice editing facilities.

Because of the security built into your accelerator, you can’t connect to mySQL from anywhere but your server. Handy for preventing attacks, but slightly painful for server management. So, you need to open up an SSH tunnel, to securely connect to the server. But first of all you need to make some configuration changes on your accelerator.

sudo nano /etc/ssh/sshd_config

change the following parameters to ‘yes’

AllowTcpForwarding yes
GatewayPorts yes

Then you need to restart the ssh daemon, with the following command

sudo svcadm restart svc:/network/ssh:default

The next step is to setup an “SSH tunnel” between your machine and the accelerator, which is basically a direct connection between the two machines. All traffic that flows along this connection is encrypted, reducing the ability for someone to sit there and listen in on what you’re doing.

The command I use for setting up the tunnel is

ssh -2 -f -c blowfish -N -C username@accelerator.ip -L 3307/127.0.0.1/3306

This sets up a connection between port 3306 on the accelerator (specifically the mySQL port) and port 3307 on your local machine. To connect to your mysql server in CocoaMySQL you just connect to port 3307 on 127.0.0.1, which then just sends everything to your accelerator.

And a screenshot of the config screen for CocoaMySQL:

Now, obviously, that ssh command is going to be slightly painful to type in every time you want to connect to your accelerator. So, here’s a handy dandy script and configuration file that makes connecting to multiple servers a breeze. All you need to do is copy the setup in the config file and change the settings for each server. And of course, mirror those settings in CocoaMySQL.

Thanks to Ben Rockwood from Joyent for the tips on the config changes needed on the accelerator

Upgrading to Capistrano 2

For no other reason than this is something I need to remember on other projects, here is a list of the changes I made when I uninstalled deprec and upgraded to capistrano 2 for deployment.

Things to do

Do this once

gem install mongrel_cluster

then in the application directory

capify .

then remove “require ‘deprec/recipes’ from the deploy.rb file

then put the following in to the deploy.rb file

namespace :deploy do task :start, :roles => :app do start_mongrel_cluster end task :stop, :roles => :app do stop_mongrel_cluster end task :restart, :roles => :app do restart_mongrel_cluster end end

Maps, Geocoding and Great Timing

On one of the sites I’ve been working on for quite a while (SchoolSeek) we’ve been wanting to add the ability for users to find out how far a school is from their current location. There are services that have been able to geocode addresses for a while, but they were either based in the US, or cost money. Not a lot of money, but free is always good.

A couple of years ago, Google Maps launched in the US and opened up a massive range of possibilities for programmers to develop cool “mashups”, which took information from one site, mashed it together with maps from google, and created a whole new website. They were all really cool, but unfortunately (at least for the non-US based of us) it was nothing more than something we could sit back and watch – which was kind of weird because the team of developers that built Google Maps was based in Sydney.

But finally the Australian version of google maps was released, and then recently integrated with Local Search, which means you can search on Pizza shotps near a particular address. Which is cool.

What is even cooler (if you’re a ruby developer), though, is the release of this: GeoKit. It integrates with all of the major geocoding/mapping services and provides a huge range of options and services. So now you can do stuff like this:

add_1=GeoKit::Geocoders::GoogleGeocoder.geocode("1 St Georges Terrace, Perth, Western Australia")
add_2=GeoKit::Geocoders::GoogleGeocoder.geocode("1 York St, Albany, Western Australia")

distance = add_1.distance_from(add_2, :units => :kms)

Which returns

389.248018478531

which is, of course, how far it is between the main streets of Perth and Albany in Western Australia.

Sawwweeeeeet.

Update

Well, that didn’t take long at all. A quick loop over the existing schools in the SchoolSeek database got the lat/long information for all the addresses. Then adding this line

acts_as_mappable default_units => :kms

to my Address model allows you to do this

@addresses = Address.find(:all, :origin => "18 Bland St, Ashfield, New South Wales", :conditions => "distance < 10")

which gives me all of the addresses within 10Kms of 18 Bland St, Ashfield. Nice.

So, basically GeoKit let me add distance searching to the site within about 45 minutes (allowing 25 minutes for me to read through the examples and such!)

Handy RubyOnRails Tips

19 Rails Tricks Most Rails Coders Don’t Know

When looking at my own Rails code and that of the community as a whole, I often see places where certain Rails techniques could have been used, but weren’t. As much for my own memory as yours, I thought I’d list down some Rails tricks and tips that can make your application or code more efficient:

Easy comma delimited files

From ~:caboose

  # this is used directly on models only
  # Usage: Customer.to_tab_delimited
  def self.to_tab_delimited(path = nil, columns = nil, options = nil)
    columns ||= self.column_names
    File.open(File.expand_path(path || "./#{table_name}.txt", RAILS_ROOT), 'w') do |file|
      file.puts(columns.join("\t"))
      # Notice this is very close to what the Array#to_tab_delimited below does, except Array only calls itself, meaning
      # it only exports it's own contents. 
      self.find(:all, options).each do |rec| 
        row = columns.inject([]) do |arr, col|
          if rec[col].is_a?(String)
            arr << rec[col].gsub(/[\t\n]/, ' ')
          else
            arr << rec[col]
          end
        end
        file.puts(row.join("\t"))
      end
    end
  end
end

Biblio

The other day there was a post by Seth Godin in which he talked about the opportunity for an online bibliography creator that allowed you to just enter the ISBN for a book, and it would automatically find all of the relevant information for you, incorporate it into a bibliography, which you could then share with whomever you wanted to. I was sort of looking for some project where I could incorporate Amazon Web Services and also start using Google AdSense. So this project basically slapped me in the head and said “Build Me”.

The upshot of all of that is that I did. Checkout Biblio – Online Bibliography Creator. It’s all very basic at the moment, but you can search for a book, build a bibliography and print it out in 5 different citation styles. There’s alot of things I need to add to it: Search by Book Name, Manual Citation Entry (for sources not listed on Amazon), other Citation types (articles, web pages, journals etc etc). Still, it’s a start.

I built it relatively quickly in RubyOnRails. I have spent most of the past few months pretty much back in the world of ASP.Net and C# for a couple of projects that I’m working on – GradDirect (I’m subcontracting for IBC) and on a new site for KVM Australia which is just about getting ready to launch (man, talking about WebCentral is a whole new post .. sigh). The upshot of it all was that I had slipped out of the RubyOnRails space, and I had forgotten just how damn cool it is. Good to know that she fits back on like a smooth glove.

Anyway, check out Biblio. Kick the tyres and leave a comment about what you thought of it.

Getting Underway with Ruby On Rails

This post originally appeared over on Spin Technologies, but I decided that it was probably more appropriate for this site. So here it is!

Almost a year ago, I started hearing about BaseCamp, a project management/organisational tool, that had been created by 37 Signals. As more information started appearing about the product and it’s development, people started talking about RubyOnRails, a new “framework” for web development. Details were relatively sketchy, documentation was around but not for the beginner and most importantly, Rails was only at 0.4 or something similar, so I had a quick peak, and then ducked for cover within the comfortable blanket I had made from ASP.Net and C#. (Oh, but I signed up for BaseCamp though!)

And then in December last year, a client that I had developed a site for, using my technology security blanket of course, came to me with an idea for a site (it eventually became World Time Capsule– a site where you can store information/documents/whatever and lock them up, for loved ones to view later). There wasn’t a lot of money for the project and so I began to think that this might be an opportune time to begin to look at RubyOnRails. I sat down and did a couple of the online tutorials that had appeared around the place (mainly blogging applications, todo lists – modelled on Tada from the ubiquitous 37 Signals!)

It quickly became clear that here was a technology that had been lovingly crafted, with amazing skill and forethought. David Heinemeier Hansson is the creator of the framework. He was initially a contractor for 37 Signals, doing work in PHP, and then when he started work on BaseCamp he decided to create a new method of developing websites. Once BaseCamp was up and running, he extracted RubyOnRails from it, did some polishing up and released it out onto the web.

It was quickly taken up by developers around the world who were getting tired of the large amounts of work that needed to be done in order to start a project, and were taken by the simplicity of the code. These developers started making suggestions and improvements, working hard to fix bugs and issues that arose, and basically working hard to create a fantastic environment. These efforts continue today, with David recently allowing Jamis Buck and Jeremy Kemper to commit changes to the official RonR source tree.

Bear in mind that all of this development is because the extra nice people at 37 Signals decided that the technology they built when creating BaseCamp would be made available to whomever wanted it, at no charge. Open Source at it’s best.

So how is Spin making use of this great platform/framework? Other than World Time Capsule, this site is built using RubyOnRails, as are several more that we’ve done recently: Global Health Source, 11 Recruitment, and Planning Your Get Away. We are currently redesigning our company Intranet using Rails, building a fully fledged eCommerce site for an up and coming clothing retailer and developing an intranet for Five Senses Coffee to help them manage their orders and interaction with their clients.

I think you could safely say that Spin is now firmly running on Rails. In an upcoming post, I’ll look at exactly how Ruby On Rails is used to build a site.