2nd Gen Accelerators, Rails and attachment_fu

File this one under WTF.

I’ve been working on an updated version of the Soil Quality website for a little while now, and recently needed to deploy the site to a staging server for testing. I ordered up a 1/4Gb Accelerator from Joyent, configured it, and deployed the app, just like I’ve done with 20+ other sites, and BOOM, straight into a brick wall.

I opened up the log files and saw this error:

** Daemonized, any open files are closed.  Look at /tmp/soilquality-mongrel.8200.pid and log/mongrel.8200.log for info.
** Starting Mongrel listening at 127.0.0.1:8200
** Define INLINEDIR or HOME in your environment and try again

Wonderfully descriptive I know, but something I’d never run into before. So, fire the the google and it turns out it’s a common error, with a common fix: just put

into your config/environment.rb file, make sure the directory exists and that it’s writeable by the mongrel, restart, and away you go.

But of course, that didn’t fix it, did it.

Much hair pulling ensued. I finally enlisted the help of Darcy Laycock and together we managed to track the problem down to the attachment_fu plugin, which was causing the problem as the mongrel process booted. OK, so now we knew where the problem lay, but what was causing it.

It turned out to be the ImageScience image processor, or more specifically the way the attachment_fu plugin and ImageScience work together ON A 2ND GENERATION JOYENT ACCELERATOR – eg the ones that use pkgsrc. Didn’t seem to cause the problem on an ubuntu machine, nor on one of the older BlastWave based Accelerators. I’m not sure why as of yet, as I was more worried about getting some sleep last night when we managed to fix the problem.

vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/processors/image_science_processor.rb

@@default_processors = %w(ImageScience Rmagick MiniMagick Gd2 CoreImage)

vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/attachment_fu.rb

Like I said, I have no real idea why this is happening at the moment, but I’ll try and work it out and update this post if I get anywhere.

Once again, thanks to Darcy for A) being a sounding board, B) helping fix the problem and C) being up and available when I needed him :)

Edge of the Web: Ben Buchanan

Although it only seems like yesterday that I got back from Sydney and the wonderful Web Directions South, things are rapidly heating up for Perth’s own Web Conference: Edge of the Web, taking place at the UWA Club on November 6 and 7, 2008 – tickets available now!.

There is a huge list of international and national speakers on the program, one of whom is Ben Buchanan. I recently conducted a short email interview with Ben, talking about some of the subjects that he’ll touch upon in his talk Hacking Humans: Advocating for a better web

Ben is just one of a range of awesome speakers who are going to be presenting at Edge of the Web on the 6th November. On the following day, there are 4 half-day workshops on a diverse range of topics.

And then, to cap the two days off in style, Friday night sees the WA Web Awards, where, in a burst of glorious self congratulation, the best of the WA Web industry are recognised for their efforts. Get your tickets here.

Web Directions South 08

So, thanks to the fine people at 5 Senses Coffee, I trundled over to Sydney for Web Directions South 08.

Thursday, 25th September 2008

Opening Keynote – Lynne Johnson

Boo. So mismatched to the audience it wasn’t amusing in the slightest. If we were a bunch of magazine publishers from 2003, then it might have been interesting, but we weren’t, and so it wasn’t.

Derek Featherstone

Nubbins. Yay!!

I’ll just pretend that the opening keynote didn’t happen, and that this was the start. It’s always good to see someone who A) has command of their field and B) is a great presenter. Totally comfortable on stage, and thoroughly knowledgeable about his topic, which he managed to make seem almost exciting, which is pretty hard for what is essentially a dry topic.

iPhone Development

Tim Lucas and Pete Ottery worked on the iPhone Site and did a good presentation on some of their experiences. Perhaps because I had already started on the iPhone site for 5 Senses and I had already encountered most of those problems, I didn’t really come away with much. But that’s cool, because it was an intro level talk, not for an iPhone Dev Conference or anything.

JavaScript Libraries/Frameworks

Yeah, good times, much laughter, fun, and frivolity, and then when Cameron rolled out the Drum Machine – FUCKING BRILLIANT.

I’m not sure if I actually learned anything, except that just about everyone is using jQuery, and those of us still on Prototype are mocked mercilessly.

August de la Rejos

Hmmm, ok, but hold the ads.

An interesting talk about the progress of user interaction with computers, as well as the psychology behind it, that had some wonderful videos outlining potential future developments, but was undermined by the fact that they were MicroSoft corporate videos. Oh well.

WebJam8

Bucketloads of awesome, mixed in with tasty beverages, fine companies that sponsored the tasty beverages, fine talks, and conversation. Mad times. Cannot wait until WebJam 9 as part of Edge of the Web

Friday, 26th September 2008

Jeffrey Veen

Hot. Wired.

Awesome talk about visualising data, and what good design can achieve. Memorable, and featuring the entirely wonderful Gapminder World

Gina Bolton

No. Not good. I don’t know if Gina has a ninja like mastery of CSS (I’m guessing she does since she consults/designs for companies like Apple and has written a book), but statements like “from what I’ve heard when talking to developers, computers start counting at 0” doesn’t make code-monkey happy.

A thorough understanding of your topic, and avoiding statements like “I haven’t checked the blogs for a month or so, so I don’t know if the CSS3 Working Recommendation has been ratified or not.” Seriously!

Michael™ Smith

Not too bad. Bit of a laundry list. Of course, any recitation of the trials and tribulations of HTML 5 isn’t really going to be anything but dry. Still, good to hear for the most part. However, the take away from this, Gina’s and Douglas Crockford’s talk is that it’s been 10 years since any successful revision to HTML, CSS or ECMAScript, which just instills wonderful amounts of trust and faith in committees.

And really? Michael™ What is this – Burning Man?

Myles Eftos.

Huzzah! The madpilot flew like a crazy bastard over the top of the dangerous territories of OpenID and OAuth. And if that isn’t enough, he launched incursions into HTTP 1.1 as well as REST and SOAP. Lucky for all he came back unharmed.

Douglas Crockford

Good. Real code. Real ideas.

And scary. (We are so totally screwed if even half of what he was saying comes to fruition.)

Mark Pesce

Oversold. A fishing basket full of “ok, but meh”. One of the apparent pitfalls of giving what, from all accounts, was an amazing presentation at WDS07 that inspired everyone who went, is that the next year expectations are particularly high. When those expectations aren’t met (probably through no fault of Marks) it leaves people underwhelmed. I didn’t necessarily agree with the central theme of the presentation (that twitter and the like are going to transform society and its’ institutions), so that didn’t help.

One thing I will say is that if you need to make a point that you aren’t going to do next year’s closing keynote, you probably shouldn’t be doing this years.

A slight down to end on.

DRINKS!!!!!

Wheee! All that needs to be said can be summed up in these two tweets that I, um, twitted.

Salad is what food eats
09:19 PM September 26, 2008 from twitterrific

Has teh drunk on. Hi meat lady!
12:18 AM September 27, 2008 from twitterrific

Would I go again?

Totally. As we are no doubt about to find out, running a conference is a tricky thing. Getting a mix of the right speakers, hoping they are all in top form, and cultivating the vibe is no doubt a difficult thing. I had a great time in Sydney: I met some great people, learned lots, felt inadequate (I mean, you’re talking to Douglas Crockford over a beer … what do you say?), and saw a Europe video on a large screen.

The Slow Decline of Internet Explorer and Slower Rise of Firefox

For the past 8 years or so, 5 Senses Coffee has been both a client of mine, and for the last 15 months, my employer. The public face of that work is the main website, which sells coffee, teas and other wonderful items. I’d like to think that the visitors to the website are a pretty good spread of the general population of the internet, not necessarily too technical/web based, and not too “Hey, what’s this internet thing all about?”

Every now and then a story will appear in my feed reader about the use of one browser being higher than another, and more often than not, that site has a skewed audience, which means that it’s users are more likely to have the latest and greatest than other, less browser obsessed users.

So, bearing in mind that this probably has no relationship to the wider world than any other site, here is the browser share of visitors to the 5 Senses Website over the past 21 months.

And the breakdowns for the various versions of Internet Explorer, where “IE x” is all versions other than IE 6 and IE 7.

and FireFox.

What is evident from these numbers is that while IE is still the dominant browser, with almost 2/3rds of market share, Firefox has been slowly and steadily increasing. But not necessarily at the same rate that IE is losing share. Firefox seems to be losing a bit of ground to other browsers, namely Safari.

The interesting number is that Safari has doubled over the sample period. True, it’s still third by more than a few lengths, but it seems to be taking some of those gains from IE. I’m guessing this is mainly due to the increased market share of Apple in general, and not some massive use of Safari on MS Windows.

Other, possibly unrelated points that can be extracted is that FireFox users are more likely to upgrade their browser than IE users. This can be seen by the almost complete drop off in FF 1.5x users at the beginning of the timeframe, and the rapid rise of FF 3.x in the last 3 months, to the point that it almost equals FF 2.x. That said, there certainly appears to be some die-hard holdouts for FF 1.x, who are disappearing at a slower rate than those still using FF 1.5.x

So, not a scientific sample and/or analysis by any stretch, but still interesting. The surprising item here is the doubling in share of Safari (which, in the authors less than humble opinion is the best browser there is.) True, it didn’t have far to go to double that share, but it still had to be done.

The upshot of all of this is that, unfortunately, there is still a large portion of the internet population still using IE 6 (approximately 20% of all visitors it seems), so it still needs to be supported, or those people need to be encouraged to migrate upwards. How do we do that? Well, your guess is as good as mine.

Yahoo! Maps - Fully Suck

Flickr has long had geotagging support, which is a great way of being able to browse photos for a particular town/region/whatever. However, if you’re in Australia, geotagging your photos with the Flickr map service boils down to blind luck. The maps are so bad as to be beyond useless, and in fact are laughable. Here is the highest resolution map for the western suburbs of Perth, Western Australia:

Great huh? I mean, that light grey blob really helps you work out where you are, as well as that thing that might be a road.

So, way to compete with Google there Yahoo!

Lolcat Sandman

I happened across a screendump of a 4Chan thread today that had lolcat pictures that matched up with the lyrics of Enter Sandman. Someone posted a comment saying that they’d love to see it with the soundtrack. So, I fired up iMovie and bingo: Lolcat Sandman.

Of course, the music copyright is owned by Metallica, but I actually had to go and buy it from iTunes because I’d deleted my “unofficially obtained” copy a while back.

Gmail and MobileMe - Living Together in Perfect Harmony

I’ve been using Gmail for about 4 years now, and it’s pretty much become the centre of my email life, with my various email accounts being funneled into Gmail. I’ve used it with both the web client, and Mail.app on my Mac via IMAP, and have been pretty happy with both.

With the release of MobileMe and the iPhone 3G I made the decision to move to MobileMe (an Apple fanboi at heart, I wanted to keep it all within the one happy family – particularly seeing as work just bought me a MacBook Air.) But I was really reluctant to stop using Gmail as the repository of all of my email, both received and sent (awesome search, archiving and just to have a backup on another server).

It’s relatively easy to setup forwarding from Gmail through to MobileMe, which you then add to your mail client (Mail.app on the Mac) and the mail will come in as per normal. But this would mean that any sent items won’t appear in Gmail, which in turn means that Gmail is only storing half of the picture. But with adrianlynch doing most of the legwork, we were able to work out a solution.

There are quite a few little things that need to be done in order to fine tune the setup, particularly if you want to use multiple email addresses. I’ll cover those, and then get to the meat of the setup.

Multiple email addresses in Gmail.

Gmail has had the ability to support multiple email accounts for quite a while, and it all works quite well. First of all, sign into gmail, then go to the settings -> Accounts tab. The bet we’re interested in is the “Send Mail As” bit. You can add all of the email addresses you want to have gmail send email for in here. Gmail will send a confirmation email to the address you enter (just to make sure that you are in fact billg@microsoft.com).

Setting up the email forwarding

There are two ways to forward email in Gmail. One is in the Settings -> Forwarding and POP/IMAP tab, which will forward every message to the address you specify. The other way is via a Filter (Settings -> Filters). I prefer to use the filter, because it allows me to get rid of some of the cruft mail that I don’t really ever want to see again (eg, one of my mail accounts flags spam as *SPAM* ... if I didn’t setup a filter to not forward this, I’d receive all of those emails in MobileMe, which is sub-obptimal to say the least).

Setting up your computer

Of course, this assumes you’re using a Mac. If not, I’m sure you’re a smart enough person to work it out :)

MobileMe setup

First up, add your MobileMe account. It’s pretty simple. Just enter your MobileMe account details (same as your old .Mac account), and it should all be jake.

If you’ve setup gmail to handle multiple email addresses, enter them all in the “Email Address” field, separated by commas.

Gmail setup

Now, you need to add your gmail account to the system, as an IMAP account. Follow the instructions on the Google Help Center

Once you have added the account Mail.app will probably start downloading all of your mail from Gmail, which could take a fair while. This may or may not be something you want: your choice.

After the mail has all downloaded, go into the Advanced tab for the Gmail account in Mail.app, and uncheck “Enable this account.”

Then, go back into your MobileMe account and set the outgoing SMTP server to smtp.google.com. All of your sent mail will now go through Gmail, which is great for archiving purposes. Because MobileMe is an IMAP server, it will automatically be copied up to the MobileMe server (because it’s storing a copy in your Sent Items folder on your Mac), which means it’s then available on all the computers you use MobileMe with, including your iPhone / iPod Touch.

Why did I enter the multiple email addresses?

If you have multiple email addresses, and you entered them all in the “Email Address” box earlier, you will now have the option to choose which address to send your mail as when you create a new message.

There’s one little gotcha here, and it’s why I mentioned setting up the addresses from within gmail before. If you try to send a message from an email address that Gmail doesn’t “know” about, it will replace the sender email with your Gmail address. This is obviously to stop you spoofing your address, but it took me a little while to work that bit out. So, just make sure that you’ve defined all the addresses that you want to “Send As”

Setting up your iPhone / iPod Touch.

Once you have the above system up and working, you can basically mirror that setup on your iPhone. You need to create both accounts under “Mail, Contacts, Calendars” in the Settings application. Then disable the Gmail account, and set the outgoing SMTP server to gmail in the MobileMe setup.

Push Goodness.

If everything has gone according to plan, you should now have your Gmail “pushing” to your iPhone, and also appearing on your computer. And if you have multiple macs, and have sync turned on for “Mail Accounts” and “Mail Rules and Smart Folders”, then you will shortly have those settings on both of your computers.

But I'm not installing ANYTHING!

Nice one Adobe.

Check out the space required, particularly when I don’t have anything checked.

Freezing Rails with Git

Now that the Ruby On Rails team has moved the codebase over to github, some of the standard rake tasks aren’t working the way that they used to. When it was on SVN, it was possible to type

and the appropriate version would be copied into your vendor/rails directory.

Now if you do that, rake downloads a zip of the edge release. Which is fine and all, but sometimes you don’t want to be on edge … like in any production site.

So, I found a screencast that goes through the process, but I thought I’d actually put the text into a post, mainly for my own reference more than anything else.

$ rails path_to_app

$ cd path_to_app

$ git init

$ git submodule add git://github.com/rails/rails.git vendor/rails

At this point, git will effectively clone the repository, so that you can then choose one of the branches to “freeze” to. Type “git tag” to get a list of all the available tagged branches. Choose the one you want and type

And that’s it. Slightly more involved than the old way, but still none too shabby.

Ev puts twitter to sleep

Random chance meant that my twitterific looked like Ev has the power to put twitter to sleep with a single tweet.

How to access US iTunes content on your Apple TV in Australia

One of the frustrating things about being an Apple TV owner in Australia is that we are missing out on a fairly major component of the experience: namely renting movies and purchasing TV shows. It’s possible to create a US iTunes account, but generally that’s involved having a US based credit card, which of course, isn’t the easiest thing to come by.

Yesterday, I found this guide to setting up an account, which has some extra steps in it that aren’t entirely necessary, if you’re prepared to actually pay for some content.

Basically, you just need to create an account on the US iTunes store. First, sign out of your iTunes account, (if you already have one of course). Then, at the bottom of the iTunes home page, select the US store

Once you have the US store home page up, click on “Sign In” in the upper right hand corner. Then you will have the opportunity to create a new account. There’s a EULA you have to accept, and then you’ll need to enter some personal information. The next screen is the payment screen. Make sure you choose “None” as your payment option.

You will need to have a US address to enter at this point. The only point to note here is that the Zip Code you enter matches the state that you have selected. Choose “Done” and you should have yourself a US iTunes Account.

All well and good you say, but that doesn’t do me any good if I can’t buy anything does it? True. What you need to do is get yourself a US iTunes Gift Card. You can easily do this by searching eBay for iTunes Gift Card=. Purchase a card. You’ll probably have to pay a small premium on the card, eg I paid US$19 for a US$15 card. Obviously the people selling the card need to make themselves a profit.

Once you’ve got the card, the seller should email you a code, which you just cut and paste into the “redeem” section on the ITMS home page. Then, you’ve got credit on your iTunes account.

So, then you just need to add your account to your Apple TV, change the country in the settings and bingo, you can start renting movies directly from the US. The HD movie quality is excellent, and on my decent ADSL connection (14Mbps) the movie starts playing in almost no time.

Happy watching.

Getting Rails, Git and Capistrano to work on a Joyent Accelerator

After several days of repeatedly smashing my head into both a metaphorical and an all too real brick wall, I seem to have managed to get git and capistrano working happily together on my Joyent Accelerator. I’m also using github for my git hosting, which threw up it’s own little challenge mid-way through the entire process.

Now, I should probably also say that I already had my site up and running using subversion, capistrano and my accelerator, so this article isn’t necessarily going to help with getting everything setup the first time. For that, you probably need to read this wiki entry.

Things You Will Need

• A github.org account
• A dedicated Joyent Accelerator (I have no idea how to do all of this on a shared accelerator. Sorry.)
• Also, I’m really only talking about RubyOnRails apps here … not too sure how applicable a lot of this is to other frameworks (it will probably help at least.)

Getting started

Assuming that you have your project in a git repository, and have a github account (and obviously an Accelerator) we can start.

Compiling git on your accelerator

Unfortunately, the first step in the process was, for me at least, a total nightmare. I’m not the biggest unix-head by any stretch, but I can do some basic tasks with a degree of proficiency. Unfortunately, I went into a dark place trying to get git compiled. One thing to note is that I’m talking about setting up the git client here, not a git server. Because capistrano executes scripts on your remote server, you need to have a copy of the client software setup for capistrano to call.

So what did I do? There are a couple of helpful threads on the Joyent Forums:

• Got git?
• Git and Cap 2.0
• Installing git

Hopefully those threads will put you onto the path of successfully compiling and installing git onto your Accelerator.

Setting up SSH keys with github and your accelerator

When you setup your account on github, you need to setup an SSH key for authentication. github has a really good tutorial on how to do this. I have a user defined on my accelerator that my website “runs” under, so what I did was to create a key for that user which gets stored into the ~/.ssh directory. I then added the contents of the id_rsa.pub key to my github account, which allows that user to access the repository.

Another tip: don’t forget your passphrase. It’s needed in the next step.

Configuring capistrano

Assuming that you have your capistrano deploy.rb file setup as outlined here there are a few changes that you will need to make to get things working with git.

I’m using Capistrano 2.2 at the moment. I don’t think it will work with earlier versions because of the relatively new git support.

Here’s my deploy.rb file:

  require 'erb'
  require 'config/accelerator/accelerator_tasks'

  set :application, "website" 
  set :repository, "git@github.com:your_username/website.git" 

  default_run_options[:pty] = true
  set :domain, 'XX.XX.XX.XX' #Your Accelerators public IP address
  set :deploy_to, "/var/www/apps/#{application}" 
  set :user, 'website_account_username'
  set :scm, :git
  set :scm_username, "github_username" 
  set :scm_passphrase, "your passphrase here" 

  role :app, domain
  role :web, domain
  role :db,  domain, :primary => true

  set :server_name, "url.for.website" 
  set :server_alias, "*.url.for.website" 

  # Example dependancies
  depend :remote, :command, :gem
  depend :remote, :gem, :money, '>=1.7.1'
  depend :remote, :gem, :mongrel, '>=1.0.1'
  depend :remote, :gem, :image_science, '>=1.1.3'
  depend :remote, :gem, :rake, '>=0.7'
  depend :remote, :gem, :BlueCloth, '>=1.0.0'
  depend :remote, :gem, :RubyInline, '>=3.6.3'

  ################################
  # Some tasks for the old server
  ################################

  task :after_deploy do
    # tasks to run after deploy
  end

  ################################
  # End tasks for the old server
  ################################

  deploy.task :restart do
    accelerator.smf_restart
    accelerator.restart_apache
  end

  deploy.task :start do
    accelerator.smf_start
    accelerator.restart_apache
  end

  deploy.task :stop do
    accelerator.smf_stop
    accelerator.restart_apache
  end

  after :deploy, 'deploy:cleanup'

It appears that the important line here is ’ default_run_options[:pty] = true ’. This means that capistrano can respond automatically for the request for the SSH Key passphrase that github replies with when you try to clone the repository.

If everything is working, you can type ‘cap deploy’ and it should all deploy nicely. If you get this error:

then there’s a problem with your SSH key and your settings on github. Make sure the key you copied into your github account is the public key for the SSH in your .ssh directory.

Hopefully, you’ll be up and running. If you have any tips, recommendations or corrections, leave a comment.

Special Valentines Day Message

From w00t

Ideas 4 - Lisa Herrod

Here’s the second video from Ideas 4 from last week. In this one, Lisa Herrod from Scenario Seven gives her “Usability is More Than Skin Deep” presentation.

Once again, there were a few teething issues with the camera. I framed it using Harriet as a guide, but it turns out that Lisa is a little bit taller, which means there’s a little bit of wandering camera action going on for the first minute as I play with the tripod.

It was a great night. I wished I could have hung around a little bit longer at the end, but I had to rush home (sick children and such – all is good now) so I missed out on the trip to Fast Eddies.

Ideas 4 - Rachel Cook

Last night, AWIA (the Australian Web Industry Association) held one of our semi-regular “Ideas” nights, Ideas 4 to be specific. Rachel Cook (minti.com) and Lisa Herrod(Scenario Seven) each gave fantastic presentations to a great crowd (about 85 people by my rough count).

Here’s Rachel’s talk. There were a few teething issues with the video, mostly to do with lighting and sound (lol), but it’s definitely watchable.

Lisa’s presentation will be posted next week (there’s an upload limit, and I shot this in HD, so it’s pretty sizeable). I’ll post a link when it’s available.