ActsAsFlinn

Updated 2008-06-19 – better support Halcyon

JSON-P Rack Handler

Juicing Ruby

I’ve been trying to find ways to squeeze all the juice out of Ruby lately. So many blog posts talk about how Rails doesn’t scale, Ruby is slow, blah blah. I had a convo with another developer at work today that went something like this:

Me: if we’re willing to do anything for performance we’d we just switch to Java

Jared: Yeah let’s not do that.

Switching to Java would be a big trade off in performance but also in development time. That’s a big trade off that none of us around here think is worth. That’s where this Rack handler comes in.

JSON-P Caching

I’ve posted before about how to use JSON-P in Rails and how to cache JSON-P in Rails with fairly decent results (500+ reqs/sec) but I felt like I could do better. Action caching always seemed like the best way to cache the full JSON output of a request but the fact that jQuery uses a dynamic callback takes action caching out of the equation.

Speeding up JSON-P

500+ reqs/sec is good and all but I felt like if I could action cache then somehow pad that cached JSON result with the callback I’d get better performance. I thought I’d see what Merb could offer on the JSON-P front. In my quest to juice Ruby my test setup looks like this: Merb, Datamapper, Memcached (the c gem), Memcached (the server) and Ebb. I’m really impressed with Merb and Datamapper in terms of development and I’m equally impressed with Rack and Ebb for performance.

I read recently it’s possible to use Rack to filter results to gzip output, which got me thinking. Why not try to do the same to pad my action cached JSON. Well it is possible and I’m squeezing out JSON-P at ~1200 reqs/sec with the JSON-P Rack Handler with my test stack.

JSON-P Rack Handler

# config/jsonp.rb
class JsonP
  def initialize(app)
    @app = app
  end

  def call(env)
    status, headers, response = @app.call(env)
    request = Rack::Request.new(env)
    response = pad(request.params.delete('callback'), response) if request.params.include?('callback')
    [status, headers, response]
  end

  def pad(callback, response, body = "")
    response.each{ |s| body << s }
    "#{callback}(#{body})" 
  end
end

Config

# config/rack.rb
require 'config/jsonp'
use JsonP
run Merb::Rack::Application.new

In Halcyon

# runner.ru
...
require 'config/jsonp.halcyon'
use JsonP
run Halcyon::Runner.new

Rails Support Soon

One last thing… whenever Rails starts using Rack, you’ll be able to use this in your Rails app.

About Me: I’m a developer with Sports Technologies, a Rails firm specializing in community focused web applications aimed at sports and entertainment. We’re always looking for Rails talent, if you’re looking to work on rewarding high profile projects with a seasoned team of professionals give us a shout.

Rails based JSON-P

For the last 2 months I've been working on part of a project for a large publisher rolling out some new web services that use Rails based JSON to display comments on static web pages. This project has been a learning experience for nearly everyone involved on the Rails side and on the front end development side. We’ve overcome a number of limitations with the lack of a safe data transport in web browsers and limitations in the way Rails handles JSON-P and cross site REST. Some of the problems are unique to the solution we’re providing but they are no doubt a thorn in other developers’ sides. This is a follow up to a post I made about a month ago called JSON-P on Rails with JQuery. Below I’ve laid out some of the Rails problems and solutions related to JSON-P, cross domain JSON using JSON-P, JSON-P with jQuery and Rails caching JSON-P.

1. JSON-P with Rails
2. JSON-P with jQuery
3. Rails Caching JSON-P
4. Cross Domain RESTful JSON-P

JSON-P with Rails 

My son: “What are you doing?”

Me: “Writing a blog post about JSON-P.”

My son: “Who is Jason P.?”

Me: “Javascript Object Notation with Padding.”

My son: “Oh.”

So as you hopefully know JSON-P isn’t some guy working in the back office of your IT department. JSON-P is a powerful data transport method in use all over the internet by some heavy hitters like Yahoo!, CNN, ESPN and lots more. JSON-P is a hack of sorts to get around a few problems like the dreaded Same Origin Policy. Plain old JSON is great but it can’t do cross site/cross domain. Short of constructing a same origin proxy JSON-P is your best option for cross site AJAX. The best part about JSON-P is you can use it as an alternative to XMLHttpRequest to transport JSON data across sites.

JSON-P Structure

JSON-P can be expressed a number of ways:

1. Embedded in an HTML request as an invisible iframe that includes a callback to instantiate JSON into the global scope. (clunky)
2. With a static callback wrapping JSON (good for server cache, bad because of browser cache).
3. With a dynamic callback wrapping JSON (harder to server cache).

Out of the box Rails best supports option #3 like so.

render :json => @chats, :callback => params[:callback]

This could result in the following JSON-P response, foo being the callback param you submitted as part of your query string.

foo(
 {"chats": [
   {"user": "actsasflinn", "created_on": "June 04, 2008", "id": 109328, "body": "Hey there Jim, I have never seen that kind of hat before!.  Where did you get it?"},
   {"user": "jim", "created_on": "June 04, 2008", "id": 109329, "body": "It's an ass hat. I got it from my ex."}
  ]
 }
)

JSON-P with jQuery 

Using the above you can use some built-in jQuery methods to make life easy once again. In case you don’t know, jQuery is the balls! jQuery’s solution to the same origin policy is support for the script tag transport. jQuery will handle the transport magic if you specify a callback=? as a query param (yes question mark - jQuery fills it with a dynamic callback name in for you). This tells jQuery to add a dynamic callback and to use padding and the script tag transport. Check out the getJSON method example using the dynamic callback query param.

$.getJSON("http://example.com/chats.json?callback=?", function(data){
  $.each(data.chats, function(i,chat){
    alert(chat.body);
  });
});

Rails Caching JSON-P 

The dynamic callback does a few important things like defeating the browser cache but it kills Rails’ ability to do the same. Action caching the resulting JSON data doesn’t work because jQuery changes the callback name with each request. Overcoming the lack of caches_action isn’ t so bad as long you use data caching within your format.json block. The to_json method is pretty intensive and it’ll kill your reqs/sec so it makes sense to cache the output from to_json. Using cache_fu something like the below will get you rolling with 500+ reqs/sec...

class Chat < ActiveRecord::Base
  acts_as_cached

  def self.recent_chats(format = nil)
    chats = find(:all, :order => "created_on desc", :limit => 10)
    chats.send(format) unless format.blank?
  end
end

class ChatsController < ApplicationController
  def index
    respond_to do |format|
      format.json do
        @json_chats = Chat.caches(:recent_chats, :with => :to_json)
        render :json => @json_chats, :callback => params[:callback]
      end
    end
  end
end

It’s not as great as action cache but it minimizes the db hit and the to_json processing time (which is hefty) reducing the response to simply padding the memcached JSON.

Cross Domain RESTful JSON-P 

If you’ve gone this far JSON-P is working wonders... unless of course you want to do AJAX POSTs (argh)! So this next bit of mojo is indeed a hack and not for the faint of heart. As you might know most browsers don’t support the HTTP DELETE and PUT methods so Rails spoofs them by making an POST and passing in a query param called _method. Wonderful but you can’t do an AJAX POST across domains. The AJAX REST Nazi says “no REST for you!” You can of course monkey patch Rails to use _method regardless of the actual HTTP method enabling you to spoof AJAX and REST across domains.

module ActionController
  class AbstractRequest
    def request_method
      @request_method ||= begin
        method = (parameters[:_method].blank? ? @env['REQUEST_METHOD'] : parameters[:_method].to_s).downcase
        if ACCEPTED_HTTP_METHODS.include?(method)
          method.to_sym
        else
          raise UnknownHttpMethod, "#{method}, accepted HTTP methods are #{ACCEPTED_HTTP_METHODS.to_a.to_sentence}"
        end
      end
    end
  end
end

Achtung! Monkey patching with the above will expose your create method without using an actual post. Imho no big deal, others might be more cautious (CAPTCHA is always an option), ymmv. On to the controller...

class ChatsController < ApplicationController
...
  def create
    @chat = Chat.new(params[:chat])
    respond_to do |format|
      if @chat.save
        format.json { render :json => { :chat => @chat }, :callback => params[:callback], :status => :created, :location => @chat }
      else
        format.json { render :json => { :errors => @chat.errors.full_messages }, :callback => params[:callback], :status => :unprocessable_entity }
      end
    end
  end
end

Using jQuery to serialize your form, a spoofed AJAX POST might look something like this.

$(document).ready(function() {
  $('#chat_form').submit(function() {
    var chat = $('#chat_form').serializeArray();
    $.getJSON("http://example.com/chats.json?_method=post&callback=?", chat, function(data){
      if (data.errors == undefined) {
        alert(data.chat.body);
      } else {
        $.each(data.errors, function(error) { alert(error); })
      }
    });
  });
});

Huzah! Easy right? It took quite a while to put all this together into a cohesive process that everyone could work with. At the onset Rails didn’t support spoofing REST, action caching JSON-P or even cross domain JSON but with a little ingenuity we made it happen and you can too. JSON-P is the solution to put your RESTful Cross Site XMLHttpRequest woes to bed.

p.s. none of the above code has been tested so if you copy and paste it and it doesn’t work, don’t complain unless you include working code.

About Me: I’m a developer with Sports Technologies, a Rails firm specializing in community focused web applications aimed at sports and entertainment. We’re always looking for Rails talent, if you’re looking to work on rewarding high profile projects with a seasoned team of professionals give us a shout.

I was initially skeptical of Passenger aka mod_rails (because it seems to good to be true) but tonight I became a believer. We recently finished work on fairly high profile project. The installation seemed to be running fine but after installing monit regular tests revealed mongrel instances were hanging or unresponsive and randomly coming back online. After 3 days of the mysterious mongrel issues, apache tweaking of the proxy config, replacing mod_proxy_balancer with haproxy, and experimenting with other backends like thin and ebb I decided to give Passenger a shot. The gem was super easy and the apache install application was super easy and very helpful. After installing Passenger the new site is rocking. Goodbye mongrel.