Handling APIs with Ruby XML Parsing
Have you ever wanted needed to write a Ruby wrapper for an XML based API?
If you have a burning desire to seamlessly exchange data over the web or you just want to use the latest Interweb 2.0 service – you’re most likely contemplating writing your API client in Ruby… yes that’s why you’re here isn’t it?
Why Use Ruby to Parse XML?
It’s a fact – Ruby kicks ass at parsing XML. You can find tons of examples of XML API clients written in Ruby.
ActiveResource parses XML and handles RESTful HTTP
The new ActiveResource Rails gem found in Rails2 makes pretty light work of handling XML APIs via REST. Unfortunately not everyone is rushing to support REST just yet. If you support a big Rails 1.2 app you can’t just run out an add ActiveResource to your project (which is my case). This post is not about REST or ActiveResource so if you’re looking for that, click the link you just clicked skipped over.
Enough with the useful useless Ruby XML facts…
Show me how to write a Ruby XML API wrapper
I’ve been working with an API recently for a third party registration system on a project we’re rolling out soon. The third party provides their API using domain scoped query URLs and HTTP GET params and returns XML documents.
Huh?
http://example.com/aaflinn/lookup?username=billlumberg&password=swingline
XML Messages
When you get a matching user/pass combination you get something like this.
<auth>
<user>
<username><![CDATA[billlumberg]]></username>
<fullname><![CDATA[Bill Lumberg]]></fullname>
<zipcode><![CDATA92131]></zipcode>
<email><![CDATA[bill.lumberg@initech.com]]></email>
</user>
</auth>
When you put the wrong password you get an error like so.
<auth>
<error><![CDATA[Invalid username/password combination]]></error>
</auth>
If you don’t enter a username at all you’ll get an error thusly.
<auth>
<error><![CDATA[No username given.]]></error>
</auth>
API Wrapper Concepts
Here are some concepts that I felt were important when I started writing my wrapper.
- Simple – it should be easy to code (I hate writing stupid code)
- DRY – if it’s worth writing the wrapper make sure it’s reusable
- Self Documenting – it should be as self documenting as possible (rdoc)
- “Exceptional Code” – it should raise errors on exceptions and handle errors from the libraries it makes use of
- Don’t Spam – Don’t abuse APIs (grrr!)
Requirements
The wrapper should use a GET query string to perform a query to the service provider passing an md5 hashed password and username combination. If the user exists parse the XML result document and return an instantiated user object based on the XML. If no user exists raise some type of rescuable error (RecordNotFound).
Additionally the wrapper should be able to create a new user. This particular service provider uses an HTTP GET query string but in some cases you might find a plain old POST like you’d see in a form or you’ll need to build XML and pass that back to the service provider (which I am not doing here). The wrapper should be able to be able to interpret error messages and determine the status of our create request in a graceful way.
Ruby API Wrapper by Example
The names have been changed to protect the innocent. I’ve edited the wrapper a bit to reduce some complexity and renamed it to hide the actual API provider. Read on in the comments of the code, I’ve done everything on that bullet list and you can read the code as you read the comments setting up just about every line of code written.
#
# Usage
#
# === setup ===
# require ‘ApiExample’
# ApiExample::account = ‘test’
# ApiExample::logger = Logger.new(‘example.log’)
#
# ===Find A User===
# user = ApiExample::User.find(‘bill.lumberg’, :password => ‘swingline’)
#
# ===Create A User===
# user = ApiExample::User.create(:username => ‘bill.lumberg’, :password => ‘swingline’, :email => ‘bill.lumberg@initech.com’)
#
require ‘base64’
require ‘digest/md5’
require ‘net/http’
require ‘rexml/document’
require ‘cgi’
require ‘logger’
module ApiExample
# Example Database Host
HOST = ‘www.example.com’
# These params are required to register a new user
REGISTRATION_PARAMS = [ :username, :password, :email ]
# ApiExample account (brand account not user account)
@@account = nil
@@success_url = nil
@@fail_url = nil
@@logger = Logger.new(STDERR)
mattr_accessor :account, :logger, :success_url, :fail_url
# Exception handling
class ApiExampleError < StandardError; end
class UnexpectedError < ApiExampleError; end
class RegistrationError < ApiExampleError; end
class RecordNotFound < ApiExampleError; end
def md5password(password)
Base64.encode64(Digest::MD5.digest(password)).strip
end
# Using Struct here allows us to make our object act similar to an
# ActiveRecord object. In this example it’s not so obvious of a pain
# in the ass it is but the real class has about 20 or so attributes
# Using means I don’t have to create attribute read and writes
# attribution – I got this idea from the Ben Vinegar’s
# <a href="http://rubyforge.org/projects/freshbooks/">freshbooks gem</a>
User = Struct.new(:username, :email, :password, :fullname, :zipcode)
# Extend the Struct attributes by adding the class and instance methods we want
class User
attr_accessor :attributes, :errors, :new_record
# class method for create a new user
# Usage:
# user = ApiExample::User.create(:username => ‘bill.lumberg’, :password => ‘swingline’, :email => ‘bill.lumberg@initech.com’)
#
def self.create(attributes = {})
object = new(attributes)
object.create
object
end
# class method for finding an existing user
# Usage:
# ApiExample::User.find(‘bill.lumberg’, ‘swingline’) # plain text will be sent md5 hashed rather than clear text
#
def self.find(username, password)
query_params = { :username => username, :password => ApiExample::md5password(password_option[:password]) }
query = query_params.collect{ |k, v| [k, v].map{ |kv| CGI::escape(kv.to_s) }.join(’=’) }.join(’&’)
# @@account doesn’t need to be encoded because we are setting internally, the other stuff is vulnerable
uri = URI::HTTP.build(:host => ApiExample::HOST, :path => "/#{ApiExample::account}/lookup", :query => query)
# I’ve got a bunch of these loggers throughout which are used to debug, remove as you see fit.
ApiExample::logger.debug("URI: #{uri}")
# Make the actual HTTP Request
result = Net::HTTP.get_response(uri)
# Parse the XML Result of the HTTP Request
response = REXML::Document.new(result.body)
ApiExample::logger.debug("RESPONSE: #{response}")
# Check to see if there is a user node, if there’s not raise an error similar to ActiveRecord
unless response.elements[’//user’].nil?
attributes = {}
# Members is the Struct method for the attributes we setup…they correspond to what
# we expect the XML to return, so lets only handle what we know
members.each do |field_name|
node = response.elements[’//user’].elements[field_name.to_s]
next if node.nil?
attributes[field_name.to_sym] = node.text # you can do casting here if you need, I did
end
object = allocate
object.attributes = attributes
object
else
# Raise an error, setting the message to what the API sets as the error in XML
# if there is no ‘error’ node in the XML set an ‘unknown error’ message
raise RecordNotFound, (response.elements[’//error’].text rescue "Couldn’t find ApiExample::User with Username: #{username} because of an unknown error!")
end
end
# instance method to setup new object ala ActiveRecord
# Usage:
# user = ApiExample::User.new(:username => ‘bill.lumberg’)
#
def initialize(attributes = {})
@new_record = true
self.attributes = attributes
end
# instance method ala ActiveRecord
#
def errors
@errors = [] if @errors.blank?
@errors
end
# instance method to get an attributes hash ala ActiveRecord
#
def attributes
butes = {}
members.each{ |member| butes[member.to_sym] = self.send(member) } # you can cast here, I did
butes
end
# instance method to set attributes via a hash ala ActiveRecord
#
def attributes=(new_attributes = {})
members.each do |member|
unless new_attributes.has_value?(member.to_sym)
# #{member}= because you might possibly write an attribute writer to handle the input to =
self.send(member+’=’, new_attributes[member.to_sym]) # you can cast here, I did
ApiExample::logger.debug("#{member}: #{self.send(member).inspect}") # for snooping
end
end
end
# instance method to create the new instantited object ala ActiveRecord
# note the difference in create and self.create are the same as in ActiveRecord
# Usage:
# user = ApiExample::User.new(:username => ‘bill.lumberg’)
# user.password = ‘swingline’
# user.email = ‘bill.lumberg@initech.com’
# user.create
#
def create
# we pass our success and fail URLs even though we aren’t using them for their intended purpose
query_params = { :success => ApiExample::success_url, :fail => ApiExample::fail_url }.merge(attributes)
ApiExample::logger.debug(query_params.inspect) # for snooping
# check to make sure all required params are included
if ApiExample::REGISTRATION_PARAMS.all?{ |param| query_params.include?(param) }
# URL Encode everything
query = query_params.collect{ |k, v| [k, v].map{ |kv| CGI::escape(kv.to_s) }.join(’=’) unless v.blank? }.compact.join(’&’)
# @@account doesn’t need to be encoded because we are setting internally, the other stuff is vulnerable
uri = URI::HTTP.build(:host => ApiExample::HOST, :path => "/#{ApiExample::account}/register", :query => query)
ApiExample::logger.debug("URI: #{uri}") # for snooping
# Make the actual HTTP Request
response = Net::HTTP.get_response(uri)
ApiExample::logger.debug("RESPONSE: #{response}") # for snooping
# Check for failure and errors
if response[‘location’].include?(ApiExample::fail_url)
# Raise an error for each message
CGI::parse(URI::parse(response[‘location’]).query)[‘error’].each do |error_message|
raise RegistrationError, error_message
end
elsif response[‘location’].include?(ApiExample::success_url)
# Change the status to reflect the fact that we’ve saved the object…this could get much more
# in-depth in terms of ensuring our data was correctly saved… you wouldn’t expect to do
# that kind of thing in an RDBMS so why here?
self.new_record = false
else
# it wasn’t the fail url, it wasn’t the success url so what the hell was it?
raise UnexpectedError, "Unknown response URL!"
end
else
# For each missing required param add an error message
ApiExample::REGISTRATION_PARAMS.each do |missing_param|
self.errors << "#{missing_param} can’t be blank." unless query_params.include?(missing_param)
end
end
end
end
end
Limit By Scope Rails Plugin
I’m pleased to announce the release of the LimitByScope plugin, the second package in the Software as a Service project.
From the README:
This plugin adds class limit, available, and capacity methods to enforce quota limits on the creation of models. The plugin is primarily for use with the acts_as_scoped plugin (part of the saas project) because it otherwise enforces a flat limit on the creation of limited models. If used with the acts_as_scoped plugin you can lock users to a scope such as host or domain then ensure they can only create the number object within their plan. The Usage section below should explain a little better.
Documentation
http://saas.rubyforge.org/limit_by_scope
Rubyforge Project
http://rubyforge.org/projects/saas
Usage
# plan attributes:
# - name
# - price
# - item_limit
class Plan < ActiveRecord::Base
has_many :hosts
end
# host attributes:
# - name
# - plan_id
class Host < ActiveRecord::Base
cattr_accessor :current
belongs_to :plan
end
# item attributes
# - name
# - host_id
class Item < ActiveRecord::Base
limit_by_scope :host, :delegate => :plan, :error => 'Quota met: #{self.class.limit}, please upgrade your plan to add more.'
belongs_to :host
end
# for best results, use with acts_as_scoped like so:
class Item < ActiveRecord::Base
acts_as_scoped :host
limit_by_scope :host, :delegate => :plan, :error => 'Your service tier only allows #{self.class.limit}, please <a href=\"/upgrade?id=#{self.host.id}\">upgrade</a> your plan to add more.'
belongs_to :host
end
class ApplicationController < ActionController::Base
...
before_filter :current_host
def current_host
Host.current = Host.find_by_name(request.subdomains.first)
redirect_to('/not_found.html') and return false if Host.current.nil?
end
...
Installation
script/plugin install svn://rubyforge.org/var/svn/saas/limit_by_scope/trunk/limit_by_scopeSoftware as a Service
The plugin is part of the Software as a Service project on Rubyforge. You can use this plugin in conjunction with something similar to the userstamp plugin to automagically limit the scope of your find, calculate, save, delete methods. It is used with other plugins as a drop in that will allow you to turn just about any rails application into a software as a service.
Acts As Scoped Rails Plugin
I’m pleased to announce the public availability of the acts_as_scoped plugin.
From the README:
This plugin wraps find in a scope based on a persistent variable. This enables you to use something like the userstamp plugin to ensure a model’s find method returns objects scoped within the current user.
Documentation
http://saas.rubyforge.org/acts_as_scoped
Rubyforge Project
http://rubyforge.org/projects/saas
Usage
# You'll need to add a user_id column to the sandwiches table
class Sandwich < ActiveRecord::Base
acts_as_scoped :user # belongs_to is included
end
class User < ActiveRecord::Base
cattr_accessor :current
has_many :sandwiches
end
class ApplicationController < ActionController::Base
...
before_filter :current_user
def current_user
User.current = User.find(session[:user_id]) unless session[:user_id].blank?
end
...
Installation
script/plugin install svn://rubyforge.org/var/svn/saas/acts_as_scoped/trunk/acts_as_scoped
Software as a Service
The plugin is part of the Software as a Service project on Rubyforge. You can use this plugin in conjunction with something similar to the userstamp plugin to automagically limit the scope of your find, calculate, save, delete methods. It is used with other plugins as a drop in that will allow you to turn just about any rails application into a software as a service.
