Handling APIs with Ruby XML Parsing
Have you ever wanted needed to write a Ruby wrapper for an XML based API?
If you have a burning desire to seamlessly exchange data over the web or you just want to use the latest Interweb 2.0 service – you’re most likely contemplating writing your API client in Ruby… yes that’s why you’re here isn’t it?
Why Use Ruby to Parse XML?
It’s a fact – Ruby kicks ass at parsing XML. You can find tons of examples of XML API clients written in Ruby.
ActiveResource parses XML and handles RESTful HTTP
The new ActiveResource Rails gem found in Rails2 makes pretty light work of handling XML APIs via REST. Unfortunately not everyone is rushing to support REST just yet. If you support a big Rails 1.2 app you can’t just run out an add ActiveResource to your project (which is my case). This post is not about REST or ActiveResource so if you’re looking for that, click the link you just clicked skipped over.
Enough with the useful useless Ruby XML facts…
Show me how to write a Ruby XML API wrapper
I’ve been working with an API recently for a third party registration system on a project we’re rolling out soon. The third party provides their API using domain scoped query URLs and HTTP GET params and returns XML documents.
Huh?
http://example.com/aaflinn/lookup?username=billlumberg&password=swingline
XML Messages
When you get a matching user/pass combination you get something like this.
<auth>
<user>
<username><![CDATA[billlumberg]]></username>
<fullname><![CDATA[Bill Lumberg]]></fullname>
<zipcode><![CDATA92131]></zipcode>
<email><![CDATA[bill.lumberg@initech.com]]></email>
</user>
</auth>
When you put the wrong password you get an error like so.
<auth>
<error><![CDATA[Invalid username/password combination]]></error>
</auth>
If you don’t enter a username at all you’ll get an error thusly.
<auth>
<error><![CDATA[No username given.]]></error>
</auth>
API Wrapper Concepts
Here are some concepts that I felt were important when I started writing my wrapper.
- Simple – it should be easy to code (I hate writing stupid code)
- DRY – if it’s worth writing the wrapper make sure it’s reusable
- Self Documenting – it should be as self documenting as possible (rdoc)
- “Exceptional Code” – it should raise errors on exceptions and handle errors from the libraries it makes use of
- Don’t Spam – Don’t abuse APIs (grrr!)
Requirements
The wrapper should use a GET query string to perform a query to the service provider passing an md5 hashed password and username combination. If the user exists parse the XML result document and return an instantiated user object based on the XML. If no user exists raise some type of rescuable error (RecordNotFound).
Additionally the wrapper should be able to create a new user. This particular service provider uses an HTTP GET query string but in some cases you might find a plain old POST like you’d see in a form or you’ll need to build XML and pass that back to the service provider (which I am not doing here). The wrapper should be able to be able to interpret error messages and determine the status of our create request in a graceful way.
Ruby API Wrapper by Example
The names have been changed to protect the innocent. I’ve edited the wrapper a bit to reduce some complexity and renamed it to hide the actual API provider. Read on in the comments of the code, I’ve done everything on that bullet list and you can read the code as you read the comments setting up just about every line of code written.
#
# Usage
#
# === setup ===
# require ‘ApiExample’
# ApiExample::account = ‘test’
# ApiExample::logger = Logger.new(‘example.log’)
#
# ===Find A User===
# user = ApiExample::User.find(‘bill.lumberg’, :password => ‘swingline’)
#
# ===Create A User===
# user = ApiExample::User.create(:username => ‘bill.lumberg’, :password => ‘swingline’, :email => ‘bill.lumberg@initech.com’)
#
require ‘base64’
require ‘digest/md5’
require ‘net/http’
require ‘rexml/document’
require ‘cgi’
require ‘logger’
module ApiExample
# Example Database Host
HOST = ‘www.example.com’
# These params are required to register a new user
REGISTRATION_PARAMS = [ :username, :password, :email ]
# ApiExample account (brand account not user account)
@@account = nil
@@success_url = nil
@@fail_url = nil
@@logger = Logger.new(STDERR)
mattr_accessor :account, :logger, :success_url, :fail_url
# Exception handling
class ApiExampleError < StandardError; end
class UnexpectedError < ApiExampleError; end
class RegistrationError < ApiExampleError; end
class RecordNotFound < ApiExampleError; end
def md5password(password)
Base64.encode64(Digest::MD5.digest(password)).strip
end
# Using Struct here allows us to make our object act similar to an
# ActiveRecord object. In this example it’s not so obvious of a pain
# in the ass it is but the real class has about 20 or so attributes
# Using means I don’t have to create attribute read and writes
# attribution – I got this idea from the Ben Vinegar’s
# <a href="http://rubyforge.org/projects/freshbooks/">freshbooks gem</a>
User = Struct.new(:username, :email, :password, :fullname, :zipcode)
# Extend the Struct attributes by adding the class and instance methods we want
class User
attr_accessor :attributes, :errors, :new_record
# class method for create a new user
# Usage:
# user = ApiExample::User.create(:username => ‘bill.lumberg’, :password => ‘swingline’, :email => ‘bill.lumberg@initech.com’)
#
def self.create(attributes = {})
object = new(attributes)
object.create
object
end
# class method for finding an existing user
# Usage:
# ApiExample::User.find(‘bill.lumberg’, ‘swingline’) # plain text will be sent md5 hashed rather than clear text
#
def self.find(username, password)
query_params = { :username => username, :password => ApiExample::md5password(password_option[:password]) }
query = query_params.collect{ |k, v| [k, v].map{ |kv| CGI::escape(kv.to_s) }.join(’=’) }.join(’&’)
# @@account doesn’t need to be encoded because we are setting internally, the other stuff is vulnerable
uri = URI::HTTP.build(:host => ApiExample::HOST, :path => "/#{ApiExample::account}/lookup", :query => query)
# I’ve got a bunch of these loggers throughout which are used to debug, remove as you see fit.
ApiExample::logger.debug("URI: #{uri}")
# Make the actual HTTP Request
result = Net::HTTP.get_response(uri)
# Parse the XML Result of the HTTP Request
response = REXML::Document.new(result.body)
ApiExample::logger.debug("RESPONSE: #{response}")
# Check to see if there is a user node, if there’s not raise an error similar to ActiveRecord
unless response.elements[’//user’].nil?
attributes = {}
# Members is the Struct method for the attributes we setup…they correspond to what
# we expect the XML to return, so lets only handle what we know
members.each do |field_name|
node = response.elements[’//user’].elements[field_name.to_s]
next if node.nil?
attributes[field_name.to_sym] = node.text # you can do casting here if you need, I did
end
object = allocate
object.attributes = attributes
object
else
# Raise an error, setting the message to what the API sets as the error in XML
# if there is no ‘error’ node in the XML set an ‘unknown error’ message
raise RecordNotFound, (response.elements[’//error’].text rescue "Couldn’t find ApiExample::User with Username: #{username} because of an unknown error!")
end
end
# instance method to setup new object ala ActiveRecord
# Usage:
# user = ApiExample::User.new(:username => ‘bill.lumberg’)
#
def initialize(attributes = {})
@new_record = true
self.attributes = attributes
end
# instance method ala ActiveRecord
#
def errors
@errors = [] if @errors.blank?
@errors
end
# instance method to get an attributes hash ala ActiveRecord
#
def attributes
butes = {}
members.each{ |member| butes[member.to_sym] = self.send(member) } # you can cast here, I did
butes
end
# instance method to set attributes via a hash ala ActiveRecord
#
def attributes=(new_attributes = {})
members.each do |member|
unless new_attributes.has_value?(member.to_sym)
# #{member}= because you might possibly write an attribute writer to handle the input to =
self.send(member+’=’, new_attributes[member.to_sym]) # you can cast here, I did
ApiExample::logger.debug("#{member}: #{self.send(member).inspect}") # for snooping
end
end
end
# instance method to create the new instantited object ala ActiveRecord
# note the difference in create and self.create are the same as in ActiveRecord
# Usage:
# user = ApiExample::User.new(:username => ‘bill.lumberg’)
# user.password = ‘swingline’
# user.email = ‘bill.lumberg@initech.com’
# user.create
#
def create
# we pass our success and fail URLs even though we aren’t using them for their intended purpose
query_params = { :success => ApiExample::success_url, :fail => ApiExample::fail_url }.merge(attributes)
ApiExample::logger.debug(query_params.inspect) # for snooping
# check to make sure all required params are included
if ApiExample::REGISTRATION_PARAMS.all?{ |param| query_params.include?(param) }
# URL Encode everything
query = query_params.collect{ |k, v| [k, v].map{ |kv| CGI::escape(kv.to_s) }.join(’=’) unless v.blank? }.compact.join(’&’)
# @@account doesn’t need to be encoded because we are setting internally, the other stuff is vulnerable
uri = URI::HTTP.build(:host => ApiExample::HOST, :path => "/#{ApiExample::account}/register", :query => query)
ApiExample::logger.debug("URI: #{uri}") # for snooping
# Make the actual HTTP Request
response = Net::HTTP.get_response(uri)
ApiExample::logger.debug("RESPONSE: #{response}") # for snooping
# Check for failure and errors
if response[‘location’].include?(ApiExample::fail_url)
# Raise an error for each message
CGI::parse(URI::parse(response[‘location’]).query)[‘error’].each do |error_message|
raise RegistrationError, error_message
end
elsif response[‘location’].include?(ApiExample::success_url)
# Change the status to reflect the fact that we’ve saved the object…this could get much more
# in-depth in terms of ensuring our data was correctly saved… you wouldn’t expect to do
# that kind of thing in an RDBMS so why here?
self.new_record = false
else
# it wasn’t the fail url, it wasn’t the success url so what the hell was it?
raise UnexpectedError, "Unknown response URL!"
end
else
# For each missing required param add an error message
ApiExample::REGISTRATION_PARAMS.each do |missing_param|
self.errors << "#{missing_param} can’t be blank." unless query_params.include?(missing_param)
end
end
end
end
end
read rails warnings!
I was dealing with an issue today were I was certain my code was right. I was getting the dreaded ActionController::DoubleRenderError error. I was sure I could just redirect_to ... and return but I failed to read the rails warning for the last day or so. The following comes literally right from the development log I’ve been staring at.
ActionController::DoubleRenderError (Render and/or redirect were called multiple times in this action. Please note that you m
ay only call render OR redirect, and only once per action. Also note that neither redirect nor render terminate execution of
the action, so if you want to exit an action after redirecting, you need to do something like "redirect_to(...) and return".
Finally, note that to cause a before filter to halt execution of the rest of the filter chain, the filter must return false,
explicitly, so "render(...) and return false".)I had a professor in college who used to tell me to read every sentence in a newspaper article because the whole story comes out in the last sentence. There it was plain as day, to halt the filter chain I need to return false, dang!
