@madpilot makes

Make scheduling jobs easier – use web services!

Web developers deal with scheduled jobs a lot. Any online application that deals with paid subscriptions needs to remind users when they need to pay up. Other apps may require data mining services to be run on a nightly basis.

This is pretty easy to do – run a cronjob under *nix or a scheduled task under windows. However, where things get tricky is when you have spent an in ordinate amount of time coding business logic into you application. Duplicating this logic for an external script to be run by cron is pretty silly, not to mention bug-prone. However there is a quick and simple solutions: Web services!

By setting up a web service that does your maintainence, you can leverage the classes and business rules that you have already written. For example, if we take a Model-View-Controller framework (such as Rails or CakePHP) we may have logic that will do data manipulations before the data is saved to the database. There is no way I would want to try and emulate what these frameworks do in an external script. By using web services, you are calling the framework natively and you avoid all of these problems.

How it works

It is really simple:

  1. Setup a webservice that will perform the functions you need to run on a regular basis.
  2. Create a simple script that calls the web services.
  3. Add this script to you crontab or task scheduler

The beauty is that you don’t even need to bother setting up the web service using SOAP or XML-RPC – REST will do the job quite nicely, especially since the web service is designed to only be used by you. However, when using rails I like to use SOAP, because it is so easy to setup and use. Here is a simple example:

  1. class HousekeepingController < ApplicationController
  2. wsdl_service_name ‘Housekeeping’
  3. web_service_api HouseKeepingAPI
  4. def maintain()
  5. # Run you maintainence script here
  6. end
  7. end

This file is the webservices controller. You would add your logic here – this may be expiring users or automatically checking email.

  1. class HousekeepinApi < ActionWebService::API::Base
  2. api_method :maintain, :returns => [:string]
  3. end

This file defines the method to the web service caller. I usually set the return to type :string so I can output statistical messages or errors that occur during the job.

  1. #!/usr/bin/ruby -w
  2. require ‘soap/wsdlDriver’
  3. include SOAP
  4. begin
  5. wsdl = ‘path_to_wsdl_file’
  6. factory = WSDLDriverFactory.new(wsdl)
  7. housekeeping = factory.create_rpc_driver
  8. out = housekeeping.maintain
  9. put_s out
  10. rescue Exception
  11. $stderr.print “An error occured: #{$!}n
  12. $stderr.print “Detailes error description, if any:n
  13. raise
  14. end

This file is called by the cronjob. You will need to replace the string path_to_wsdl_file with the path to the real WSDL file. You get get the WSDL file from a Rails web service by querying /:controller/wsdl (so in this example http://www.yoursite.com/housekeeping/wsdl – pipe the returned xml to a file and save it.

Then it is just a matter of adding the housekeeping_run.rb command to you cronjob! Cron will even email you the results that get returned by your service – Nifty!

Nice an easy, eh?

Busy, busy, busy

Phew! What a week.

I’ve started at Bam, and I’m trying to get all of my outstanding contracts completed and trying to organise the WA Web Awards (I’m the sub-committee chairperson).

A few quick announcments:

  • WA Web Awards entries are open. It’s $25 for Port80 members, $10 for Student Members, and $35 for non-members.
  • 88 Miles is but days away from having a full web-service API released. It’s all working, I’m just finishing up the unit tests.
  • Port80 Melbourne is starting up next week. It is piggy-backing on a Ruby on Rails user group meeting. Go and see Miles’ blog for details.
  • There has been some interest in starting a Ruby on Rails special interest group on the back of Port80 over here, so I’m seriously looking in to that. Miles’ suggestion for a name was caboose. I like it


Creating web services in Ruby on Rails

I’ve been working on setting up web services in Ruby on Rails for 88 Miles and have discovered somethings that may be useful to other users. It may save someone some time, as the documentation for this particular area is a little sparse.

1. Putting a colon(:) in front of a return type indicates a primative – i.e an integer or string. If you want to return a complex type, leave off the colon

  1. api_method :find, :expects => [ { :id => :int }, { :user_name => :string }, { :password => :string } ], :returns => [ Shift ]

This will return the “Shift” Object. You can return Arrays by using two square brackets:

  1. api_method :find_all, :expects => [ { :project_id => :int }, { :user_name => :string }, { :password => :string } ], :returns => [[ Shift ]]

This will return an array of “Shift” objects.

2. Raising an exception with only a message invokes a SOAPException (and probably a XML-RPC error too – haven’t tried that yet), which is a much better way of handling errors than returning integers or strings (For those playing at home, check out the “fault” body of the SOAP message)

  1. raise Exception, “This is the string that gets returned”

3. If you are trying to access the webservices via WDSL, simply add /wsdl at the end of the URL


4. You can generate the scaffolds for a webservice using the generate script:

# script/generate web_service controller_name

Using fcgid instead of fastcgi with Ruby on Rails

I have fallen victim to the fastcgi zombie process issue. It would seem that there is a problem with FastCGI running under apache2, where many zombie processes can not be killed, which eventually fills up the process table, slowing down the server. My guess is that is using up all of the sockets, thus stopping any other processes from taking connections (THis is speculation as I haven’t really investigated fully)

Any way – after a quick google, I found that other people have been successfully using the fcgid module, which is a binary compatible replacement for mod_fastcgi. Generic instructions can be found on this blog – however, the following instructions are suitable for Gentoo – it also allows you to use the .htaccess file, giving a finer site control:

1. emerge mod_fcgid (I used the ACCEPT_KEYWORDS=”~x86″ to get the latest version)

2. edit /etc/conf.d/apache – add the flag -D FCGID

3. modify the virtual host in the apache.conf file to include:

Options ExecCGI


DefaultInitEnv RAILS_ENV production (For production mode obviously)

4. modify /etc/apache2/modules/20_mod_fcgid.conf to include:

IPCCommTimeout 40
IPCConnectTimeout 10

(I put it immediately after the LoadModule directive)

5. modify the AddHandler fastcgi-script .fcgi line in [Rails_app_root]/public/.htaccess to read:

AddHandler fcgid-script .fcgi

6. Restart Apache.

I’ll keep you posted on whether it fixes the problem :)

Tagging in Ruby on Rails – easy as you like!

One of my projects required tagging, and I was about to write my own engine, when I can across this article. This library from the article turned adding tags into a five minute job.

Kudos to Dema, the developer of it.

Ruby on Rails validator plugin for password fields

I’m working on a site that allows users to create password protected accounts (Pretty much like every other website around) and I needed a way to check that the password field and confirm password field were the same.

I saw this as an opportunity to play with plug ins :)

You can download the validates_equality_of plug here.

Instructions for use:

  1. Untar the file into the plugins directory of your rails application
  2. You now have access to the validates_equality_of method


The method is called in the following manner (assuming you have fields called password and confirm_password)

validates_equality_of :password, :with => :confirm_password

You can override the message in the usual way by supplying a :message attribute.

Hope someone find it useful.

Ruby, FastCGI and Virtualhosts

Now that that MadPilot Productions site is live, I wanted to run FastCGI. Even though the site is relatively low traffic, I was intrigued to see whether the speedup would be noticable. It is. Unfortunately, the literature I have seen hasn’t really made it clear how to set this up on a server that runs virtualhosts, so here is how it works.

To make a ruby on rails site run in production mode you need to add the following line to the apache.conf file:

FastCgiServer /path/to/site/public/dispatch.fcgi -initial-env RAILS_ENV=production -processes 15 -idle-timeout 60

However, putting this between VirtualHost tags fails. The solution is simple really – put it OUTSIDE the VirtualHost tag. You can have multiple entries in the config file. Since I split my virtual hosts into seperate files (ala Debian, even though I run a Gentoo server) I just drop it in bottom on the file below the closing VirtaulHost tag.

As an aside – Gentoo has a slight problem in that there may throw an error about User/Groups when starting. To fix, put the lines

User apache
Group apache

before the FastCgiIpcDir /tmp/fcgi_ipc directive in the [number]_mod_fastcgi.conf file that is located in /etc/apache/modules.d/ directory.

MadPilot is on Rails

The MadPilot Productions website now has a new look and a new scripting engine driving it – Ruby on Rails!

Leave your comments and tell me what you think :) 

Ruby on Rails highlighting in Dreamweaver

I found this site that outlines how to setup code highlighting and hinting in Dreamweaver MX.

In short – the instructions for code highlighting is here and the xml for code completion is here. (I would duplicate it here, but my version of wordpress doesn’t allow xml :( )

The only thing I did different was to create a file called Ruby.xml in the CodeColoring directory with and cut and paste the code colouring xml inbetween codeColouring tags (Have a look at one of the other XML files to get what I mean).

Unfortunately, I still haven’t gotten the .rhtml files highlighting correctly. I’ll keep you posted.

Sub-directories on rails

I haven’t been able to find this anywhere in literature (I’m sure it is somewhere, I just haven’t figured a good enough google search string to find it), but I wanted to be able to use sub directories to partition different areas in Ruby on Rails.

e.g /admin /admin/articles etc

Ruby on rails uses the following mappings between the URL and the controller:


which maps to a AcitonController class called controller which has a method called action

I wanted to have multiple a controller directories:


You do this by creating a sub directory under the controller directory (in this case called controller) and creating file called subcontroller_controller.rb – The class declaration of this file should be Controller::SubControllerController < ApplicationController.


So, if you wanted to create a controller called articles in the sub directory admin you would create:

  1. The subdirectory “admin” under the controllers directory
  2. The file articles_controller.rb in the controllers/admin directory
  3. Declare the class as: Admin::ArticlesController < ApplicationController

You also need to create the corresponding .rhtml files in a sub directory in the views folder…