Long running Threads in Rails and metaprogramming fun
Posted 11 June 2008 @ 3am | Tagged background, code, rails, ruby, thoughts, threads
Disclaimer: This post contains evil (but highly fun!) code. Proceed at your own peril…
I was recently designing an application that needed to execute some long running requests against an external host. If you’ve ever tried doing something like this in Rails, you’ll find your mongrels will block up waiting for the request to complete, bringing the experience for all other users to a halt.
I wanted to dispatch my long running request, return to the user, and then poll for results using AJAX. There are many ways to do background tasks in Rails, most of which require running an out of process background server with which you will communicate over some sort of queue or memcached. There’s BackgroundRb, Bj, workling, and so on, but this seemed overkill for my problem.
After reading a post on using Ruby Threads, I decided to be brave and try this approach. I implemented a simple action which would spawn a thread and proceed to return the result whether it was ready or not. This action is polled via AJAX and on the next poll the result will be correct. The pseudocode looks something like
#spawn a thread
precache_the_results
# This action throws DataNotAvailableException
# if file is missing/unreadable
results = read_cached_results
rescue DataNotAvailableException
# This tells me that when I load the page
# I should invoke an ajax a couple seconds
# later to check for results again
flash[:update_right_away] = true
ensure
respond_to do |wants|
# render an RJS update with the results
end
end
Thread.new { expensive_action_outputs_to("file.txt")
}
end
Because I didn’t join the Thread to the request thread, it lives on after the request completes, which is just what I needed. Since the code inside my Thread is a call to an external provider and doesn’t write to the database, I am not concerned with ActiveRecord threading issues.
The only problem with this approach is that in development mode, Rails likes to reload your classes on every request. But if your thread runs past the request lifetime, the class that’s running it may be unloaded while it’s running, wreaking all sorts of havoc. But Ruby allows us the power to be truly evil:. What if I just prevent Threads from doing what they want to in development mode? Turns out I can!
if == 'development'
block.call
end
end
end
This code is defined in the class where I’m doing the magic. Do NOT just slap this into your environment.rb as you’ll horribly break the Rails startup logic. There’s probably a slightly smarter and safer way to do this by using a Factory pattern to create the threads and explicitly specifying the implementation you want. But this is my party and I’ll monkeypatch if I want to.
So..comments, suggestions, complaints? Is this going to die horribly in production? I guess we’ll have to see!
17 Comments