Delayed Job
In a project I'm working on, when a report was published it would generate a PDF (using the very convenient Prawn) and then send that PDF to half a dozen subscribers. This took a while (around 5 seconds) and made the response to the user a little slow. Actually, it was even worse than that. If I sent the PDF to 5 subscribers then I generated the PDF 5 times (and again each time it was viewed online).
My first step was to write the PDF out to the local filesystem so that I only ever had to generate it once. For further peace of mind, I also copied the PDF to S3.
I then chose Delayed Job (the CollectiveIdea fork) to start processing everything in the background. Now, when a report is published then the a job is created and the user gets a snappy response.
Following advice from Ryan Bates (Railscast 171) I started to think about what to do if the PDF publiction job failed half way through. The risk here is that if you email 5 people and the fifth one fails then the whole job fails and it is rerun a bit later. This would mean repeated emails to the first 4. My first plan was a new table that recorded which PDF had been sent to whom - then it occurred to me that I could just 'spawn' new jobs for each separate email. That way, only the failed email gets reattempted. So now I have a primary job that gets run which generates the PDF and then creates a handful of other jobs that actually send the emails out. Nice.
P.S. Exactly one year since my last post and its title is "Delayed Job". How appropriate.
posted on November 30 2009