That's more like it!

Yesterday, as a result of the performance issues, we fast-tracked part I of a new higher performance architecture (HPA). HPA part I was deployed yesterday evening, and it worked very well. All the feeds were polled and the emails sent out before 3:00 am eastern this morning. Yay!

For the technically inclined, FeedBlitz's single-threaded emailer was replaced by a multi-threaded version of itself (note that FeedBlitz's MTA / SMTP server was not changed in this process; what was updated was the application layer code that sends mail to the MTA). Bottom line: faster.

Now, there's more room for improvement in the mailer - we made these changes in a day, so could only make the lowest risk performance changes - but despite this and the recent growth in FeedBlitz, the mailer managed its best performance in nearly three weeks. And three weeks ago, FeedBlitz was much smaller than it is today. So, for now, we're going to leave the mailer alone since the changes are working pretty well - no sense in perturbing it further - and going to look at part II of the HPA.

Part II? Improving the performance of the core feed poller. It too is single threaded at the moment. A multi-thread poller will yield major throughput improvements - better, I suspect, than the equivalent changes in the mailer - given that polling performance is heavily dependent on the performance of underlying network connections and remote systems. So there's plenty of idle time to take advantage of. We'll see.

Meanwhile, thanks for bearing with us - onwards and upwards!