Well, as many of you have pointed out, hashtags.org has been down since July 10th - nearly a month. This is because we rely on Twitter’s XMPP service to recieve tweets. They took the service down on the 10th and since then we’ve been waiting, patiently.

I have emailed the folks at twitter several times, but I haven’t been able to get a solution from them. In the mean time we are going to try a few other ways to get the data from them - even if it means banging on the public RSS feed every 2 seconds.

I assure you that we will be back, it’s just a matter of when Twitter gets it’s things in order.

There has been some confusion about the hashtags registration process. Why do we require you to follow the bot account? Why can you just parse the public feed like other sites do? I am here to answer these queries.

Why do we require you to follow the bot account?

It boils down to the fact that we want hashtags to be an opt-in service. If you don’t want the bot to look at your messages, it didn’t. We also figure this would develop a more active user base. I mean how many people are going to use hashtags unless they know about the site? Another benefit, that we didn’t realize at the time, was that this process allowed us to handle gradual growth rather than just letting open the flood gates of twitter.  So that is why we have you follow the bot.

Why can’t you just parse the public feed like other sites do?

Simple, it just doesn’t really work. Have you ever looked at the public feed page and tried refreshing? Even if you click twice in a row, the results are almost always completely different. What that means is you inevitably miss some messages. In order to do this reliably we’d have to have multiple bots hammering the public xml feed and parsing those. Parsing xml, in general, is really slow too. We made the decision to not hammer twitter’s servers and instead took the passive approach of parsing the bot’s xmpp feed.

What’s next?

Well the beta version we’re working on runs off of the public jabber feed. So we’ll be processing every single message that comes in. The only problem is that it requires a lot more computing power. We’re processing a whole lot more messages. The beta site, feature wise, is pretty complete. Right now Ben is working on optimizing our database calls and caching the hell out of everything. Our biggest problem, if you want to call it that, is that we’re getting to much client work. Since there’s no current value proposition in hashtags itself, it’s hard to dedicate too much time to it. Any suggestions or question you have post in comments and we’ll give them a look.