DrunkenBlog

Of Google Talk, Jabber/XMPP, and putting a candle in the window

The Google Blog has a quick rundown on their new Open Federation initiative for Google Talk, which -- as a user and a geek -- is starting to get me seriously excited as well as amused.

Admittedly some of the amused part comes from nugget-san having spent a bunch of time wiggling around in the service and getting it working and learning that Google Talk has apparently been queuing up all the authorization requests that users have attempted since they went live last August. Since he's posted about these types of things before, once the servers went hot he got over three dozen authorization requests from poor souls who now had no idea who he was, leading to his day being littered with random, "Hey, who are you?" messages from GTalk users.

It's also a little weird to hear an engineer nonchalantly mention "flipping the switch" on a network of that size, but not bad-weird, and I've been meaning to talk about Jabber a bit...

Years ago I used to chat via IRC and MUDs (EOTL -- w00t) and such with friends, and then chat rooms on the net were born and it became a lot easier to talk to people from wherever they were -- especially not those especially technically inclined. Then I got sucked into instant messaging, and over time started to find I was doing more and more with it, especially as spam started to make my actual inbox more of a chore. However, I kept getting annoyed as I tried to integrate it further into my workflow...

1. As a user, I had to be on x service to talk to x person. If I was on x service and wanted to talk to y person, I had to get them onto service x or fire up service y myself -- assuming it was available in a decent fashion on my platform of choice. This meant I'd end up with an AIM, ICQ, Yahoo, and MSN client all running. They all sucked up their own resources, and I kept having to switch between them depending on who was talking at me, let alone screen and Dock real estate issues.

2. It peeved me that different services have these arbitrary limits set on how many people I can have in my list because it quickly became one of the limiting factors in actually making it a larger part of my workflow. I originally thought of maxed-out buddy lists as girls flaunting their boobies on webcams in forums, but then I started smacking the walls. I don't talk to them all every day or even every week or every month, but when I need to talk to them I need to know who's available and getting the message that my list is full means I have to decide who now won't be instantly available.

   To be fair, each person in a buddy list has to be stored on the server and then manipulated for use, which was happening on the service's dime. x kilobytes or something might not seem extreme until you start multiplying it by millions of users. As a user, the simple solution seemed to be:

   • If my needs are so above the moving average, let me pay something to pay make up for it.

   • Allow me to foot the dime on my own. Many people on the net started out with free email accounts, and then decided they needed more than what was being offered in terms of storage or max length. Let me setup my own with a few hundred megs, so I can go away for a week and people can drop as many offline messages as they want and I can have as many people on it as I want.

   Yeah, wasn't going to happen.

3. It so peeved me that nothing worked quite the same from service to service, even if it can seem a little strange to be griping about competitive advantages. Yahoo had offline messaging, as did ICQ, but AIM didn't. Each had different headroom in terms of contact lists, and outside of a few all had different lexicon for emoticons. It's a small amount of headspace that has to be devoted to keeping track of it, but you still did, and it was still annoying.

4. I'm so sick of saying, "I'm w on AIM and x on Yahoo and y on MSN and z on Google Talk and f on ICQ," and it's equally annoying getting all of those things from someone else.

Fairly basic stuff, and like I said, it's from a user point of view where I want messages being sent to an identity and not what particular client I happen to be logged into that's tickling the server right then and there. I'm not 5 people, I'm one person (Ok, perhaps two but that's too weird to think about this early) and as a user I want that consolidated, just as I don't want to deal with multiple address books on various computers. This isn't about what the services want, although you have to take it into account to know what has to occur to get there -- it's about what the user wants.

[1] was solved by the various ilk of multi-protocol chat clients, but that still left the rest, and when I'd mention some of this around, I got pointed towards Jabber, which is based upon XML. It didn't take long to realize that Jabber kinda blew (I'd dropped down the rabbit hole on XMPP versus SIMPLE and such at the time) because if you dropped onto the mailing lists there'd be much furor over everything except the fact that no one was using it outside of specialized areas.

Things I kept encountering while pinging around, which are worth taking with a grain of salt because they're from awhile ago and may or may not be applicable today and who knows, may have not been the whole picture back in the day:

• Someone joked to me that Jabber was a massive general purpose notification system for routing events via XML that happened to have a few bylines in the protocol that made it useful for instant messaging. It gets compared to Growl often for a reason. Whether fairly or unfairly, this rubbed a lot of people the wrong way and turned a lot of devs off of it. It can be a big thing to wrap your head around for a someone just wanting to whip up cool IM stuff.*

• There were a ton of complaints at the time that the spec was... under-specc'd and vague in areas it shouldn't have been vague in for things people really wanted to do, especially those who wanted to "bring it" when it came to competing to the existing chat services. When that happens, and people are looking at getting on board, you start to invite competition where it doesn't necessarily need to occur and lots of treading water that doesn't need to occur -- see RSS versus Atom.*

• *There's a certain pink noise that comes when you're just pinging around regarding technologies people feel strongly one way or another about, whether they've really used them or not, so take the above with a grain of salt as I was about to give up. For ever person I talked to going on about how the spec wasn't xyz, another was going on about how the whole thing was way too bloated with too much overhead and why the hell can't they just send a damn UDP packet.

• A lot of the implementations at the time kinda sucked, and maybe still have real issues -- I haven't poked in awhile. Realize that Jabber/XMPP is a protocol solution, you're free to code up your own clients and your own servers, all with varying degrees of features and bugs. At the time, they were generally short on features and long on bugs.

  I had seriously un-fun experiences with the server, made better by when I was playing with ejabberd, but even there I'd find an error message in the logs and couldn't for the life of me find it within any of the docs, but at least I got it going without overt weirdness -- although seeing the passwords stored directly on the server rather than say, a hash with the cleartext transmitted over encryption from the client to authenticate against it, wigged me the hell out because there had to be a great reason for why but for the life of me it seemed insane. That left the clients, and.. it wasn't pretty.

• 95% of the current people who have heard the term Jabber in relation to instant messaging did so when they heard about GTalk or Apple adding support for it within iChat. Its public awareness was a rounding error -- I heard weird asia-specific protocols coming up more often from general users than Jabber.

Not a good situation: Few people I knew used Jabber, and if I made them aware of it and urged them to pick up a Jabber ID all the clients they'd need to use would kinda suck in comparison to the other services, let alone the features, and they would have the same problem convincing their friends. I was having to do way too much explaining on how they could actually register and get a jabber account.

It may seem like I'm bashing the state of where things were, but it wasn't so much the current state of things at the time as how little was being seen to fix them from the outside. Jabber was going to conquer the world, but it felt as though the people on the lists just sort of assumed it'd happen magically one day once people realized the greatness of it.

I encountered some lone screamers in the process of losing their voice, but the fact that no one was actually using it, let alone why and what might change that, didn't seem to be of all that much consequence.

Jabber just gave me that vibe of those Really Cool Things™ that are right around the corner, but end up being DOA because they never quite get the traction and eventually something comes along that rhymes with what the original Really Cool Thing™ and cuts off its growth path. It doesn't matter if one acorn hits the ground before the other, if the other happens to get all the sunlight and water it'll choke the first, and the reality of using it as a user, for most users, wasn't anywhere near where the hype was saying it was going to be and progress was hard to see from the outside.

Bygones, as sometimes if you build it, they do come.

Probably the biggest thing people have to wrap their heads around when it comes to Jabber (or earlier services which I'm sure inspired Jabber) versus the traditional instant messaging that we're used to is centralized versus distributed. I covered some of the basics of this (as well as things like NAT traversal) back in the "Will iChat AV stay peer-to-peer, or moving server-side" section back in iChat AV Bittage.

As a cliffnotes version, something like AIM or Yahoo or MSN is decidedly centralized. You pick a service name which is lodged in their servers, and you fire up a chat client which then communicates through them. If you read the linked entry you'll see there may be some peer-to-peer stuff going on when you hit file transfer or A/V functionality on the servers, but I don't want to confuse the issue too much -- it's centralized.

With Jabber, you start out with a JID, with the format of:

username@host/resource

This could be something akin to drunkenbatman@myjabber.net, but it could also be something akin to drunkenbatman@mac.com (both are examples only). Where it becomes distributed is that while there are... mainstay... servers you can default to login to, you could just go ahead and setup your own server with your own host outside of myjabber.net or mac.com, and if you chose, it could communicate as a node on the larger system, aka IM cloud.

Many people were already doing this with Mac OS 10.4 Server, because it came with a Jabber server and iChat in Tiger allowed one to use Jabber -- set it up, have your mates and coworkers all point it towards the server, and you had your own private IM server, but if you wanted it could join the cloud and send messages to anyone with a JID that's not cut themselves off from the cloud -- if they aren't online, it just gets stored until they are, or forwarded to where they want it to go. You may have noticed the /resource part of the JID above, which actually allows a user to setup something akin to:

drunkenbatman@myjabber.net/home
drunkenbatman@myjabber.net/bubblebath
drunkenbatman@myjabber.net/mobile

Most of the time the resource stuff would be transparent to the other user unless they specified, but with priorities it gets Uber™ and allows you to be logged in on the same node on the same JID. It can start to get complex, but for the most part with well-designed interfaces it's abstracted and you'll just get the functionality you want.

You may have noticed it's remarkably similar to email, but email lacks presence -- for instant messaging you don't want to just fire off messages and know they'll get to the individual at some point, you also want to know to know whether they're online right now and will get them: presence.

With the centralized services, your buddy list is stored on their servers -- for Jabber it's the same thing, except each server has its own user list, and each user has a roster. Your roster is what'd you'd consider your buddy list, and while it's accessible from other servers if you need to login to them, it's easily movable from one to the other, although you have to reauthorize your contacts -- otherwise anyone could add anyone they liked by modifying and moving their roster.

If you add someone's name to your list, a presence subscription handshake occurs. Think of this like Yahoo Messenger or MSN, where you get a notification that someone would like to add you and you can authorize or decline. If you are both on the same server, this all happens internally, however I could be on one and you could be on a more public server, and they'd connect via the cloud and do what they need to do.

Once the magic has been worked and they're in your roster, the servers automatically take care of everything for you -- when you login with your client, your server communicates to the servers it needs to through the cloud to know who is online and who isn't, and vice-versa if someone on another server has you in their roster. Again, if it's internal, it just queries itself. There are obviously pro's and con's to a centralized versus distributed approach, as there is to everything, but once it's all setup the convenience to the user is dramatic even with say, chatter.

To wrap back around to the original link, as this should now make sense, the big deal is that Google has now connected Google Talk to the cloud. But wait you say, people were already using iChat to talk to GTalk?

Well they were, but that was different, and it was long enough ago that most people forget they had to go sign up for an ID or use their current account, because not a lot of Mac users -- or anyone for that matter -- have gravitated towards Google Talk yet because it doesn't really do a whole lot the other services don't. You could use iChat or any of the multi-protocol clients that supported Jabber to connect to Google Talk, however you did so with an actual GTalk login and you had to specifically enter Google's Jabber server addresses.

Now that they've flipped the switch and connected to the cloud, you can connect to them -- and by extension their users -- from your own Jabber server. It doesn't mean it'll be perfect as not all servers support the same things, let alone client-side features, but it's a big step.

There are other aspects to Jabber/XMPP, like gateways and transports which are just as cool as what I glossed over, especially once they're abstracted from the user. If you're on something like a Jabber server and want to talk to someone on an MSN client, your problems are:

• MSN has its own chat servers, which aren't directly talking to any Jabber servers or recognizing your Jabber ID.

• Jabber talks XML, while other services use their own formats.

Transports, or gateways, take care of this by basically being conduits you can install within your server -- or tap into a publicly available one -- which tunnels the traffic where it needs to go and converts what you're sending into something the other service can understand and work with. As a good example, because iChat now talks Jabber, it's not difficult to set it up so that -- if you have an MSN account -- tunnel through a transport and use iChat to talk on MSN. It's not "native", but it works, and transports can be used for all sorts of things.

If you're behind a firewall, it can go out over HTTP and -- wait for it -- even use AJAX between the clients for chatting, so it keeps the connection open instead of polling back and forth. It's totally web 2.0, and we're very much just still scraping the protocol level. Because it's just a protocol, a specification to be implemented, you can get highly creative on the server side for how you handle things, just like different web browsers use entirely different engines to render HTML/DOM. It's not extraordinarily difficult to setup a fault-tolerant cluster of machines for your Jabber domain, so that if one goes down everything is fine.

It's just so cool, but really, it's still not worth much to a Normal.

You know an IM service sucks when websites exist to connect people who want to actually use it. iChat AV is a good example, as I remember well when the iSight first came out and all these Mac users were on the mailing lists asking if random strangers with an iSight would connect to them to actually use it.

Then -- and I swear to god -- iChat AV directory websites popped up to fill the need for Mac users who had bought one yet had no one to chat with. Sure, it kinda-sorta worked with AOL's official AIM client for awhile, but it never really worked, and it's dead for the Triton release -- and yes, there are directory websites out there for Jabber users who want to talk.

Sucks may peeve some people, but it doesn't matter if your cell phone is twice as thin as a RAZR if it only connects to other RAZR's, but you need to talk to the rest of the world. Very neat tech, but a seriously-sucky solution, as this is the old network effect in action -- the more people you can connect to with a device the more valuable it becomes. If there's a phone out there that's twice as thick as the RAZR, but connects to everything, it may be boring tech but it's a comparatively great solution.

There are actually millions of Jabber users out there, probably tens of millions now, however you rarely really encounter them in general life. If you're a corporation or an institution, and decide you want instant messaging internally, you often either have a choice of one of the Triumvirate's enterprise solutions or something akin to Jabber.

It's doing well there -- quite a bit of cash can be turned over for Jabber solutions and their support. However the key is internal solutions, as in most cases half the point is that the worker bees aren't meant to be talking outside of the network -- average people don't go home and use Jabber. Google is the current hostess with the mostess, but Google Talk hasn't exactly bumped up the Jabber protocol's share in the general populace up from a rounding error in any way.

You might be surprised at the amount of Google employees who are, or rather aren't, using it -- even for internal communication. Google employees aren't pod people that were hatched this year, and brought their own lives and contacts with them -- as a casual example, consider the freaky percentage of ex-AOL employees at Google, that they still talk to their former coworkers, that they had been using their former company's service for years, and on the whole AIM isn't generally short on features that Google Talk happens to be offering. The network effect is strong.

I wouldn't say that I've been waxing poetic in this post, but I think it'd be fair to say I was rubbing the promise shiny more than what is the current reality. This magically delicious communication epoxy doesn't necessarily have all the pieces where you'd want them to be -- it's a protocol after all.

Adoption is easily the longest-term problem for Jabber in general, but there is a hell of a lot of work to be done on all fronts. I've heard getting some of the more advanced features in Mac OS 10.4's Jabber server can be problematic to say the least, and even in things like iChat I noticed if a transport disconnects it doesn't automatically notice it -- you have to specifically disconnect and reconnect. Then there are a bunch of different flavors of Jabber servers running around with different levels of feature support, and it isn't as though Google Talk doesn't have its share of bugs.

There are a ton of things that could use work, but my goal isn't to hammer on it but rather to concrete the idea that even if everyone in the world was wanting to switch over tomorrow, they'd probably be in for a rude awakening. As would we, because we'd be the ones hearing them bitch about missing features and buggy implementations -- but these are all things that can be worked out with the sweat and sleepless nights of coders, as adoption really will be the long-term problem.

The network effect is strong, let alone plain old user inertia, which is where something has to be pretty compelling to get you to move away from what you know works and is fast becoming a commodity (except for voice over IP, but that's a whole different post). I know people that have never used anything but MSN Messenger -- it came with their computer and just about everyone they know has an MSN account. That's going to be hard for anyone to overcome, let alone millions upon millions use Yahoo or AIM exclusively because it's just what they know.

The current weird situation can't last forever, just as it didn't last forever back in the days of internet services. At some point it's just going to make fiscal sense for the more centralized IM parties to cede the lower layer and try to focus on other revenue avenues atop them. However, with the current rate of traction, it's going to be one long, slow slog towards adoption, excepting:

• Someone buys entry into the market with one of the currently larger services, and rather than settling in as the overlord, asks the others to interoperate.

• Some other catalyst is able to raise Jabber/XMPP's userbase to a critical mass, slowly putting pressure on the more centralized services to change their game and interoperate.

Someone like Apple Computer might come to mind as a catalyst, as they have a small yet sizable base they're good at leading by the horn, but it isn't as though their base isn't also affected by the network effect. Linux comes to mind, but if we're going to wait for it to get a critical mass of desktop share for it to affect the outcome we're in for way longer than I want to wait -- we've already been waiting for far, far too long.

Ah, well. Sometimes when you build it, they really do come.

So there's that.

 Posted by drunkenbatman
    January 19, 2006, at 09:26 AM