this morning techcruch decided to roundup folks waxing philosophical about the scalability problems with twitter, and decentralizing twitter by creating some new web site or service. and that’s entirely the problem with their thinking.
why their approaches will not work
people are thinking of this as a web problem. twitter is not a web problem. the trouble begins when they try to use web for something it was never really meant to do.
twitter is a one-to-many architecture trying to be a many-to-many distribution model. they are the hub of a really massive wheel that will eventually shear clean from the axle upon which it spins. what we really need is a tried and proven way of doing many-to-many distribution of real-time data.
realtime data is not a web idea. the web is where data gets stored and presented; it piles up like old newspapers until we finally get around to bundling them up with string and hauling them out by the curb. the best the web can muster is near-real-time. how fast can you grab a feed, how often do you reload your browser.
realtime data is in a protocol, a network. if you want to store it, figure that out later. how you will present it, figure that out later. separate the distribution of data from its presentation and lifetime. deliver, then present. (then store).
distributed delivery? scaling? network: IRC
the model of the IRC network is as near as perfect for ephemeral data as one can find. and it’s been proven for decades. nearly infinitely scalable to users (add more servers). geodiversified locations for servers. short messages being sent to #channels on particular topics (sound familiar)?
the huge, glaring, and supremely important difference is this: the people responsible for the network absolutely and completely do not care about the lifetime, storage, or presentation of the data. they are only concerned with distribution.
people on individual channels consume the data and if they choose to store it, they do so (irc logs pile up like their homonymous wooden cousins). they read new information in real-time. they extract quotes and publish them to the web. they compile statistics and publish them to the web.
the model for scalable, many-to-many distribution of real-time information is IRC. now we need to think about the format. plain text is fine for console jockeys (myself included) but the web crowd is a bit more sophisticated.
universal, portable, adaptable information format? data: ATOM
we need a data format that is extensible beyond the author, title, and content. we need something that’s nationally agnostic, can be extended to support geo-location. what more needs to be said? it’s sophisticated enough for 140 character messages, that’s for sure. i know my twitter homepage looks like a graphical IRC client with the addition of user icons. mix in a bit of latitude and longitude and now we’re talking.
more about why their approaches will not work
i think the main reason why nobody thought about this yet is that they’re all too young, too new, and too enterprising. i’m probably guilty of that too. i’ve been sitting on this idea for several years now, trying to find a way to build a business around it.
everyone is thinking “what can we invent?” because invention nowadays equals entrepreneurship, venture capital, business success, cars, romance, glory, and all the spoils. (or some subset of the above).
nobody makes (legitimate) money from IRC. if you talk to an IRC operator (the administrators responsible for maintaining servers), you will find that their personal contributions, if calculated in billable hours, would be tremendous (enough to purchase some of the aforementioned entrepreneur’s bounty).
forget free lunch
we’re thinking: how can we get rich, or at the very least, eat lunch, by solving a problem. our “forefathers” in the internet world as it were, hadn’t concerned themselves with such things. they typically had corporate benefactors who paid the bills, but they were free to invent protocols that fixed a problem and met a need, without regard to how to patent, market, and monetize them.
bottom line: in order to “decentralize twitter” you need to separate the network for data from the presentation and storage of data.
Scott Hanselman touches on this idea referencing Russell Beattie’s Peep using Jabber as a distribution model. i hadn’t read a lot about Jabber, but it’s interesting. i still think these guys are too web-focused, but keep an eye on where they run with this. they’re starting to see the light.
Posted on May 5th, 2008 by mike
Filed under: business, future, inventions, lexicon, software, web | No Comments »