Comic Update: The Zombie Link Apocalypse
Posted by Kyle Weems on April 20, 2009There’s been a lot of discussion (read that: unholy firestorm) lately about URL shorteners, link rot, and solutions to prevent a zombie link apocalypse where a large portion of the Internet’s links suddenly become useless undead anchors within a single fortnight. Today’s comic presents a pair of survivors trying to live in such a world gone wrong, starring Jeremy Keith.
For anyone who’s new to the fray, this is basically what’s going on: As most of you know, popular social services like Twitter exist. Twitter uses TinyURL as a shortener for URLs tweeted to help keep character counts down. Third party shorteners, while convenient, pose the threat of someday going under. This will result in millions of hyperlinks on the Internet that would no longer work, causing link rot. As we all should know, link rot is bad.
Those taking the long picture, such as Jeremy Keith (who’s view of the long picture is much more than most considering his thoughts of concepts such as the Long Now, which is even farther ahead than I can even bother thinking about), are already suggesting solutions today to prevent link rot from killing the Internet’s usefulness in that grim day when these shorteners disappear. One such solution is rev=”canonical”, which Jeremy has discussed at least twice now. It seems like a reasonable option, and far preferable to a rabble of links that rot with no backup plan attached.
Now, by and large the resulting debate hasn’t been whether short url-related link rot is a problem. The real argument is whether the proposed solution’s use of rev is somehow confusing, or whether good alternatives already exist, etc. However, hidden throughout these discussions are little comments like this one from Shelley Powers:
I hate to break it to the folks so worried, but it will probably be a cold day in hell before anyone digs into Twitter archives. Most of us can’t keep up with the stream of tweets we get today, much less worry about yesterday’s or last week’s.
I don’t think that Shelley is actually trying to say “there is no problem,” but I can’t help but feel that the comment is fairly short-sighted, and reflective of a certain prevailing mindset that the issue at hand isn’t largely a serious one because of the medium that’s most exposed: Twitter and other social networks. After all, we each have our own blogs, right? All the smart people on the web are home-brewing their own websites, correct?
I think we sometimes forget how we, as developers, represent a small subset of the Internet’s population. And that in addition to our own blogs and sites we use these services ourselves. The fact is, I like to favorite tweets to follow important links later. If I follow someone interesting, I’m inclined to dig through their past posts to see interesting things they’ve talked about. The first hit on Google for “Kyle Weems” that is me (and not a basketball player or yo-yo champion) is (for better or worse) my Twitter account. Heck, the vast majority of my site’s non-direct traffic comes from links via Twitter, some of which are months old. Regardless of what you think of the quality of these networks, they generate a large amount of content and connect to a lot of content elsewhere. If suddenly all the links went dead, it would kill off a great deal of the web’s existing site-to-site traffic.
So although I’ve yet to build my own URL shortener, I clearly think it’d benefit me to do so because URL shortening is a problem that will affect me (and the rest of us) negatively in the future if we continue to use third party shorteners. Although I don’t know if rev=”canonical” is the proper solution, I think we need to ensure that we focus on a quick implementation of something. Even if the functionality to make use of it isn’t there yet, coding for it now will prevent more headaches later. The pace at which new links are being generated each day is staggering, and the sooner we turn things around, the less tragic the zombie link apocalypse will be for us all.
Tags: Comic, jeremy keith, link rot, rev, tinyurl, twitter, URL shorteners, zombies
I have to agree that a lot of the references to my site also come from twitter or are direct links from outside sources (Which I would consider, twhirl, tweetdeck, and the many other twitter programs out there to be the likely suspect for said traffic). However, as someone who is not a developer… I have no good suggestions to fixing the problem, so I’ll leave that to you and the smarter people
I was mostly sitting aside and watching the conversation about link rot and happen, only dropping the odd remark from time to time on Twitter or elsewhere.
There is some truth that the mere possibility of persisting all this information we persist today, has convinced us that it is also valuable or will be valuable at some point in the future. We don’t save every phone conversation or SMS we send, or even every face-to-face conversation we have with a friend at the cafe, but with the Internet we seem to put things in another frame entirely.
In reality, not just short links rot. Long links also rot, pages are removed, sites disappear, domains expire, it is a part of a natural cycle. When information is really useful, there is usually an active effort to preserve it, mirror it, and make it available in different media and formats.
Short links are a bit of a sensitive subject, however, since their providers are turning into a single point of failure for linked content, and so the perceived danger of sudden significant damage has propelled the conversation about rev=”canonical” and its synonyms forward.
As I commented elsewhere, people seem mostly concerned about how to spell the attribute name and value, while I see much larger problems in general. Will short link providers harm their business by respecting rev=”canonical”? Would people know enough to install browser plugins supporting it? Will they know enough to know how to support this on their own sites?
And also, some scenarios seem not well covered by this mechanism. A rev=”canonical” link is relevant for the entire page. This means you can’t specify a short link to a fragment identifier inside that page, unless — as I was hinted by Chris Shiflett in a conversation — you append the full fragment identifier to the short link itself, thus making it a not very short link. Also links are created in advance, instead of on-demand, which creates a link “garbage” problem for service with highly dynamic URLs like Google Maps.
So, the problems are many, solutions are hard to come by. As an attempt to do something, I created this little experimental service: http://lll.jugai.com
It is essentially taking 4 short link providers and creating 4 short links with the same name pointing to the same URL on all of them. Hence adding automatic mutual “link backup” if any three of those services go down in the future. I’m not claiming this solves the problem entirely, I’m leaving this to the community to judge, but hopefully it demonstrates there is more than one way to tackle this, and I hope future discussion also covers the many alternative solutions we don’t consider right now.
[...] Bamboo Juice, which helps illustrate his impression of thinking long term (which I touched on in my last comic update). I’d like to point out in particular the line: “Think about what you would put on the [...]