Dealing with information overload 2008-04-11


I've been thinking a lot about dealing with information overload recently. WIth the ever-increasing hype for sites like Twitter and FriendFeed, neither of which I use, and a steady stream of Facebook invitations, LinkedIn requests and invites to a continuously growing set of social networking sites and bizarre (to me) services that all add some form of social networking services, I'm more and more fatigued. I hardly keep up with even my feeds and my e-mail, never mind my IM accounts - messages from people keep building up for days before I answer them. And the thing is, I'm not actually connecting to large numbers of people. I'm fairly anti-social and notoriously bad at keeping in touch with people I've worked with in the past etc. (it's nothing personal folks - I'm happy to hear from people, I'm just rarely initiating contact myself because I'm always deep into focusing about something or other). I need better tools to manage it. Friendfeed aims to combine lots of streams of data into one huge river. The problem is that my challenge isn't to get a single view - it's to effectively manage whats there: What should I read? What should I ignore? How do I find past information? What do I need to respond to? What should I keep and what should I discard completely? In other words I need a personal search engine and my own personal recommendation or classification engine. If I had time I'd build one. Building a small scale search engine for this data is trivial - there are tons of packages like Sphinx that are "good enough", so the problem is mostly ranking (and that is by no means trivial for data like this, that for many elements can be as short as a single line, and that doesn't have enough internal linking for something like pagerank to work). Building a classifier is also fairly trivial - but the problem is similar: Small snippets of data + training. Again, that's not a trivial problem. But even a badly flawed one would be vastly superior to nothing. Getting something like that to "product quality" is hard. Getting something that is more than marginally usable for personal use should be doable, and mostly limited by my continuous lack of time. So I guess I'll paraphrase Abbie Hoffman:

-Steal this idea

Pretty please? I want someone to build a service that'll do this for me, damn it. Even if I have to install a local client to capture some of the data. Yes, it will have a ton of privacy and security implications, and no, I wouldn't trust just anyone with it. Let's summarize: Yeah, it's a tall order.

blog comments powered by Disqus