Michael Fagan asks: "Maybe you should exclude the consistently popular links"

I thought this was worth talking about... Before I begin, let me present to you Exhibit A. The file containins a list with numbers of occurrences on the left side, link URLs on the right side, and is sorted by frequency. To a certain degree, ignoring the most popular links sounds like it might work. The top few links go to sites like Blogger/Blogspot and MovableType, which certainly hold little actual meaning. You go down the list, and you see that the same applies for links like google.com or the W3C validator.

But then, at position number 13, like a truck in thick fog, a link to The Onion appears without warning. What should we do? Can we disregard that? Personally, I don't think so. Even though a link to blogdex (a few places further down) doesn't hold much significance, I think a link to The Onion does say a thing or two about the blog (and blog author(s)).

Anyway, that's the main reason why I'm not so sure weeding out the top N popular links would work. That doesn't mean I have a better solution either... Perhaps what I need is some sort of feedback loop, so people can democratically determine which links are "significant" and which ones aren't. If I could add in a feedback mechanism and feed it to an AI algorithm, that might work... Hmm.

Time to hit Crtich for his AI book.