Jon Kleinberg's word bursts

 by Martin Belam, 19 February 2003

This article in the New Scientist about Jon Kleinberg's work - Word 'bursts' may reveal online trends - makes it sound like it is just the same thing that the BBCi homepage, Aaron Schatz's Lycos 50 and Google Zeitgeist are doing already with search terms.

And very similar to David Sifry's work at Technorati, where what you could call 'link bursts' within blogs are generating great stuff at his 100 most interesting recent blogs and 100 interesting newcomers lists.

But if Kleinburg has come up with an effective way to apply this technique to large sets of documents it could be very useful & interesting indeed. I'd want to see some rather more substantial examples of it in action than the ones given by New Scientist though:

'Kleinberg analysed all the annual state of the union addresses given by US Presidents since 1790...from 1930 to 1937 a spike in the use of the word "depression" is seen'

Not a real shock finding I guess.

It seems to me from a distance, however, that the technique may be much better suited to smaller sets of documents or to searching over intranet sets, where there is a common theme and a semi-controlled vocabulary, rather than applying it to the web as a whole.

Keep up to date on my new blog