P2P browsing during surge events

 by Martin Belam, 10 June 2006

Well, I don't know how it was for you, but here on Crete in the run-up to the opening game you couldn't get on to fifaworldcup.yahoo.com, or indeed yahoo.com, for love nor money. I don't know if that was a worldwide phenomena, or whether my little rock in the Med just exceeded the maximum number of connections or bandwidth back to the mainland.

It called to mind though something that was mentioned in a thread on Digg about the demise of Pirate Bay, which wondered if you could host a website on a peer-to-peer basis. That would be so useful at the time of huge surge events like the World Cup finals.

If all of the things that you might usually cache, like the stylesheet and furniture graphics, could actually be distributed by browsers on a peer-to-peer basis, all Yahoo! would have to actually serve would be diffs of their main content, greatly reducing the volume of data and number of connection their servers farms were having to handle.

Oh well, I guess they've got four years to think of something cleverer before the opening game next time around.

1 Comment

This already exists in a number of ways: Freenet, at http://freenet.sourceforge.net/ , is a way of P2P-hosting entire websites - 'freesites', since they're not on the standard internet - in a way that is entirely anonymous. There are a number of issues with this: it makes copyright infrigement very easy (since nobody is 'hosting' the files as such), and clearly where copyright infringers can post stuff without getting caught, so can more unsavoury people. And, up-to-date information is difficult: it can take days to update sites, given the nature of their publication.

The Coral Content Distribution Network - www.coralcdn.org - allows you to mirror entire websites. This is good for 'The Digg Effect' or 'The Slashdot Effect' where the original sites are too busy to cope with more requests. This appears to be quite good in coping with up-to-date information; apparently going to http://fifaworldcup.yahoo.com.nyud.net:8080/06/en/ will fix your worries. (That points to the English version, assuming your Greek isn't quite that good!)

But you're right: this is one of the current failings of the internet infrastructure. For broadcasters especially - who have to cope with incredible fluctuations in traffic when they tell two million people to 'go to the website now' - it's a difficult task to ensure that your infrastructure will scale adequately.

Keep up to date on my new blog