Newspaper "Site Search Smackdown": Round 4 - The Daily Express vs The Times

 by Martin Belam, 11 April 2008

I'm running a series of smackdowns between British newspaper site search engines, to test how fresh their indexing is. The Daily Mail triumphed over The Sun in Round 1, and in Round 2, The Independent emerged victorious over The Telegraph, getting a perfect 10 out of 10 in the process. Yesterday was a low scoring Round 3, with The Guardian just edging out The Mirror, by 7 points to 6. Today I'll be finishing off the inter-newspaper contests with a smackdown between The Times and The Daily Express.

The Times vs The Daily Express

The rules

The rules of the contest are quite simple. Go to a newspaper homepage at around 9am UK time. This is after the print edition has hit the streets, and when the online version has had overnight to publish the paper's main news. Take a note of the ten most prominent online headlines. Then use the newspaper's own search engine, and type in each of those ten headlines exactly as they appeared. Newspapers score a point if the story comes up in the #1 slot for that search query. Doesn't sound too exacting a test? You'd be surprised how often newspaper search engines fail to deliver...

The Times

The last redesign of The Times was noted for a rather idiosyncratic use of CSS classes. Would the site search hold up to the paper's traditional offline reputation for quality? The short answer was emphatically "No".

The first three headlines I tried to find via site search all drew blanks - "Olympic torch fears after Chinese kill eight", "Fanatics targeted Canary Wharf and nuclear sites" and "Terror returns as Mugabe thugs vow to fight". It was only with my fourth search that I got a positive outcome, locating a story about NHS property developments.

With just one successful search out of the opening four, The Times looked set to post a score lower than The Sun's 5 and The Telegraph's 4½. Two sports stories rallied the score a little, as both "Pietersen wants players to grab IPL bonanza" and "Arsenal to benefit from Liverpool changes" produced the right stories at the top of the results. However, a story about Chelsea wasn't in the index, and business stories about Yahoo! and the Halifax also got no results.

In the end, The Times only managed to return four of the top ten stories of the morning if you searched for their exact titles on The Times search engine. At 4 out of 10, it was the most dismal performance by a paper so far. Surely the Daily Express had this round in the bag?

The Daily Express

I had very low expectations of search on The Daily Express. The impression that this area of the site is neglected is given by the fact that the most popular searches haven't changed for over 50 days now, and that the suggested search term for users has been 'e.g. Princess Diana' for years.

I wasn't much surprised therefore to see the paper stumble at the first hurdle. The day's lead story for the Express was MPs expenses, and the fact that "You have to pay Brown's TV Licence". This had been indexed, but a search for the headline only listed the story as the second most relevant result.

Then, much to my astonishment, the Daily Express absolutely breezed the rest of the test. Whether the story was about the Duke of Edinburgh, Coleen, Naomi or Renee, the Daily Express cruised past the score of The Times. In total 9 headline searches returning the relevant story in the top slot. With just the Brown TV Licence blip, it meant the Daily Express scored 9½ out of ten, and pipped the Daily Mail to second place overall.

The verdict

This was a complete reversal of how I expected this contest to pan out, with the Daily Express having an easy victory over The Times, which produced the worst results of any newspaper. The full table of merit was as follows:

PositionNewspaperScore
1.The Independent10
2.Daily Express
3.Daily Mail9
4.The Guardian7
5.The Mirror6
6.The Sun5
7.The Telegraph
8.The Times4

In Round 5

This whole contest was sparked by wondering whether Google's "Search in search" functionality improved the user experience for people searching for newspaper articles. In the next round of the Newspaper Site Search Smackdown, I'll be throwing three Google-powered services into the fray, and pitting them against the results you get from the newspapers themselves.

1 Comment

Times Online articles often have different headlines on the news index page and on the article, usually due to lack of space or search friendliness. If you search for the front page headline it will only display the article if all the words are contained in the indexed headline and body copy - this isn't always the case. If you ran the test again with the on page headlines the result would be better for Times Online (and probably the Telegraph as well).

I'd argue that this test doesn't really prove how fresh or relevant newspaper indexing is, especially against Google News, because it doesn't reflect general search behaviour and depends on all headline versions being identical. A more accurate test would be to search for the subject mentioned in the story to see if the article is both indexed and appearing prominently in the search results, or to use the article headline only.

Keep up to date on my new blog