Sniffing out bias and censorship on the BBC News site

 by Martin Belam, 25 October 2006

A mail on the BBC's mailing list yesterday pointed me to News Sniffer. It is a site that monitors online news output to detect revision and bias. At the moment it has two main sections - "Watch Your Mouth" monitoring the BBC News message boards, and "Revisionista" monitoring edits to output on the BBC, Independent and Guardian website. I'm always intrigued by systems that mix automation with human input, and the News Sniffer also allows users to "recommend" revisions which display bias.


Striking a balance between one size fits all and tailoring the way the system parses the HTML is a tough one, but I think the Revisionista script could benefit from a couple of tweaks to more accurately reflect the semantics of the page being scraped.

One of the articles from the BBC News site that has had the most votes for bias is this one. The difference as it appears on News Sniffer is that in the middle of the article the phrase "Many Palestinians feel Mr Blair has ignored their suffering" has been deleted and the phrase "Mr Blair met with the families of captured Israeli soldiers" added.


Embedded in the text this looks like a clear attempt to move from a negative image of the British PM to a more positive and sympathetic to Israel image. However, if you look at the original article on the BBC News site, this isn't content from the article itself, as News Sniffer records it, but the captions for the photgraphs included in the article, which have changed. Going back through the versions you can see that the article always had one picture from each "side" of the conflict.


Likewise News Sniffer currently records as revisions when pull-quotes from the message boards are changed, or when the Have Your Say call to action is edited, or a link added. Whilst they are clearly changes to the published article, they are not in the same league as paragraph long revisions of text.


These do exist, as this article about Cuba clearly illustrates.


It is, of course, terribly hard for machines to rank the importance of a particular change, but I'd be keed to see the site experiment with some kind of "interestingness" algorithm about the edits. You can't say for certain that an edit of only two or three characters is insignificant - it will probably be a grammatical or typo correction, but clearly the addition of 'not' to a sentence would only be a small change in the character count but a big semantic change. I'd like to see News Sniffer flag up really big changes more though, where whole paragraphs have been added or taken away.

The idea of monitoring BBC News for changes is one that has been done before for the homepage alone as one of the prototypes - "BBC News front page archive" by Matthew Somerville - and I know one member of BBC staff who built a similar system for inside the BBC that showed up differences in articles as they were republished.

On the News Sniffer blog I saw that the person behind the site had recently contacted Biased BBC about it. I loved some of the comments it generated, where initial glee quickly turned to mistrust as they realised the person behind News Sniffer was "left leaning" and generally a "not-we". One commenter suggested that the News Sniffer site would turn out to be biased itself. It took me a few minutes to get my head around the concept of being instinctively biased against a site that sets out to detect bias in a news source you already believe to be biased :-)

I found the "Watch Your Mouth" section very interesting indeed. It attempts to capture all the comments that have been moderated off the BBC News "Have Your Say" section.

The "Watch Your Mouth" section has a chart of the most censored threads. I'd prefer to see this expressed as a percentage as well as just the raw numbers of comments, as you'd expect busier threads to attract more moderation. Some of the posts have clearly been pulled because they are low quality or break the house rules about flaming people - like this one:

How large is your ecological footprint?

>>>>My footprint is huge and I like it that way. How dare anyone try to control what I consume.
Angie, USA

>>Typical american attitude. Selfish, arrogant and greedy.
ali denk

Gee, Ali, will sanctimonious stereotyping save the planet? Are you living in a cave, using no energy and eating grass? If not, get over yourself.

Others seem pretty lightweight to get 'censored'. A comment on the thread "What's your favourite tranquil spot" which was pulled was 'I love the silence at Old Trafford when the visiting team scores a goal'. I laughed out loud, but it got deleted from the BBC site. Of course, out of context you can't see if this individual user had a difficult pattern of behaviour (e.g. posted anti-Manchester United comments in every single thread regardless of the topic), or whether this was one post as part of an ongoing feud in the thread. However, it does seem like a lot of pretty innocuous content is getting deleted.

I was also intrigued by the legal position here. According to the BBC's Have Your Say rules, users

agree to grant the BBC a perpetual, royalty-free, non-exclusive, sublicenseable right and license to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, perform, play, and exercise all copyright and publicity rights with respect to any such work worldwide and/or to incorporate it in other works in any media now known or later developed for the full term of any rights that may exist in such content, consistent with privacy restrictions set forth in the BBC's Privacy Policy.

Clearly the user hasn't given permission for News Sniffer to reuse the content, and I wondered where the legal responsibility lies if the BBC has pulled comments from Have Your Say for legal reasons (e.g. copyright infringement or potential libel proceedings) and it is republished with a permanent link on News Sniffer.

I think it would be interesting to also do this kind of monitoring on a range of forum's associated with news sites. They all have their standards of moderation, and it would be good to get some kind of cross-media "percentage zapped" score. Of course, this only works on post-moderated boards, and will never answer the question "I submitted loads of comments to news organisation x, and they never publish them because they are biased about y".

From the comments left by the author on Biased BBC, I understand the code producing the site is going to be released as open source, allowing people to run it themselves, and tailor it to their own media interests.


BBC have your say has many fishy things going on. Ive many times put comments on that languish in "moderation" for a couple of days while other comments are published. Then my comment gets published on page 20 and nobody reads it.
They are also so slow in general moderation that the "readers recommended" is a farce as you get people reading only the first page of most recent. so any comments that happen to appear as most recent on the infrequent moderation get the most recommedations. This is the compounded by allowing the user to recommend comments from the "readers recommended" so if you're up top from your head start you get more readers seeing and recomending you.
Personally I wont be commenting anymore, its a waste of time time.

The HYS debate on the Israeli stikes in Gaza during December 2008 had an interesting pattern. As of Tuesday 30th, two posts were 'battling' for top slot in Reader's Recommended with in excess of 1700 each. Wednesday 31st, these two have gone and another story is top, out of nowhere!

Suddenly, we have reader's recommended supporting the Pro-Israeli pogrom.

Astounding, very fishy - and not for the first time either.

I posted a comment to HYS which was pointing out a rather inconvenient truth of bank bailouts and questioning whether people shouldn't seek safer options for their savings than our rather precarious-looking high street banks.

After the first two days while other comments were being moderated and published I asked them when my would likely be released. No answer.

It was never published, and still sits 'Awaiting Moderation', eight months after the debate was opened!

The BBC seems to get very sensitive in HYS debates on two issues: Israel/Gaza and property. If there is a HYS on either of these topics and you say something the BBC doesn't like but can't censor, expect it to languish in 'Awaiting Moderation'.

Newsniffer is great but it only detects what the BBC has outright rejected. If it languishes in 'AM', the BBC have the plausible deniability that they "didn't get round to looking at it".

basically, HYS is used to paint a not-very-truthful picture of grass-roots opinion. In fact this technique is called Astroturfing (fake grass roots opinion - geddit?!) and the BBC are masters of astroturfing!

Bob's comment about "reader's recommended" with pro-Israel comments ending up on top. Recently there was a HYS on whether people were worried by the recently announced Russian arms buildup. I was not suprised that there were plenty of people saying that Russia should not be criticised on this because the US spends so much more. What was truly amazing was that I counted down to the 27th most popular comment before I found one that did not support the Russian arms buildup.

Since a very large proportion of readers must be from NATO countries it is suprising how many of them seemed to be saying "It is absolutely right that they should point more nuclear missiles at me." Very fishy.

I posted regular on HYS, the posts have ended up on other websites as a source of amusement
bit upsetting actually
the internet is evil

Keep up to date on my new blog