"Filtering" user-generated content on the BBC News site
Late last week someone left a comment on one of my old pieces about the BBC's moderation policies, asking my opinion on some of the things mentioned in Peter Horrocks' recent speech about user-generated content and citizen journalism. Bryan has left comments on currybetdotnet before, and we've exchanged email. We don't agree on everything by a long chalk, but he's always been very pleasant about it, as I hope I have.
I started writing a response, but once it got more involved I realised it was probably worth putting up as a new post, rather than just tacking it onto the end of an article that is nearly a year old now.
For context, here is Bryan's comment:
"Hi, Martin,
You might be interested to know that The Editors blog is still fumbling and stumbling in its apparent attempts to sort out the 'technical' problems that have plagued it pretty much since its inception. I put 'technical' in quotes since I just cannot see how the problems could possibly drag on for months turning into years if they were simply technical in nature and I have long suspected that the BBC was actually trying to filter out 'undesirable' comments and commentators - not because they break any rules, but because they are not PC. Paranoia? Conspiracy theory? Possibly. But have a look at this:
'We need to be able to extract real editorial value from such contributions more easily. We are exploring as many technological solutions as we can for filtering the content, looking for intelligent software that can help journalists find the nuggets and ways in which the audience itself can help us to cope with the volume and sift it'.
It's from Peter Horrock's recent Value of citizen journalism post.
As someone who is clued up about these matters, perhaps you can indicate what this filtering process involves?
Over the weekend following his article, I tried to send Mr. Horrocks a lengthy comment, in two parts. It wouldn't go through, just hung. So I sent it on the Monday and it went through OK but was not published, perhaps proving my point. So I published it on the breakaway blog from Biased BBC
Any thoughts you have on this matter would be much appreciated".
Without doing a point-by-point analysis, I was struck by three things that I thought worth mentioning.
The first is on the issue of the technical problems with the BBC's blogs. Now, I'm a big fan of applying Occam's Razor. It could be that the BBC didn't implement their blog platform with enough forward capacity. Or it could be that the BBC implemented their blog platform with enough capacity, but then make it run badly in order to control the types of comments posted. I know which seems more likely to me.
Robin Hamman wrote a lengthy post about the issues they have had, as Bryan rightly points out, pretty much from the moment the blogs achieved any kind of popularity. It appears that when the BBC heavily customised the version of Movable Type they were using, they did so in a way that has prevented them going on to implement some of the usual defence measures against spam.
I can sympathise with that. Even on a minor personal blog like currybetdotnet, wave after wave of spam aimed at the Movable Type comment system caused my server to max out repeatedly and database to crash, and I've got a fraction of the audience of the BBC's blogs. I've been able to try different solutions and play around with my installation - and have got it much more stable now, at the expense of people having to put in a codeword to submit a comment.
The BBC doesn't tend to play fast'n'loose with live systems like I can on my own. Plus I can pretty much guarantee that at the project prioritisation meeting where #1 on the agenda is getting the iPlayer launched before Xmas in time for the marketing, and #19 on the agenda is having a look at the blog servers, the resources stop well before item #19 gets reached.
Effectively the BBC's blog platform is hostage to some ill-starred technical decisions early on. It is anecdotal evidence I know, but it isn't just the general public trying to post comments who are affected. On one of my articles last year for the BBC Internet Blog, James Cridland and I had to continue a discussion about a post on our own sites, since neither of us could manage to get a comment submitted and published. He's BBC staff and I was trying to reply to an article I had authored!
[UPDATED: James points out in the comments and on this post on his blog that this wasn't the case. I appear to have conflated our discussion about the BBCi Search impartiality 'glitch' with attempting to reply to some comments on the BBC Internet Blog by Andrew Bowden and Lizzie Jackson. My apologies]
The second point I was interested in taking up is Horrocks' use of the word 'filtering'. I feel, reading through his speech again, that two separate issues have become conflated in the minds of some people. He opens his talk discussing the internal debate in the BBC about the audience reaction to Bhutto's death. The BBC considered switching off Have Your Say. Horrocks then goes on to talk about content 'filtering', and I bleieve that some people have assumed that this 'filtering' is about what appears on the site in public.
I'm not entirely convinced that is what Horrocks was saying. I believe that he is talking about using 'filtering' to improve the value that user-generated content brings to the newsgathering and news production process.
To use an example entirely divorced from politics and religion, so we don't get sidetracked by any unfortunate use of phrase on my part, think about the final day of the football transfer window. The BBC encourages user's of 606 to text and message with their sightings of footballers at training grounds, airports, taxi ranks and football club, to give the day a sense of fun and urgency. The BBC gets thousands of messages on the topic, mostly consisting of banter and wishful thinking.
But somewhere in there may be the one genuine first sighting of Jermaine Defoe with a lawyer who someone knows represents Portsmouth. How does the BBC get that information out of the big bucket of user-generated content coming in, most of which is inconsequential fluff, and into the hands of Radio Five Live's football correspondents as an exclusive.
To go back to the Bhutto example Horrocks cited, I think what he meant was that in amongst the 15,000 comments were statements from potential eye-witnesses to the assassination, and people from Bhutto's life. He wants to find a way, and technology is one avenue worth exploring, that this content can be put quickly into the hands of the newsgatherers, rather than just be one of 15,000 messages looked at by poorly paid part-time moderators.
I don't believe that Horrocks was suggesting that stuff should be filtered off the site, more that the process of getting the good stuff into the hands of journalists for follow-up needs to be faster.
What sort of format might such a filtering technology take? Well, there are a couple of reasonably straightforward steps that could be employed. One would be to think of a spam filter in reverse.
Spam filtering works by recognising words and phrases like 'viagra', 'casino' and 'I am the widow of', and trapping those messages. To look for eyewitness accounts, for example, you might filter in reverse for phrases like 'I saw' and 'We heard', or things that indicate proximity like 'We were n metres away', and highlight those.
You could also run some IP filtering on your comment submissions. If you were looking for eyewitness counts of a train-crash in Greece, you could pretty much discount anything where the IP address originated outside of that country. As a result you would very likely cut down the number of comments that journalists had to look through in order to tease out the 'hidden gems' Horrocks believes are in there.
That isn't to say that the BBC won't do less of 'Have Your Say' in the future. I would have thought that there must be some senior level editorial consideration given to the fact that user-generated content can have a big impact on how a site or organisation is perceived.
Whilst I'm a big champion of news sites utilising user-generated content, and think that some people have done it really well, I happen to agree with Horrocks that reading through a lot of the stuff published on 'Have Your Say' is monotonous and unhelpful to people's understanding of an issue.
At least Speak Your Branes means I don't have to read it much myself anymore to find the amusing stuff. When I do look, I don't particularly find the posts from users to be particularly skewed one way or another. The very first comment on Horrocks' posts suggest HYS has been hijacked by the BNP, but I know very well that the general experience of people on Biased BBC is that it is hard to get their comments published, because, they assume, they don't toe the BBC party line. I tend to see what I would expect on an Internet forum, a large mix of very strongly held views shouting each other down as idiots. It isn't quite the viper's nest of 'Comment Is Free', but it is getting there.
I wanted to pick up a third and final point from Horrocks' speech, or at least the reaction to it. I notice some comment has been made about his use of the word 'agenda':
"We cannot just take the views that we receive via e-mails and texts and let them dictate our agenda".
As I've said before, one of the things that I often thinks lets down those who believe that the BBC demonstrates bias is a lack of precise focus. There are some really interesting threads in Horrocks' speech, but noting his use of the word agenda as evidence of BBC bias is almost beyond trivial. Of course the Today programme and the 10 O'clock news have an agenda.
If they didn't have an agenda they would be dead air.
It seems, frankly, self-evident that Horrocks is talking about the slate of stories and running orders that go into the BBC's bulletins, rather than an 'uber-reaching ideological agenda'. I suspect you could only believe he meant the latter if you'd never worked in news broadcasting, and already believed it to be true anyway.
The overall impression I got from Horrocks' speech is that there are still areas of the BBC unsure of the value that all this user-generated content adds to the BBC News site. It seems to me quite clear that the BBC couldn't go back to the old days of the 'email-and-publish' model. But it seems equally clear that publishing thousands upon thousands of user comments that hardly anybody reads doesn't make for a particularly compelling user experience.
Striking the balance between giving people full participation, producing a quality readable site, and getting consistent journalistic worth out of user-submitted content is, I think, something that no online news site I've reviewed in the last couple of years has yet got right.
I left comments on the BBC Points of View which weren't breaking any rules and got banned. They refused to tell me why I was banned so I registered again. Everytime I registered I ended up getting banned without explanation. After about the sixth time I finally got a reply from the mighty BBC telling me it was because I had registered six times so I replied to them pointing out that they hadn't given me a reason for the first banning and I've never had a reply ;)
Basically the mighty BBC just doesn't like people giving their honest opinions about the BBC if they go against the grain.
Martin, thanks for that response. I was preparing one of my own when I decided to have a look at the "Speak Your Branes" site from your link and found this:
That attracted a number of comments in a similar vein to this one:
It’s all gone pear shaped since the Holacaust [sic] Industry got taken over.
Wanting to reserve judgement I had a browse around the site but I was eventually forced to conclude that it is nothing but a bundle of crude, far-left attempts at "humour" with quite vicious undertones. I avoid sites like this one like the plague and I'm amazed that you recommend it.
While dwelling on this thought for a while, I was struck by the contradiction between one BBC guy being amused by the mocking of conservative HYS contributors and the other claiming to be "looking for intelligent software that can help journalists find the nuggets" among HYS contributions.
Would Horrocks go panning for his nuggets on the right-wing side of the stream? I'm pretty sure the idea would not even occur to him.
That said, I was close to being half-convinced by your insistence that Horrocks was not talking about an ideological agenda but a practical one. I accept that the term "agenda" is very broad and could have an innocent meaning in this context. But in the same breath Horrocks mentioned the BBC's "take on a story." Are you convinced that is also totally divorced from ideology? I'm not.
Now I must admit I'm scratching my head over all of this and trying to resist the conclusion that the BBC is more a private club than a public broadcaster and has no genuine interest in feedback from "citizen journalism" apart from that which reinforces its own worldview.
I'll give you one example from a few months back of the BBC's ideologically-driven agenda influencing its output. The World Service picked up on the results of a suspect left-wing Israeli poll claiming that the majority of Israeli Jews were racist towards Arabs. Despite the fact that a news broadcast was not the appropriate slot for this story, the World Service broadcast it on the hour and half-hour at least through an entire afternoon (I first tuned in around 13:00 UTC) and right up to close to midnight, trying to ensure that whoever tuned in from whatever time zone across the planet would hear about it:
[See here]
Now if that is not bias, I don’t know what is.
I didn't comment in situ on the BBC Internet blog for a number of reasons - none, actually, to do with any tech problems.
For good news about the tech problems, and for why I'm more comfortable posting and not commenting, I refer the honourable gentleman to this post
My bad James - faulty recall. I know that both Andrew Bowden and Lizzie Jackson left comments on one post of mine that deserved a fuller response, and I wasn't able to post it. I must have conflated that with our conversation - apologies.
I'm interested as to why you think it wasn't a news item - because of who commissioned it, how it was carried out, or because of what the results were?
I tend to find a lot of news stories based on 'studies' of this, that or the other to be quite tenuous when you look at who commissioned them and the methodology, however they are quite the staple of journalism these days
Regarding Speak Your Branes - I have to say that I subscribe by RSS, so the content of the comments completely passes me by, I only ever see the original post. It can be pretty crude, but I don't see it as particularly political in a left-wing / right-wing way. I find it often neatly punctures the pomposity of a certain shrill type of British tone of voice wailing "why oh why oh why" about some minor non-calamity. Sometimes, it must be said, just by swearing at them - but perhaps the fact that I find that funny betrays my East End of London background. Along with Dilbert and Worse Than Failure, it provides a little bit of light relief in the midst of my daily feed reading.
Probably all three. It was quite extraordinary to have the results of a poll by a far-left organisation with highly suspect motives as the second most important news story for half a day. Was there really nothing else going on in the world on that day to fill the slot? The BBC grabbed hold of that story like a three-year-old with a new toy and would not let it go. The obvious intent was to demonise Israeli Jews as “racist” against Israeli Arabs and whip up anger against Jews in general since many of Israel’s enemies don’t differentiate between Israelis and Jews.
Do I overstate my case? Possibly, but until I see the BBC blaring out the results of the Palestinian polls that show a majority of Palestinians approve of suicide bombings against Israeli civilians, I think I’ll simply rest my case.
Do I object to the results of the poll? Not if it can be shown to be accurate. But that is highly debatable. The BBC has an obligation here to give us background information on the nature of the organisation that conducted the poll. It didn’t, and this is in keeping with the BBC’s habit of describing right-wing organisations as such but omitting to label left-wing organisations so as to create the impression that the left-wingers have broad legitimacy and support when this is clearly not the case.
Since you say that polls are a staple of news journalism I’ll look out for them. But I doubt that I’ll find them on the hourly and half-hourly news bulletins, which is the spot where the World Service’s news “editor” chose to place the Israeli poll. This is simply propaganda.
As far as commissioning goes, I’m not sure if you mean the people who commissioned the poll or the BBC. I don’t know much about the process by which events become news, but it is evident here that a left-wing organisation had a hotline to the BBC. Wouldn’t it be funny if the BBC itself commissioned that poll?
Re the reply from James Cridland it will be good news if the ongoing problems with the BBC blogs can be resolved. Also, maybe someone can do something about the BBC “Complaints” site. It’s difficult to navigate, looks like it hasn’t been updated in ten years and once you have sent your complaint, you don’t get a reference number to help in following it up. Here’s a personal experience of trying to use the BBC’s complaints facility:
Link #1
Link #2
I meant to add, regarding the Branes site, I suppose everyone has their own sense of humour. You probably wouldn’t enjoy Jackie Mason. I find him hilarious.
The blog problem isn't just technical, it's a complete absence of 'customer focus'.
If they were bovvered, the 502 error message would be user friendly (and not even say '502'). It would warn you on the page of potential technical problems and - behind that - resorting to Help would answer why your comment failed. These aspects are just crass and inept and obviously untested.
The blogs appear to be about soliciting feedback and comment but the attitude to so-called 'technical' problems just ruins their actual purpose (although I notice that Hamman has acknowledged they're still stuck in publishing so 'customers' have yet to enter their thoughts ...).
Well, here we have the attention of the esteemed Martin Belam and James Cridland, both of whom have been/are involved in these technical issues for the BBC.
If they cannot/have not been able to/will not be able to solve these problems I can only assume that there is no will on the part of the BBC itself to solve them.
Having an opinion and being able to fix things are very different things! I haven't been a full-time member of staff, or done any technical work at the BBC since December 2005, and even when I was there I never worked directly on the blogs side of things.
Just to tie up the loose ends of my comments on this thread, I reiterate what Peter Horrocks had to say in his Value of citizen journalism about the BBC's reaction to the public's Have Your Say comments on the Bhutto murder:
But our real question concerned the editorial value of the comments and how far they should influence our coverage more widely. And the answers to that were: very little and hardly at all.
And:
So the urge to constrain debate, as with the initial Bhutto response, needs to relax. We're going to just have to let it go and just make hosting this material less significant for us and audiences.
http://www.bbc.co.uk/blogs/theeditors/2008/01/value_of_citizen_journalism.html
Well, it looks like the BBC has now certainly made it less significant by severely limiting the number of topics people can comment on on the Have Your Say site:
http://news.bbc.co.uk/2/hi/talking_point/default.stm
There are only three topics currently open for debate on the international version of Have Your Say, whereas there used to be between ten and fifteen topics open on any given day as recently as a month or so back. This curtailing of feedback coincided with a new design of the Have Your Say page, with a ridiculous amount of space devoted to the two comments in the text bubbles:
http://news.bbc.co.uk/2/hi/talking_point/default.stm
Sneaky of the BBC to distract attention from their reduction of topics by a new look? How could I think that of them?!
This was a classic case of If it ain't broke don't fix it. And so very typical of the BBC.
But I thought the complaints about HYS were that it was broken, because of long moderation queue times, and because the pullquote from a discussion often differed from the viewpoint expressed in the 'most recommended' comment?
Surely the two changes address that. Less topics presumably mean more moderators focussed on each topic, and less of a backlog. And I think the two speech bubbles are there so that whatever way the debate has been swinging on the board, the BBC can show two opposing views from the one debate. The fact that this will lead to one bubble being 'Hamas are murdering scum' and the other being 'The IDF are murdering scum', and consequently everybody being unhappy seems by-the-by.
Your second point is a good one. I must concede that I had not registered it. It really is an improvement on the old system of pushing the comments that fall in line with the BBC's point of view, even if it is a very small minority view.
However, I disagree that by reducing the number of topics the BBC has allowed the moderators to cope better with the volume of comments. they've probably reduced the number of moderators as well. Whatever the case, have a look at this topic
Here's the debate status:
Total comments: 4385
Published comments: 1094
Rejected comments: 193
Moderation queue: 3097
It's been going for four days and is quite a hot topic, what with events in Gaza and Jerusalem. So it seems to me that the moderators could do a bit better than publish a third of the comments.
That said, I've never felt that the BBC is obliged to publish every comment it receives. I don't think the public has an automatic right of publication on any media. But I think the BBC needs to pull its socks up quite a bit on HYS.
Regarding the reduced number of topics, I can't see how less is better in this case.
The "moderators" are allowing all those comments to languish in the "moderation queue" as they so often do. I suspect it's because they can't find enough anti-Israel comments to publish. Scan the first ten or so pages of the "Most Recent" comments and it's evident that the majority by far are anti-Israel. (The anti-Israel comments, many of them quite vicious and propagandist, even outnumber pro-Israel and neutral comments taken together.) Scan the first ten or so pages of the "Most Recommended" comments, on the other hand, and it's evident that the majority by far are pro-Israel.
One would expect there to be at least a rough equivalence between the popularity of the pro-Israel comments and the number of pro-Israel comments actually appearing on the site. Instead, the reverse is the case. What do we glean from this? The "moderators" are deliberating trying to push anti-Israel propaganda by publishing a disproportionate number of anti-Israel comments. I've seen this countless times on Have Your Say.
I've probably mentioned this before here, so forgive me if I'm repeating myself, but if the BBC genuinely wants to uncover the bias of the people it employs, it should have a look at this HYS.
BBC staff will do anything to bash Israel, including lie and distort.
Well, here we go. The results of a startling new poll by Palestinians of Palestinians has revealed that a large majority supports the terror act on the Jerusalem seminary in which eight young Israeli students were gunned down:
This increase in radicalism is certainly newsworthy but somehow only the Israeli poll on the apparent racism of Israeli Jews found its way onto the BBC website. The Palestinian poll is curiously absent. I don't know about the World Service because I haven't been listening to it lately but I seriously doubt that the poll occupied a place of honour on its newscasts for hours, as did the Israeli poll. In fact, I doubt it was reported at all by the BBC.
Palestinian "resistance" is hallowed ground to the BBC and it wouldn't dream of publishing negative facts about majority Palestinian support for terror.
Just thought I'd let you know, Martin, sort of to tie up loose ends. The BBC is biased to the hilt against Israel. But don't take my word for it. Look at the evidence.
"Increase in radicalism"? Palestinians support Palestinian terror against Israel? Sounds like dog bites man to me.
Funny, having made such a song and dance about a poll that reflects negatively on Israelis, the BBC seems to be distinctly allergic to publishing the results of polls that reflect negatively on Palestinians.
That's yet another one that is nowhere to be found on the BBC website.
Martin, you are the one who insisted on February 10th that polls like these are "the staple of journalism these days." I scratched my head for a while over your last comment and then gave up trying to understand it. If you really believe that Israeli radicalism is newsworthy but Palestinian radicalism is somehow not newsworthy then you fit in well at the BBC.
Obviously I have to respond, because you know I can't bear to let anyone else have the last word on currybetdotnet....
I think you are missing out the 'new' bit of 'news'. I don't think it is news that a survey of a population that basically voted for Hamas at their last 'election' contains a large proportion who supports the terrorist tactics that Hamas stand for.
Now, if the survey showed a steep rise in support, that would be news. And I hope that one day journalists will be able to report a survey that shows a steep decline in the level of support for murder and insurgency amongst Palestinians. That will most certainly be news, and very welcome worldwide.
But all the Jerusalem Post report shows me is that support for terrorism amongst a sample of Palestinians has remained broadly the same within the admitted margin of error of the survey over the last few years. That isn't news, is it? I mean, we all saw the shameful scenes of Palestinians celebrating and praying in the street with joy after the murders at the Mercaz Harav seminary earlier this year.
The licence is actually a government set reception licence, starting with the introduction of radio over 80 years ago. The developing technology had to be funded, which eventually added pictures to get the TV you see today. The technology is still developing, and the BBC has been charged by the government (the one voted in by the British public) to invent, develop, and introduce the digital radio, TV, and HDTV services you are now getting. Perhaps more importantly, the licence means YOU own the BBC – not shareholders or advertisers, which makes it totally independent, and has earned it a worldwide reputation for truth and unbias. ITV News could not run a story, say, on McDonalds dodgy burgers if McDonalds were in the ad break! Plus NO ADS in you favourite prog! (post continued after this short break – you won’t hear that on the BBC, lol!)…