BBC News
After the Boston Marathon bombing and the killing of a British soldier on the streets of Woolwich in London, it emerged that suspects were known to the security services - prompting concern from critics. But how feasible is it for the spies to monitor everyone on their watch list?
Michael Adebolajo and Michael Adebowale's names have become well known since they became key suspects in the killing of Drummer Lee Rigby in a street attack in Woolwich on 22 May.
But it has transpired that they were already familiar names to the British domestic intelligence service, MI5. The UK Parliament's Intelligence and Security Committee is to investigate the agency's actions in relation to the case.
Boston bomb suspect Tamerlan Tsarnaev, meanwhile, was questioned in 2011 by the FBI amid claims he had adopted radical Islam.
But is it practical - or even possible - to keep close tabs on every person who comes to the attention of the security services?
No, according to Dame Stella Rimington, former head of the of MI5. To see that, she says, you just have to look at the numbers.
It's not known how many people are on the terrorism watch list in the UK. But it has been said to be around 2,000.
Continue reading the main story
More or Less: Behind the stats
Listen to More or Less on BBC Radio 4 and the World Service, or download the free podcast
To keep a constant watch on just one of those people, you would need a team of at least six surveillance operatives, Dame Stella says. But of course they couldn't work 24 hours a day, so you would need three teams of six.
And those operatives couldn't just sit outside a suspect's house. So, you'd need an additional person to, say, sit in a nearby house, and alert the team of six when the suspect left the house.
Then there's the control centre, where staff receive information from the mobile operatives and give them directions. And finally, there's a desk officer in charge of the case.
"Doing that 24 hours a day, seven days a week - well, you do the sums, it's an awful lot of people," Dame Stella says.
And if 2,000 people were to be followed like that, we'd be talking about 50,000 full-time spies doing nothing but following suspected terrorists. That's more than 10 times the number of people employed by MI5. The numbers don't add up.
Boston bomb suspect Tamerlan Tsarnaev had been questioned by authorities
As a matter of simple economics, then, it's not possible to follow every suspicious character around the clock. Surveillance technology offers an alternative - but it also presents mathematical problems of a different kind.
Imagine that the intelligence services had unlimited resources and could monitor everyone's phone lines.
Imagine they could detect would-be terrorists within the first three words they utter on the phone with a 99% degree of accuracy.
There would just be one small problem, according to Howard Wainer, a professor of statistics at the Wharton School of the University of Pennyslvannia in the United States.
Continue reading the main story
More from the Magazine
Before they were Tamerlan and Dzhokhar Tsarnaev, they were Black Hat and White Hat - two young faces in baseball caps, identified in grainy video footage as the prime suspects in the Boston Marathon bombings.
Those bombs would kill three people and wound more than 200, and Black Hat and White Hat were identified soon after their images were released. A day later, one was dead and the other in hospital.
The crucial role that video footage played in this case has many Americans re-evaluating the role of CCTV and other surveillance tools in public spaces.
Suppose there are 3,000 terrorists in the United States, he says. If the software is 99% accurate, you would be able to pick up almost all of them - 99% of them. However if you were listening to everybody - all 300 million US citizens - 1% of the general population are going to be picked up by mistake.
"So mixed in with the 3,000 true terrorists that you've identified are going to be the three million completely innocent people, who are now being sent off to Guantanamo Bay," Wainer says.
That is, for every terrorist you would have 999 innocent, but very angry people.
In reality, Wainer says, your terrorist detector would be nowhere near 99% effective. But, on the other hand, the security services do not in fact monitor everyone.
Still, if you narrow your target population to the point that there is one actual terrorist per 100 people wiretapped, and assume a 90% effective test, the chance of a false positive remains high. Even when someone triggers an arrest, Howard Wainer says, the odds are 11 to one that they're not a terrorist.
It's hard to predict who will become radicalised, and who will go on to commit an act of violence, according to Nigel Inkster, who used to work for MI6, the UK's secret intelligence service abroad and is now employed by the International Institute of Strategic Studies, a London think tank.
"With the wisdom of hindsight, everything always looks clear, but when you are looking at the situations as they unfold, you are operating in a climate of considerable uncertainty," he says.
Continue reading the main story
“Start Quote
Intelligence services can strangle themselves if they have too much information”
Dame Stella RimingtonFormer head of MI5
Technological advances mean intelligence services can use computers to sort through lots of information to help target the right people.
Algorithms are used to search social media for key words, or to spot suspicious patterns in airline travel records, for example.
But these techniques may be of limited use, according to Louise Amoore, a professor at Durham University specialising in data and security.
"You may already begin to think about how the algorithms used to detect possible risky connections might be adapting," she says.
"For example, post-Boston there may be more attention in the US to travel to particular parts of the world, perhaps including Chechnya and Dagestan. We could imagine, post-Woolwich, that there might be greater attention in the refining of algorithms to think about patterns of travel and links to deportation.
"But of course, it's using data from past events. Our research is suggesting that the tuning of the algorithm reflects almost always past events."
In other words, the algorithms are always fighting the last war.
And, as for the useful information that can be gleaned from the data, there is a risk it can be lost in the noise.
Big data is one of the challenges that security and intelligence organisations around the world now face, according to Nigel Inkster.
Continue reading the main story
More on Woolwich and Boston attacks
"You're able to amass large quantities of data, beyond what is possible for one individual or group of individuals quickly to analyse and assimilate," he says. "Algorithmic approaches are being adopted to try and triage this information, but I don't think you are ever going to be able to develop algorithms which can substitute for or improve upon human judgement."
Dame Stella points out that this is a well-known problem - it happened to the East German Stasi. They "overdosed" on information, she says.
"Intelligence services can strangle themselves if they have too much information, because they can't sort out from it what they need to know and what they don't need to know. So in all this search for information you've got to be pretty focused and targeted."
