Could TDCs problems be bad fact finding?
Written by Matt B · first published 27/11/2009
A recent TDC survey polled 400 people about their opinions but was the survey unduely biased? I think it was.
I was recently forwarded a press release which appears to have come from Thanet District Council (TDC) announcing that Operation Clean Sweep was universally popular. While I have no fixed opinion on the subject I do object to the way in which the data was collected.
1. This press release came out at the begining of November but the survey runs until the 18th of December. Rule number one in statistical sampling never tell the target population what everyone else has been saying if you want them to tell you their opinion and not shift it to fit with the "popular" view of things. This press release undermines the value of the final findings by biasing the whole survey towards the early results.
2. This data collected came from 400 out of about 125,000 people a statistically insignificant sample size. That is just 0.32% of the population of Thanet. A bigger number than that vote BNP and we do not consider them a significant reflection of the area's opinion.
3. Thanet Community Safety Partnership spent a week getting 400 people's opinion from sites around Thanet that ammounts to 80 a day which must be taken to roughly equal 80 per point in Thanet visited. again this sample rate is too low. Furthermore these people were being asked if they supported or liked something that the question asker was part of promoting. Hardly a nutuural assesment.
4. We are not told how many results were rejected. If, for example, a large percentage were rejected we should be told why - on what grounds. How many people did they speak to and how many results were counted towards the total. I could easily speak to 5,000 people over a few months and then reject every result that did not support my views and claim that after talking five thousand people 90% of results support the view I want to push. Even if I only had 10 valid results left. I did speak to 5,000 people and my percentage is now 90% in favour.
5. We are not told how many people they spoke to in total and what percentage offered an opinion. This again shows that the data was not well reported. Did they speak to 400 people but only get 50 surveys filled out? Did they fill surveys out for the people they spoke to? What controls were used to make sure a representative sample was obtained?
6. Those hostile to such groups would not have approached the bus thus the survey was naturally bias in its sampling methods with a strong positive only self selection built in tot he way it was run. If I take a survey of people that like me or owe me a favour and ask do you support the idea that Matt should be given ten grand a year by the council for being a smashing bloke I would get a large percentage saying "yes". The general public would be more likely to answer "who the f*** is Matt?"
7. The press release makes use of strong prejudicial language by describing a worry as "nuisance youths". This is mentioned twice along with two other issues thus raising it's profile and likelihood of being remembered in the mind of the reader and then the press release goes on to urge people to fill in a survey. All this after using language likely to make these issues much more prominant in the mind of the survey taker than would have normally been the case.
8. We are not told what portion of that 400 were actually worried about dog poo, young people, bikes on pavements, litter and other issues. For example if only twenty people mentioned these things then they should not be trumped as significant but the language of the press release makes it sound as if everyone was bothered about them.
I do not want to give you the idea that I am against solving problems related to young people hanging around with nothing much to focus on and the problems that arise. I am not going to tell you that dog crap on the pavement has not been an on going problem for as long as anyone can remember. It is a problem that has not gone away since the 1980's which is the earliest that I am able to personally recall such things. Watching out for dog-poo is part of walking in Thanet and has been for many, many years (at least 20 years).
What upsets me is that this looks like it is one of two things: (1) a badly managed fact finder - or - (2) an attempt to push an agenda to make it appear to be in response to public opinion. Which one you believe boils down to how skilled at manipulation you feel the authorities behind this actually are.
I was recently forwarded a press release which appears to have come from Thanet District Council (TDC) announcing that Operation Clean Sweep was universally popular. While I have no fixed opinion on the subject I do object to the way in which the data was collected.
Why do I object to this survey so much?
1. This press release came out at the begining of November but the survey runs until the 18th of December. Rule number one in statistical sampling never tell the target population what everyone else has been saying if you want them to tell you their opinion and not shift it to fit with the "popular" view of things. This press release undermines the value of the final findings by biasing the whole survey towards the early results.
2. This data collected came from 400 out of about 125,000 people a statistically insignificant sample size. That is just 0.32% of the population of Thanet. A bigger number than that vote BNP and we do not consider them a significant reflection of the area's opinion.
3. Thanet Community Safety Partnership spent a week getting 400 people's opinion from sites around Thanet that ammounts to 80 a day which must be taken to roughly equal 80 per point in Thanet visited. again this sample rate is too low. Furthermore these people were being asked if they supported or liked something that the question asker was part of promoting. Hardly a nutuural assesment.
4. We are not told how many results were rejected. If, for example, a large percentage were rejected we should be told why - on what grounds. How many people did they speak to and how many results were counted towards the total. I could easily speak to 5,000 people over a few months and then reject every result that did not support my views and claim that after talking five thousand people 90% of results support the view I want to push. Even if I only had 10 valid results left. I did speak to 5,000 people and my percentage is now 90% in favour.
5. We are not told how many people they spoke to in total and what percentage offered an opinion. This again shows that the data was not well reported. Did they speak to 400 people but only get 50 surveys filled out? Did they fill surveys out for the people they spoke to? What controls were used to make sure a representative sample was obtained?
6. Those hostile to such groups would not have approached the bus thus the survey was naturally bias in its sampling methods with a strong positive only self selection built in tot he way it was run. If I take a survey of people that like me or owe me a favour and ask do you support the idea that Matt should be given ten grand a year by the council for being a smashing bloke I would get a large percentage saying "yes". The general public would be more likely to answer "who the f*** is Matt?"
7. The press release makes use of strong prejudicial language by describing a worry as "nuisance youths". This is mentioned twice along with two other issues thus raising it's profile and likelihood of being remembered in the mind of the reader and then the press release goes on to urge people to fill in a survey. All this after using language likely to make these issues much more prominant in the mind of the survey taker than would have normally been the case.
8. We are not told what portion of that 400 were actually worried about dog poo, young people, bikes on pavements, litter and other issues. For example if only twenty people mentioned these things then they should not be trumped as significant but the language of the press release makes it sound as if everyone was bothered about them.
I do not want to give you the idea that I am against solving problems related to young people hanging around with nothing much to focus on and the problems that arise. I am not going to tell you that dog crap on the pavement has not been an on going problem for as long as anyone can remember. It is a problem that has not gone away since the 1980's which is the earliest that I am able to personally recall such things. Watching out for dog-poo is part of walking in Thanet and has been for many, many years (at least 20 years).
What upsets me is that this looks like it is one of two things: (1) a badly managed fact finder - or - (2) an attempt to push an agenda to make it appear to be in response to public opinion. Which one you believe boils down to how skilled at manipulation you feel the authorities behind this actually are.
Think you can do better? · read more items categorised Politics · Don't miss a thing: Subscribe Today · Help: what's a feed? · This site is operated on set principles of policy and disclosure read them here
Waste and recycling collections may be suspended say council 
