The smokescreen of stats in medicine, and how to reduce crime

Here is a s

uperb article about the failure of doctors to understand statistics, especially in screening, which leads to completely wrong recommendations. 

Do doctors understand test results?

http://www.bbc.com/news/magazine-28166019

Gigerenzer, the clever guy at the centre of this article, suggests a standard set of questions to ask your doctor after he/she has prescribed whatever treatment/screening they urge you to do:

What are the alternatives?

What's the benefit and what's the harm?

Please tell me this in terms of absolute numbers. If 100 take this medication and 100 people don't, what happens after five years?

I am particularly struck by the third question.  I think this would be a lovely thing to apply to education research papers.

If you take the effect our English literacy program ReadingWise English has on 11-year-olds who have reading problems, and increase the term to say, 10 years, the answer is likely to be something like this:

If a struggling 11 year old does our program, he's going to have an average bump of 1 year in reading age.  If his pre-programme reading age was 9, you would expect him to go through secondary school learning very little, go into unskilled employment.  Or be chronically unemployed.  And the chances of the chronically unskilled ending up in prison after 10 years are x per 1000.

Whereas the bump of just one year in reading age would be enough to give him/her the confidence to actually pay attention to his/her secondary schooling, and get at least some vocational training, and therefore his/her chance of ending up in prison would be y per 1000.

So what are x and y?

There is a study by researchers at Northeastern University in the US which used a range of census data to find this out.  They concluded that "about one in every 10 young male high school dropouts is in jail or juvenile detention, compared with one in 35 young male high school graduates." 

If we extrapolate these numbers, we would get:

• If 100 stuggling readers do NOT do ReadingWise English, 10 end up in trouble with the law.
• If 100 struggling readers DO do ReadingWise English, 3 end up in trouble with the law.

That is a massive difference. 

Yes, yes, you are saying, but that's the USA, not the UK. 

You're right, and I need to research these numbers for the UK, BUT I think it's an entirely predictable that a short sharp boost in literacy for struggling readers in UK secondary schools could make a serious impact on the crime rate.

The conventions of Randomised Control Trials are too limiting

I just read a research paper slamming "synthetic phonics" and extolling "whole language".

Why do academics have to come down on one side or another like they are supporting a football team?

This to me epit​omises a fundamental problem that wrecks scientific research - we can't cope with cognitive dissonance.

Our culture requires us to assess stuff on one scale from good to bad, and then come up with a binary conclusion - it's good, or it's bad. 

What's wrong with a curate's egg - good in parts, bad in other parts?

Synthetic phonics (one of my pet subjects) has lots of good points.  "Whole language" (the other team) has lots of good points.  They are not mutually exclusive. 

The problem with most research is it's looking for a holy grail of "your one single hypothesis is probably true".   Everyone bow to the Probability Deity.

It's present in our justice system - a person is either guilty or not guilty.  What if he's partially guilty?  I know you can't be partially pregnant.  But you can be partially guilty.  You can be partially right.  You can be an almost complete shit and still be nice to your mother.

This problem besets research into vital development procedures which effect the lives of millions.

Some very smart people from MIT, I think it was, did a study which found that the benefits of microfinance initiatives are not proven.

That is so not helpful.

What we need is to know all the aspects of microfinance that are beneficial, and to what degree.

And what aspects of microfinance as it is now practiced are not beneficial, and what can we do about it.

We ourselves (ReadingWise) just did an RCT where we came up with the probability that our literacy program worked.  (It did, and remarkably well.)

That's all very well.

But what are its good bits, and how far do they work?  What aspects of it don't work so well?  That's what we need to know.  Shades of grey, please.  But we are trapped in an academic culture that disapproves of that.  As it so happens, I'm writing a supplementary paper where I can go into these shades of grey.  But to get a research trial published, you have to follow a very restrictive set of conventions which just don't help illuminate the issue you are trying to explore.

So please, academics, it's time to drop this convention of proving just one hypothesis with one parameter.  It's just so arbitary.  So 20th century.  

Test yourself - is Luis Suárez (the footballer with a habit of biting people) a good or bad person?  You probably thought bad, because he bites people.  But maybe he looks after his Mum and donates lots of money to orphanages.  See - nothing is completely black and white.

  

Sociopathy in the workplace and family

There is a superb article about prosperity and the market by Hanauer and Beinhocker in Democracy Journal:

http://www.democracyjournal.org/31/capitalism-redefined.php?page=all

In summary, it addresses the following problem.  Given that  markets are not perfectly efficient, and people are not perfectly rational, and money is a poor measure of prosperity, how do you measure prosperity, and how do you increase  it?  They suggest solutions are the markers for prosperity, not financial wealth.  So Western society is prosperous because it has medical solutions, transport solutions etc etc.

All very nice, and I basked in a glow of intellectual satisfaction as I read it.

Then in the comments section, I saw the following by a chap called George Wells:

The article fails to consider the effect of sociopaths and their behavior. The sociopaths have convinced us that corporations, and by implication, all business has a singular purpose: to make the capitalist more wealthy. Anything, and everything that stands in the way of pure profit is wrong even if it means delivering harmful products or not producing beneficial ones because they are not sufficiently profitable.

Why is it that humans reward sociopathic behavior? That, to me, is the real question.

Yeah, George!  You hit the nail on the head.

(If you think his judgement is too harsh, go read Ben Goldacre.)

I think the challenge for us chattering classes in the behavioural science arena is to solve the problem of sociopathy in the workplace and the family.

The Case for Making Criminals Literate

The total cost of crime against individuals and households in England and Wales is something like £36 billion per annum.

Assume 40% of the prison population have a serious literacy problem.   (Most studies show it higher, but we are being conservative here.)

Assume that you can make half the illiterates in prison literate at a cost of, say, £1000 per head.

There are 88,000 prisoners currently in England and Wales.

So making half of the illiterates literate would cost 88,000 * 40% * ½ * £1000.

That’s around £18 million.

Let’s assume that making half the illiterates in prison literate keeps 1% of them out of prison and reduces the cost of crime by 1%.  That’s a saving of £36 billion * 1% = £360m.

So by spending £18 million on making them literate, we save £360 million.

Not too shabby, is it?

There’s two assumptions here you might want to challenge.

Firstly, can you make half the illiterates in prison literate?  Yes, our program ReadingWise English will certainly work on at least 50% of the inmates.  Last year we did a research trial on English schoolchildren with reading difficulties, which showed that the older the child and the worse the difficulty, the higher the degree of improvement.  We have extensive experience with making adults literate in other countries (80,000 at the last count) so we know we can do it.

Secondly, can it be done for £1000?  Well, actually, it can be done for far less.  The program can be done on a computer or tablet WITHOUT INTERNET and needs to be supervised by a lay person with 2 hours training from someone certified by my company. 

So let’s say it takes 80 hours to make a criminal literate, and there are 10 prisoners in the class.  One supervisor in 50 weeks can handle 25 * 10 = 250 prisoners a year.  But let’s assume a 50% failure rate.  Then one supervisor can make 125 prisoners a year literate.

Let’s assume the lay person is paid £40k per year and equipment for the class costs £10k per year.  That’s a total of £50,000 per year.

So the cost is £50,000 / 125 prisoners = £400 per prisoner.  So why did I assume £1000 per prisoner?  Because I’d like to be paid £600 per prisoner into my own pocket on a results-only basis!

An algorithm that predicts number of prisoners from percentage of 10/11-year-olds that cannot read

From today's Guardian - an article by Neil Gaiman:

http://www.theguardian.com/books/2013/oct/15/neil-gaiman-future-libraries-reading-daydreaming

I was once in New York, and I listened to a talk about the building of private prisons – a huge growth industry in America. The prison industry needs to plan its future growth – how many cells are they going to need? How many prisoners are there going to be, 15 years from now? And they found they could predict it very easily, using a pretty simple algorithm, based on asking what percentage of 10 and 11-year-olds couldn't read. And certainly couldn't read for pleasure.

According to an article in today's New York Times, 

Dr. Mark Olfson, a professor of clinical psychiatry at Columbia University Medical Center, noted that from the time they develop major depression, it takes Americans eight years on average to seek care.

Why? Why? Why? Why? Why does it take so long?

http://www.nytimes.com/2013/06/20/business/in-head-hunting-big-data-may-not-be-such-a-big-deal.html?pagewanted=2

In an interview in the New York Times today,  a senior HR person at Google says two very interesting things.

Firstly:

Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.” The interesting thing about the behavioral interview is that when you ask somebody to speak to their own experience, and you drill into that, you get two kinds of information. One is you get to see how they actually interacted in a real-world situation, and the valuable “meta” information you get about the candidate is a sense of what they consider to be difficult.

Some of us Metapsychology therapists have known for a while that if you concentrate on exploring with Tell me a condition you have had to face, how have you handled it?, powerful stuff is unleashed, and it's a good candidate technique for handling a personality disorder. 

Secondly - and before you read this, you need to know that GPA stands for "Grade Point Average" and is how every American's college career is statistically assessed - to a fetishistic degree:

One of the things we’ve seen from all our data crunching is that G.P.A.’s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there’s a slight correlation. Google famously used to ask everyone for a transcript and G.P.A.’s and test scores, but we don’t anymore, unless you’re just a few years out of school. We found that they don’t predict anything.
After two or three years, your ability to perform at Google is completely unrelated to how you performed when you were in school, because the skills you required in college are very different. You’re also fundamentally a different person. You learn and grow, you think about things differently.
Another reason is that I think academic environments are artificial environments. People who succeed there are sort of finely trained, they’re conditioned to succeed in that environment. One of my own frustrations when I was in college and grad school is that you knew the professor was looking for a specific answer. You could figure that out, but it’s much more interesting to solve problems where there isn’t an obvious answer. You want people who like figuring out stuff where there is no obvious answer.