Topic: Less Research is Needed

[Glock32]

H/T: Small Dead Animals

This is a somewhat lengthy and academic read, but it argues an important but unpopular viewpoint: sometimes the right conclusion is that less research is needed. Virtually every study that makes it into mainstream reporting ends with the bromide "more research is needed."  The author makes the point that if repeated inquiries into a particular experimental line yield negative or ambiguous results, the proper conclusion is that perhaps something is wrong with underlying assumptions or methodology, not that there is an automatic need for more research.

As I think more on this and other matters, I have decided that "nuance" needs to be an honorary 4 letter word. It is the codeword that represents modern society's self-regard, the way it has fallen in love with its own mental abstractions be it in the fields of science, law and justice, or politics. Take the controversy surrounding global warming for instance. Virtually all of the dire warnings come from predictive models, and we've had pretty much two decades where actual observed data contradict the predictions of these models. In statistics a model's validity comes from its predictive force. The AGW crowd have a pretty poor batting average with their models, to say the least. This means the models are useless, either because they are based on faulty assumptions or are collecting variables that have minimal significance. Yet that is not the conclusion being reached, and it goes beyond the AGW debate. This is how research is being conducted in other fields too.

Are we simply training new generations of technically adroit sophists, clever and even intelligent, but the inverse of the old forest vs. trees fallacy? Rather than the classic -- unable to see the forest for the trees -- we seem to be producing people who can't see the trees for the forest. SDA excerpted the best parts of the article:

    On my first day in (laboratory) research, I was told that if there is a genuine and important phenomenon to be detected, it will become evident after taking no more than six readings from the instrument. If after ten readings, my supervisor warned, your data have not reached statistical significance, you should [a] ask a different question; [b.] design a radically different study; or [c] change the assumptions on which your hypothesis was based.

    In health services research, we often seem to take the opposite view. We hold our assumptions to be self-evident. We consider our methodological hierarchy and quality criteria unassailable. And we define the research priorities of tomorrow by extrapolating uncritically from those of yesteryear. Furthermore, this intellectual rigidity is formalized and ossified by research networks, funding bodies, publishers and the increasingly technocratic system of academic peer review.

    [...]

    Whereas in the past, any observer could tell that an experiment had not 'worked', the knowledge generated by today's multi-variable mega-studies remains opaque until months or years of analysis have rendered the findings - apparently at least - accessible and meaningful. This kind of research typically requires input from many vested interests: industry, policymakers, academic groupings and patient interest groups, all of whom have different reasons to invest hope in the outcome of the study. As Nic Brown has argued, debates around such complex and expensive research seem increasingly to be framed not by régimes of truth (what people know or claim to know) but by 'régimes of hope' (speculative predictions about what the world will be like once the desired knowledge is finally obtained). Lack of hard evidence to support the original hypothesis gets reframed as evidence that investment efforts need to be redoubled.[2] And so, instead of concluding that less research is needed, we collude with other interest groups to argue that tomorrow's research investments should be pitched into precisely the same patch of long grass as yesterday's.

Do read the rest: http://thehealthcareblog.com/blog/2012/06/25/less-research-is-needed/

[Weisshaupt]

H/T: Small Dead Animals
  The author makes the point that if repeated inquiries into a particular experimental line yield negative or ambiguous results, the proper conclusion is that perhaps something is wrong with underlying assumptions or methodology, not that there is an automatic need for more research.

But how will I justify the continuation of my grant?

[Alphabet Soup]

In health services research, we often seem to take the opposite view. We hold our assumptions to be  self-evident.

With a generation (or more!) of skulls~full~of~mush indoctrinated in outcome-based education how can anyone expect anything different?

Even in my generation I was exposed to this drivel. We were encouraged to claim a premise (say, "The moon is made of cream cheese"), state it in the form of a hypothesis (which is really only formalizing the language of a wild-assed guess) and then devise some structural tests that would confirm that guess. ("Let's see....the best way to confirm my hypothesis would be to sample the moon's surface. But it would be too  expensive to do that. So instead I will compare pictures of the moon to some pictures I took down  at the cheese shoppe. See - they are identical, thus proving that the moon is indeed made of cream cheese!").

Brilliant. 

[LadyVirginia]

YES! YES! YES!

The underlying assumption is that the hypothesis is correct and it's up to the researcher to prove it.

Pretty much anything the government throws money at falls under this.  Public education is a perfect example.

I'm for less research!