Most people are familiar with most of the basic elements of the scientific method: Observe a phenomenon, form a possible explanation, make predictions following from that hypothesis and design experiment after experiment to test those predictions until you have reasonable evidence to support (or disprove) your idea. This process represents humanity’s best attempt to rid ourselves of the baggage of our egos; our biases, our experiences, our preconceived intuition of how things should be, and try to get at the truth of how things really are. Though not everyone may acknowledge or accept this i.e. post-modernists.
However, people may be less familiar with the next steps in the chain: Peer-Review and Publication. This is, in principle at least, more important than a hoop to jump through and something to pad a scientist’s CV. Openness is at the heart of science and it has to be that way, for several reasons.
One is that this task of bypassing our own assumptions and intuitions is too large a task to be left to one person, or group of people. The person whose hypothesis it is, who did the experiment, is too close to the matter. Not only because of the rather common desire to be right (though there are many stories in science of people happy to be proven wrong), but also because if you are immersed in the nitty-gritty details of the work you may not be able to see the whopping great flaw in your logic, methods, assumptions etc. It may take someone looking at the situation from a completely different field to see the figurative pink elephant in the vacuum chamber.
Another one, that (in my opinion) is often overlooked in the academic pressure to publish new results, is that people need all the details of your methods, results and conclusions if they are to repeat your experiment. It’s not enough for you to repeat your experiment 3 million times if there is something wrong with the equipment: a dodgy fibre-optic cable, perhaps? Someone else has to do the same (or similar) thing in a different place and get the same results for people to be really convinced that you are truly examining something about the universe and not the effects of stray fields from the microwave oven. This is also the way that scientists police themselves and catch frauds as in the infamous case of Jan Hendrik Schon.
The key point is that truly open space for ideas to be challenged and attacked is necessary to build a respectable consensus. Nothing can ever be completely proven beyond any doubt. Eternal Truth, with a capital T, doesn’t exist and to do science one must accept that all knowledge is provisional. Any theory, no matter how successful, can be overturned if enough evidence and experiments present themselves. Of course, that begs the question: Who decides what is enough evidence? The short answer is no one. If you want your idea to be taken as a serious explanation of natural phenomena, you need to convince people of it. This is not a democracy, you can’t buy, win or charm votes. It’s a meritocracy and the only thing that counts is the strength of your evidence (this is what disqualifies creationists, climate denialists, homeopaths and other bullshit peddlers from the game).
You have to throw your work to the wolves and let them tear at it. If your work stays standing, if the work of other scientists stand in support of it, then you may convince enough people to form a “consensus”. This is what draws the line on the current moment saying “This is the way it is (we think).”
Now, this is probably all very idealistic and is definitely very idealised. I’ve heard enough horror stories from colleagues of slow editors, non-sensical replies from referees and preference for buzzword articles. I well aware that the current academic publishing system is not perfect. It suffers from numerous flaws, such as publication bias, and often seem the driving force behind scientists more than the science itself. However, I wrote what I did above to declare, and to reaffirm to myself just how important openness is to science and the key role peer-reviewed journals play in that.
Think of it something like how, after watching the news and seeing the morons who get elected to government, you might need to think about how democracy should work to motivate your arse to go out and vote.
The reason for this affirmation and this post is a couple of interesting issues pertaining to Openness and the current system of Peer-Review.
The first concerns the production of mutant strains of avian flu. Those words are strong enough to make some people uncomfortable already. The science involved and what can be discovered from these is bound to be amazing and of very high impact, though I’m not a biologist so cannot comment too much. The question is: should such a study be published in full with all the details of how these strains were created? Remember, part of the reason these details are revealed in most papers is so the results can be replicated. However, the costs of a functioning microbiology lab aren’t what they used to be and there are a lot of angry/deluded people out there with very malleable consciences.
The details of this issue can be read here at Nature.
This raises the question about “How open is too open?” I sincerely believe that no area of science should ever be put off-limits. It is a necessity that people explore every aspect of life, the universe and everything. Even if we may never get any answers, even if we strongly suspect we never will, we can never let ourselves give in to it. Just as we will never know everything, we can never truly know what’s “unknowable,” and the moment we think we do, science stops! If, for example, we could never understand or explain what happened in the first nanosecond after the Big Bang we still have to keep trying. If we stopped, shrugged and said it was unknowable, we might not find out what happened in the second nanosecond or the third. Each piece of knowledge is always worth the effort to find it and if we box it off as impossible to know, that effort will grind down to a stop. This, incidentally, is a large part of why religious “answers” are so ultimately unsatisfying.
However, this does not mean that every result needs to be shared. Most (sane) people would accept the need to keep certain things secret, in military circles for example. No one expected people on the Manhattan Project to be publishing their results. So what happens if everyday academics “stumble” across military level science, or at least potentially dangerous stuff? If I strain and ponder a bit, I could probably think of a few other things that maybe humanity might be better off not know if the data existed (well, maybe just the one). Should there be limits on this academic openness?
To be honest, I don’t have a bloody clue. I didn’t know when the issue of this research came up a few months ago and my wife asked my opinion. I can see that there would be good reasons to withhold some data or some results because of potentially dangerous implications. You could also probably guess my reluctance based on how important I feel the spirit of openness is to the scientific process. My cynical nature also makes me suspicious on what criteria would be used to judge what was “Too dangerous” and who would do the judging. As I said, the scientific community needs open discourse, in part, to try to overcome the short-sighted egos of individuals and small groups. It seems counter-productive to this effort to hand censorship power to which ever group is set up for the task. Also, just as we can’t box away the “unknowable” due to the risk of losing knowledge, can we box away the dangerous and risk losing research findings that could potentially be of great benefit to society?
It’s above my pay grade to know such things, but I guess it’s probably a good thing that the WHO recommended that the paper be published in full.
It is interesting to note that the scientist’s doing the work did decide to halt their endeavours to allow the debate concerning the potentially risks and benefits of this line of research.
The second line of thought is linked to a growing debate about the business of peer-reviewed publications. The full-story can be read in this APS news piece. The cliff note version is: There are two bills dueling in the US House of Representatives; one seeking strike down and one seeking to expand, current laws stating that research article, stemming from public funding via the NIH must be made publicly available on PubMed within a year. This is rubbing some of the journal publishers the wrong way, as they make their money by charging institutions for access to the articles. This political debate is stirring the growing sense of disgruntlement within the university community concerning the role of private businesses in academic publishing.
The details are well-summed up in the APS article I linked to. For my part, although I admire the desire to make scientific results available to the public, I do wonder is full-access to the article is necessary or even preferable. If, we only consider the public angle, i.e. the responsibility to let the taxpayers know what is being done with their money (what, why and what has been found) perhaps lay-summaries would be a more suitable approach. The simple fact is that scientific literature requires a lot of specialist knowledge, familiarity with technical jargon, statistical measures of significance etc. More or often than not, I don’t feel qualified to understand papers in my own field. If scientists were to write summaries or abstracts of their work specially aimed at a lay-audience; detailing what they found, how they found it, possible areas of future research and (perhaps most importantly) why it matters, this may be more effective at informing the public than simply putting the paper online with no further comment. The Cochrane Collaboration, stalwart defenders against quackery, seems to be leading the way in such summaries. Other advantages to such a policy, would be forcing scientists to write with the aim of engaging the public in their work, something far too many in all fields seems to neglect if not openly disdain. One would also hope that it would avoid public figures sneering at things like “Fruit-fly research,” without a clue of how valuable such work is.
However,there is something very wrong with the current publishing status quo. Currently, universities not only have to pay large sums in subscription fees to access scientific journals, but academic must also pay to have their work published in those journals. The authors cede the copyright of their work, data and images to the publishers. Also, the scientists who serve as referees in the peer-review process for these journals receive no payment for this effort, it is entirely voluntary.
A good run-down of the problem was written by George Monbiot in the Guardian.
Though I know that the staff of such publishers put in a lot of hard work, I can’t help but feel the current arrangement reads like something straight out of Marx, with one group responsible for a large part of the means of production and another reaping the profit. The publisher’s themselves also seem to resort to the tried-and-true business defence form “We agree but don’t live the government forcing us to do things or getting involved with out business.” Please! Where do they think the money for the subscriptions come from? The same public funds that are the source of research budgets and other university costs. At the very least, the current system seems to be a giant waste of money that could be better spent in hiring scientists, equipment or in improving teaching.
It’s also help if publishers would stop bringing out all those extra titles, allowing them to charge more for their various bundles (with access to different journals grouped like access to cable channels) . Surely, if something was not suitable to be published in Nature or Nature Physics etc. someone else should get to publish it rather than being bumped down the road to Nature Comms
Can’t say I know the answer. That’s well-beyond me. I’m not sure a world archive of peer-reviewed journals is practical in a non-Utopian world. Any global collaboration seems to get quickly bogged down by politics and division, the integrity of the scientific method and peer-review needs no be kept clear of such things (though that itself may be only possible in Wonderland). Also, quality products need money so charging for something in the process is not unreasonable. However, do universities have to pay quite so much? Or to pay to be published and to read? Must academics also sacrifice the copyrights on their work and data to the publishers too??
Tricky problem. Something about this needs to change, but I’m glad I’m not the one who has to solve it.