Almost six months ago I wrote a post about the report of a bacterium collected from Mono Lake that could substitute arsenic for phosphorus in such crucial biological molecules as DNA. The announcement raised the possibility that living things could construct building blocks quite different from those we were used to seeing.
However, in the weeks and months following the initial publication of the research report online in Science Express, several scientists began to share their concerns and criticisms online. Blog posts with comments shot back and forth. People raised their concerns about the methodology of the arsenic group and complained about NASA's over-the-top PR. Some researchers wrote letters to Science , which sent them out for review, eventually accepted them for publication, and asked the original researchers to provide a response. Eight comments and the response appeared online yesterday.
This is all good. It's the way science is supposed to work. Ideally, major holes and problems with a research paper will be addressed by reviewers before the paper is published, but after publication there will always be other scientists with special expertise who can offer their own commentary. This is the process of post-publication review, made more powerful by the way online communities allow a doubter to spread the word. Questions and criticism often spur the original authors either to do more experiments to confirm their findings or to change their hypotheses to address the objections.
Science is far from perfect, but it earnestly works to perfect itself, learning from mistakes along the way. This doesn't seem to happen with, say, the social theories behind public policy. If only there were such an efficient and agreed-upon way to address flaws in social theories thrown too quickly before the public, or a way to dispassionately point out when enacted policies fail to produce the expected results! Then maybe politics could progress, erratically but always tending forward, as science does.