How Science Should be Done

Science and academic life at large have changed out of recognition since the Second World War under the influence of rapid growth, increasing government control and the politicisation of academia and the allocation of research grants. And so the world of science is in a bad way and academic studies in the philosophy and social studies of science have not helped.

The philosophy of science took an unhelpful turn in the 1930s when the subject became an academic specialty and the school of thought known as logical positivism or logical empiricism became embedded in the universities of the Anglosphere when the leading European exponents occupied prestigious chairs after they fled from Hitler. The most popular reactions to that orthodoxy were Kuhn’s paradigm theory and social studies of science but they were equally unhelpful because they became dominated by cultural Marxists and fellow travellers. 

The Scientific Method is not a contribution to the academic literature—it is much more important and helpful than that. It is actually more than one book, in a single set of covers. One of the books is a practical handbook or an operating manual for serious scientists. In the language used by Gordon Tullock in his classic work The Organization of Inquiry (1965) the authors’ curiosity is driven by the quest for truth or practical value, unlike the “normal scientists” whose curiosity is induced by their conditions of employment.

Scott Armstrong has dedicated his life to discovering useful scientific results. He has spent fifty-two years as a professor at the Wharton School at the University of Pennsylvania with sorties to deliver guest lectures at universities in twenty-seven countries. Kesten C. Green moved into academia at the University of Adelaide to follow his interest in research after thirty years as an entrepreneur. From the window of his home he looks into the grounds of the Waite Agricultural Research Institute, where I did my first postgraduate research after moving from the University of Tasmania to Adelaide. The Waite Institute is co-located with the CSIRO Soils Division and a generation ago this was a world-class centre of excellence in rural research, before the universities and the CSIRO were taken over by content-free managers with political agendas.

The authors begin with a survey of the problems that afflict science at present and they end with practical suggestions for improvement that can be taken up by the range of stakeholders in the scientific enterprise. There are chapters on assessing the quality of scientific practice, the problem of advocacy, concerns with the effectiveness of peer review and the complications that arise with government funding. The positive suggestions are offered to university managers, journal editors, governments, courts, the media and interested individuals.

The core of that part of the book is a series of checklists to help scientists to navigate on the journey from the beginning of a research career through the stages of research projects to the publication and dissemination of the findings. All of the elements of the process should become second nature to researchers who are well taught and well supervised early in their careers, but unfortunately not enough supervisors bother to master the art and science of supervision.

The first list is “self assessment of self-control” in a chapter on “what it takes to be a good scientist”. The prospective scientist is advised to think hard about the pros and cons of a research career. Armstrong was influenced by the 1925 novel Arrowsmith by Sinclair Lewis, which could be prescribed as a text on the trials and tribulations of idealistic researchers.

Arrowsmith, the hero of the book, is a radical and independent medical researcher who adheres to strict principles of scientific method. The central drama of the book is his discovery of a special kind of cell in the blood that destroys bacteria and his dilemma in the face of an outbreak of bubonic plague on a Caribbean island. His scientific principles demand that he should test the therapy before its mass use on the island, even at the expense of lives that might be saved. There is a message for our public health officials, especially when the disease in question is not a plague but a variant of influenza.

As for the exhilaration of advancing the frontier of knowledge, the authors cite a survey of students in the doctoral programs in economics at eight leading universities in the US. The survey found that 18 per cent of the students experience moderate or severe depression and 11 per cent think about suicide in a two-week period. Not surprisingly, economics is dubbed “the dismal science”.

There is a short checklist on identifying important problems and a long list of things to attend to in planning and executing data collection and analysis. After drawing out conclusions, the scientist turns to disseminating the findings. There are progress reports and seminar papers on preliminary findings along the way but the critical products to maintain tenure and ongoing grants are papers published in peer-reviewed literature. Hence the talk about “publish or perish” that has been a refrain since the 1960s.

The checklist starts with “Explanation of findings and why they are credible and useful”. It has to be said that many claims of usefulness in the softer sciences tend to strain credibility. These are the projects that regularly attract criticism from conservative critics of the major grants allocated by bodies such as the Australian Research Council.

The largest checklist concerns writing the paper. By the time researchers get to this point they should be thoroughly familiar with this particular literary form, but familiarity does not guarantee that the beginner or even experienced scientists will do a good job. The most important suggestions are at the end of the list: “Use editors to improve clarity and rewrite until the report is clear and interesting.” By editors I presume they mean colleagues, preferably experienced in report-writing.

At the Waite Agricultural Research Institute in the 1960s there was a valuable practice to have all papers rigorously reviewed by two people using the kind of scrutiny that the paper would receive from referees at the journal when it was submitted. One reviewer was drawn from the specialists in that field to check the technical aspects of the paper, while the other was selected from a neighbouring field to see if it was written clearly enough to be understood by people outside the specialty.

The last three checklists are concerned with sales and marketing. Some of the most conscientious and committed truth-seekers overlook this work because they find it distasteful or beneath the dignity of scholars. It certainly calls for a very different skill set from research itself and neglecting it probably didn’t matter so much a generation ago when there were fewer scientists and good work would usually be picked up by influential colleagues and given due attention including all-important citations. This is no longer the case unless researchers are fortunate with their colleagues, otherwise the necessary connections will have to be reached by well-organised efforts to contact and cultivate them.

The book is clearly written. It contains the fruits of many years of dedicated work in research and the evaluation of research findings. This makes it a valuable addition to the shelf of working scientists and anyone else with a serious interest in science as a source of knowledge or a social institution.

The Scientific Method: A Guide to Finding Useful Knowledge
by J. Scott Armstrong & Kesten C. Green
Cambridge University Press, 2022, 200 pages, $43.95

Rafe Champion is an independent scholar who lives in Sydney. His books include collaborations with Ruth Park