Books

How Science Should be Done

Science and academic life at large have changed out of recognition since the Second World War under the influence of rapid growth, increasing government control and the politicisation of academia and the allocation of research grants. And so the world of science is in a bad way and academic studies in the philosophy and social studies of science have not helped.

The philosophy of science took an unhelpful turn in the 1930s when the subject became an academic specialty and the school of thought known as logical positivism or logical empiricism became embedded in the universities of the Anglosphere when the leading European exponents occupied prestigious chairs after they fled from Hitler. The most popular reactions to that orthodoxy were Kuhn’s paradigm theory and social studies of science but they were equally unhelpful because they became dominated by cultural Marxists and fellow travellers. 

The Scientific Method is not a contribution to the academic literature—it is much more important and helpful than that. It is actually more than one book, in a single set of covers. One of the books is a practical handbook or an operating manual for serious scientists. In the language used by Gordon Tullock in his classic work The Organization of Inquiry (1965) the authors’ curiosity is driven by the quest for truth or practical value, unlike the “normal scientists” whose curiosity is induced by their conditions of employment.

Scott Armstrong has dedicated his life to discovering useful scientific results. He has spent fifty-two years as a professor at the Wharton School at the University of Pennsylvania with sorties to deliver guest lectures at universities in twenty-seven countries. Kesten C. Green moved into academia at the University of Adelaide to follow his interest in research after thirty years as an entrepreneur. From the window of his home he looks into the grounds of the Waite Agricultural Research Institute, where I did my first postgraduate research after moving from the University of Tasmania to Adelaide. The Waite Institute is co-located with the CSIRO Soils Division and a generation ago this was a world-class centre of excellence in rural research, before the universities and the CSIRO were taken over by content-free managers with political agendas.

The authors begin with a survey of the problems that afflict science at present and they end with practical suggestions for improvement that can be taken up by the range of stakeholders in the scientific enterprise. There are chapters on assessing the quality of scientific practice, the problem of advocacy, concerns with the effectiveness of peer review and the complications that arise with government funding. The positive suggestions are offered to university managers, journal editors, governments, courts, the media and interested individuals.

The core of that part of the book is a series of checklists to help scientists to navigate on the journey from the beginning of a research career through the stages of research projects to the publication and dissemination of the findings. All of the elements of the process should become second nature to researchers who are well taught and well supervised early in their careers, but unfortunately not enough supervisors bother to master the art and science of supervision.

The first list is “self assessment of self-control” in a chapter on “what it takes to be a good scientist”. The prospective scientist is advised to think hard about the pros and cons of a research career. Armstrong was influenced by the 1925 novel Arrowsmith by Sinclair Lewis, which could be prescribed as a text on the trials and tribulations of idealistic researchers.

Arrowsmith, the hero of the book, is a radical and independent medical researcher who adheres to strict principles of scientific method. The central drama of the book is his discovery of a special kind of cell in the blood that destroys bacteria and his dilemma in the face of an outbreak of bubonic plague on a Caribbean island. His scientific principles demand that he should test the therapy before its mass use on the island, even at the expense of lives that might be saved. There is a message for our public health officials, especially when the disease in question is not a plague but a variant of influenza.

As for the exhilaration of advancing the frontier of knowledge, the authors cite a survey of students in the doctoral programs in economics at eight leading universities in the US. The survey found that 18 per cent of the students experience moderate or severe depression and 11 per cent think about suicide in a two-week period. Not surprisingly, economics is dubbed “the dismal science”.

There is a short checklist on identifying important problems and a long list of things to attend to in planning and executing data collection and analysis. After drawing out conclusions, the scientist turns to disseminating the findings. There are progress reports and seminar papers on preliminary findings along the way but the critical products to maintain tenure and ongoing grants are papers published in peer-reviewed literature. Hence the talk about “publish or perish” that has been a refrain since the 1960s.

The checklist starts with “Explanation of findings and why they are credible and useful”. It has to be said that many claims of usefulness in the softer sciences tend to strain credibility. These are the projects that regularly attract criticism from conservative critics of the major grants allocated by bodies such as the Australian Research Council.

The largest checklist concerns writing the paper. By the time researchers get to this point they should be thoroughly familiar with this particular literary form, but familiarity does not guarantee that the beginner or even experienced scientists will do a good job. The most important suggestions are at the end of the list: “Use editors to improve clarity and rewrite until the report is clear and interesting.” By editors I presume they mean colleagues, preferably experienced in report-writing.

At the Waite Agricultural Research Institute in the 1960s there was a valuable practice to have all papers rigorously reviewed by two people using the kind of scrutiny that the paper would receive from referees at the journal when it was submitted. One reviewer was drawn from the specialists in that field to check the technical aspects of the paper, while the other was selected from a neighbouring field to see if it was written clearly enough to be understood by people outside the specialty.

The last three checklists are concerned with sales and marketing. Some of the most conscientious and committed truth-seekers overlook this work because they find it distasteful or beneath the dignity of scholars. It certainly calls for a very different skill set from research itself and neglecting it probably didn’t matter so much a generation ago when there were fewer scientists and good work would usually be picked up by influential colleagues and given due attention including all-important citations. This is no longer the case unless researchers are fortunate with their colleagues, otherwise the necessary connections will have to be reached by well-organised efforts to contact and cultivate them.

The book is clearly written. It contains the fruits of many years of dedicated work in research and the evaluation of research findings. This makes it a valuable addition to the shelf of working scientists and anyone else with a serious interest in science as a source of knowledge or a social institution.

The Scientific Method: A Guide to Finding Useful Knowledge
by J. Scott Armstrong & Kesten C. Green

Cambridge University Press, 2022, 200 pages, $43.95

Rafe Champion is an independent scholar who lives in Sydney. His books include collaborations with Ruth Park

11 thoughts on “How Science Should be Done

  • STJOHNOFGRAFTON says:

    The Scientific Method is elegant yet simple. Its steps are easy to follow but rigorous in their prosecution. The Scientific Method is, or uesd to be taught early on in high school, usually in year seven. The right stuff of good education. But that’s usually where it ended because what usually followed on was some kind of dogma based on the Theory of Evolution which may have gone something like this in the classroom: “Well, that’s the Scientific Method covered, our next topic is how complex animal life arose over billions of years from the primeval slime which resulted when lightning struck primitive gases like methane and ammonia to form the first soup of proteins”. All plausibly deniable of course, but a subtle paradigm shift has occured. The Theory of Evolution has become the Law of Evolution and our students have gone from being taught how to think to being taught what to think. This is a dangerous flaw in education because it short circuits critical thought and dumbs down the enquiring mind. As a result, The Scientific Method has become a forgotten chapter in the Educational Cookbook of Mass Indoctrination.

    • rosross says:

      Eloquently put. Modern academia betrays science and its methodology in ways previously unknown. And the corruption increases because science is no more than a tool of vested agendas. Most medical research is funded by the pharmaceutical industry which will never be objective or indeed, truly scientific. Fox in charge of the henhouse writ large. Covid demonstrated that in spades.

  • Daffy says:

    It was once the role of the scientist to be sceptical: of his or her peers, opponents, and self. Now we have science as the endeavour to quash scepticism and boost the party line, the ‘consensus’ so-called. A consensus can look after itself, what we need to advance knowledge is sceptical inquiry that brooks no hesitation or opposition.

    • rosross says:

      Science, real science, does not do consensus. Science works to proves theories but continues to embrace questions and so theories change.

      Having said that, any reading of science history and medical history, makes it clear that change is slow and often it takes a century or more for a theory to be even considered, let alone accepted. Science is a system run by mere mortals and they defend their ‘turf’ to their death as often as not.

      The German physicist Max Planck said that science advances one funeral at a time. Or more precisely: “A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.”

  • rosross says:

    While not covered in the article but possibly covered in the book, there are other factors which compromise the scientific system of enquiry today.

    1. Science sold its soul to Government and corporate interests long ago and became a profit-driven industry instead of an information gathering resource.

    2. More than a century ago the scientific system of enquiry set itself up as an alternative to religion. Unfortunately it set itself up not against the best of religion but the worst, fundamentalism, i.e. that which could easily be decried. As is the wont with humans, for science is nothing more than a system invented and run by humans, the shadow effect came into play with such stark polarity and increasingly science became in attitude and form, unconsciously, that which it condemned, fundamental and intolerant in nature. It became in essence a new form of religion, in orthodox approach, with all the flaws that entails.

    3. Science is a system of enquiry and it can only know, not understand, just know, that which it can measure. This is extremely useful where things can be reduced to the material and mechanical and utterly useless if not destructive where they cannot. And where they cannot is a realm which larger than that where things can be reduced to the material and mechanical. This tunnel vision means the scientific system lost perspective and that has increased even more so as the whizz-bang, techo-toy, gizmo successes of the scientific system has fed hubris and encouraged ignorance.

    4. Any system founded in ego, pride and elitism, as has been the case for the scientific system of enquiry will fall and fall hard at some point and more so when profit is the determinant.

    We need science, but we need good science, and as we have seen in the time of Covid, and in the time of Climate change theories, science has become perhaps the most dangerous system at work in the world, and not the beneficial system it believes itself to be.

    • ianl says:

      Full of straw men, that comment (as are the others above it). Far too many to list, but a few of the most egregious:

      1) careless confusion between the concepts of hypothesis and theory. And then confounding this with the even more careless use of the concept of “Scientific Law”. These concepts are very well defined and conflating them is destructive. As Rafe C says – read Popper;

      2) what we cannot (yet) measure cannot be understood – it can be hypothesised, conjectured, tested. Claiming “understanding” of things so far immeasurable is the trademark of propaganda;

      3) that some (perhaps many) scientists have attempted to cheat these simple, straightforward guidelines does not tarnish the scientific method, but merely presents the commonality of homo sapiens’ baser nature.

      A horrid example in my own sphere of geoscience was the deliberate efforts of a Professor of Geology to “find” critical fossils exactly where he had predicted they should be. This serendipity occurred so often that eventually he confessed to the deceit – but the truly horrid part was that he was unable to remember all the times he had done it. The geological database was to remain corrupted at unknown points, or else everything he’d published was to be expunged. Yet the discipline of geoscience remains an integral part of scientific method because attested field evidence keeps mounting;

      4) my own personal observation: the segment of scientific method most disliked by arm waving straw men is the end activity, where an hypothesis is rigorously tested with prediction. Why disliked ? Because it may destroy the hypothesis, and thereby damage vanity and perhaps the flow of research funds. “Climate Science” is right upon this junction and has been for thirty years or so.

      • rosross says:

        You said:
        1) careless confusion between the concepts of hypothesis and theory. And then confounding this with the even more careless use of the concept of “Scientific Law”. These concepts are very well defined and conflating them is destructive. As Rafe C says – read Popper;

        I have read Popper and the point is modern science dismisses and in essence does not tolerate philosophy. The issue is not me or others reading and appreciating Popper but the fact scientists do not. There is no conflation just a recognition of that reality.

        You said: 2) what we cannot (yet) measure cannot be understood – it can be hypothesised, conjectured, tested. Claiming “understanding” of things so far immeasurable is the trademark of propaganda;

        No, making the point that the scientific method measures but measurements do not mean understanding is valid and a limitation to the scientific method. There is nothing wrong with that as long as we factor in that limitation. Science can only measure and that provides data but not necessarily understanding. As an example, the effects of gravity can be measured and effectively so but science does not understand what gravity is, nor in any substantive way, how it works, let alone Why.

        You said: 3) that some (perhaps many) scientists have attempted to cheat these simple, straightforward guidelines does not tarnish the scientific method, but merely presents the commonality of homo sapiens’ baser nature.

        you miss the point. I suppose you must. It is not that some or many scientists fail on this count but that the modern system of science has no place for it and has thus become increasingly flawed and dangerous. The system is now designed to cheat such critical guidelines.

        You said: 4) my own personal observation: the segment of scientific method most disliked by arm waving straw men is the end activity, where an hypothesis is rigorously tested with prediction. Why disliked ? Because it may destroy the hypothesis, and thereby damage vanity and perhaps the flow of research funds. “Climate Science” is right upon this junction and has been for thirty years or so.

        One could make a case that the modern scientific system of enquiry is so often devoid of rigour that little can be trusted and as you cite, never more so than with Climate science. I would add, much of modern medicine, particularly vaccines falls into the same category.

        As long as the system is corrupt, inefficient and unreliable then so will be those who work in the field. Fix the system and you have some hope of restoring real science.

  • Rafe Champion says:

    Question. Where are Karl Popper’s ideas when we need them?

    Answer. Here. https://www.amazon.com/s?k=rafe+champion&ref=nb_sb_noss_1

  • talldad says:

    Just bought a copy from Booktopia.

    Many thanks for the review, Rafe

  • Macbeth says:

    It’s over ninety years since on my way to school, I walked past what we called Peter Waite’s paddock.

  • guilfoyle says:

    There is a disconnect in philosophy and science. The universities often emphasise modern philosophers – who may be term ideologues and scientific teaching acts as if it simply sprang from a light of ‘enlightenment’ that suddenly occurred when modern man progressed past the backward and superstitious dark fog of medieval primitiveness. This was a direct result of propaganda for reasons not necessary to go into here. However, to cut off the historical origins is to impoverish the handing on of knowledge. The ‘scientific method’ was developed, primarily, by a 13th century Benedictine monk and was developed by later, often clergy or religious, to the point that newton observed that he was standing in the shoulders of giants- not a pat, pompous meme, but the truth.
    As St John of Grafton adverted to, the adherence to the ‘scientific method’ became formulaic once science was co-opted to the ‘theory’ of evolution- which necessarily involved the polarisation of religion against science. The ‘theory’ of evolution was something that was developed and then evidence searched for/seized upon; is still not defined completely and anyone who questions or points out inconsistent facts is immediately shouted angrily down or patronised- does this sound like science, or ideology? The theory of evolution is desperately needed by the materialists of the world and no opposition will be countenanced. So scientific totalitarianism didn’t simply arise with Covid – it arose in the 1850’s with the theory that is still looking for supporting evidence.

Leave a Reply