Hanging on to the Edges

Front matter

Essays on Science, Society and the Academic Life

DaNieEL NETTLE

To access digital resources including: blog posts videos online appendices and to purchase copies of this book in: hardback paperback ebook editions

Part Three 11. 12. 13. 14

Waking up and going out to work in the uncanny valley Staying in the game

Acknowledgements Index

215 231

247 249

Introduction

Those with any imagination and understanding are filled with doubt and indecision.

-Bertrand Russell, New Hopes for a Changing World?

This is, in many respects, an anti-book. Books have a clear, unitary central message. The message is set out clearly in the opening chapter; seen growing up, fighting off rivals and doing all kinds of good deeds in a series of episodes in the middle; and then triumphantly restated at the end. Books come from certainty and self-confidence: the world is simpler than you thought! Anti-books, on the other hand, grow from critical self-reflection, compromise, and doubt. They cross and re-cross a complex landscape, trying to see its features from as many angles as possible, pointing out commonalities and false friends, abandoning one path and trying another. Their central message, if there is one at all, cannot be summarised in a sentence, but perhaps emerges, unsuspected, from an entanglement of detailed local engagements. It is a set of value commitments as much as a claim.

In 2016 I realised, with some alarm, that I had been working for over twenty years (twenty years!) at the interface between the biological and the social sciences, trying to cross the gulf that still tends to separate those two great human endeavours. What conclusions did I have from all this effort? None clear enough, right now, for a book; but plenty for an anti-book. I had been downcast for years that where other people had grand, bold theories or sweeping claims to make their names

1 Russell, B. (1951). New Hopes for a Changing World (London: Allen & Unwin, p. 5).

with, I did not. I had a lot of reading and thinking behind me; a lot of experimentation with different methods and ideas, without entirely nailing my colours to any of them; a lot of ‘both sides have useful things to contribute’ sentiments; a lot of reasonably good-humoured scepticism; and a great deal of respect for the craft. Running through all this was a diffuse sense of slight disappointment: in private moments, I could see that none of the theories espoused out there in the literature, especially those espoused by me, quite lived up to their promise. The big breakthrough had not quite come. When was I going to discover my gift?

It was only latterly that I realised: disappointment, good-humoured scepticism and the ability to see something valuable on both sides are gifts, of a sort. At any rate, if they’re what you’ve got, they’re what you've got. I resolved to reflect on human nature in a way that did not suggest closure, overstatement or facile answers, yet still offered something useful beyond the status quo. More than that, I wanted to find a way of writing more honestly about the academic life. The published record of books and papers airbrushes out a lot of the true nature of this life. Generally, the more influential and prestigious the publication, the more severe the airbrushing is. Readers can see only the tiny subset of thoughts and experiences that makes it through the filtering and signalling processes usually involved in publication. The quotidian mass of unpublished rumination is less cocksure, more imaginative, and in some important sense, truer. The excision of all the doubt and exploration from the final product both biases the scientific record, and gives novice scholars a completely unrealistic sense of what the academic life is really like. I have here tried to find a way of writing that is more open, more like an authentic conversation, than academic papers generally allow for. Over the course of the writing of this book, the search for the authentic voice became part of the substance as well as the style. As such, I hope the reader will forgive me an informality of tone, a periodic recourse to flippancy, and a certain self-involvement, in what follows.

Hanging on to the Edges consists of fourteen essays, written in 2016, 2017 and 2018, and originally published separately on my website.” My

2 www.danielnettle.org.uk

Introduction 3 intention was that each essay could be read in a single sitting (ignoring the footnotes unless you are keen to follow up sources), and each would stand alone. There are, however, plenty of connections between them if you want to make them. They are organized into three groups. Part lisa set of critical journeys through the current terrain of the human sciences. Respectively, these examine the recurrent tendency for researchers to over-claim for their theories (How my theory explains everything); the pernicious and persistent maintenance of a false distinction between the ‘social’ and the ‘biological’ (What we talk about when we talk about biology); the continuum from social theories that see humans as too self-determining and independent-minded, to those that don’t see them as self-determining and independent-minded enough (The cultural and the agentic); the perils of conceiving of culture as something like DNA and its change as something like natural selection (What is cultural evolution like?); and finally, the whole question of what a theory is, and what a good one would look like in the case of human behaviour (Is it explanation yet?).

Part II turns to the topic of poverty, particularly the consequences of poverty within affluent societies. This is one of my main specialist interests, and a topic to which the human sciences must turn with renewed vigour. Poor people know what it means to be hanging on to the edges, in many different ways. Just as the experience of poverty is insufficiently discussed, researchers struggle to conceptualise the causes and consequences of poverty in sensible ways. Thus, these essays deal as much with how to theorise about poverty as the empirical reality of what poverty is like. The mill that grinds young people old examines the link between poverty and ageing, and begins to raise my general claim that the behaviours of poor people typically make pretty good sense given the conditions under which they have to live. Why inequality is bad argues that economic inequality is bad for well-being, though not necessarily for the reason people usually say. Let them eat cake! considers the role of hunger in the consequences of poverty, arguing that before we turn to more abstruse and symbolic arguments, we consider the visceral: poor people in affluent societies simply can’t afford to eat well, and this could explain a lot. The worst thing about poverty is not having enough money argues, as its title suggests, that our explanations for the behaviour of poor people should start not with their intrinsic traits, culture, or life skills, but with the elephant in the room: their scarcity of material resources. Evidential support for this argument comes from what happens when this material scarcity is lifted, even if that uplift is completely at random. Finally in part II, Getting your head around the Universal Basic Income examines the case for that particular form of social transfer, focussing both on its potential advantages and the intuitive reasons people find it problematic.

In part III, I return to the academic life, its institutional organization, and how one might navigate its shoals. I have always been an advocate of inter-disciplinarity, and this is the subject matter of the first two essays. In The need for discipline, I try, with more success than I anticipated, to force myself into the view that specialist disciplines have value and transmit useful skills. In Waking up and going out to work in the uncanny valley, I reflect on the difficulties of doing inter-disciplinary work for a living, relating this to the categorical proclivities of the human mind. The penultimate essay, Staying in the game, is a slightly battle-scarred reflection on how to survive in academia. The final essay, Morale is high (since I gave up hope) links the personal search for the good life to the broader difficulties swirling around science, which concern reproducibility and incentives for discovering truth.

For all I have said about this being an anti-book, there is a positive vision for the human sciences in there somewhere, albeit not spelled out in capital letters. I believe in the eventual unity of all knowledge. I believe in the capacity of science to discover the truth about the world, including about this extraordinary, self-conscious ape, and the various ways his social life on this planet might be organised. To realise its potential, science will need to be done rather better than it currently is. I look forward to a world where researchers entertain ideas without building fiefdoms; keep their ears and minds open; treat each other with courtesy and respect; seek out the places where their ideas and knowledge break down rather than comfortable confirmation of what they already believe; and admit when they don’t know, or are wrong. This is also a world where we academics studiously avoid the traps of ordinary-language dichotomies and institutionalised ways of thinking; apply ourselves willingly to problems that really affect human wellbeing; are prepared to advocate certain social arrangements over others where we feel that our knowledge warrants doing so; and assume as

Introduction 5 much humanity and nuance in the lives of people different from us (other classes, other cultures, other times) as we do in our own. That is a world worth working for.

In closing, let me read to you from a letter written by the great Paul Feyerabend to the future readers of his final book.’ The letter was never used in that book, but was discovered on a floppy disk (remember those?) after Feyerabend’s death. He writes:

In a few pages, you will find a story written in a style you may be familiar with. There are facts and generalizations therefrom, there are arguments and there are lots of footnotes. In other words, you will find a (perhaps not very outstanding) example of a scholarly essay. Let me therefore warn you that it is not my intention to inform, or establish some truth. What I want to do is change your attitude.

I can think of no better introduction to Hanging on to the Edges.

3 The letter is reproduced in Ian Hacking’s introduction to the fourth edition of Feyerabend's Against Method: Outline of an Anarchist Theory of Knowledge (New York: Verso, 2010, p. xv).

Part One

1. How my theory explains everything: and can make you happier, healthier, and wealthier

One has consistently to think against oneself — to make matters as difficult for oneself as one can.

-Jon Elster!

The quintessence of science is doubt. In other kinds of belief systems, the statements held to be true are hardened into dogma, declared as absolute and certain. In science, they are held provisionally, constantly questioned, continually refined and replaced. At any moment, new observations or reasoning will lead to them being qualified, revised or thrown out entirely. In dogmatic belief systems, all our epistemic effort is directed towards the confirmatory instances of what we believe; see, I went to sacred waterfall yesterday, and today my cold is better. In science, by contrast, all our epistemic effort is directed to the anomalies; the cases where the prediction is not met; where the theory breaks down; the puzzling inconsistencies that help reject incorrect claims or stimulate the development of a whole new paradigm.

This is fine, and so far, uncontroversial. But I spend a lot of my time reading what scientists write, and I have to ask myself: why, then, do they seem so damned sure of themselves? Where is all this doubt that is supposed to be so great? Did these people not get the memo about the Enlightenment? It seems like doubt hardly gets a look-in in your average scientific paper or book. If it appears at all, it is mainly levelled at other people’s claims. At best it plays cameo parts: for example in the opening section or two, clearing the stage of clutter for the author’s grand proposal to enter. And grand it is: here is my paradigm, here is what we have shown, here are all the things my theory can do. Come into my camp; what we have in here is good. In other words, individual

1 Elster, J. (2007). Explaining Social Behavior: More Nuts and Bolts for the Social Sciences (New York: Cambridge University Press, p. 13), https://doi.org/10.1017/ CBO9781107763111 bits of scientific rhetoric look uncomfortably unlike how science is supposed to work, and awfully like how persuasion is done in nonscientific realms.

The overplaying of one’s own correctness, and the underplaying of doubt, have anumber of different flavours. In empirical papers published in the primary scientific literature, the principal flavour consists of finding a positive result, building the paper around it, and discussing it as if it definitely represented a real and important generalization about life. By ‘positive’ here, we usually mean some pattern that can have a Statistician’s p-value attached to it, preferably a rather small one. A small p-value is taken to mean that the said difference or association is unlikely to be due to chance, and hence represents an important discovery about the world. The problems with this way of reasoning are well documented. For a start, there is a good argument that ‘significant’ p-values do represent chance in most instances. Hang on, you say, the p-value is defined as the probability that the result is due to chance; so if p is small, then chance is an unlikely explanation. But here it pays to distinguish between the probability of p being small given that the finding is false, and the probability that the finding is true given that p is small. They are not the same thing.

To see why, let us say that the discovery of new and surprising things about the world is rare. By now that ought to be the case. Anyway, studies tend to be small-scale (to be low in what is known as statistical power), and so often fail to detect effects even when those effects are really there. So if you do experiment after experiment, maybe only one in a hundred correctly detects a genuine new and surprising result. Fine. But, by definition, one in twenty will produce a p-value less than 0.05, the usual cut-off for declaring a result ‘significant’. So, for every hundred experiments you perform, you will get a ‘significant’ p-value about 5 times by chance, and about one time because you have made a genuine discovery. So, 5 out of every 6 ‘significant’ p-values are in fact due to chance. As a very famous paper put it: most published research findings are false.’

I have some quibbles with the details of this argument. For a start, most published findings are not false. The data are not false. I think almost all scientists describe the data honestly. Rather, it is the inference that robust truths about the world have been discovered from these small p-values that is false in many cases. Also, people do not choose the experiments they perform at random, but go where pilot data and other information suggests there may be something going on. So the rate of true discovery may be higher than the argument implies. And then the argument neglects that many of the ‘significant’ p-values reported in the social and behavioural sciences are neither due to chance, nor discoveries about the world, but just reflect semantics? An example is the literature on the personality trait of neuroticism as a predictor of clinical depression. This is a perfectly serious topic of investigation. However, the predictor (personality trait neuroticism) is measured with a questionnaire containing questions like ‘Do you often feel blue? ’, and the outcome (depression) is measured with a questionnaire containing questions like ‘In the last two weeks, have you felt blue? ’. It would be an odd world indeed where you didn’t get an association with a ‘significant’ p-value in a study like this, because the null hypothesis (people who say they often feel blue are no more likely than anyone else to say they have felt blue in the last two weeks) is somewhat nonsensical. Its failure doesn’t tell you much about humans beyond, perhaps, a basic capacity for self-consistency. It tells you more about the semantics of the questionnaires. But certainly this ‘finding’ is not a falsehood.

These quibbles aside, the general burden of the criticism holds. A small p-value does not mean that anything has been demonstrated that is repeatable. The picture is in fact rather worse than I have suggested so far. This is because of ‘researcher degrees of freedom’. Basically, for any reasonably complex dataset, there are several different statistical analyses that could be performed. For example, there may be several slightly different choices for the outcome measure; several nuisance

2 Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine 2: 0696-0701, https://doi.org/10.1371/journal.pmed.0040215 3 Arnulf, J. K. et al. (2014). Predicting survey responses: How and why semantics shape survey statistics on organizational behaviour. PLoS ONE 9: e106361, https:// doi.org/10.1371/journal.pone.0106361 variables that either could or could not be adjusted for; and several subsets of the data that could be discussed. As a result, the researcher has quite a few different goes at getting than all-important small p-value, meaning that the probability of finding one by chance is typically rather a lot higher than 0.05.4

An illustration of the power of research degrees of freedom comes from a recent study of large clinical trials of treatments and preventative interventions for cardiovascular disease.’ Starting in the year 2000, the researchers carrying out these trials had to ‘pre-register’ their planned analyses in a public database of trial protocols. This means that they had to say, in writing for anyone to see, in advance of having looked at any data, what the critical comparisons would be, and what they would consider to constitute evidence that the intervention had a positive effect. Prior to 2000, there had been no requirement to do this; the researchers collected their data, analysed their data, and then wrote up their papers. To put the following findings into perspective, these trials are not two-bit minor research projects. These are large-scale, publicly funded, medically important evaluations done by teams of eminent clinicians and biomedical scientists.

Prior to the year 2000, 57% of the trials detected a ‘significant’ benefit of the intervention they were studying. After 2000, 8% did. The pre- and post-2000 trials did not obviously differ in any other respect than the requirement for pre-registration of planned analysis. It could of course be the case that human cardiovascular physiology changed in some fundamental way around the turn of the millennium, much as Virginia Woolf had claimed about human character on or around December 1910.° Somehow I doubt it though.

All in all, then, it should be no surprise that when studies in the behavioural and life sciences are replicated, we don’t consistently see

4 Simmons, J. P, L. D. Nelson and U. Simonsohn. (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22: 1359-66, https://doi. org/10.1177/0956797611417632

5 Kaplan, R. M., and V. L. Irvin. (2015). Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS ONE 10: 1-12, https://doi.org/10.1371/ journal.pone.0132382

6 Woolf, V. (1924). Mr Bennett and Mrs Brown (London: The Hogarth Press).

the patterns reported in the originals; the patterns argued so confidently to be real and to be a vindication of this or that theory.” A great deal has been written about this recently. It makes uncomfortable reading. The scales are falling from our eyes.

As consumers of science, we have to shift our focus from thinking about individual findings as instruments of truth, and instead thinking in terms of a slowly-evolving population of findings.’ If one study finds that eating broccoli is statistically associated with less depression, that’s not news. Once the population of scientific studies starts to contain finding after finding associating, in diverse ways, broccoli with reduced depression, that starts to be something interesting; maybe even heading for knowledge counter. Even once there it will continue to be dissected and its causal basis probed. It’s important for the media to understand that the development of knowledge is a gradual, population process, else it looks from the headlines like science is changing its mind all the time. This invites a kind of scepticism and hostility about the whole enterprise. Say I am trying to work out the proportion of cars on the roads that are red. Imagine if I issued a press release when I have observed a red car: ‘Study finds that all cars are red!’. Then researchers at a rival university could do a debunking study. They observe a car. It’s not red. ‘Study finds that no cars are red. Earlier study was flawed)’. Both parties increase their scientific visibility. But the truth is: neither study is decisive (that’s why science is hard); and yet every decent study very slightly increases the precision of our collective knowledge (that’s why science works).

What interests me the most is not the near-universal over-selling that goes on in scientific papers describing the results of primary empirical studies. It is the programmatic over-selling you find when academics write their ‘big idea piece’, or BIP. The BIP is often a book, or more rarely, a long-form discursive article. If a book, it may be aimed at a

7 Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science 349: aac4716, https://doi.org/10.1126/science.aac4716 8 McElreath, R., and P. E. Smaldino. (2015). Replication, communication, and the population dynamics of scientific discovery. PLoS ONE 10: e0136088, https://doi. org/10.1371/journal.pone.0136088 more general audience than just the few research specialists in the field; the so-called ‘popular’ or ‘trade’ book. It’s the author's chance to cement their position in the minds of the community; to recruit future devotees; to make their mark. (By the way, the pronouns in the preceding sentence are not grammatical errors, but none other than ‘singular they’, the American Dialect Society’s word of the year for 2015. ‘Singular they’ will henceforth be used throughout this book. And by the other way, let me pre-empt your inevitable thought: I am as guilty as anyone else of the sins to be described below.) The BIP has a recurrent four-act structure.

Act One: There are huge problems to be solved. The science is in disarray. Our existing theories are bad or incomplete.

Act Two: Heroically, through alarms and excursions, I’ve come up with this theory. It overturns the bad theories and completes the incomplete ones. It solves the problems. Here are some arguments in its favour.

Act Three: Here is another thing my theory is good at. And another. The good things about my theory go beyond the problem I set out to solve. Within a broad domain, it is quite possibly the long-awaited theory of everything.

Act Four: You can apply my theory to life. Unification of the sciences? My theory. Economic uncertainty? My theory. Unlucky in love? You’ve guessed it.

We can know these acts by nick-names, each of which tells you what the problem is. They sound like the syllabus for a class on bad reasoning:

Act Four: If all you have is a hammer, everything looks like a nail

The BIP is, more or less, a systematic over-statement of the relative merits of the position the author is peddling. I say relative merits, because not only is the author's position inappropriately bigged up; the alternatives are inappropriately belittled too. It is not enough for my theory to succeed; others must fail. Usually, the existing theories are mis-represented to the point of denying their daily successes and the valid insights on which they were based. There is a tendency here to go for the tall poppies. Presumably you get more points for flaying a big theory than a little one. The favourite target of all seems to be the biggest: Darwin’s theory of evolution. As John Welch has recently pointed out, there is a quite a cottage industry of setting out to declare Darwin’s theory fundamentally inadequate/incomplete/incorrect, usually as Act One to the author’s own BIP.?

You can see why. If you start out by saying: ‘evolutionary theory is basically fine as far as it goes, but I want to talk about phenomenon X somewhere in the world of living things’, then you are just a worker at the mill. You pay the rent but don’t get your picture on the cover of Nature magazine. If you start out by saying: Darwin’s theory fundamentally requires refinement/completion/replacement (by the thing I want to talk about), then you are an intellectual giant-killer, and people pay attention. But some of the claims that follow can be a little overcooked. Often they present a phenomenon that is, directly or indirectly, the outcome of genetic evolution as if it challenged our fundamental understanding of the process of genetic evolution. You may have seen examples: Richard Dawkins says evolution is all about selfish genes, but what about mutualism/mirror neurons/mariachi music? They’re really important! Evolutionary theory as we know it can’t be right! It’s not that mutualism/mirror neurons/mariachi music aren’t interesting or important, of course. Nonetheless, I can’t read this kind of BIP without the image coming to my mind of a philosopher of science, somewhere across the world, weeping quietly in order not to wake their spouse.

Whereas in Act One, all other theories are caricatured and overcriticized, in Act Two, the author’s own big idea is suspiciously free of problems. It accumulates credit for things that are indeed consistent with it, but are probably consistent with many other theories too. And then we only get to visit the cases that show it to its best advantage. The odd thing here is that the author ought rationally to believe that their own

9 Welch, J. J. (2016). What's wrong with evolutionary biology?. Biology & Philosophy 32: 263-79, https://doi.org/10.1007/s10539-016-9557-8 theory, too, is likely to turn out false. Philosophers even have a name for this reasonable inference: the pessimistic meta-induction.”” The grounds for the pessimistic meta-induction go like this: the vast majority of explanatory beliefs about the world that humans, including scientists, have entertained through our history have turned out to be wrong. So faced with a new one, a reasonable being’s assumption should be that it too will turn out to be wrong. We should thus entertain a certain detachment and vigilance toward it. Yes, it could be interesting, worth thinking about, but at least in the current form, it’s probably wrong and it’s not going to be around for very long. The authors of BIPs are good at applying the pessimistic meta-induction to all the other theories; they just can’t take the extra step and apply it to their own.

In some cases, Acts Three and Four become comical: the second half as farce. In Act Three, The Theory, extended further and further beyond any basis it had in the technical literature, becomes more and more under-specified, under-evidenced and under-grounded. It’s not even a promissory note. It’s a vague promise to meet you with an important package at some unspecified time in the future in a pub near Chingford. But by this point the author is three months late with their manuscript and subsisting on a diet of their own rhetoric. And then Act Four. You can almost hear the literary agent: ‘Well, they like it, but it’s a bit academic. It has a better chance of breaking through if readers could see how to apply it to their daily lives. Could you put in a chapter about how to make practical use of your big idea in internet dating/choosing a pension plan/promoting world peace? After all, we’ve got a big advance to pay off’.

This is not how science ought to work, is it? Surely the BIP should be a little bit about (a) the common features of different existing intellectual approaches to a problem, with a view to how they can be synthesised; and a lot about (b) the failures of our current understanding: the

10 See Doppelt, G. (2007). Reconstructing scientific realism to rebut the pessimistic meta-induction. Philosophy of Science 74: 96-118, https://doi.org/10.1086/520685. Note that in the philosophy of science, the pessimistic meta-induction is discussed as an argument against the realist view of science (that is, the view that science can gradually come to approximate the objective truth about the world). I am using it in a more informal and uncontentious sense: that a lot of individual scientific ideas have turned out to be wrong.

anomalies, the failed predictions, the problems, the things that don’t yet fit. That’s where the scientific action is.

The reasons why BIPs over-state their cases do not seem too difficult to understand. Do we need to point to anything more than ordinary human self-interest? People want to get their papers published. They want their grants funded. They want status. They want their book to make a stir. And a few of the best ‘popular’ or ‘trade’ books about science have made their authors surprisingly large amounts of money. So we have a perfect incentive set for ambitious writers and thinkers to over-sell their wares. What else would we expect them to do?

There’s been alot written lately about changing the incentive structure of science, for example so that the acceptance of data for publication does not depend on the size of the p-value. This obviously makes sense. It will make it harder to ignore anomalies and negative replications, and that in turn will allow more airtime for appropriate doubt. But it will not eliminate the BIP-problem (if we decide that it is a problem, a question to which I will return). BIPs are typically statements of broad theories or paradigms. In the human sciences at least, broad theories or paradigms seldom make individual ‘line-in-the-sand’ predictions that can be decisively judged to have failed. They make meta-predictions: I predict that it will prove more useful in approaching a topic you wish to study if you use my framework to make your predictions, than if you don’t use my framework. And it’s obviously hard to defeat this meta-prediction: maybe the meta-prediction is false, maybe you just didn’t use the framework correctly (yet). And who knows what it would have looked like if you had used a different framework anyway? So there’s plenty of wiggle-room for writers of BIPs to construct cases for their pet theories, and scope for ailing big ideas to persist.