Why is it that big, consequential life decisions — whether to have kids, marry a specific person, pursue one career path over another — can feel so agonising? It seems like a stupid question. Obviously, it’s because they really matter. Yet, on closer inspection, that can’t be the whole story.

All sorts of other decisions “really matter”, too: whether to seek medical help when you break your leg; whether to use an oven glove to handle an extremely hot dish; whether to park your car on a level crossing. But they’re not agonising at all. They’re so straightforward that it sounds strange even to think of them as decisions.

All right, then. Perhaps what makes decisions agonising is that they matter and there is too much uncertainty to know which option to choose. That sounds more reasonable. But as the US psychology professor Tania Lombrozo points out, it’s still a bit weird. If no single option clearly stands out — if they’re roughly equally appealing and you can’t reduce the uncertainty by doing further research — then your decision doesn’t much matter. You could just flip a coin. Or, as Lombrozo puts it: ‘Hard decisions should be easy.’

This is Fredkin’s paradox , proposed by the computer scientist Edward Fredkin, whose colleague Marvin Minsky quoted him as follows: ‘The more equally attractive two alternatives seem, the harder it can be to choose between them — no matter that, to the same degree, the choice can only matter less.’

Every bone in our bodies rails against the idea of choosing a spouse or a career the way you’d choose between spaghetti bolognese or pizza margherita. Yet to the extent that you’re unable to know how things will turn out, overthinking is futile: it can’t affect what economists call your ‘expected utility’. Certainly, it’ll turn out to have mattered in hindsight — but by then it’ll be too late. Hence the paradox: we fret and stew, as if hoping through sheer effort to see into the future. In the worst case, we end up choosing none of the potentially good options, but a definitively bad one — paralysis — instead. That is the fate of ‘Buridan’s ass’, the hypothetical donkey, positioned equidistantly between hay and water, that is hungry and thirsty in equal measure and stays rooted to the spot, thus starving to death.

Merely knowing about Fredkin’s paradox probably won’t reduce your tendency to overthinking: it’s too deeply conditioned for that. But it can provide solace, I think, when a decision turns out to have been bad. If you’re tempted to berate yourself, wondering how you could have chosen such a disastrous job, or spouse, or place to live, you can remind yourself that you truly couldn’t have known. Your agonised decision-making process may have made it feel like you were weighing the pros and cons, a task you should have performed better, but really you were taking a stab in the dark.

The philosopher Alan Watts once observed that the process we call “deciding” — moving gradually towards a resolution — is frequently no such thing. Rather, it’s just a period of flipping back and forth between options, followed by a sudden, intuitive, semi-random choice. We might as well own up to that.

Steven Johnson’s new book, Farsighted, makes the case against “going with your gut” when it comes to life-altering decisions — and argues that one crucial tactic is to generate more options than you think you have.

Guardian News & Media