Some people sniff the air and smell an alarmingly foul whiff of the 1930s. The rise of demagogues and “strongmen”; the resurgence of authoritarianism, nationalisms and fundamentalisms; the denigration of expertise and the celebration of ignorance; scorn for consensus-builders and pragmatic compromise; the polarisation of politics towards venom-spitting extremes. Haven’t we seen this horror movie before?

No, argues David Runciman in this scintillating treatise about representative democracy and its contemporary discontents. Donald Trump is “an old man with the political personality of a child”, but he is not “a proto-Hitler”. We are not reliving the first half of the 20th century in Europe. Vladimir Putin presides over a “parody democracy” in Russia, but he is not Stalin. Some of the symptoms of democratic decay may seem familiar, but the disease is different. We make a potentially fatal mistake if we think that history is just repeating itself. Gaze obsessively into the rear-view mirror and we won’t see the true threats on the road ahead.

He is right to register “widespread contemporary disgust with democratic politics”. Some of the sources that he identifies will be familiar to readers of the burgeoning literature on the malaise afflicting the more mature democracies. Voter confidence has been sapped by governments that struggle to deliver the underlying contract to spread prosperity sufficiently widely and fairly that everyone has the sense of a stake in society. It is not surprising that many Americans were discontented enough to choose the wild ride of Trump when you consider that the average real wage in the United States has been stagnant for the past 40 years. The internet, far from being the elixir of democratic accountability and engagement that utopians once imagined, has poisoned the well. Opposed sects promote conspiracy theories in their rival echo bubbles rather than engage in reasoned debate around an agreed set of facts. Democracy has become more venomous – and at the same time more toothless. Governments flounder in the face of the disruption unleashed by the tech titans of Silicon Valley and subverters tilling the troll farms run out of the Kremlin. Short-termist politicians are inadequate to the task of tackling existential threats to humanity, such as climate change, because thinking about the end of the world “is too much for democracy to cope with”.

Runciman is gloomy because one of his key contentions is that representative government has lost the capacity to reinvigorate itself. In the opening decades of the 20th century, support for democracy was widened by extensions to the franchise and the foundation of welfare states. He offers the provocative thought that democracy also thrived – even depended upon – “chaos and violence” because they “bring the best out in it”. The second world war demonstrated the benefits of democracy when confronted by nazism. The cold war – this is my suggestion, not his – advertised why liberal capitalist democracy was superior to totalitarian communism.

For all its flaws, where democracy has enough time to put down roots it proves durable. He offers the example of Greece, a country that was ruled by a military junta within living memory. Despite a series of flailing governments presiding over a dire economic situation that has inflicted enormous stress on the population, the Greek military has not put a bootcap out of its barracks.

Yet Runciman finds this not a reason to be cheerful, but another cause for concern. A thought-stimulating strand of his case is that the resilience of the mature democracies is at the heart of their failings. “Stable democracies retain their extraordinary capacity to stave off the worst that can happen without tackling the problems that threatened disaster in the first place.” Despite the look-at-me title of the book, he doesn’t think democracy is over. Rather, he contends it is suffering a “midlife crisis”. In the American iteration, “Donald Trump is its motorbike”. His presidency could end in “a fireball” but it is more likely to be looked back at as a phase of decline that is “simply embarrassing”.

For all its manifest and manifold imperfections, democracy has a better record than any rival form of government

Runciman’s flair for turning a pithy and pungent phrase is one of the things to admire about his writing. The cogency, subtlety and style with which he teases out the paradoxes and perils faced by democracy makes this one of the very best of the great crop of recent books on the subject. What he doesn’t offer is solutions, bluntly admitting “I do not have any”. There is penetrating diagnosis here, but no suggestion of a cure. He considers the alternatives and rightly finds them wanting. The Chinese experiment with authoritarian capitalism may look seductive to those who think economic expansion is all that matters to a society, but can the repressive Beijing model survive the inevitable day when growth slows down? Government by experts, “the rule of the knowers” or “the epistocracy”, was advocated by Plato and is still promoted by those who regard citizens as too stupid to be trusted with making decisions. The public wouldn’t wear that and “intellectuals” are just as prone to making terrible mistakes as the crowd. Runciman seems attracted to the idea that technological advances could offer some form of “liberation”, but comes to the equivocal conclusion that this “includes all sorts of potential futures: some wondrous, some terrible, and most wholly unknowable”.

I share a lot of his anxieties, but ultimately he didn’t persuade me to subscribe to his underlying despair. For all its manifest and manifold imperfections, democracy has a better record than any rival form of government at sustaining free, innovative, peaceful and prosperous societies. Yes, democracy is often messy, clumsy and ineffectual. Yes, voters sometimes empower ghastly rulers. Yes, democracy is looking tired at this moment in its history. But almost despite himself, and without saying it this explicitly, Runciman seems to accept that there is something special about democracy. One of its great merits is the capacity for self-questioning and self-correction, which is lacking in other systems of government whether they be tyranny by emperor, colonel, president of the praesidium, priests or data. Democracy can go wrong, but it has the flexibility to put itself right. As Runciman acknowledges, “democratic politics assumes there is no settled answer to any question” and this “protects us against getting stuck with truly bad ideas”. As Tocqueville put it: “More fires get started in a democracy, but more fires get put out too.”

I finished this book feeling more hopeful than I thought I would be and the author probably expects his readers to be. Democracy can change its mind and by doing so it can improve its circumstances and prospects. This is a precious quality that contains within it the possibility of renewal. Donald Trump is not the end of democracy’s story.

• How Democracy Ends by David Runciman is published by Profile (£14.99). To order a copy for £12.74 go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99