Despite the clear warnings in Iraq and Afghanistan of the dangers of destroying an existing state, the USA overthrew the Ghaddafi regime in Libya, leading to the collapse of the state, civil war, and a flow of migrants across the Mediterranean that has driven right wing extremism in Europe. In the process, the Obama administration also engaged in blatant deceit of Moscow and Beijing, destroying whatever remained of US diplomatic credibility in those capitals. And the USA has not even been able to prevent one set of allies in the region from boycotting another ally – Qatar – which is home to a vital US air base!

The current preferred explanation for the latest disasters on the part of the US establishment is the personal behaviour of President Trump – and indeed, in all the history of American diplomacy there may be nothing to equal Trump’s latest statements on Turkey and the Kurds for illiteracy, irresponsibility and absolute stupidity. In the wider world, some of Trump’s instincts appear to be good. Despite all his bluster, he shrinks from actual war, and he opened direct talks with the North Koreans and the Taliban that ought to have been initiated by Washington many years ago. Yet as these examples demonstrate, he appears incapable of the most minimal consistency or steadiness in his approaches, or indeed of grasping the basic dynamics of any given international relationship.

However, the concentration on blaming Trump is also a way for the US and Transatlantic establishments to excuse themselves for the series of disastrous and sometimes criminal decisions (and lack of decisions) by the previous three Democratic and Republican administrations. This pattern has its roots in the decay of the US political system and political establishment at home, including the power of lobbies and their money over US policy in key areas; the retreat of area studies in academia and think tanks, leading to sheer ignorance of some of the key countries with which the USA has to deal; the self-obsession, self-satisfaction and ideological megalomania that in every dispute leads so much of the US establishment and media to cast the USA as a force of absolute good, and its opponents as absolutely evil; and the failure – linked to these three syndromes – to identify vital an secondary interests and choose between them – a failure that has led the USA to the cvrazy position of confronting China, Russia, North Korea, Iran, Turkey and ISIS all at the same time while alienating the countries of the European Union.

However, it must also be acknowledged that the structures of the Middle East itself have also been responsible for many disasters; and that the appalling bitterness, complexity and interconnectedness of the region’s problems would have challenged even the wisest and most far-sighted of would-be hegemons. It is not only that almost every state in the region is threatened from within by some combination of failure of socio-economic development, lack of political legitimacy, sectarian strife and rebellious minorities. The way in which these divides cut across the region means that almost every state has the capacity to pose an existential threat to other states, or at least is perceived to have this ability. The resulting set of fears and hatreds could be described as paranoid, except that in many cases they are in fact well-based.

The Middle East recalls, though on a much larger and more dreadful scale, the situation in the southern Caucasus by the mid-1990s. The newly independent states of the region had got into such a terrible mess, and fought (and generally lost) so many wars, that there was a widespread assumption among Western journalists covering the region that the result would be the creation of a new Pax Russica, with the agreement of most of the region’s exhausted inhabitants. But Moscow was never able to solve the problem – which probably was in fact insoluble – of how to incorporate Azerbaijan and Georgia in a new Russian-dominated security system without comprehensively betraying the Armenians, Ossetes and Abkhaz. With the threat of Russian military power severely diminished by failure in Chechnya (just as the threat of US ground invasion has become empty since the disaster in Iraq), no such Pax Russica could be established.

At the same time, given its own interests and the risk of new wars, Russia could never simply withdraw from the southern Caucasus. Nor – as both Obama and Trump have found – can the USA simply withdraw from the Middle East without causing yet more disasters. The answer, if there is an answer, lies in a fourfold US strategy: of the elimination of hatreds and disputes left over from the past; of trying as far as possible to play the role of an even-handed “honest broker” in local conflicts; of reducing US and Western dependence on oil, allowing distancing from Saudi Arabia; and of seeking a concert of power with Russia, China and India to combat terrorism and contain conflicts.

President Obama did try partially to achieve the first two goals, through the nuclear deal with Iran and his public recognition that Iran and Saudi Arabia will have to share influence in the region. But a combination of the outrageous public stances of Khatami’s successor, Iranian President Mahmoud Ahmedinejad (in office 2005-2013) with bitter resistance from the Israel lobby, the Republican Party and many Congressional Democrats meant that Obama was only able to achieve the nuclear agreement towards the very end of his presidency, after which it was promptly torn up by Trump; and no more than any other US president was Obama able to achieve balance between Israelis and Palestinians or Turks and Kurds.

As for co-operation with Russia and China, this is ruled out both by wider hostility on the part of the USA and by the congenital incapacity of the US establishment to treat any other state as an equal. Nor indeed is there much evidence that China wishes to play a greater geopolitical role in the Middle East. As a Chinese official once told me, referring both to US failures and to Chinese awareness that they have no solutions for the regions’ conflicts, “Why would we want to get involved in that mess?” Given the modern history of the Middle East, it is hard not to agree with him; and as the Russian administration attempts to spread its influence in the Middle East, and to develop good relations with Saudi Arabia and Iran simultaneously, Moscow too would be well-advised to keep this in mind.