Millions of black-clad Shiainfo-icon pilgrims are converging on the holy city of Kerbala for the Arbaeen religious commemoration, the largest annual gathering of people anywhere on earthinfo-icon. Walking in long columns stretching back unbroken for as much as 50 miles, sleeping and eating in tents erected by supporters beside the road, the event has become an overwhelmingly powerful display of Shia belief and solidarity.

The Arbaeen coincides this year with the final defeat of Isis, the movement that slaughtered Shia in their tens of thousands and aimed to overthrow the Shia-dominated government in Baghdadinfo-icon. The Syrian armyinfo-icon announced today that it has captured the last Isis-held town in Syriainfo-icon, Albu Kamal, its victory coming a few days after Isis was driven from western Iraqinfo-icon.   

Arbaeen is the living symbol of the rise of the Iraqi Shia, a highly significant development in the Middle Eastinfo-icon, but it has happened only recently. Karim, 48, a tribal leader from Najaf, who provides free foodinfo-icon for the pilgrims, recalls that when he first took part in an illegal Arbaeen walk under Saddam Hussein, "we had to take a roundabout route by the river [Euphrates] and try to keep hidden because, if we were caught, we would put in prison or executed".

The Arbaeen has provided many modern-day Shia martyrs, murdered by Saddam Hussein, al-Qaeda and Isis, but its purpose is to mourn the martyrdom of Imam Hussein, the revered Shia leader, killed in the battle for Kerbala in AD680. The long ritual walk to his golden-domed shrine in that city - some walkers spend 10 or 12 days on the road from Basrainfo-icon or Kirkuk, others two or three days from Najaf - comes on the 40th day of the mourning periodas religious fervour reaches its peak among the faithful.

Shia cities, towns and villages all over Iraq empty out during a 20-day period as their people take to the roads in an elaborately organised and well protected mass movement not seen anywhere else in the worldinfo-icon. Estimates vary of the total attending, from highs of 15-17 million to a low of 6-7 million, but it includes at least two million Iranians whose numbers are easier to calculate because they require documentation to enter Iraq. Mohammed al-Hilli, the author of a book entitled The Arbaeen: the Walk, says that "the city of Kerbala can only contain two or three million people at one time, but, since pilgrims are coming and going over a long time, the total attending will be much higher."  

The pilgrims carry black, green, red and white flags, with the black flag of mourning for Imam Hussein by far the most common. Vast numbers of them decorate permanent brick buildings and temporary tents which are used for praying, eating and sleeping along the three main routes leading to Kerbala. Once pilgrims were lucky if they got rice and bean stew - "there was nothing but muddy water to drink" recalls one early participant - but everything is now highly organised with copious supplies of food, small clinics and even dentists all working for free. The care of pilgrims is regarded as a religious duty.  

This year there are more red, white and black Iraqi national flags evident than before, indicating a shift towards greater identification with the Iraqi state by the Shia, traditionally marginalised by the Sunni since Ottoman times and before. When the Shia-dominated government took power in Iraq in 2005 it was the first time the Shia had held power in any country in the Arab world since the Fatimids in Egyptinfo-icon were overthrown by Saladin in the 12th century. It is only now that they have started to look comfortable in their new role. 

All religions have their martyrs, but for the Iraqi Shia they come from the present as well as the distant past. Lamp posts fifty yards apart along the 45-mile Najaf-Kerbala road each have a different picture of a soldier or civilian killed by Isis or al-Qaeda. The same is true of other routes to the holy city. Once pilgrims risked death from Isis ambushes, but the roads are more secure. Major General Qais Khalaf, military governor of Kerbala, Najaf and Diwaniyah provinces, says "there was just suicide bombing in this area 18 months ago, when three people were killed." He believes Isis no longer has the base areas or the level of support it needs to launch big attacks.

The mood of the pilgrimage is one of intense piety and communal solidarity, though Shia clerics keep emphasising that the pilgrimage is dedicated to peace. Asked if the Iraqi security forces' victory over Isis had an effect on the gathering, Shia clerics said there has been an improvement in morale and self-confidence. "Who does not want more security?" asked Kamil Kadar, a pilgrim taking part in the walk. After 40 years of wars and emergencies, Iraqis are wary of good news, always suspecting that developments will turn sour as they have done so often in the past.  

Sayyid Alaa al-Moussawi, a senior Shia cleric who is head of the Office of the Shia Endowment, says that Iraq seems to be entering a period "of greater harmony with its neighbours, notably Saudi Arabiainfo-icon, Turkeyinfo-icon and Iraninfo-icon". Iraqis see this as crucial because it is the combination of domestic insurgency with financial and military support from foreign states that has kept Iraq in a permanent state of warinfo-icon and emergency.   

Pilgrims from all over the world head for Najaf, Iranians being much the most numerous, but they also come from Pakistaninfo-icon, Indiainfo-icon, Azerbaijaninfo-icon, Lebanoninfo-icon and places where the Shia community must be tiny. Many of the pilgrims are teenagers or in their twenties. Asked what the pilgrimage meant to him, one 19-year-old visitor from London said: "It means my whole life to me. It means one small step towards heaven."  

Shia identity is beginning to blend with Iraqi national identity as Iraq looks less like a failed state. There is strong sympathy for Shia struggles in countries like Syria, Lebanon, Bahraininfo-icon and Yemeninfo-icon. Inside Iraq, there is strong popular backing for the Hashd al-Shaabi or Popular Mobilisation Units as a sort of Shia national guard. Nationalism may be on the rise but it is still trumped by religioninfo-icon as a cause Iraqis will die for.  Asked about the Hashd, one observer said: "I don't think you could have defeated Isis without using ideologically-driven fighters like the Hashd."

The first time I saw Arbaeen walkers was in April 2003 when USinfo-icon soldiers looked in perplexity and with suspicion at bands of young men, often carrying green palm fronds rather than flags, walking to Kerbala from all over Iraq. This was happening just after the US invasion and the pilgrims were walking unconcernedly past burned out Iraqi tanks and military vehicles. The incoming administrators of the US-led occupation paid the walkers no attention, though they were an early sign of the Shia piety and determination which was to shape the future of the country.  

The US shows little sign of having learned much about the Iraqi Shia community in the 14 years since the invasion. It certainly still underestimates them. The Arbaeen was not a victory rally for the Shia as Isis is defeated, but they may well feel that, in Iraq, their day has come.