The television channel's news editor Ben de Pear said he finally decided to pull the team out after six men appeared at his hotel room door early on Sunday demanding entry.
He said the men claimed they were from the immigration police and accused the journalists of breaching their visa conditions by "attempting to enter the president's palace".
"It was an intimidating experience," he wrote on Channel 4's website in a posting Sunday, saying the campaign of intimidation made it impossible to work.
"People often get visits in this country and are bundled off in white vans never to be seen again," he said.
The team was repeatedly targeted by protesters over the channel's award-winning documentary "No Fire Zone: Sri Lanka Killing Fields".
The Commonwealth summit, which ended late Sunday afternoon, was overshadowed by a row over alleged war crimes by the Sri Lankan military during the conflict against Tamil separatist rebels that ended in 2009.
British Prime Minister David Cameron threatened to push for an international inquiry into the allegations, while Sri Lankan President Mahinda Rajapakse said he would not be pressured over the issue.
A UN panel in 2011 found as many as 40,000 civilians may have been killed in the final months of the civil war, mostly due to Sri Lankan military action. Colombo denies its troops killed any civilians.
During its six days in Sri Lanka, the news team was stopped from travelling to a former war zone to speak to ethnic minority Tamils, who say they are still suffering four years after the war ended.
The government said it was for their own safety.
"Our phones and laptops have been monitored, our hotel was under instructions to tell the authorities every time we left, and it has been made clear to us by those who have bravely wanted to meet that many of them are then spoken to themselves," de Pear said.
Media rights groups say Sri Lanka remains a dangerous place for journalists despite the end of the war.
In 2009, the Sunday Leader newspaper's editor, a staunch critic of the government, was shot dead near his office just outside the capital.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.