"I have always believed that there should be a time limit for the disposal of mercy petitions pending before the President," Mr Singh told the Press Trust of India.
Delays in disposal of mercy petitions do not help anyone and nor do they serve any purpose, he said.
"It is because of this I am of the view that Afzal Guru should be hanged soon," the Congress leader said.
On the controversy surrounding Union Home Minister Sushil Kumar Shinde's remarks on "Hindu terrorism" at the recent Jaipur 'Chintan Shivir', he said those who had gone to meet the Prime Minister and sought the release of terrorists must also apologise.
Mr Singh claimed that senior BJP leaders had earlier met the Prime Minister to secure the release of some people arrested for having indulged in terror attacks. "They must also apologise for doing so," he said.
Stating that there was no need to talk of terrorism in religious terms, he said, "A terrorist is simply a terrorist, irrespective of the religion to which he belongs."