The convicts are lodged in a jail in Belgaum, Karnataka for killing 22 policemen in a landmine blast in 1993. They were given the death sentence in 2004 by the Supreme Court.
The Centre, represented by Attorney General GE Vahanvati, opposed the request to stay or suspend the death sentence for Veerappan's men arguing that the landmine blast was a crime against the state.
Last week, President Pranab Mukherjee rejected their mercy petitions.
The convicts - Gnanaprakash, Simon, Meesekar Madaiah and Bilavendra - have appealed to the Supreme Court to commute their sentence since there was an inordinate delay of nine years in deciding on their mercy petitions.
The Supreme Court today agreed to hear their appeal. On Wednesday, it will decide which judges should consider the petition.
A bench of two Judges is already hearing a similar matter. Three people sentenced to death for the assassination of former Prime Minister Rajiv Gandhi, have asked the Supreme Court to commute their sentence to life in prison because it took 11 years for their mercy petition to be rejected.
Another death row prisoner, Devender Pal Singh Bhullar, convicted of a bomb blast in Delhi in 1993, has pointed out that his mercy petition was pending for eight years before it was rejected in 2011.
Lawyers for Mr Bullar and for the men convicted for Rajiv Gandhi's killing say that the delayed decisions for their mercy petitions are cruel, dehumanize the convicts, and violate Article 21 of the Constitution (Protection of life and personal liberty).
"This government has executed more people since November 2012 than in the previous 10 years. To continue such a regressive trend would be truly shameful," said G Ananthapadmanabhan, chief executive of Amnesty International India.
Veerappan, who killed nearly 180 people, most of them police and forest officials, was killed in a police encounter in 2004. Security forces had been trying to catch him for nearly two decades. His wife, V Muthulakshmi, who was arrested in the landmine case and later acquitted, has spoken in support of the four men on death row.