His comments came in the wake of the Archbishop of Delhi referring to a "turbulent political atmosphere" in the country and appealing to launch a prayer campaign ahead of the 2019 general elections.
"I have not seen the statement (of the Archbishop) verbatim but all I can say is that India is a country where there is no discrimination against anyone on the basis of caste, sect or religion. Such a thing cannot be allowed," Mr Singh said on the sidelines of an event of the Border Security Force (BSF) in New Delhi.
During the event, the home minister said the government will not allow any breach in the unity of the country.
"Sometimes questions are asked to us. We will not compromise on the unity, integrity and sovereignty of this country at any cost and this is our top priority. We are also committed to strengthen the bonds of amity, affinity and harmony in our society," he said.
Citing "turbulent political atmosphere which poses a threat to the democratic principles enshrined in the Constitution and the secular fabric of our nation," the letter stated, "It is our hallowed practice to pray for our country and its political leaders all the time but all the more so when we approach the general elections."
"As we look forward towards 2019, when we will have a new government, let us begin a prayer campaign for our country," the letter stated.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections. Election results will be out on May 23.