Mr Singh was speaking at a function in the capital when he reportedly expressed concern that the widespread use of the English language in the country has resulted in the loss of "our language and culture" as hardly anyone speaks in Sanskrit now.
The BJP prides itself on its reach among the younger and more literate population of the country through vehicles like the social media, many of who use English as their chosen language of communication.
Information and Broadcasting Minister Manish Tewari said this morning, "I sometimes feel like laughing at our friends. On one side, their vision document is outsourced to people who don't speak any language other than English. Is this medievalism or hypocrisy?"
The Congress minister was alluding to reports that the BJP is prepping its vision document for the 2014 general elections with the help of several leading intellectuals and academics of the country.
Mr Tiwari said the debate whether pre-schoolers should be taught English so that they can use the Internet had also taken place in other countries.
"This attempt to create a dispute over language or saying that one language is better or worse than another, doesn't strengthen the country and is not expected from a responsible political party," he said.