"One on hand we are trying to bring peace in this country. We are also taking steps against injustice to minorities as also against infiltration. But, in the midst of all this, we have got an investigation report that be it the RSS or BJP, their training camps are promoting Hindu terrorism. We are keeping a strict vigil on all this," Mr Shinde said at the All India Congress Committee or AICC session in Jaipur.
He said bombs were planted in Samjhauta Express, Mecca Masjid and also a blast was carried out in Malegaon. In all these incidents, the involvement of right-wing extremist groups is suspected.
"We will have to think about it seriously and will have to remain alert," he said.
Attacking Mr Shinde, BJP spokesperson Mukhtar Abbas Naqvi told reporters in Delhi, "Their (Congress') destructive mindset is reflected in the statement of the Home Minister. The statement he has given at the Chintan Shivir is very objectionable. It's not only unacceptable but also dangerous."
He said Mr Shinde's statement was aimed at disrupting peace and harmony in the country.
The BJP demanded an immediate apology from the Congress leadership. "Sonia Gandhi should apologise. Rahul Gandhi and Home Minister should also apologise, otherwise there will be serious consequences. This is not accepted. Such baseless things when said by Home Minister amounts to giving clean chit to real terrorists."
RSS spokesman Ram Madhav said Mr Shinde's remarks were "highly irresponsible" and made only to please his party bosses.
"If a Congress leader had said something like this it wouldn't have mattered that much, but if the Home Minister of the country is saying something like this, it is highly irresponsible. This is all about playing votebank politics," he said.
"We feel ashamed that he (Shinde) belongs to Maharashtra," said Shiv Sena spokesperson Sanjay Raut in Mumbai. If the Home Minister has any evidence to back his remarks, he should show it, Mr Raut said.
"Sometimes I feel that Congress has become a group of cowards. I am saying because terror strikes are repeatedly happening in the country, terrorists are mushrooming here," Mr Naqvi said.
Confronted by the BJP, Mr Shinde later clarified, "This is saffron terrorism that I have talked about. It is the same thing and nothing new. It has come in the media several times."
Mr Shinde's remarks were defended by several Congress leaders. Congress General Secretary Digvijaya Singh said, "It is not Hindu terrorism but Sangh-backed terrorism".
"I am 100 per cent with Shinde on this. This isn't a secret. I want to thank the Home Minister for saying what everyone knows but do not have the courage to say," party leader Mani Shankar Aiyar said.
"He (Shinde) meant right-wing terrorism. There is no Hindu or Muslim terrorism," added Union Minister Rajiv Shukla.
Mr Shinde said that infiltration from Pakistan, insurgency in the north east and Naxalism were the main challenges before the country.
"Infiltration is from Pakistan, insurgency is in some states in the north east and Naxalism is another challenge. Environment is being created against peace but my department will do its complete work," he said.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.