He said this while criticising the Pinarayi Vijayan-led LDF government for stating in the Supreme Court that a controversial Muslim marriage case in the state did not warrant a National Investigation Agency (NIA) probe.
"Is it fair and right for the Kerala government to move the Supreme Court to say that you don't allow NIA, we are doing our job."
"Is the Kerala Government doing its job?," Mr Prasad asked at a press conference here.
His comments came days after the state government informed the Supreme Court that its police had conducted a "thorough investigation" into the conversion of a Hindu woman to Islam and her subsequent marriage to a Muslim man and did not find material warranting the transfer of probe to the NIA.
The top court had on August 16 directed the NIA to probe whether there was a wider pattern of alleged 'love jihad' in the case in which the Hindu woman converted to Islam and later married Shafin Jahan, the Kerala Muslim man.
Mr Prasad, who was in Kerala to attend the Jan Raksha Yatra organised by the state BJP against "jihadi terror and red terror," said that whenever he comes to the state, he hears stories and there is a widely held perception that many young boys and girls have eloped, misguided by radical elements.
He also expressed concern over a protest march of Muslim Ekopana Samithi to Kerala High Court on March 30 this year in protest against a judgement cancelling the marriage conducted by a qazi between a Muslim man and Hindu woman.
"Is it right to organise a dharna against a High Court judgement? What kind of culture is happening and what action was being taken by this government? As the Law Minister of India, I am concerned," he said.
On instances of youths being radicalised after conversion, he asked "Why is it happening? Is it happening in the air or there is certain rationale and fear and apprehension for that.
"That is question the government needs to answer. My charge is that the Kerala government is very soft on terrorism and their patrons," he said.
"I have got the evidence to show that," he said and cited an instance in Kerala in which the Chief Minister and senior CPI(M) leader Pinarayi Vijayan allegedly shared the dais with Abdul Nasar Madani, who was arrested in connection with Coimbatore bomb blast, during an election rally in 2009.
"How could the topmost leader of the CPI(M) share a dais with a person who was a principal conspirator in killing of 38 people?" Mr Prasad asked.
The Minister said there has to be some consensus between the Centre and states on the issue of terrorism and national security.
Noting that India's fight against terrorism and radicalisation is being appreciated the world over, Mr Prasad said, "Therefore, purely for vote bank politics, if you seek to compromise on the issue of terrorism, that is a serious matter. Only in that larger light I sought to raise this question," he said.
Mr Prasad said there were radical movements, seeking to radicalise younger minds in Kerala for extremist purposes. "I would expect from the government that the wide held perception of apprehension must be dispelled by fair investigation," the Minister said.