"When we hear about these rapes, our heads hang in shame," the PM said in an unscripted speech from Red Fort in the capital. (Highlights of PM Modi's speech)
"Those who commit rape are also someone's sons. You should stop them before they take the wrong path," the PM said at a time when sexual violence against women and children has ignited national outrage and debate.
"Even when they are only 12, young girls are always being asked so many questions by their parents, like 'where are you off to?'. But do these parents ask their sons where they are going?" he asked. "The law will take its own course but as a society every parent has a responsibility to teach their sons the difference between right and wrong," the PM said.
After the fatal gang-rape of a young student on a Delhi bus in 2012, new laws were introduced to punish sexual offences. But gang-rapes have been reported since then in Mumbai, considered the country's safest city for women. Last month, a six-year-old was raped in her school in Bangalore, triggering national anger and protests.
In May, two teenage cousins were found hanging from a mango tree after they were gang-raped in their village in Uttar Pradesh. The girls, who were aged 14 and 15, were attacked while going to the toilet in fields after dark as -- like hundreds of millions of Indians -- they did not have a toilet in their own home.
"We are in the 21st century and yet there is still no dignity for women as they have to go out in the open to defecate and they have to wait for darkness to fall. Can you imagine the number of problems they have to face because of this?" the PM said. "People may criticise me for talking about toilets from the Red Fort. But I am from a poor family, I have seen poverty first hand. For the poor to get dignity, it has to start from here."
Mr Modi said India should strive to ensure that every household should have a toilet within the next four years and pledged to ensure that all schools had separate toilet facilities for girls and boys.