The Associated Chambers of Commerce and Industry of India (ASSOCHAM) says overall tourist arrivals are down 25 percent year-on-year, with holidaymakers opting instead to visit other Asian countries such as Malaysia and Thailand.
The fatal gang-rape of a 23-year-old Indian student by six men on a bus in New Delhi in December sparked outrage at the country's treatment of women, and since then there have also been other widely reported attacks.
A Swiss cyclist was gang-raped in Madhya Pradesh last month, while a South Korean tourist was allegedly drugged and raped in the same state in January by the son of the owner of a hotel where she was staying.
Such incidents have "raised concerns about the safety of female travellers to the country," said D.S. Rawat, secretary general at ASSOCHAM, which surveyed 1,200 tour operators from different cities.
The figures contradict monthly estimates compiled by the Indian tourism ministry, which found an increase in foreign tourists arriving in early 2013, although figures for March were not yet available.
February saw arrivals increase by 1.6 percent compared with last year, according to the ministry, still far lower than the 7.9-percent growth logged in February 2012.
Gour Kanjilal, executive director of the Indian Association of Tour Operators, also questioned the scale of the drop in visitors reported by ASSOCHAM, but said some potential visitors had been in contact to discuss concerns.
"There is a decline whenever any such incident takes place. Tourism becomes the first casualty," he told AFP.
The ubiquitous "Incredible India" marketing campaign has helped raise the number of foreign visitors over the past decade to about 6.6 million in 2012 -- albeit still far behind destinations such as China and Malaysia.
The concern over sex crimes and the lurid reporting of the crimes is a blow to government attempts to boost the tourism industry, especially during a period of lagging economic growth.
Rawat said deteriorating standards of safety and security were the main reasons for the apparent drop in tourists this year, although the global economic slowdown was also a factor.
Travel advisories from various countries stressing the need for their citizens to take care in India has "further aggravated" the situation, said the release from the industry body on Sunday.
If found nearly 72 percent of tour operators had a number of cancellations in the last three months -- usually a busy tourist season -- especially by female visitors from countries such as Canada, the United States and Australia.