The thieves knocked down the 16th-century entrance to a temple in the sprawling ruins of Hampi, 350 kilometres (220 miles) from the Karnataka state capital Bangalore, a district superintendent said.
"We have formed two police teams to hunt down the culprits who vandalised the 15-feet entrance tower assuming that priceless treasure was hidden in its foundations," Chandragupta, who uses one name, told AFP.
The state archaeology department was trying to verify if any treasure in the form of gold or silver coins and priceless antiques was indeed buried under the pillars as popularly believed, Chandragupta added.
Located on the banks of Tungabhadra River, the ruins of Hampi were listed by UNESCO as a world heritage site in 1986 for their grandeur and breathtaking medieval architecture.
The ruins are a favourite with local and international tourists as well as pilgrims and historians keen to know more about Hampi, once the seat of the mighty Vijayanagara empire, which between the 14th and 16th centuries was south India's wealthiest and most powerful Hindu kingdom.
The treasure hunters dug around one of the four pillars of the structure and tried to lift it, causing the structure to collapse.
Local police inspector Venkateshulu said tools including crowbars and hammers were left behind by the gang after their raid on Saturday.
A dozen security men have been deployed to guard the vandalised spot.
Hampi Mahesh, a member of a group which fights for protection of the heritage site, told the Hindustan Times newspaper that treasure hunters were repeatedly trying to steal antique pieces.