Pasupathi Pandian, founder of Devendra Kula Velalar Kootamaippu (Federation), was sitting outside his house in an isolated locality when he was attacked and killed, they said.
Police said they suspected the gang monitored the Dalit leader's movements before attacking him.
Meanwhile tension spread as news of the Dalit leader's death spread.
Pandian's wife Jesintha was killed in a bomb attack about two years back in Tuticorin district.