Mr Sreekumar was accosted by the group when he was about to leave the venue of a public function where he was a guest speaker. He was threatened, verbally abused and jostled around, the police said.
"If I did not have a human chain of protection around me, they would have even physically attacked me. But I am not scared," Kureepuzha Sreekumar said.
The arrests took place this morning, after the 62-year-old poet filed a complaint with the police. "Of the six persons arrested, one is a BJP and panchayat member," said local police officer B Asokan.
Condemning the attack, the Chief Minister wrote in a Facebook post, "The RSS attack on Kurishapuzha Sreekumar is condemnable. 6 BJP-RSS activists have been arrested... the state government will do everything to muzzle such attacks on freedom of expression.
The wall was cutting off access to an adjacent field on which public functions were held. The Dalits objected, pointing out that the field was public land and a wall would cut off their access to it. The wall was dubbed a "caste wall" by Dalits -- indicating it was being built to keep out the lower castes.
On Sunday, a protest was planned in the area, but the protesters did not get clearance from the police. As they gathered, a police crackdown began. While it was on, activists of Rashtriya Swayamsevak Sangh, the ideological mentor of the BJP, jeered and raised anti-Dalit slogans, the poet said in his speech.