Thakor Solanki, general secretary of SC/ST the Employees Union of Maharaja Sayajirao University of Baroda, claimed that they had gathered at the venue to pay their tributes before the BJP leaders.
The members of the Dalit community raised slogans at the GEB Circle area on Race Course against other party leaders and Ms Gandhi, the Women and Child Development minister who was in the city to attend a number of events.
She arrived at Ambedkar's statue along with BJP MP Ranjanben Bhatt, city mayor Bharat Dangar, BJP MLA Yogesh Patel and others.
The members of the Dalit community, led by Mr Solanki, started raising slogans against them. An argument broke out between them and the police personnel, but no untoward incident took place.
"We told the police that since we had gathered there much before the BJP leaders came, it was our right to pay tributes first. However, police prevented us from garlanding the statue stating that as per the protocol, it was the mayor's right. This angered our members," Mr Solanki said.
"The statue and the atmosphere in the GEB circle area was polluted after the arrival of Gandhi and other BJP leaders. Therefore, we cleansed Ambedkar's statue with milk and water after it was garlanded by the BJP leaders," he added.
Before Ms Gandhi's arrival, Jivraj Chauhan, general secretary of SC/ST cell of BJP's state unit, was also gheraoed by the Dalit members, who shouted slogans against him, forcing him to leave the venue.