Police said that some party workers also waved black flags as PM Modi's motorcade approached the venue from the helipad where his aircraft touched down. The prime minister was in Daman to lay the foundation stone of some projects.
Police officials said that no one had been detained for the protest.
The president of the Daman and Diu unit of the Congress, Ketan Patel, however alleged that police had cordoned off his house and had detained party workers who had gathered there.
Mr Patel said that he, along with these workers, had planned to march to the programme venue from his house but the plan was foiled by the police.
He said it was to overcome this situation that the party decided to release black balloons near the venue.
Mr Patel said that the police allowed them to carry on with the protest march after the prime minister's programme concluded.