A scuffle erupted between BJP supporters and the agitators when Ghosh, heading a party delegation, arrived at the venue in the evening where a 'Vijaya Sammelani' of party workers was scheduled to be held.
As soon as he arrived, the agitators shouted slogans demanding that the BJP delegates leave the hills and showed them black flags. The BJP supporters were far outnumbered by the agitators, who beat them up.
The BJP state chief tried to pacify the agitators but failed. He was then heckled following which he cancelled the meeting and lodged a complaint at a nearby police station.
Mr Ghosh told reporters, "All of a sudden they rushed to the dais and heckled everybody including me. I tried to pacify them but they kept shouting slogans and pushing others. I left the venue and walked to the nearby police station."
He said, "On the way my party supporters, who were with me, were brutally beaten up by the agitators."
Darjeeling Superintendent of Police Akhilesh Chaturvedi said, "There was scuffle between two groups and a few people were injured. A case has been registered and we have started the investigation. The culprits will be arrested soon."
Rallies were taken out by the BJP in Kolkata as well as in Hoogly, Burdwan and several other places in the state to protest against the heckling of Ghosh. The party demanded immediate arrest of the culprits.
An angry Mr Ghosh alleged that the attack on him and the BJP leaders was "pre-planned" and that he was not provided any security despite being an MLA.
"It was pre-planned attack. And I think this planning has its roots in Kolkata. The administration and police were just mute spectators as our supporters were beaten up and I was heckled. I was not given any security cover. I have lodged a police complaint about the incident," he said.
Mr Ghosh claimed that the agitators were supporters of expelled Gorkha Janmukti Morcha (GJM) leader Binay Tamang. Mr Tamang denied the allegation.
Asked whether Mr Ghosh was provided security for the meeting, Mr Chauturvedi said, "The police was outside the venue.
We had also provided security for those programmes which were held earlier and we had information. He (Ghosh) will be provided complete security."
Reacting to the incident, state Parliamentary Affairs Minister Partha Chatterjee said, "The BJP had gone there to foment trouble when peace was returning to the hills. Instead of taking police personnel along with him, he (Ghosh) had taken hired goons. If he doesn't ask for security then how can the police provide security cover?"
Senior north Bengal Trinamool Congress (TMC) leader and Minister Gautam Deb said, "It (the scuffle) was an outburst of anger of the hill people against the sinister designs of the BJP."
He claimed that the people of the hills were with the BJP and the GJM and that was why their leader received "so much" support wherever they were going.
He said hundreds of people in the hills greeted the BJP leaders and shared their problems. The response was tremendous, he said.
Earlier in the day while entering Darjeeling from Kalimpong, Ghosh and his team faced protests by supporters of the Gorkha National Liberation Front (GNLF), who like some other hill parties demanded that the BJP leaders leave Darjeeling immediately.
Mr Ghosh and his team were also shown black flags and asked to refrain from disturbing the "peace and stability" in the region.
They had faced protests yesterday too by supporters of the Jan Andolan Party and Binay Tamang during their visit to Kalimpong. Posters with the messages such as 'BJP go back' and asking the hill people to boycott the party's delegation were put up.
The state BJP chief had, however, refuted the GNLF allegation that he was in the hills to foment trouble and claimed that the protests against the BJP delegation were fuelled by the ruling TMC.
Mr Ghosh claimed it was due to his initiative that peace has returned to Darjeeling. "I talked to the Union home minister, after which peace returned to the Darjeeling hills," he said.
The BJP team is scheduled to visit various places in the hills and neighbouring Sikkim and hold meetings with its workers and the GJM leadership.
Expelled GJM leader and vice-chairman of the Board of Administrators (BoA), Anit Thapa also accused the BJP of trying to foment trouble in the hills.
"What was it (BJP) doing for the three-and-a-half months when the hills were burning? Now that peace has returned, it wants to create trouble," he alleged.
GNLF spokesman Niraj Zimba later said his party had protested in the morning as it felt the BJP was trying to destablise the peace in the hills.
Mr Zimba, however, condemned the violence that took place in Darjeeling and said in democracy Mr Ghosh has every right to conduct a political programme.
A BJP leader said, "We want immediate arrest of the culprits. The way the attack took place suggests that it has been a planned one and the administration was a silent spectator."
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.