Mr Chowdhury, who is the Junior Minister for Railways, was present when a group of Congress workers used lathis or batons and stones to smash flower pots and the windshield of a jeep. (Watch video here)
Mr Chowdhury has said he is "ready to go to court or be arrested." Last night, he told NDTV that what took place does not amount to vandalism and that he "will not apologise to anybody."
The minister climbed onto the jeep that was attacked and urged his supporters to calm down, but was unsuccessful.
The trouble began when the District Magistrate in Behrampore, Rajiv Kumar, said he was not available to receive a written complaint from Mr Chowdhury about the death of a Congress worker in police custody
Mr Chowdhury who was addressing a rally in the area then led a group of Congress workers to the District Magistrate's home.
15 minutes after the ransacking began, the police arrived at the District Magistrate's home-cum-office, but was outnumbered by the mob.
The crowd was pacified only after the District Magistrate met the Congress delegation and accepted its memorandum.
Over the years, Murshidabad has been largely dominated by the Congress party, with neither the Trinamool Congress nor the Left able to make its presence felt in the district.
Mr Chowdhury has been winning the Lok Sabha seat in Berhampore since 1999.
By-elections to the Rejinagar assembly seat in Mr Chowdhury's constituency, in Murshidabad district, will take place on February 23.