The All India Mahila Congress said PM Modi should apologise to the nation and to Ms Chowdhury for his "Ramayana" comment.
The remark belittled the office of the prime minister and was an insult to every woman in the country, Mahila Congress president Sushmita Dev said.
Ms Chowdhury had burst out laughing when PM Modi claimed in the Rajya Sabha yesterday that the concept of Aadhaar was mooted in 1998 by then home minister L K Advani.
The loud guffaw prompted PM Modi to say he had not heard such laughter since the serial 'Ramayana'. Demonic laughs often figured in the series, telecast in the late eighties.
"Ramayana serial ke baad aisi hansi sunne ka saubhagya aaj jaake mila hai. (After Ramayana serial, we've now had the privilege of listening to this kind of laughter)," he said.
Ms Dev described the comment as "utterly sexist" and "downright distasteful".
"I express my shock at the utterly sexist and downright distasteful comment against an honourable Member of Parliament, made by none less than the honourable Prime Minister of India," she said in a statement.
She said PM Modi should apologise to Ms Chowdhury and the nation "for speaking in a manner which encourages disrespectful behaviour towards women".
Ms Dev also said she was not surprised by the remark because the prime minister had never understood the role a woman played as an equal partner in society.
She also hit out at the RSS, saying that its thought process militated against the idea of equal positions for women in society.
Ms Dev went on to add that RSS chief Mohan Bhagwat had reportedly said women should restrict themselves to doing household chores.
"It is this RSS training, as a pracharak that reflects in the PM's attitude and comments," Ms Dev said.