At a programme organised by traders in Lucknow, a person while raising concerns related to the business community mentioned that both PM Modi and BJP chief Amit Shah also belonged to the same section of the society.
"Tell them to make laws in favour of the community...
Amit Shah is from our community, but Modi is a 'Teli'," Mr Agarwal said prompted a strong reaction from some members of the audience who were apparently not pleased at the remarks.
Reacting strongly, UP BJP chief Mahendra Nath Pandey said, "We condemn the attempt made by SP general secretary and Rajya Sabha MP to link the name of the prime minister with a community, and demand that he should tender an apology."
In a statement, Mr Pandey said the prime minister of India does not belong to any particular caste or community and represents all the 125 crore people.
Later, speaking to reporters after the programme, Mr Agrawal seemed to defend his comments.
"Every person has a caste, and there is nothing new in this. If any leader is not telling his caste, then public will tell his caste...Let me give you a piece of advice, if you want to known about your three generations, then contest an MP or MLA election. The opposition will tell what you don't know.
We are not talking about the caste of the prime minister, people are talking about it," the SP leader, who has made controversial statements in the past, said.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.