Addressing the All India Mahila Congress-organised "I am Courage" event, he said that the Congress would build pressure on the ruling party for an early passage of the Women's Reservation Bill or the Constitution (108th Amendment) Bill in Parliament.
Mr Gandhi, who will formally take over as the Congress president later this week, said as the party "don't have a woman president any more, we have to compensate for that".
He said that the party had exerted pressure on the government for bringing pro-people changes in the Goods and Services Tax, which he had earlier described as 'Gabbar Singh Tax'.
"The same way we will give a clear message to the government that it will have to bring women reservation. The Congress party will not give any choice to it. We will fulfil the role of the opposition party in doing so," he said.
This was Mr Gandhi's first address at a public function after being elected the Congress president.
He said he has done so since he accorded importance to women, who will have to play an important role in spreading the Congress ideology across the country.
Mr Gandhi said on one hand is the ideology of the BJP and the RSS, where women have no role and cannot enter their functions, while on the other side is the Congress ideology.
Hinting at a larger role for women in the Congress party, he said, "I have come here to give one clear message that the biggest change in the party will be for women."
Mr Gandhi's comments for a bigger role for women in the party came amid speculation that his sister Priyanka Gandhi Vadra may also play some active political role in the party under his leadership.
The Congress president-elect said there should be a road map for the coming times and will have to systematically and actively prepare women at every level in the party to contest elections and ensure they have their representation.
"We have to mention Congress President Sonia Gandhi Ji who has pushed your agenda, sometimes silently but mostly openly and she has helped push a number of women forward.
Unfortunately we don't have a woman president anymore. So we have to compensate for that and see woman chief ministers in states. We will work on that together," he said.
Mr Gandhi also lauded the Congress general secretary and secretaries in Gujarat, who launched a vigorous campaign in the state and connected very well with the workers and the people of the state, and said their act needs to be replicated in other states too. "I have seen the way the merger of the Congress state unit and people of Gujarat, I want such work to be replicated in every state - respect public and hear them out and together systematically contest elections," he said.
He also lauded the role of Sheila Dikshit as the Delhi chief minister for many years for which the party respects her, but who had to suffer a "nasty and untruthful" campaign against her.
Among others who were present at the function include Leader of Opposition in Rajya Sabha Ghulam Nabi Azad, former union ministers Anand Sharma and and Mahila Congress chief Sushmita Dev.
Former Lok Sabha Speaker Meira Kumar and former union minister Mohsina Kidwai inaugurated the "I am Courage" workshop today.