"Why shall I leave BJP? What this party means to me, (it) possibly does not mean to anyone (else). I have given my blood and sweat for this party. From 2004 to 2014, we were fighting against the then ruling party (Congress)," Mr Sinha said.
"Then some others came. But now that they have come, they must deliver on the promises they made. And if they don't, I will raise those issues whether I'm with them or against them," he said.
On whether his criticisms could hurt the BJP government in elections, Mr Sinha said, "Governments are formed to fulfil the promises they made in the election manifesto. But if it is not happening, then it is also a responsibility to bring the party on the right track."
Mr Sinha, who held the Finance and External Affairs portfolios in Prime Minister Atal Bihari Vajpayee's cabinet between 1998 and 2004, said the Rashtra Manch or National Forum floated by him is not an organisation but a movement over the issues which are impacting the country at the moment.
"It is already pan-India since it saw participation of representatives from other political parties, farmer associations and many intellectuals from the country. We are creating ground for the movement and we have to reach the masses to inform them on the issues, so that improvements can be made," he said.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.