Mr Jaitley is a Rajya Sabha member from Gujarat.
Mr Sinha, who has been criticising the Modi government's economic policies, during an interaction with reporters, also alleged that the GST was rolled out without giving due consideration to each and every aspect.
Indian economy received "two shocks" in the form of demonetisation and GST in quick succession, said Mr Sinha, a former union finance minister.
Mr Sinha was invited to poll-bound Gujarat by the activists associated with 'Lokshahi Bachao Andolan' (Save Democracy Movement) to share his views about the current state of economy and impact of demonetisation and GST. "Our finance minister is not from Gujarat, as he was elected to Rajya Sabha from here. And, he is a burden on the people of Gujarat. If he was not elected from here, a Gujarati would have got the chance," Mr Sinha said while responding to a question.
Mr Sinha said the finance minister believes in only one rule, "that is 'chitt bhi meri patt bhi meri', means heads I win, tails you lose."
The minister takes credit for everything, even if something is not implemented properly. If proper attention was paid while fixing the GST rates, such "anomalies and anarchy" could have been avoided, Mr Sinha said.
"He cannot take credit for imposing on the country a deeply flawed tax system, and people of this country are well within their right to demand that he should lose his job," the veteran leader said.
On Mr Jaitley's earlier remark that Mr Sinha is looking for a job at the age of 80, Mr Sinha, who insisted that he is not part of any 'Margdarshak Mandal', said he is still fit unlike those "who give their speech while sitting", an apparent reference to Jaitley, who had sat down in between his budget speech due to health reasons. "People who give their speeches in sitting position are talking about my age," he said.
Without taking any name, Mr Sinha said attempts were made by some people to create a rift between him and his son - Minister of State for Finance Jayant Sinha.
Yashwant Sinha had written an article, criticising the Modi government over the state of economy. In that article, Mr Sinha, who was the finance minister under the Atal Bihari Vajpayee-led NDA government, had criticised Mr Jaitley over the "mess the finance minister has made of the economy" and said he would be failing in his national duty if he did not speak up.
The BJP veteran also claimed that the decision of demonetisation has failed to serve its purpose. He also accused PM Modi of shifting goal posts. "There is still a question mark on the success of demonetisation. Be it black money, or fake currency or eliminating terrorism, all these problems are still there."
He even suggested to form a committee under the guidance of GST architect and former chairman of 13th Finance Commission, Vijay Kelkar, to find out the flaws and include the suggestions of that committee in the next union Budget.