Mr Jaitley made the comment at a two-day meeting of public sector bankers convened by the Narendra Modi government to suggest a roadmap for reforms. The meeting ended on Saturday.
"There is a need for far greater autonomy being given to them," Mr Jaitley said at the meeting, which was also attended by Prime Minister Narendra Modi and Reserve Bank of India Governor Raghuram Rajan.
"There are unacceptable levels of NPAs (non-performing assets) in some cases. And the banks have to be given sufficient amount of leeway...to deal with commercial issues with a commercial mindset."
State-run banks recorded the highest level of stressed loans at 12.9 per cent of their advances in September last year, while the same ratio for private sector banks was at 4.4 per cent, according to RBI data.
Mr Modi told the bankers there would be no interference from the government, according to a statement.
More than two dozen state banks have been constrained by a pile of bad loans and governance issues. They also lag private sector rivals in profitability.
For years, political interference and union opposition have thwarted major reforms such as mergers and lowering of the government's holding to give more autonomy in decision-making.
The bankers suggested the government, whose stakes in the state-run banks range from 56 per cent to 84 per cent, should transfer its holding to a new investment company and over time cut its ownership to below 51 per cent.
The bankers also recommended creation of "independent high performing boards" that would drive capital raising and acquisition strategies. They also pitched for minimum interference by various government institutions.
Financial Services Secretary Hasmukh Adhia said Mr Jaitley told the public sector bankers their suggestions would be looked at "very positively". He did not elaborate.
If reformed, India's big state lenders offer investors the best exposure to any sustained upswing in the economy, analysts say.
Copyright @ Thomson Reuters 2015