Ms Sharma, CEO since 2009, had asked the board to shorten her next term that is set to begin on June 1, the lender said.
While Axis Bank was silent on Sharma's replacement, the new CEO's most pressing task will be to tackle the lender's high level of soured assets and deteriorated financials.
"We hope the new person is an outsider, allowing fresh thoughts and strategy and to reestablish old links and rebuild credibility with investors," Jefferies analysts led by Nilanjan Karfa wrote in a note.
Axis has seen its gross bad loans jump more than six times over the last three years to Rs 25,000 crore ($3.85 billion) as of end-2017. Investors have also been concerned after a central bank audit last year unearthed Rs 5,633 crore more in bad loans than what the bank had originally reported for the fiscal year ended March 31, 2017.
Most Indian banks have seen their bad loans surge over the past three years, leading to higher scrutiny and a crackdown by the regulators. Several other bigger lenders have also reported additional bad loans after central bank audit.
In Axis's case, several run-ins with regulators in the recent past have added to the woes.
The CEO change also presented the "best opportunity" for forging a combination with Axis's smaller rival Kotak Mahindra Bank which last year was speculated to be eyeing Axis for a takeover, analysts at Nomura wrote in a note, adding they expected "limited pushback" from Axis for a merger this time. Axis had last year dismissed reports of a deal calling them baseless speculation.
Kotak Securities said it expected the bank to move further towards de-risking the business by increasing the share of deposits and loans to retail customers and better-rated companies.
It upgraded the stock to "add", retaining a target price of Rs 600, citing current valuations after recent underperformance.
In afternoon trade, Axis Bank shares were trading 4.7 per cent higher at Rs 542, outperforming the banking sector index that gained 0.9 per cent and the main Mumbai market index that was up 0.3 per cent.
The stock is still down about 4 per cent so far this year.