The founder of Bandhan Financial Services - Bandhan means "bonding" - must transform his microfinance company into a bank in just 18 months to serve India's millions of "unbanked poor" under the licence terms.
"For a microfinance company, getting a banking licence is like achieving the Holy Grail," Mr Ghosh said.
India's central bank created a stir last month when it handed one of its first banking licences in a decade to the microfinancier based in Kolkata, bypassing heavyweight contenders such as billionaire tycoon Anil Ambani.
The second licence went to India's biggest lender to road projects, IDFC.
Reserve Bank of India governor Raghuram Rajan wants these new banks to penetrate poverty-hit rural areas to promote lending and saving among farmers, small businesses and others whom traditional banks have been reluctant to serve.
Just 35 per cent of India's adults have bank accounts, a rate the central bank calls "pathetic".
Mr Ghosh aims to open nearly 700 bank branches in 22 states and start with at least 10 million savings accounts.
Under the licence conditions, one in four branches must be in towns with fewer than 10,000 people. But Mr Ghosh plans to go further, setting up 80 per cent of his branches in rural areas, although there will also be "signature" branches in cities to raise Bandhan's profile.
The 53-year-old has come a long way from serving in his father's sweets shop.
Money was scarce at home when he was growing up, but he earned a master's degree in statistics. He then worked for non-governmental organisations where he saw close-up exploitation of the poor by moneylenders who charge up to 200 percent a year for loans.
Dismayed by the predatory loan sharks, he created in 2001 a small lending outfit with Rs. 2 lakh in capital and two employees.
At first he borrowed from friends to have funds to lend. But he finally got his big break when the state-run Small Industries Development Bank of India put up capital - allowing him to expand his loan portfolio.
Bandhan slowly became India's largest microfinancier, focusing on the northeast and then expanding across the country.
"I'm not an economist, I'm not a banker - I learn from my customers," he said.
Mr Ghosh recruited poor people to run his branches, declaring only they "understood poverty" and could identify the needy.
As Bandhan becomes a bank, Mr Ghosh said some staff will need training to deal with more sophisticated financial products. Just three-fifths of his employees reached the final grade in school.
"Having a degree has never been the most important qualification for a job with us," Mr Ghosh said.
The beauty of becoming a bank means cheaper lending rates. Now, as a microfinancier, Bandhan must borrow from other financial institutions to make loans, which means its interest rates are as high as 22 per cent annually.
But as a bank Bandhan will have its own resources from depositors.
Mr Ghosh will attempt to use Bandhan's 52 lakh borrowers to build his savings account base. He already has a Rs. 5,700 crore loan portfolio and a staff of 12,961 working in his more than 2,100 microfinance branches.
"If we get two family members from our client base opening accounts, we'll automatically have 1 lakh accounts," said Mr Ghosh.
But Bandhan's bank licence is key for another reason - it signals redemption for India's microfinance sector. Two years ago, alleged strong-arm debt recovery tactics by the then biggest microfinancier, SKS, drove some farmers to suicide and brought the sector to the brink of collapse.
Microlenders have since been forced to significantly drop their lending rates thanks to strict new guidelines.
The risk for Bandhan as it expands is that most of its borrowers are unsecured - they have no assets to pledge against their debt.
But Bandhan already enjoys a top credit grading and a near zero default rate because under its current system an entire local community pledges to keep watch over one of its members who is borrowing.