The deal was reached after intense last-minute negotiations to settle a dispute between India and China over the headquarters of the new bank, which will have initial capital of $100 billion to invest in infrastructure projects.
"The agreement towards setting up the BRICS New Development Bank is a significant step. I am happy the initiative announced in 2012 in Delhi has become a reality," PM Modi said.
BRICS is a group of emerging economies and draws its name from the five member countries - Brazil, Russia, India, China and South Africa.
Negotiations to create the bank dragged on for more than two years as Brazil and India fought China's attempts to get a bigger share in the lender than the others. Negotiations had also been stalled over who would host the bank - China, India or South Africa.
There were also tough negotiations over shareholding. China argued that the economic strength of a member nation should be the criteria for contribution to the bank - a higher contribution will automatically mean greater control. But India demanded that each member have an equal share.
BRICS is also considering starting a joint infrastructure fund with initial capital of around $10 billion, a source close to the discussions told Reuters on Tuesday. The money would be invested in infrastructure projects within the BRICS countries, including the construction of roads, bridges and airports.
The BRICS summit is Mr Modi's first multilateral engagement since he took over as Prime Minister in May this year. He met Chinese president Xi Jinping on Tuesday morning.