Mergers and consolidation may be among options for financial institutions to boost profitability, Mr Kuroda said in an unusually frank call for bolder steps to deal with Japan's over-crowded regional banking sector.
Faced with low inflation and tepid economic growth, many central banks like the BoJ, the Federal Reserve and the European Central Bank, have adopted unconventional monetary easing steps since the global financial crisis in 2008.
While the measures themselves were necessary to revive growth, the resulting plunge in interest rates have hurt profits at financial institutions by narrowing their margins, Mr Kuroda said.
"A new challenge has emerged in the form of low profitability at financial institutions," Mr Kuroda said, adding that rapid growth in shadow banking and new financial technology were bringing big changes to the global banking environment.
"These developments suggest that a different kind of financial crisis could happen in the future," he told an international conference on deposit insurers on Thursday, without elaborating.
Mr Kuroda said the problem of low interest rates hurting bank profitability was a global one, pointing to bad loans piling up at some European banks and headwinds plaguing Japanese banks, for instance, from sluggish lending driven by an ageing population.
"For the financial system to ensure future stability, it is becoming more and more important in the long term to think about possible responses to low profitability at financial institutions," he said.
Four years of aggressive money printing by the BoJ have failed to pull Japan sustainably out of stagnation, forcing the central bank to revamp its policy framework to one better suited for a long-term battle with deflation.
But the attempts to revive Japan's anaemic consumer spending through unconventional monetary policy have created new problems for the central bank in its dealings with markets and financial institutions.