The smoke was noticed on the fourth floor of the six-storeyed building at around 4.40 pm, the fire brigade officials department said.
Nearly 300 people were present inside the building, the bank official said, adding, all of them were evacuated safely.
"We were busy winding up the daily work, when we heard a blast-like sound but ignored it as it seemed a trivial issue. Meanwhile, a fire fighting system kicked in and our colleagues started shouting about the blaze, so we left the office immediately," another bank official said.
The 100-year-old building houses the main Mumbai branch of Bank of India.
More details awaited.