Minister Anisul Huq said the Anti-Corruption Commission (ACC) will investigate "almost all of the allegations" levelled against Sinha by the country's president Abdul Hamid.
"Nobody is above the law," Huq told reporters after the top court issued the statement, sparking a media uproar.
He, however, said "it is the prerogative of the President to take (the subsequent) actions if the charges are proved".
The government's row with the judiciary sparked in July this year when the top court delivered a verdict declaring void 16th constitutional amendment, scrapping parliament's authority over impeaching Supreme Court judges.
In an unusual move, the country's Supreme Court (SC) said yesterday the top court judges earlier this month decided not to sit in the bench of Sinha over 11 "grave" charges brought to their notice against him by the President.
The Supreme Court issued a statement as Sinha overnight left for Australia, saying he was "embarrassed" over the controversy surrounding his July ruling. He also denied claims by the government that he was sick.
"I am not sick (as claimed by the government) and I am not fleeing either," Sinha said while leaving, adding that his departure was "temporary".
"What he (Sinha) said (ahead of his departure) is a lie," Huq said reading out Sinha's letter to the president in which he had conveyed to him about his sickness and willingness to go abroad.
Asked if Sinha could resume office on return after the one-month leave, Huq answered in negative, saying "he does not have the scope to resume office until the disposal of the allegations".
Huq, however, said since the Chief Justice's office is an "institution" nothing would be done in haste and all steps would be taken in adherence to the law of the land.
The Supreme Court statement said President Hamid on September 30 had invited all the five top court judges to Bangabhaban presidential palace barring the chief justice and held a long discussion.
The five judges, it said, then met the chief justice at his official residence on the same day and raised the matter seeking his explanation.
"But despite long discussion the five judges of Appellate Division didn't get any acceptable explanation or reply from him (Sinha) and so all of them clearly conveyed to him that until the disposal of those charges it will not be possible for them to share with him the bench to deliver justice," it read.
Sinha told them that "he will resign" but he would let them know about his final decision on October 2.
"(But) on October 2, 2017, without informing anything to those aforesaid judges, he filed an application to the honourable president seeking one month leave and the president subsequently approved it," the statement said.
The July verdict declaring void 16th constitutional amendment was hailed by lawyers as a crucial step towards a secular judiciary in the Muslim-majority nation.
The dispute grew in the subsequent weeks as several senior government leaders virtually attacked Sinha over his comments blasting the government for its reaction and referred to Pakistan's instance where former prime minister Nawaz Sharif was stripped of his premiership under an top court ruling.
Prime Minister Hasina accused him of defaming parliament and president and "humiliating" Bangladesh by referring to Pakistan's instance premier's removal saying "he (chief justice) should have quit (and) the most humiliating thing is the comparison with Pakistan which is intolerable".
Sinha, Bangladesh's first Hindu chief justice whose tenure expires in January 2018, in his written statement said he was a "bit worried about the independence of the judiciary".
He complained that the judge who became the acting chief justice to perform the "routine" job in his absence was encouraged by the government to bring changes to the Supreme Court administration soon.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.