Television footage showed thick smoke pouring from the Hao Han, a 1,999-tonne cargo vessel, as the ship listed badly in waters off Osaka, in Western Japan.
Japan's coastguard, which has recently been heavily involved in a standoff with Beijing over disputed islands in the East China Sea, pulled the crew to safety and was tackling the blaze.
"There were no injuries among the crew," a coastguard spokesman said, adding the first information about the accident had come in on Thursday night.
The cargo vessel, which left Kagoshima in southern Japan last Friday and docked at two other Japanese ports before sailing for China, was loaded with 1,000 tonnes of scrap metal, he said.
The accident comes as a bitter territorial dispute between Japan and China over the ownership of an island chain showed little sign of abating.
China took the row to the UN General Assembly on Thursday, with Foreign Minister Yang Jiechi accusing Tokyo of stealing the islands, known as Diaoyu by Beijing and as Senkaku in Japan.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.