The three attackers escaped in a Hyundai car from the Imam Hussain mosque in Verulam town on the outskirts of the eastern port city.
Police said the men entered the large mosque after midday prayers and stabbed three people.
"They further set certain rooms alight and fled in their getaway vehicle," police spokeswoman Nqobile Gwala said in a statement.
"The motive of the attack on the three men is unknown at this stage. Verulam police are investigating three counts of attempted murder and arson."
Paul Herbst, spokesman for the private IPSS medical rescue service, told AFP that the imam "who had his throat slit, passed away a few minutes ago due to his injuries."
"There were three people in the mosque -- a caretaker, a worshipper and the imam and they were held up by three armed men," Herbst said after visiting the scene.
"The priest was tied up and thrown to his knees and his throat was slit. The other two men were also stabbed -- one was stabbed in the abdomen and the other in the groin of his left leg.
"Before the assailants left the mosque, a petrol bomb was thrown into the mosque and it was set alight.
"The attackers managed to get out of the mosque. The imam was in a critical condition. He was treated by paramedics on the scene and transferred (to hospital).
"Fire services were on scene and put the blaze under control."
Prem Balram, spokesman for the Reaction Unit South Africa security company, said he had arrived at the mosque to find two victims covered in blood lying in the forecourt of the mosque.
He said a third victim had been attacked inside the building, which was then set alight, and had jumped out of a window.
"Eyewitnesses said three attackers with guns had used knives. One knife was left behind," Balram told AFP.
"There have been theft and robbery incidents at mosques before, but not like this, when nothing appears to have been taken."
South Africa has a history of relative religious tolerance. About 1.5 percent of the country's population is Muslim.
The Muslim holy month of Ramadan is due to start next week.
"It is the first time anything like this has happened in South Africa, let alone in KwaZulu-Natal province," Faizel Suliman, chairman of the SA Muslim Network, told AFP.
"It was a sudden attack and at this point we don't have enough information. Speculation about the motive is quite dangerous at this point."
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.