The whales were among a pod of 10 spotted by locals stranded along Ujong Krueng beach in Aceh province Monday, with one seen farther out in the water.
Rescuers tried to push the nine beached whales back out to sea as hundreds of curious locals looked on, some snapping pictures.
Five were pushed back and survived but four died, some with their mouth agape as small waves crashed over their giant bodies.
Rescuers tied ropes to the tails of some of the mammals to pull them to deeper water.
"We had problems evacuating them due to a lack of experience and equipment," Aceh nature conservation agency head Sapto Aji Prabowo told AFP.
"But this is a good lesson for us because Aceh is a crossing point for marine mammals so in the future we should be prepared to deal with situations like this."
An official autopsy will be conducted on the whales to determine the cause of death, but Prabowo said the giant gray creatures may have followed their leader to shore or lost their way.
One whale had wounds that suggested it had been injured by a coral reef, he said, adding that the four dead will be buried close to the beach.
This is the second time in the last few years that sperm whales have been found beached in northernmost Sumatra island, with one found dead on a beach in Banda Aceh in 2016.
Also last year, eight pilot whales died after a mass stranding on the coast of Indonesia's main island of Java.
They were among a group of more than two dozen short-finned pilot whales that had come ashore during high tide.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)