By morning, ISIS had withdrawn to residential neighbourhoods in the east of the city, the Syrian Observatory for Human Rights said.
"ISIS withdrew from most of Palmyra after laying mines across the city. There are still suicide bombers left in the eastern neighbourhoods," Observatory head Rami Abdel Rahman told AFP.
"Government forces have not yet been able to enter the heart of the city or the eastern parts."
Palmyra's ancient ruins have long been listed by UNESCO as a world heritage site.
Before ISIS entered the city in May 2015, they boasted temples, colonnaded alleys and elaborately decorated tombs that were among the best preserved classical monuments in the Middle East.
But the terrorists launched a campaign of destruction against them, the scale of which was fully revealed when government forces briefly retook the city with Russian support last year.
Satellite imagery has shown that ISIS has demolished more monuments since it recaptured Palmyra from government forces in December.
"There are no ISIS fighters left in most of the Old City, but it is heavily mined," the Observatory chief said.
Supported by Russian air strikes and ground troops, government forces have been battling through the desert for weeks to reach Palmyra.
On Wednesday, a senior military source in Damascus told AFP the army had reached a strategic crossroads leading into Palmyra.
"This crossroads is the key to entering the city," the source told AFP.