Khader Adnan of the Islamic Jihad faction was taken into custody near al-Aqsa mosque in the Old City in East Jerusalem, which Israel annexed as part of its capital - a move not recognised abroad - and where Palestinians seek independence.
A police spokeswoman said he was detained for lacking an Israeli entry permit and would be deported to the occupied West Bank, where he lives.
"He is being questioned and, when that concludes, will be transferred to the (Palestinian) territories," she said.
On Sunday, Adnan was released from an Israeli prison after staging a 56-day hunger strike in protest at being held without trial under so-called administrative detention, a method Israel says it employs as a security measure to prevent violence.
Israel arrested Adnan, 37, last July for the 10th time. Both sides had feared that his death from starvation would hurt a shaky Gaza truce or spur further violence.
Adnan is a known Islamic Jihad figure in the West Bank, captured by Israel in a 1967 war and among territories where Palestinians seek statehood. Like Islamist Hamas, Islamic Jihad opposes peace deals between the Palestinians and Israel and advocates the destruction of Israel.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.