Prime Minister Benjamin Netanyahu's cabinet voted to remove the metal detector gates after a meeting lasting several hours convening for a second time on Monday after they had broken off discussions a day earlier.
The security cabinet accepted "the recommendation of all the security bodies to change the inspection with metal detectors to a security inspection based on advanced technologies and other means," a statement from Prime Minister Benjamin Netanyahu's office said.
Tensions escalated sharply since Israel installed the metal detectors at entry points to Al-Aqsa mosque compound in Jerusalem after two police guards were shot dead by gunmen there on July 14.
Palestinians view the new security measures as Israel asserting further control over the site. They have refused to enter the compound in protest and have prayed in the streets outside instead.
Israeli authorities said the metal detectors were needed because the July 14 attackers smuggled guns into the site and emerged from it to shoot the officers.
The decision to remove the metal detectors follows talks between Mr Netanyahu and Jordan's King Abdullah II.
Jordan is the official custodian of Muslim holy sites in Jerusalem.
(With inputs from Reuters and AFP)