Heavy machinery and cranes were moved to the compound, located about 800 yards from the elite Pakistan Military Academy, this evening as security agencies closed all roads leading to the house in Bilal Town, local residents told the media.
Powerful spotlights were put up and a large number of security personnel, including army soldiers, were deployed in the area before the demolition began at around 9 pm.
"Osama compound at Bilal Town in Abbottabad city has been dismantled," state-run Radio Pakistan announced on Twitter.
The private Geo News channel reported that authorities first tore down the room on third floor where the world's most wanted man was gunned down by US Navy Seals.
Local residents and reporters said authorities had directed people living in houses near the compound not to go out on their rooftops.
Dunya News channel aired grainy footage of a section of a balcony of the three-storey building being demolished.
The Pakistani military had taken control of the compound shortly after the al Qaeda chief was killed in a pre-dawn raid by US troops on May 2 last year.
There were rumours on at least two occasions in the past that the compound would be demolished so that it did not become a shrine for jihadists.
The media was given limited access to the compound after it was briefly handed over to local police by the army for a few days in May last year. However, soldiers were redeployed at the site and people were barred from approaching it.
Pakistani intelligence agencies had thoroughly searched the compound before giving American officials limited access to it last year.
Media reports had said that the CIA maintained a safe house for a small team of spies who conducted extensive surveillance on the compound over a period of months.
In the past, Pakistani security agencies had demolished Jamia Hafsa, a seminary affiliated to the radical Lal Masjid in Islamabad, after a military operation in the mosque in 2007.
The walls of bin Laden's compound were higher and thicker than those of every house in Bilal Town.
The compound consisted of three portions a big open area for farming, a built up structure and a lawn on the north side that was smaller than the farming area.
There was no other compound of the same size in the area. The land for the compound was bought by Arshad Khan, one of two Al Qaida couriers killed with bin Laden.
The house's gas and electricity connections too were in his name. Arshad Khan and his brother Tariq Khan were tribesmen and belonged to the Wazir tribe of South Waziristan Agency bordering Afghanistan.
Noor Mohammad, the contractor who built the compound in 2005, said Arshad Khan had paid him Pakistani Rupees 750,000 for the project.