Musharraf, who lives in Britain in self-exile, said the "rogue" lower-level members of the powerful ISI and military might have known about bin Laden's location during the last year of his Presidency six years ago.
"It's really appalling that he was there and nobody knew. I'm certainly appalled that I didn't know and that intelligence people from that time onward didn't know for six years that he was inside.
"And there is no excuse for this great, massive slip-up. And an investigation is in order and people must be punished for this big lapse," Musharraf told ABC News.
As a policy, the army and ISI are fighting terrorism and extremism, including Al Qaeda and Taliban, he said. "But rogue element within is a possibility," he said.
According to the former President, there were three attachment commanders in Abbottabad in the past six years and he could not imagine all three knowing and harbouring bin Laden.
Regardless of who knew what, according to Musharraf, was the fact that the US raid was a possible violation of Pakistan's sovereignty and that there was never a deal struck during his tenure to allow the US to make a unilateral attack on Pakistan's soil if bin Laden was found.
While calling the Al Qaeda chief's six-year residence in Abbottabad a "big blunder" on the part of Pakistani intelligence, Musharraf warned the United States that if it continues to alienate Pakistan as it did in the bin Laden raid, the US will be the "loser."
"You want to alienate Pakistan, you will be a loser," he was quoted as saying.
Follow NDTV for latest election news and live coverage of assembly elections 2019 in Maharashtra and Haryana.
Subscribe to our YouTube channel, like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.