The revelation shocked Washington just three days after the re-election of US President Barack Obama and shortly before Petraeus had been due to testify on the CIA's alleged failure to properly protect a US consulate in Libya.
"After being married for over 37 years, I showed extremely poor judgment by engaging in an extramarital affair," Mr Petraeus said in a message to CIA staff, released to the media.
"Such behavior is unacceptable, both as a husband and as the leader of an organization such as ours."
Neither Petraeus nor the CIA explained exactly why he felt he had to step down over the affair, and whether his liaison presented a purely personal problem or raised security issues in his sensitive work as spy chief.
The affair came to light as the FBI was investigating whether a computer used by Petraeus had been compromised, the New York Times and other US media reported, citing government officials.
NBC News and other media reported the Federal Bureau of Investigation was investigating Paula Broadwell, who published a favourable biography of Petraeus, "All In: The Education of David Petraeus," for possible improper access to classified information.
Unnamed officials told the New York Times that Petraeus's lover was Broadwell, a former Army major who spent hours interviewing Petraeus for her book. She offered no public comment on the revelations.
The resignation comes amid criticism in some quarters of Petraeus over his response to the deadly attack in September on the US consulate in Benghazi, which killed the US ambassador to Libya and three other Americans.
Petraeus was due to face a tough grilling in a series of closed-door classified hearings in Congress next week, with some lawmakers questioning the former general's handling of the incident.
Intelligence officials have defended Petraeus and his officers over the incident, saying the CIA moved swiftly to rescue the Americans under attack at the Benghazi compound.
Michael Morell, Petraeus' deputy at the country's lead spy agency, will serve as acting director and will be appearing at the hearing instead.
Obama expressed his "utmost confidence" in Morell's leadership.
The president, fresh off his electoral triumph, reportedly had no inkling that the CIA chief was about to resign until Thursday morning.
When he met with Petraeus later that day, Obama refused to accept the resignation straight away, saying he would think about it overnight, the New York Times said.
But in the end, Obama concluded he could not push Petraeus to stay on, according to the Times.
As the White House faced an unexpected vacancy at the top of the spy agency, speculation on a possible successor focused on John Brennan, the White House counter-terrorism adviser and CIA veteran who has played an instrumental role in Obama's drone war against Al-Qaeda militants.
The most celebrated military officer of his generation, Petraeus, 60, took over at the CIA just over a year ago after retiring as a four-star general.
He was credited by some with rescuing a failing US war effort in Iraq in 2007, after then president George W. Bush ordered a surge of troops into the country.
Obama later turned to him to lead a similar surge of American forces in Afghanistan in 2010, leaving a top post as commander of all US forces in the Middle East to do so.
But Obama chose not to promote Petraeus to the US military's top job as chairman of the Joint Chiefs of Staff, as White House officials remained wary of the media-savvy general who had pushed for more troops and more time in the Afghanistan war.
It was a rare setback for the ambitious, hyper-disciplined officer, but he embraced the CIA job, adopting a much lower public profile.
His military background, however, sometimes clashed with the intelligence agency's culture and there was friction at times with the congressional committees that oversee the spy services.
After accepting the CIA chief's resignation, Obama hailed his "extraordinary service."
At the CIA, he had worked to shift the spy agency to a more "balanced" approach to intelligence gathering, after an intense focus on terror threats after the September 11, 2001 attacks, said Bruce Riedel, a former CIA officer and fellow at the Brookings Institution think tank.
"He had already begun to position the agency to live in the post-911 world," he told AFP.
Senator John McCain, a loyal supporter who championed the general's surge strategy in Iraq, heaped praised on Petraeus.
"General David Petraeus will stand in the ranks of America's greatest military heroes.
"His inspirational leadership and his genius were directly responsible -- after years of failure -- for the success of the surge in Iraq," McCain said.
Petraeus has long been suspected of harboring political ambitions but he has repeatedly fended off questions about a possible White House run.
Intensely competitive and known for his sharp intellect, he graduated from the US Military Academy at West Point in 1974, where he met his wife, Holly, the daughter of West Point's superintendent.