Paula Broadwell and husband Scott were captured by television cameras returning to their North Carolina home on Sunday after hiding out at her brother's house in the capital Washington since the scandal broke.
"She's devastated and deeply regrets the damage that's been done to her family along with everyone else," ABC News reported, quoting an unnamed friend of the Broadwells.
Petraeus, America's most celebrated military leader in a generation, stepped down on November 9 as head of the CIA after admitting to an affair with Broadwell, a counter-terrorism expert and lieutenant-colonel in the Army Reserve who had co-authored a biography of the general.
The FBI stumbled upon the affair when Jill Kelley, a Florida socialite and friend of the Petraeus family, asked investigators to look into threatening emails that turned out to be from an apparently jealous Broadwell.
The probe uncovered potentially "inappropriate" emails between Kelley and the top allied commander in Afghanistan, General John Allen, that are the subject of a separate inquiry that has delayed his nomination to the post of NATO's supreme allied commander in Europe.
Television pictures showed Broadwell, wearing dark glasses and clutching one of her two sons in her arms, as she walked up the stairs from her car to enter her North Carolina home on Sunday.
"At some stage I'll have a statement for you guys, just not right now," husband Scott told reporters as he fetched groceries from the trunk.
Last week, FBI agents searched the Broadwell home and seized several boxes of documents.
Prosecutors are still deciding whether to pursue charges against Broadwell, 40, for mishandling classified documents or allegedly taking secret files from government buildings.
Petraeus told friends he and Broadwell, who he met in 2006 during a talk at Harvard University, began their affair last November, two months after he took up his CIA post, and that he ended things "about four months ago."
The acclaimed general, who was credited with turning around the war in Iraq and went on to become the commander of US and allied forces in Afghanistan, has been married to wife Holly for almost four decades.