The heavily armed gunmen from a group known as "Signatories in Blood" had been holed up in the In Amenas gas complex since they took up to 41 foreign worker hostage in a dawn assault on Wednesday.
"The (army) assault took place mid-morning. Eleven terrorists lost their lives along with the foreign hostages," the security source told AFP.
"We think they were killed in retaliation" for the army attack,the source said.
The gunmen, cited by Mauritania's ANI news agency, said on Saturday they were still holding "seven foreign hostages," denying claims of more.
"Signatories in Blood," led by Algerian Mokhtar Belmokhtar, a former senior Al-Qaeda commander in north Africa, were demanding an end to French intervention against Islamists in neighbouring Mali, ANI reported earlier.
Belmokhtar also wanted to exchanging American hostages for the blind Egyptian sheikh Omar Abdul Rahman and Pakistani Aafia Siddiqui, jailed in the United States on charges of terrorist links.
But State Department spokeswoman Victoria Nuland said "the United States does not negotiate with terrorists."
On Friday, an Algerian security official had said troops were trying to reach a "peaceful" end to the crisis, before "neutralising the terrorist group that is holed up in the plant and freeing a group of hostages still being held there."
Amid a virtual news blackout in Algiers, harshly criticised by local media, world leaders took a tough stand on the fate of the remaining hostages.
US Defence Secretary Leon Panetta said Washington would "take all necessary steps to protect our people" from the threat of Al-Qaeda-affiliated militants in north Africa.
"Whether or not that involves assisting others with military operations, whether it involves developing in a cooperative way operations there, those are areas that I think remain to be decided," he told the BBC.
Earlier, US Secretary of State Hillary Clinton told a news conference with Japanese Foreign Minister Fumio Kishida of her deep concern "about those who remain in danger. Utmost care must be taken to preserve innocent life."
At least one American had already been confirmed dead before Saturday's final assault.
On Friday, the militants gave a breakdown of three Belgians, two Americans, one Japanese and a Briton, although Belgium said there was no indication any of its nationals were being held.
An Algerian security official had put the remaining number of foreign hostages at 10.
Norway's Statoil, which jointly operates the In Amenas site with Britain's BP and Sonatrach of Algeria, said two Norwegians have been found alive but six others remain unaccounted for.
In Tokyo, Prime Minister Shinzo Abe ordered his government to do everything possible to ensure the safety of those Japanese unaccounted for in "an extremely despicable" incident that "can never be forgiven."
France, which said on Saturday that 2,000 of the 2,500 troops it had pledged were now on the ground in Mali, said on Saturday that no more of its citizens were being held.
Of the French intervention in Mali, Foreign Minister Laurent Fabius said at a meeting in Abidjan on Saturday that "France was obliged to intervene very, very rapidly, otherwise there would have been no more Mali."
Algerian news agency APS quoted a government official as saying the kidnappers, who claimed to have come from Niger, were armed with machineguns, assault rifles, rocket launchers and missiles.
This was confirmed by an Algerian driver, Iba El Haza, who said the hostage-takers spoke in different Arabic dialects and perhaps also in English.
"From their accents I understood one was Egyptian, one Tunisian, another Algerian and one was speaking English or (another) foreign language," Haza told AFP, two days after escaping during the army's Thursday attack.
"The terrorists said: 'You have nothing to do with this, you are Algerians and Muslims. We won't keep you, we only want the foreigners.'"
Algeria's El Watan daily quoted a former military officer as justifying the army's assault on Thursday, saying: "All hesitation is forbidden when the future of the nation is at stake or being threatened."
The final death tolls, of both foreign and Algerian hostages and of gunmen, was not yet known.