"There were over 500 women and hundreds of children," Kaltuma Wari, who was released earlier this week, said by telephone from the Borno State capital, Maiduguri, on Thursday evening.
There was no independent corroboration of the figure or indication of how many were still being held but another woman and a survivor who escaped have also said hundreds of women and girls were held.
"Boko Haram took around 300 women and kept us in a school in Baga," one unnamed woman was quoted as saying by Amnesty International in a statement released on Thursday.
"They released the older women, mothers and most of the children after four days but are still keeping the younger women."
Civilian vigilante Yanaye Grema, who escaped after hiding for three days in the town, said he met four women, including one with a baby on her back, as he fled through the bush last week.
"They told me they were among hundreds of women that were abducted by Boko Haram and detained," he told AFP earlier.
Wari, 40, said Boko Haram fighters took her to a girls' boarding school in Baga after she went into the streets to look for her husband and six of her nine children.
The women and children were kept in dormitories, classrooms and in the open, despite the cold weather caused by the seasonal winds, the Harmattan, she added.
"They didn't touch any woman but they paid more attention to young women. They kept watch on them and they were always accompanied by gunmen wherever they went, even to the bathroom," she said.
"Some of us turned hysterical and I was one of them. They got fed up with us and (on Wednesday) around 2:00 pm (1300 GMT) they singled us out and asked us to leave the town," she said.
"We were around 100, all of us mothers," she added. "They would never allow any young woman to leave."
Hundreds of people, if not more, are feared to have been killed in the attack on Baga that may prove to be the deadliest in the conflict.
But the detention of hundreds of young women is likely to increase mounting concern around the world about the scale of the attack.
Kidnapping of young women and girls has been an established tactic in the six-year insurgency, including last April, when 276 schoolgirls were abducted from the town of Chibok.
Fifty-seven managed to escape but 219 are still being held.