The drastic measure was taken because, according to the child protective services, the couple were not bringing the children up properly. What did they do wrong? They fed the children with their hands and the infants slept in the same bed as the parents.
"My son was sleeping with my husband. They said he should sleep separately from your son," said Mrs Bhattacharya.
"Feeding a child with the hand is normal in Indian tradition and when the mother is feeding with a spoon there could be phases when she was overfeeding the child. They said it was force feeding. These are basically cultural differences," said Mr Bhattacharya.
Recently, the Indian Embassy in Oslo stepped in and an officer even met the children, though the parents were not allowed to.
Norway's Child Protective Service is a powerful body charged with protecting the rights of children living in difficult family situations. But there are many reports of excesses.
"There has been a report in UN in 2005 which criticized Norway for taking too many children in public care. The amount was 12,500 children and Norway is a small country," said Svein Kjetil Lode Svendsen, lawyer.
The Bhattacharyas' visas expire in March. If they don't get their children by then, the couple will be forced to stay on.
"What we have thought is we are not going to apply for visa for my children. But if the case doesn't get sorted we have to apply because we cannot come back to India without our children," said Mr Bhattacharya.
It's a nightmare that, at this moment, seems without end.