Gurmukh Singh, who is married to an American citizen and has two US-born daughters, was taken into custody by federal agents after failing to get a stay in his case, his attorney and family said.
The 46-year-old Sikh taxi driver from Punjab sneaked into the United States without a visa through the border with Mexico in 1998.
He later applied for asylum, citing religious persecution. But his family and lawyer say he failed to properly pursue the case and was ordered deported.
Singh married a US citizen in 2010 but his deportation case resurfaced when he applied for a residency visa in 2012 given his new status.
He was subsequently jailed for about five months but after rights activists posted his bail, he was released and his case has since been going through the appeals process.
His family said in the past several years, he had been checking in with the office of Immigration and Customs Enforcement (ICE) on a regular basis pending the outcome of his appeals against the deportation order.
Alexis Perez Nava, with Resilience Orange County, a group helping with his case, said Singh was detained on Monday after the 9th Circuit Court of Appeals denied a request to quash the deportation order.
"We are completely devastated. This has completely broken us apart, emotionally and physically," Singh's 18-year-old daughter Manpreet told AFP.
"Watching him emotionally break down is probably the last thing any daughter wants to see," Manpreet said, her voice breaking.
"He just looked at us this morning and said 'I don't know what is going to happen to us any more. Please forgive me and take care,'" she added.
Lori Haley, an ICE spokeswoman, said Singh was ordered detained after his case had "undergone exhaustive review at all levels of our nation's legal system."
She added that while the priority for federal authorities was to focus on criminal aliens who pose a threat to public safety, those like Singh who violate immigration laws were not exempt from action.
"All those in violation of our nation's immigration laws may be subject to arrest, detention and, if found removable by final order, removed from the United States," she said in an email.
Singh's detention comes amid a clampdown by the administration of President Donald Trump against undocumented immigrants.