PM Modi witnessed the groundbreaking ceremony which was live-streamed to the Dubai Opera House where he was interacting with the Indian community.
He thanked Crown Prince of Abu Dhabi Mohamed bin Zayed Al Nahyan on behalf of 125 crore Indians for the construction of the grand temple.
"I believe this temple will not be only unique in terms of architecture and splendour, but will also give a message of 'Vasudhaiva Kutumbakam' (world is one) to people across the world," he said.
Temple Committee members had presented the temple literature to PM Modi and Mohamed bin Zayed Al Nahyan last evening in Abu Dhabi.
This is the first stone temple to be built in Abu Dhabi off Dubai-Abu Dhabi highway.
PM Modi is visiting the United Arab Emirates for the second time after his 2015 trip to the country.
The temple will be hand-carved by Indian temple artisans and assembled in the UAE. It will be completed by 2020, and open to people of all religious backgrounds.
It will be the first traditional Hindu stone temple in the Middle East, said a spokesperson from the BAPS.
Bochasanwasi Shri Akshar Purushottam Swaminarayan Sanstha (BAPS), is a socio-spiritual Hindu organisation set up in 1907 that runs more than 1,100 temples and cultural compounds around the world.
The temple will incorporate all aspects and features of a traditional Hindu temple as part of a fully functional, social, cultural and spiritual complex.
It will replicate the BAPS temple in New Delhi and the one under construction in New Jersey.