Three of the four, who said they were members of ISKCON, were women, the officer added.
The shrine and its periphery areas were declared a "no flying zone" by the district administration in 2014 after a quadcopter took aerial videos and photographs of the temple and uploaded them on social media.
A round-the-clock patrolling team has also been deployed by the four towers surrounding the temple to prevent such incidents.
The four devotees from the International Society for Krishna Consciousness were operating the drone from the roof of the Radha Ballav mutt, close to the temple, the senior officer of the Singhadwar Police Station said.
"During questioning, they said that they were not aware of the 'no-flying' rules in the area and were trying to photograph the shrine for creating a road map to the place," he said.
They also said the map was primarily meant for a congregation in Puri next month of ISKCON devotees, Sarthak Sarangi said, adding that all four had valid passports and visas.