"You can wear anything, whether it is yellow, red or green. There is no restrictions with regard to clothing. There is no guideline from our side. We have set up an inquiry into the matter," Tourism and Culture Minister Mahesh Sharma told the Press Trust of India.
The minister asserted that there are no restrictions with regard to the clothing for visiting the world famous 17th century monument.
He, however, said that a tourist guide or a policeman could have asked them to remove the saffron scarves and the truth will emerge after the probe.
When they were about to enter the monument, they were reportedly asked to remove the scarves.
The incident evoked protest from some Hindu outfits including the Bharatiya Janata Yuva Morcha (BJYM) which said it hurt their religious sentiment.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.