The court said no one can be allowed to play with the ecological safety of the 17th century monument of love that attracts millions of visitors annually.
The restaurant, still under construction, was being built on a green belt without No Objection Certificates, it was said.
According to the amended rules of Ancient Monuments Protection Act of 1958, construction within 500 metres of a monument is strictly prohibited.
Earlier, a team of experts visited the area and submitted its findings to the NGT.
Meanwhile, the Archaeological Survey of India (ASI) has sent notices to hundreds of property owners who had encroached government land or built structures without permission.
An official said demolition of some identified structures would be taken up soon.