Three people are missing, an official said, adding the eight injured people have been admitted to a community health centre at Chinyalisaud town in Uttarkashi district.
Some of the passengers also fell into Bhagirathi river river which runs just below the gorge. Uttarkashi district officials including district magistrate Ashish Srivastava rushed to the spot, but could not speed up rescue efforts due to darkness.
Rescue efforts took time and went on till morning since the hill was steep near Bhagirathi river. Teams of the State Disaster Management Response Force (SDRF), Uttarakhand Police and the Indo-Tibetan Border Police (ITBP) went down from the rear side of the gorge and rescued eight passengers.
The bodies recovered near Bhagirathi river so far are yet to be identified, police said.
"My prayers and solidarity with those who lost their loved ones in the unfortunate bus accident in Uttarkashi. May the injured recover soon," PM Modi said in a tweet.
Uttarakhand Chief Minister Trivendra Singh Rawat announced ex gratia of Rs 1 lakh for the families of those who died, and Rs 50,000 for those injured in the accident.
The Uttarakhand government has asked officials to identify dangerous zones on the routes to the annual "chardham yatra", install speed monitors and warning signage, and construct crash barriers.