The project is seen as emblematic of warming relations between New Delhi and Tokyo, who are seeking to combat China's growing regional influence.
The high-speed link, which will use Japanese technology to connect the main city of PM Modi's home state with the country's financial capital, is scheduled for completion by December 2023.
"I hope to enjoy the beauty of India through the windows of the bullet train with Modi on my side when I come to India in a few years," Mr Abe told a ceremony in Ahmedabad.
His visit comes just after a lengthy standoff between India and China in a strategically important area of the eastern Himalayas, across the border at Sikkim.
The aircraft is expected to boost India's maritime capabilities.
PM Modi welcomed Mr Abe at the airport with his traditional big hug before the Japanese premier and his wife changed into traditional Indian clothes.
Japan is providing 85 percent of the cost of the new train link in soft loans.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.