Chennai-born Pichai, who returned to his alma mater after 23 years, said his comment led to "temporarily" shutting down of the mess.
"I came from Chennai. I learnt Hindi in school but I never spoke it much. So, just listening to how people are speaking, I just thought you address people this way. So one day, there was someone in the mess and I had to call him. I called him 'Abey Saale'," he reminisced to an audience of over 3,000 people.
"In the first couple of weeks, I thought you call people that way. Next thing I know is the folks in the mess were quite upset and I think they temporarily closed down the mess," he added.
The Google chief, who met his wife Anjali on campus, said the romance was tough because access to the girls' hostel wasn't easy.
"Someone would stand outside and very loudly say, 'Anjali, Sundar is here for you'. It wasn't exactly a pleasant experience," he said.
He added that while technology, especially mobile phones, has changed the world, it has left certain things unchanged.
Talking about his college days, Mr Pichai said that like any other college goer, he stayed up late at night and missed classes in the morning.
He spoke of being interviewed by Google in 2004 and he thought Gmail was an April Fool's joke.
It wasn't until halfway through the process that he realised it was a real thing, he said at the discussion moderated by InfoEdge MD and CEO Hitesh Oberoi.
A cricket and soccer fan, Mr Pichai revealed that Infosys founder Narayana Murthy was his idol during college days and that he loved watching Sachin Tendulkar play.
A student asked Mr Pichai what they would have to do to replace him. "I'd say, be careful what you wish for, but happy to discuss over a cup of chaai," he said.
Election Results for Lok Sabha Election 2019 will be out on May 23. Get the latest election news and live updates on ndtv.com/elections. Catch all the action on NDTV Live. Like us on Facebook or follow us on Twitter and Instagram for news updates from each of the 543 parliamentary seats for the election 2019