A refugee from war-torn Yugoslavia, he eked out a living working for the elite Ivy League school. But Sunday was payback time: The 52-year-old janitor donned a cap and gown to graduate with a bachelor's degree in classics.
As a Columbia employee, he didn't have to pay for the classes he took. His favourite subject was the Roman philosopher and statesman Seneca, the janitor said during a break from his work at Lerner Hall, the student union building he cleans.
"I love Seneca's letters because they're written in the spirit in which I was educated in my family - not to look for fame and fortune, but to have a simple, honest, honourable life," he said.
His graduation with honours capped a dozen years of studies, including readings in ancient Latin and Greek.
"This is a man with great pride, whether he's doing custodial work or academics," said Peter Awn, dean of Columbia's School of General Studies and professor of Islamic studies.
"He is immensely humble and grateful, but he's one individual who makes his own future."
Mr Filipaj was accepted at Columbia after first learning English; his mother tongue is Albanian.
For Mr Filipaj, the degree comes after years of studying late into the night in his Bronx apartment, where he'd open his books after a 2:30-11 p.m. shift as a "heavy cleaner" - his job title. Before exam time or to finish a paper, he'd pull all-nighters, then go to class in the morning and then to work.
On Sunday morning in the sun-drenched grassy quad of Columbia's Manhattan campus, Mr Filipaj flashed a huge smile and a thumbs-up as he walked off the podium after a handshake from Columbia President Lee Bollinger.
Later, Mr Filipaj got a big hug from his boss, Donald Schlosser, Columbia's assistant vice president for campus operations.
Bollinger presided over a ceremony in which General Studies students received their graduation certificates. They also can attend Wednesday's commencement of all Columbia graduates, most of whom are in their 20s.
Mr Filipaj wasn't much older in 1992 when he left Montenegro, then a Yugoslav republic facing a brutal civil war.
An ethnic Albanian and Roman Catholic, he left his family farm in the tiny village of Donja Klezna outside the city of Ulcinj because he was about to be drafted into the Yugoslav army led by Serbs, who considered many Albanians their enemy.
He fled after almost finishing law school in Belgrade, Yugoslavia's capital, where he commuted for years by train from Montenegro.
At first in New York, his uncle in the Bronx offered him shelter while he worked as a restaurant busboy.
"I asked people, which are the best schools in New York?" he says. Since Columbia topped his list, "I went there to see if I could get a job."
Part of his $22-an-hour janitor's pay still goes back to his brother, sister-in-law and two kids in Montenegro. Mr Filipaj has no computer, but he bought one for the family, whose income comes mostly from selling milk.
Mr Filipaj also saves by not paying for a cellphone; he can only be reached via landline.
He wishes his father were alive to enjoy his achievement. The elder Mr Filipaj died in April, and the son flew over for the funeral, returning three days later for work and classes.
To relax at home, he enjoys an occasional cigarette and some "grappa" brandy.
"And if I have too much, I just go to sleep," he says, laughing.
During an interview with The Associated Press in a Lerner Hall conference room, Mr Filipaj didn't show the slightest regret or bitterness about his tough life. Instead, he cheerfully described encounters with surprised younger students who wonder why their classmate is cleaning up after them.
"They say, 'Aren't you...?'" he said with a grin.
His ambition is to get a master's degree, maybe even a Ph.D., in Roman and Greek classics. Someday, he hopes to become a teacher, while translating his favorite classics into Albanian.
For now, he's trying to get "a better job," maybe as supervisor of custodians or something similar, at Columbia if possible.
He's not interested in furthering his studies to make more money.
"The richness is in me, in my heart and in my head, not in my pockets," said Mr Filipaj, who is now an American citizen.
Soon after, the feisty, 5-foot-4 (1.63-meter) janitor picked up a broom and dustpan and went back to work.