Referring to the last month's rare press conference by the top four judges of the Supreme Court, Mr Roy in his farewell speech on Friday said "Judges must stand united to instil confidence in the public."
Justices Jasti Chelameswar, Ranjan Gogoi, MB Lokur and Kurian Joseph held an extraordinary press conference on January 12 to voice their criticism of how major cases were assigned by the Chief Justice. The four judges are the senior most in the country after the Chief Justice.
Stressing on the unity within the judicial system, Mr Roy said that "if judges are not together extra-legal elements will take over and that will destroy the institution."
"There is bound to be difference of opinion, but that should not create differences in heart," he added.
"If we lose paramountcy of judicial process then rule of law will be undermined," he said.
Mr Roy was given farewell on his last working day on Friday by the Supreme Court Bar Association.
Of the four judges who went public against Chief Justice Misra, the three judges -Justices Ranjan Gogoi, Madan B Lokur and Kurien Joesph - were present at the farewell function.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.