Referring to the last month's rare press conference by the top four judges of the Supreme Court, Mr Roy in his farewell speech on Friday said "Judges must stand united to instil confidence in the public."
Justices Jasti Chelameswar, Ranjan Gogoi, MB Lokur and Kurian Joseph held an extraordinary press conference on January 12 to voice their criticism of how major cases were assigned by the Chief Justice. The four judges are the senior most in the country after the Chief Justice.
Stressing on the unity within the judicial system, Mr Roy said that "if judges are not together extra-legal elements will take over and that will destroy the institution."
"There is bound to be difference of opinion, but that should not create differences in heart," he added.
"If we lose paramountcy of judicial process then rule of law will be undermined," he said.
Mr Roy was given farewell on his last working day on Friday by the Supreme Court Bar Association.
Of the four judges who went public against Chief Justice Misra, the three judges -Justices Ranjan Gogoi, Madan B Lokur and Kurien Joesph - were present at the farewell function.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.