"If you really want some of these institutions (like CBI and CVC) to deliver, you must take a risk and take courage to make them constitutional," he said, while speaking at a session at the World Economic Forum (WEF).
CBI and CVC are not independent authorities and that was why people write about them as "handmaiden of government", he said, adding "if Lokpal is expected to function with autonomy and total independence you will have to guarantee a constitutional mandate".
Mr Rai further said that as far as CBI is concerned, "It is not independent of executive and that's why very often everybody writes that CBI has become handmaiden of government in power".
"Look at CVC. It is another body much feared, much maligned, but still a statutory body. It is not a constitutional body," he added.
The Comptroller and Auditor General (CAG), which is a constitutional body, has brought to light various corruption-related issues with regard to allocation of 2G spectrum, coal blocks, conduct of Commonwealth Games etc.
These reports, however, had evoked sharp reaction from the government.
Referring to corruption, Mr Rai said that it can't be eliminated by enacting Lokpal law. However, he added, "If we can be conscious of that, we can reduce the cronyism or nepotism to the extent that we can tackle this hydra monster called corruption."
Mr Rai further said citizens and corporate too would need to discharge their responsibilities in promoting transparency and combating corruption.
"We require a paradigm shift. The paradigm shift is not totally with the government but it is also responsibility of citizens and citizens groups. All that we need today is keep our own waters clean", he said, adding there was a need to introduce transparency in all departments of the government and regulators.
With the implementation of the Right to Information Act, he said, the administrative executives and politicians have become conscious that their decisions could be questioned by the public and they would be answerable.