"If we ban gold imports, the smuggling will go up... there are already indications that smuggling of gold has gone up in the last three months," Rangarajan said.
He was addressing the silver jubilee celebrations of RBI-promoted Indira Gandhi Institute of Development Research.
"I believe gold imports will come down when inflation goes down," he added. Gold is considered as a hedge against price rise.
Rangarajan said he has been told by revenue department officials that the seizures of smuggled gold have gone up in the recent past.
In the same event, Reserve Bank of India governor D. Subbarao reiterated his concerns over rising gold imports and citied its pressure on current account deficit.
He, however, said: "RBI has not put any new restriction on gold imports. Our announcement to stop lending for gold purchase was a reiteration of decades-old decision."
Rangarajan said last year gold imports touched $60 billion, of which $15-20 billion was due to high inflation as people use the precious metal as a hedge against inflation.
RBI deputy governor Subir Gokarn had recently called for dematerilisation of the metal to discourage its physical possession, hoping that this move will trim imports.
Traditionally, India has been the world's largest consumer and importer of gold. Last year, imports were to the tune of 969 tonnes.
However, doubling of the excise duty to 4 per cent in the last budget and the curbs that RBI imposed on gold loan value (down from 85-90 per cent of the value of jewellery to 60 per cent) and banning banks from funding gold purchase by loan companies have led to drop in imports.
In April-October, imports declined 35 per cent year-on-year and overall imports are set to drop over 17 per cent to 800 tonnes this year.