Mr Gandhi - whose father, grandmother and great-grandfather have all served as prime minister - is widely expected to lead Congress into the polls after being elevated to the post of party vice-president last weekend.
But only 22 percent of voters surveyed by the magazine India Today said he would make the best prime minister - well behind the hardline Hindu nationalist politician Narendra Modi who had a 36-percent rating.
The same poll showed the left-leaning Congress and its allies are on course to get 152-162 seats in the 552-seat Lok Sabha while the right-wing Bhartiya Janata Party (BJP) and its allies were projected to get 198-208 seats.
Mr Modi, who is the chief minister of Gujarat, is widely seen as pushing to lead the party into the polls amid a fierce internal power struggle.
42-year-old Mr Gandhi has often appeared less than hungry for power after rejecting repeated requests to take on ministerial responsibilities.
But he is now his party's official number two behind his mother Sonia and a senior minister said on Wednesday that he was the party's "consensus" choice to be its prime ministerial candidate in elections due early next year.
Prime Minister Manmohan Singh, who is 80, is not expected to stand in 2014.
The BJP chose veteran Rajnath Singh as its new president earlier this week after the scandal-plagued incumbent Nitin Gadkari abruptly resigned.
But analysts expect Mr Modi, who was re-elected as Gujarat chief minister for a third time last month, to launch a bid for the party leadership in the coming months and few expect Rajnath Singh to be the final candidate.
Mr Modi, whose humble roots are in sharp contrast to Mr Gandhi's privileged upbringing, remains a hugely divisive figure as he was at the helm in Gujarat during religious riots in 2002 in which some 2,000 people were killed.
While the survey made grim reading for Congress, it also represents a blow for the BJP as regional parties appear to be the chief beneficiaries of voters' frustration.
According to government forecasts for the fiscal year ending in March, India's economy is growing at less than six percent a year - its worst performance in a decade.
Congress has also been hit by corruption scandals but allegations surrounding Mr Gadkari have also damaged the BJP's standing, according to the survey of 12,823 people.