Conte, 53, a lawyer and political novice, picked for prime minister by the anti-establishment Five Star Movement and far-right League seeking to form a coalition, had to present a list of ministers that the head of state would agreed to before his government could seek approval by parliament.
"I have given up my mandate to form the government of change," said Conte to reporters after leaving failed talks with President Sergio Mattarella.
Conte's decision to step aside leaves Italy in a political crisis nearly three months after March's inconclusive general election.
Following Conte's exit, Mattarella has summoned Carlo Cottarelli, an economist formerly with the International Monetary Fund, for talks on Monday, with a temporary technical government on the table as Italy faces the strong possibility of new elections in the autumn.
Cottarelli, 64, was director of the IMF's fiscal affairs department from 2008 to 2013 and became known as "Mr Scissors" for making cuts to public spending in Italy.
However, he will struggle to gain the approval of parliament with Five Star and the League commanding a majority in both houses.
"They've replaced a government with a majority with one that won't obtain one," said Five Star leader Luigi Di Maio to supporters at a rally near Rome.
Savona Choice Sinks Deal
Mattarella confirmed that the nomination by Five Star and the League of Paolo Savona for economy minister saw the end of Conte's brief mandate.
In his latest book, "Like a Nightmare and a Dream", 81-year-old Savona calls the euro a "German cage" and says that Italy needs a plan to leave the single currency "if necessary".
"I accepted every proposed minister apart from the minister of the economy," Mattarella told reporters.
A former judge of Italy's constitutional court, Mattarella has refused to bow to what he saw as "diktats" from the two parties which he considered contrary to the country's interests.
The president said that he has done "everything possible" to aid the formation of a government, but that an openly eurosceptic economy minister ran against the parties' joint promise to simply "change Europe for the better from an Italian point of view".
"I asked for the (economy) ministry an authoritative person from the parliamentary majority who is consistent with the government programme... who isn't seen as a supporter of a line that could probably, or even inevitably, provoke Italy's exit from the euro," he added.
Mattarella said Conte refused to support "any other solution" and then, faced with the president's refusal to approve the choice of Savona, gave up his mandate to be prime minister.
The leaders of Five Star and the League, Luigi Di Maio and Matteo Salvini, were infuriated by Mattarella's refusal to accept Savona, a respected financier and economist.
Salvini, who was Savona's biggest advocate and a fellow eurosceptic, said on Sunday that Italy wasn't a "colony", and that "we won't have Germany tell us what to do".
"Why don't we just say that in this country it's pointless that we vote, as the ratings agencies, financial lobbies decide the governments," a livid Di Maio said in a video on Facebook.
Later on Italian television he called for impeaching Mattarella.
"I hope that we can give the floor to Italians as soon as possible, but first we need to clear things up. First the impeachment of Mattarella... then to the polls," Di Maio said.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)