The open conflict within Russia's leadership follows the announcement over the weekend that Prime Minister Vladimir Putin plans to return to the presidency next year and Medvedev would then take his old job as prime minister. Russia will have a presidential vote in March, but Putin is sure to win.
The departure of Kudrin is likely to unsettle investors and further shake Russia's markets. A close Putin ally, he has been finance minister since 2000 and his tight hold over the budget has been seen as key to Russia's economic stability.
"It is difficult to see how Mr. Kudrin's resignation can be anything but market-negative," said Neil Shearing, chief emerging markets economist at Capital Economics Ltd in London. "With oil prices starting to slide and financial markets still jittery, now is not a good time for the government to lose its arch fiscal hawk."
On Saturday, Kudrin said he would refuse to serve in the government if Medvedev was made prime minister because of policy disagreements he had with him, including plans to substantially boost military spending.
Addressing Kudrin on Monday, Medvedev called the minister's remarks "irresponsible chatter" and "improper," especially since they were made while the minister was in Washington for meetings of the International Monetary Fund and the World Bank.
"If you disagree with the course set by the president and being implemented by the government, you have only one choice: Resign," Medvedev said.
Kudrin said he would decide only after talking to Putin.
"You can seek the advice of whomever you want, but as long as I'm president, such decisions are made by me," Medvedev retorted.
The Kremlin said Medvedev signed a decree on Kudrin's resignation. State news agencies reported that Kudrin confirmed that he had quit.
Nikolai Petrov, a political scholar at the Carnegie Moscow Center, said Putin may have wanted to spare Kudrin from taking responsibility for the higher taxes and cuts in social spending that Russia faces after a parliamentary election in December and a presidential election three months later.
"Putin cherishes Kudrin too much to put him in the crossfire of unpopular measures," Petrov said.
The 50-year-old Kudrin had been mentioned as a possible future prime minister, and Petrov said Putin could still appoint him to the post sometime in the future.
Anton Struchenevsky, a senior economist at the Moscow-based investment bank Troika Dialog, described Kudrin as "the restraining element against populist policies" in Russia.
"All of Russia's macroeconomic achievements are Kudrin's work," Struchenevsky said. "Kudrin is a highly influential figure, and I don't see anyone of similar weight who could replace him."
Some market analysts speculated that Kudrin's departure could have a greater effect on Russia's economy than the 2012 presidential election itself.
"It is unlikely that Mr. Kudrin's replacement will share his predecessor's credentials and clout," Shearing wrote in a note to investors.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.