The party's powerful 107-member national executive committee (NEC) met for 13 hours at a hotel outside Pretoria, and decided it would "recall" Zuma from his job as national president, several local independent media said.
"It took a brutal 13 hours, but the ANC's national executive committee has decided to recall President Jacob Zuma as head of state," the Times newspaper said citing unnamed sources.
Other media reported that the party will write to Zuma ordering him to stand down as president of Africa's most advanced economy, after he had asked for a few more months in office.
Earlier, state broadcaster SABC said the ANC had given Zuma 48 hours to turn in his resignation.
There was no official announcement and ANC officials were not reachable to confirm the reports.
The party committee can "recall" the head of state, essentially forcing him to step down, but he is under no constitutional obligation to obey.
He will then likely be ousted via a parliamentary vote of no-confidence within days.
African National Congress chief Cyril Ramaphosa reportedly left the meeting that ran late into the night to meet Zuma at his official Pretoria residence. His motorcade was seen returning to the venue of the party meeting at midnight after about an hour. Three hours after that, the talks closed.
Ramaphosa, the de-facto president-in-waiting, had been in negotiations with Zuma who rejected an earlier request from party leaders to step down more than a week ago.
"We know you want this matter to be finalised," Ramaphosa, 65, told a party rally in Cape Town on Sunday to rapturous cheering.
"We know you want closure... Because our people want this matter to be finalised, the NEC will be doing precisely that."
South African opposition parties on Monday called for early elections as the ANC's leadership battle ground on.
The parliamentary speaker announced that an opposition request for a no-confidence vote against Zuma on Tuesday was still being considered on Monday evening.
Zuma's presidency has been marred by corruption scandals, slow economic growth and record unemployment that have fuelled public anger.
He was scheduled to stand down next year after serving the maximum two terms after coming to power in 2009.
Zuma's hold over the ANC was shaken in December when his chosen successor -- his former wife Nkosazana Dlamini-Zuma -- narrowly lost to Ramaphosa in a vote to be the new party leader.
The ANC has insisted there will be no delay to the budget speech, which is due on February 21.
Sunday's rally was part of ANC celebrations marking 100 years since Nelson Mandela's birth -- as well as efforts by Ramaphosa to revive the party's tainted reputation ahead of next year's general election.
Zuma, 75, has not spoken publicly since February 4 when he was first asked by the party top leadership to resign.
In 2008, his supporters pushed out then-president Thabo Mbeki over allegations of abuse of power.
Under Zuma, the ANC won less than 54 percent of the vote in local elections in 2016 -- its worst electoral performance since coming to power with Mandela at the helm in 1994.
Ramaphosa is a former trade unionist who led talks to end apartheid rule in the early 1990s and then became a multi-millionaire businessman before returning to politics.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)