"I think Baghdadi's alive... and I'll believe otherwise when we know we've killed him," Mattis told Pentagon reporters.
"We are going after him, but we assume he is alive."
There have been persistent rumors that Baghdadi has died in recent months.
The Britain-based Syrian Observatory for Human Rights, a longtime conflict monitor, last week said it had heard from senior ISIS leaders in Syria's Deir Ezzor province that Baghdadi was dead.
With a $25 million US bounty on his head, Baghdadi has kept a low profile but was rumored to move regularly throughout ISIS-held territory in Iraq and Syria.
The Iraqi native has not been seen since making his only known public appearance as "caliph" in 2014 at the Grand Mosque of Al-Nuri in Mosul, which was destroyed in the battle for Iraq's second city.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.