Trump surprised Pentagon leaders in July by announcing via Twitter the ban on transgender people serving "in any capacity", reversing a plan launched by his predecessor Barack Obama that would see the military accept openly transgender recruits.
Trump said at the time that the integration of transgender troops would result in "tremendous medical costs and disruption," and issued a formal memorandum last Friday on the issue, saying the ban should be in effect from March 23, 2018.
But the memorandum gave Mattis discretion on how to handle transgender people already serving in the military.
The administration was facing lawsuits by transgender groups and service members, and on Monday the American Civil Liberties Union also filed a suit on behalf of several transgender troops challenging Trump's order.
In a statement Tuesday Mattis appeared cautious about implementing Trump's order.
He said the Department of Defense would establish a panel of experts and develop an implementation plan with a focus "on what is best for the military's combat effectiveness leading to victory on the battlefield."
In what appeared to be a barb against the slow movement in the White House to approve all the senior staff he needs, Mattis said "the soon arriving senior civilian leadership of the DOD will play an important role in this effort."
In the meantime, he said, existing policy on currently serving transgender troops would not change.
On Monday Pentagon officials declined to say whether there had been any studies or anecdotal reports on the impact of transgender people in the military, and they also declined to reveal any estimates of the number of transgender troops. Estimates run from the low thousands to as many as 15,000.