Harold Bornstein said the "raid" took place on February 3, 2017, two days after The New York Times quoted the physician as saying he had prescribed Trump a hair growth medicine for years.
"They must have been here for 25 or 30 minutes. It created a lot of chaos," Bornstein told NBC, saying the incident made him feel "raped, frightened and sad."
He identified Trump's aides as longtime personal bodyguard Keith Schiller, who until September 2017 worked at the White House as operations director; Alan Garten, a lawyer from the Trump Organization family real-estate firm; and an unidentified third man.
Bornstein said the original and only copy of Trump's charts, including lab reports under the president's name and various pseudonyms, were taken.
But the White House downplayed the incident as "standard."
"It would be standard procedure for the president, a newly collected president's medical records to be in possession by the White House Medical Unit and that was what was taking place," spokeswoman Sarah Sanders told reporters, insisting that what happened was not a raid.
Bornstein said that Trump dropped him after the Times article appeared.
"I couldn't believe anybody was making a big deal out of a drug to grow his hair," he told NBC. "It certainly was not a breach of medical trust to tell somebody they take Propecia to grow their hair."
NBC said the 70-year-old Bornstein was Trump's personal doctor for more than 35 years.
During Trump's then improbable run for president, Bornstein predicted the tycoon would be the healthiest president ever elected, according to a letter released by the campaign in December 2015.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.