Benjamin Smith, 44, of upstate New York was arrested in Manhattan by the U.S. Secret Service, according to a criminal complaint filed in federal court.
"Bush will get his," Smith screamed as he was taken into custody, according to the complaint. Later, when asked about his marital status, he told agents he was divorced and "working on a relationship with Barbara Bush."
At a preliminary hearing, Assistant U.S. Attorney Andrea Griswold told U.S. Magistrate Judge Henry Pitman the government believes Smith was referring to the former president's daughter and not Bush's mother, who shares the same name.
It is a crime under federal law to threaten a current or former president.
On Thursday, the man's mother called police to report she had found a threatening note in the home she shares with her son and that a rifle was also missing, the complaint said.
"I'm going to work for George W. Bush and the Pentagon," the note said, according to the government. "I have to slay a dragon and then Barbara Bush is mine."
The Secret Service tracked him using his cell phone to Manhattan, where they arrested him early Friday morning. It was not clear why Smith went to New York City or where the former president and his family were at the time.
Smith's defense lawyer, Peggy Cross-Goldenberg, said the note and Smith's outbursts did not constitute a "true threat."
The judge also conceded it was "unclear" a jury would find Smith guilty.
"Admittedly, there is some inconsistency in the notion that the way to win Barbara Bush's affections is to kill her father," Pitman said.
© Thomson Reuters 2014