In an announcement at the White House, Mr Trump described the soft-spoken Mr Powell as a smart and committed leader who would build on Ms Yellen's achievements in steering the US economy after the recovery from the 2007-2009 financial crisis.
"If we are to sustain all this progress, our economy requires sound monetary policy and prudent oversight," Mr Trump said as Mr Powell looked on. "We need strong and steady leadership at the US Federal Reserve... He will provide exactly that."
Mr Powell has worked alongside Ms Yellen for the past five years, backing her direction on monetary policy and, in recent years, sharing her concerns that weak inflation justified a continued cautious approach to raising interest rates.
Ms Yellen's four-year term as Fed chief ends in early February 2018. She will be the first US central bank chief not to be renominated to a second term since 1979.
Mr Trump on Thursday lauded Ms Yellen's stewardship but did not say why he decided to pass her over for another term. The Republican president said he was impressed by Mr Powell's experience in the private-sector and "real-world perspective" to government.
"He understands what it takes for our economy to grow," Mr Trump said.
Mr Powell, who in the last 25 years has done a prototypical Washington circuit of government, private, and think tank jobs, pledged to be attuned to emerging financial risks and the impact the Fed has on average Americans.
He will take over an economy that has been expanding for more than eight years and one that boasts an unemployment rate at more than a sixteen-and-a-half-year low.
"Monetary policy decisions matter for American families and communities. I strongly share that sense of mission and am committed to making decisions with objectivity and based on the best available evidence," Mr Powell said in brief remarks after Mr Trump's announcement.
His nomination now goes to the Republican-controlled Senate for confirmation.
"I'm encouraged by President Trump's choice," Senate Majority Leader Mitch McConnell said in a statement that pledged "timely" consideration of the nomination.
There was little apparent market reaction to Mr Powell's nomination, which had been expected. Investors were largely focused on the release of details of a Republican plan to broadly change the US tax code.
Ms Yellen, a Fed veteran who has served at all levels of the sprawling central bank system, said in a written statement that she would work with Powell "to ensure a smooth transition".
Mr Trump's decision, after a broad and very public search, offers what analysts said was a classic compromise, allowing him to select his own Fed chief while getting continuity with the policies of the central bank run by Ms Yellen.
"The kernel of what this boiled down to is that in selecting Powell, (Trump) has all but selected Yellen," said Sarah Binder, a political science professor at George Washington University and author of a recent book on Fed politics. "There is not really much daylight, if any."
In June, Mr Powell laid out both a defence of the Fed's gradualist path and a critique of those, including some of his competitors for the Fed job, who argued the central bank had increased the risk of high inflation and other problems.
Mr Trump on several occasions has said he would prefer rates to stay low, a position apparently at odds with some of those who were on his short list, particularly Stanford University economist John Taylor and former Fed Governor Kevin Warsh. Top White House economic adviser Gary Cohn also was a contender.
Mr Powell has been a reliable supporter of the consensus forged by Ms Yellen on the policy-setting Federal Open Market Committee, and likely will be seen as a less risky choice with the economy growing solidly and US stock markets near record highs.
The Fed has raised rates twice this year and is widely expected to do so again next month.
But Mr Powell has gone further than his colleagues in calling to relax some of the stricter regulations imposed after the crisis, also important to Mr Trump. Mr Powell can now pursue that end along with Mr Trump appointee Randal Quarles, the Fed's new vice chair for supervision.
Though he will be the first Fed chief since the late 1970s without an advanced degree in economics, Mr Powell brings market insights, Fed board experience and Republican ties that analysts say will likely make for a smooth confirmation and transition.
Under President George HW Bush, Mr Powell oversaw policy on financial institutions and debt markets as an undersecretary of the Treasury. From 1997 to 2005 he was a partner at the Carlyle Group, a private equity firm, and focused on public debt dynamics while at the Bipartisan Policy Center think tank.
Ms Yellen is entitled to remain as a Fed governor until 2024, though previous central bank chiefs have traditionally not stayed once a successor was in place.