Scheer, a low-profile 38-year-old former speaker of the House of Commons, on Saturday captured the leadership of the right-leaning Conservatives on the final ballot.
The closeness of the result reflects a split between members who want to debate on abortion and same sex marriage - both of which are legal in Canada - and those who prefer to focus on fiscal issues.
"Every kind of Conservative needs to have a home in our party and feel welcomed," Mr Scheer told a late-night news conference on Saturday.
The result also marked another setback for populists who took heart after Republican Donald Trump won the US presidential election in 2016.
Earlier this year, hardline parties failed to win power in France and the Netherlands. Hard line Conservative leadership contenders - including one who proposed would-be immigrants pass a "Canadian values" test - did poorly.
Kathy Brock, a political science professor at Queen's University in Kingston, said those candidates hit a false note.
"Canadians have always been more receptive by a significant degree than Americans to immigration," she said in a phone interview.
Scheer, a personable man who, like Trudeau, smiles a lot and has a young family, has a little over two years to raise his profile and prepare for an election in October 2019.
"Andrew Who?" said a front-page headline in Sunday's Toronto Star.
A Nanos Research survey last week gave the Liberals a 14-point lead over the Conservatives, enough for a crushing win in 2019.
"It's not enough for the Conservatives just to elect a leader that could turn things around for them. They actually need the Liberals to make mistakes in order to move the numbers," pollster Nik Nanos said in a phone interview.
Scheer praised former Conservative prime minister Stephen Harper, who governed from 2006 to 2015 and frustrated social conservatives by shutting down all talk of abortion and same sex marriage.
Scheer, pressed repeatedly on abortion, said he would not formally reopen the debate, but added Parliamentarians "have the right to speak out on issues they believe".
Liberal legislator Adam Vaughan, who observed the leadership campaign, said Scheer was "a radical extreme social conservative" whom voters would shun.