Two members of the World Sikh Organisation of Canada were challenging a unanimous motion adopted in the national assembly in February 2011.
Balpreet Singh and Harminder Kaur did not want to part with their kirpans as they headed into a legislature hearing to submit a brief in January 2011.
The motion stated that security personnel had the right to refuse entry to anyone who did not want to remove the religious symbol.
Originally, they argued the motion was unconstitutional but then changed their position to say it was legal but non-binding.
But Quebec Court of Appeal Justice Patrick Healy rejected their arguments in a decision on Monday as he upheld a lower-court ruling that said the national assembly has the right to establish its own rules in accordance with parliamentary privilege, The Toronto Star reported yesterday.
The court ruling coincided with Prime Minister Justin Trudeau's visit to India where his government is reaching out to the minority community to mend fences.
Superior Court Justice Pierre Journet affirmed the authority of the legislature to "exclude kirpans from its precincts as an assertion of parliamentary privilege over the exclusion of strangers," the report said.
In his decision, Mr Healy referred to a Supreme Court ruling that said a provincial legislature could invoke the privilege to exclude strangers to prevent journalists from filming in the precincts of the assembly.
"The Supreme Court confirmed that these general principles formed part of Canadian constitutional law and held specifically that the privilege to exclude strangers is entrenched in the Canadian Constitution," Mr Healy wrote on behalf of a three-member panel.
"I make no comment whether the assembly's exercise of the privilege to exclude the kirpan is a wise decision. I say only that it is a legal exercise of this category of privilege. If the appellants wish to challenge it, the proper forum is the assembly itself," it said.
"We haven't reached a decision, he said in an interview. Obviously, the judgment just came out on Monday and we received it late in the day yesterday, so we're still reviewing it. But an appeal is certainly one of our options and we're considering it very seriously," Mr Singh said.
Julius Grey, one of the lawyers who represented the plaintiffs, also said he is not ruling out an appeal with the Supreme Court.
"I don't agree with the judgement and I believe it is highly appealable," Mr Grey said.
In 2011, several Sikhs were denied entry into the Quebec legislature amid a heated debate over multiculturalism and where to draw the line when it comes to tolerating cultural practices.
Security officials at the Assemble Nationale refused to let them in because they were wearing kirpans.
The Canadian prime minister is on week-long visit to India where he will be meeting Punjab chief minister Captain Amarinder Singh.
The Trudeau government is reaching out to mend fences with Mr Singh who has accused it of including Sikh separatists in its Cabinet, CBC News.com reported.
Sikhs make up less than two per cent of India's population. But among Indian-Canadians, they form the largest group and have the greatest political clout; all four of Trudeau's Indian-Canadian ministers are of Sikh origin.