Organised by a pro-Taliban faction of Jamiat Ulema-e-Islam (JUI), headed by Maulvi Ismatullah, some 450 participants at the rally chanted: "Long live Islamic jehad and long live Mulla Omar!" an AFP reporter at the scene said.
Many of them were also carrying portraits of Osama bin Laden.
The JUI's central secretary for information, Maulana Abdul Sattar Chisti, paid tributes to Osama bin Laden.
"We are observing death anniversary of an Islamic hero, who was killed by American and Pakistani agents," he said.
"Osama bin Laden's martyrdom gave a new impetus to jehad against forces inimical to Islam," he added.
Osama bin Laden, the founder of Al Qaeda and mastermind of the 9/11 attacks on the United States, was killed on May 2, 2011 in a secret US Navy SEAL operation in a walled-off compound in the Pakistani garrison town of Abbottabad.