AAP legislators accused Mr Sharma of using "unparliamentary and undignified" language in the House.
"I suspend O P Sharma for the remaining days of the session," Assembly Speaker Ram Niwas Goel said after a motion, moved by AAP MLA Somnath Bharti, was passed by a voice vote.
This is the second time Mr Sharma has been suspended from the House. Earlier, he was suspended for two sessions for allegedly using derogatory language against AAP MLA Alka Lamba in November 2015.
Mr Bharti said the language used by Mr Sharma was "unparliamentary and undignified", and moved the motion.
The AAP MLAs alleged that Mr Sharma threatened police action against them.
Mr Sharma allegedly said to the AAP MLAs: "You will have to go to jail if you make a fake (EVM) machine", while referring to a live demonstration of a dummy EVM machine in the House last year.
Leader of Opposition Vijender Gupta argued with the Speaker over Sharma's suspension and sought to know under which rule of the House he was punished.
Mr Sharma termed his suspension "muzzling" of the opposition's voice by the ruling AAP.
"It is an unconstitutional step aimed at muzzling the voice of the opposition. When we oppose the presence of Gahlot, who has been disqualified by the President, in the House, we are ousted," Mr Sharma told reporters.
Mr Gupta said he would appeal to the Speaker to reconsider his decision.
On March 16, the first day of the Budget Session, all four BJP MLAs were marshalled out of the assembly when they objected to the presence of Mr Gahlot in the House.
According to the opposition, Mr Gahlot has no right to attend the proceedings of the House as he is among the 20 AAP MLAs who have been disqualified by President Ram Nath Kovind in the office-of-profit case.