During the zero hour, Mr Rao stood in the Well holding a placard that read "Help Fight For Justice to Andhra Pradesh" right in front of Mr Naidu's seat.
The chairman repeatedly asked Mr Rao to go back to his seat saying he will not tolerate such behaviour.
Some members urged the chairman to proceed with the listed business, but Mr Naidu ruled that the House was not in order if even one member was in the Well.
He said he had two alternatives before him - adjourn the proceedings or name the member.
The chairman asked the Congress, Mr Rao's party, to persuade him to go back to his seat.
Leader of the Opposition and senior Congress leader Ghulam Nabi Azad said his party does not approve of Mr Rao's behaviour.
He said that rule 255 states, "The Chairman may direct any member whose conduct is in his opinion grossly disorderly to withdraw immediately from the Council and any member so ordered to withdraw shall do so forthwith and shall absent himself during the remainder of the day's meeting."
Mr Naidu then directed Mr Rao to either go back to his seat or withdraw from the proceedings of the House for the rest of the day.
Mr Rao ignored his party colleagues who asked him to return to the seat. He chose to withdraw from the House, terming the entire episode as "very unfortunate".
He then picked up his belongings and walked out of the House.
The House took up the listed business thereafter.
Follow NDTV for latest election news and live coverage of assembly elections 2019 in Maharashtra and Haryana.
Subscribe to our YouTube channel, like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.