"He broke down and said the party is like his mother, it would be difficult for him to leave the BJP after so many decades," said sources close to the 68-year-old.
As he sobbed, senior leaders like Finance Minister Arun Jaitley went to him, consoled him and steadied him, the sources said.
Mr Naidu finally did accept the party's decision.
Mr Shah told the veteran politician that he was the choice of most in the BJP and RSS (the party's ideological mentor Rashtriya Swayamsevak Sangh). The Prime Minister too was not ready to let go of him but had to be persuaded, the BJP chief counseled Mr Naidu.
This morning, when Mr Naidu filed his nomination papers, it was in the reassuring presence of top leaders including PM Modi and several ministers and allies.
"I lost my mother at a very young age of one year and six months. Then I treated my party as my mother and they have really brought me up to this level. It is very painful to leave the party... That's why I became a little emotional," he told reporters.
Mr Naidu, who was one of the seniormost ministers in PM Modi's government, also quit his ministries today.
"From being constantly with people, among people, for people...moving to a different constitutional responsibility will have its own issues. I hope I will be able to do justice...I do not belong to BJP now," he said.