Mr Khan is said to be a direct descendent of Khwaja Moinuddin Chishti, the legendary Sufi saint.
In a statement on Monday, he said, "My family and I have taken a pledge on this day that we will never have beef for the rest of our lives."
He also appealed that all Muslims take the initiative and give up beef to "honour the religious sentiment of our Hindu brothers" and reestablish the legacy of goodwill among communities.
The cow should be named the national animal, he suggested.
"The cow is the symbol of religious belief. Not only the government,, but it is also the duty of every follower of the religion to protect them," he said.
He also spoke about the "triple talaq" or the practice of Muslim men saying 'Talaq' or divorce thrice for an instant divorce, which will be examined by the Supreme Court soon after a clutch of petitions challenging it. The practice is against the Quran and irrelevant today, said the Ajmer cleric.
There are several provisions in the Quran, he added, to ensure self-respect and dignity to women.
The cleric's comment comes in the middle of a debate over Uttar Pradesh Chief Minister Yogi Adityanath's action against illegal slaughter houses - which has sparked uncertainty among meat shop owners - and a new law cleared in Gujarat to punish cow killing with life term.
Heads of various Sufi shrines from across the country were present when Syed Zainul Abedin Ali Khan spoke.