Turkey angrily denounced the demonstration as "unacceptable". Many demonstrators carried symbols of the outlawed Kurdistan Workers Party (PKK) which has battled the Turkish state for over three decades in a continuing insurgency.
Tensions are already running high between Berlin and Ankara after German authorities refused to allow some Turkish ministers to campaign in the country for a "yes" vote in the April 16 referendum that would hand Mr Erdogan an executive presidency.
Significantly more people turned up than organisers had been expecting for the rally, which took place ahead of the annual Newroz festival when Kurds mark the traditional New Year.
Saturday's protest march in Frankfurt went off peacefully, a police spokesperson said.
Some of the participants carried flags and banners of the outlawed PKK, as well as portraits of the group's jailed leader Abdullah Ocalan, who is serving a life sentence in Turkey, calling for his release.
Police said no banners or flags were confiscated so as to not provoke the crowd, but added that photos had been taken which could lead to future prosecutions.
Mr Erdogan's spokesperson Ibrahim Kalin said in a statement that the presidency "condemned in the strongest terms" the fact that the rally had been allowed to go ahead.
"It is unacceptable to see PKK symbols and slogans... when Turkish ministers and lawmakers are being prevented from meeting their own citizens," he said.
He said the "scandal" of the Frankfurt demonstration showed that some EU countries were actively working in favour of a "no" vote in the critical referendum.
More than 40,000 people have been killed since the PKK launched its insurgency against the Turkish state in 1984.
The group is listed as a terror organisation not just by Turkey but also the European Union and the United States.
The Turkish foreign ministry said in statement that Germany's toleration of a rally with symbols of a group that it itself regards as a terror outfit was the "worst example of double standards".
Mr Erdogan on Monday accused German Chancellor Angela Merkel of "supporting terrorists", in a spiralling diplomatic row.
Turkey has long accused Germany of providing refuge to Kurdish and other militants.
A Merkel spokesperson described Mr Erdogan's jibe as "clearly absurd".
Mr Erdogan has also accused Germany of "Nazi practices" for blocking his ministers from speaking to Turkish voters resident in Germany.
Germany is home to the largest Turkish diaspora in the world, many of whom are of Kurdish origin.