Rally organizers had invited several far-right speakers who were confined to a small pen that police set up in the historic Boston Common Park to keep the two sides separate. The city avoided a repeat of last weekend's bloody street battles in Charlottesville, Virginia, where one woman was killed.
Police estimated that as many as 40,000 people packed into the streets around the nation's oldest park.
Officials had spent a week planning security for the event, mobilizing 500 police officers, including many on bikes, and placing barricades and large white dump trucks on streets along the park, aiming to deter car-based attacks like those seen in Charlottesville and Europe.
The rally never numbered more than a few dozen people, and its speakers could not be heard due to the shouts of those protesting it and the wide security cordon between the two sides. It wrapped up about an hour earlier than planned.
Protesters surrounded people leaving the rally, shouting "shame" and "go home" and occasionally throwing plastic water bottles. Police escorted several rally participants through the crowds, sometimes struggling against protesters who tried to stop them.
Some people dressed in black with covered faces several times swarmed rally attendees, including two men wearing the "Make America Great Again" caps from President Donald Trump's campaign.
The violence in Charlottesville triggered the biggest domestic crisis yet for Trump, who provoked ire across the political spectrum for not immediately condemning white nationalists and for praising "very fine people" on both sides of the fight.
On Saturday, Trump on Twitter praised the Boston protesters.
"I want to applaud the many protestors in Boston who are speaking out against bigotry and hate. Our country will soon come together as one!" Trump tweeted. "Our great country has been divided for decades. Sometimes you need protest in order to heal, & we will heal, & be stronger than ever before!"
Thirty-three people were arrested, largely for scuffles in which some protesters threw rocks and bottles of urine at police dressed in riot gear, the Boston Police Department said.
"There was a little bit of a confrontation," Police Commissioner William Evans told reporters, adding that "99.9 percent of the people who were here were here for the right reasons."
Several protesters said they were unsurprised that the "Free Speech" event broke up early.
"They heard our message loud and clear: Boston will not tolerate hate," said Owen Toney, a 58-year-old community activist who attended the anti-racism protest. "I think they'll think again about coming here."
U.S. tensions over hate speech have ratcheted up sharply after the Charlottesville clashes during the latest in a series of white supremacist marches.
A growing number of U.S. political leaders have called for the removal of statues honoring the Confederacy, with civil rights activists charging that they promote racism. Advocates of the statues contend they are a reminder of their heritage.
Organizers of Saturday's rally in Boston denounced the white supremacist message and violence of Charlottesville and said their event would be peaceful.
Republican U.S. Senate candidate Shiva Ayyadurai spoke at the rally, surrounded by supporters holding "Black Lives Matter" signs.
"We have a full spectrum of people here," Ayyadurai said in a video of his speech posted on Twitter. "We have people from the Green Party here, we have Bernie (Sanders) supporters here, we've got people who believe in nationalism."
Protesters also gathered on Saturday evening in Texas. In Dallas, where a Lee statue was vandalized overnight, about 3,000 people rallied near City Hall to demonstrate against white supremacy.
"Tear them down," they chanted, referring to statues of Confederate figures.
A man who appeared waving a Confederate flag was quickly surrounded by demonstrators. "Shame on you," they chanted. Police officers escorted the man away a few minutes later as the crowd cheered.
In Houston, a chapter of Black Lives Matter organized a rally to call for the removal of a "Spirit of the Confederacy" monument from a park.
While Boston has a reputation as one of the nation's most liberal cities, it also has a history of racist outbursts, most notably riots against the desegregation of schools in the 1970s.
Karla Venegas, a 22-year-old who recently moved to Boston from California, said she was not surprised that the Free Speech rally petered out so quickly.
"They were probably scared away by the large crowd," Venegas said.
(This story has not been edited by NDTV staff and is auto-generated from a syndicated feed.)