Passengers lying on stretchers littered the pier near South Street Seaport, attended to by firefighters and rescue workers who rushed to the scene of the 8:43 am (1343 GMT) hard landing.
"People were thrown into the air and onto the ground," passenger Elizabeth Banta told CNN. "There were definitely many people on the ground who were not moving."
Some of the injured were taken to the hospital, while others were treated at the scene, she said.
A police department spokesperson said one of the 57 people injured appeared to be in critical condition.
"The boat was backing up and hit something and that's when everything went crazy," said one witness, a construction worker who declined to be named, who was working nearby when the 400-passenger ferry slammed into the pier.
The National Transportation Safety Board tweeted that its investigators were gathering information about the cause of the crash.
Television images showed damage to the side of one end of the ferry, called Seastreak Wall Street, which departed from Atlantic Highlands, New Jersey.
In October 2003, a Staten Island ferry crashed into a maintenance pier, killing 11 people and injuring dozens more, and the same vessel, the Andrew J. Barberi, was involved in a second accident in May 2010 that injured around 40 people.
The ferry pilot in the 2003 crash and his supervisor were sentenced to more than a year in prison each for their roles in the accident. The pilot, Richard Smith, had passed out at the helm.
The ferry that crashed on Wednesday is run by Seastreak, a privately owned company that also holds the Interlake Steamship Company, Mormac Marine Group, Inc., and Moran Towing Co., the largest tug and barge operator on the East and Gulf Coasts, according to the company's website.