JPMorgan, Bank of America and Citigroup were all down between 1.3 per cent and 2.0 per cent. The financial sector was down 0.96 per cent and was the biggest drag on the S&P 500.
China's official factory activity gauge expanded only marginally in May, data showed, while a private survey showed conditions deteriorated for a fifteenth straight month.
"This data is a barometer, to many, of economic activity and growth," said Andre Bakhos, managing director at Janlyn Capital in Bernardsville, New Jersey. "Any hiccup, any speed bump in China will thwart upside activity."
US consumer confidence dipped, data on Tuesday showed, while a survey on business activity in US Midwest also underwhelmed.
That does not bode well for the Institute of Supply Management's manufacturing survey for May, due later in the day, with traders saying that a weak reading could see chances of a rate hike in June recede.
Traders are now pricing only a 15 per cent chance of a hike in June, down from a probability of around 32 per cent on Monday, according to the CME Group FedWatch tool.
The Federal Reserve caught investors off guard earlier this month when it signaled its next rate hike could be as soon as its June 14-15 meeting.
At 1335 GMT (7:05 p.m. in India), the Dow Jones industrial average was down 119.33 points, or 0.67 per cent, at 17,667.87. The S&P 500 was down 11.64 points, or 0.56 per cent, at 2,085.32. The Nasdaq Composite was down 21.99 points, or 0.44 per cent, at 4,926.06.
Oil majors Exxon and Chevron were down about 1 per cent as crude prices fell on expectations of OPEC (Organization of Petroleum Exporting Countries) inaction on output due to its focus stays firmly on market share, while concerns about China weighed on the demand outlook.
"The market also keeps an eye on oil as another economic barometer. Weak oil, weak economy. Strong oil, stable economy," Bakhos said.
Nine of the 10 major S&P sectors were lower, with the material index's 1 per cent fall leading the decliners.
Shares of Nike fell 3 per cent to $53.52 after a Morgan Stanley downgrade on higher chances of a slowdown in US sales due to competition and a weakening US market.
Handbag maker Michael Kors rose 6.2 per cent to $45.23 after posting its strongest sales growth in a year and announcing a new share buyback.
Demandware soared 56 per cent to $74.87 after Salesforce.com agreed to buy the company in a $2.8 billion deal. Salesforce was down 1.1 per cent at $82.79.
Declining issues outnumbered advancing ones on the NYSE by 1,939 to 724. On the Nasdaq, 1,443 issues fell and 742 advanced.
The S&P 500 index showed one new 52-week high and no new lows, while the Nasdaq recorded nine new highs and seven new lows.
© Thomson Reuters 2016