Even with the losses, the Standard & Poor's 500 index finished the year up 11.4 per cent, or 13.7 per cent when dividends are included. It was the sixth straight year of gains for the stock market.
Oil, by contrast, had its worst annual performance since 2008, ending down 45 per cent for 2014 after a sharp slump in the second half of the year.
The market's annual gain exceeded even most optimistic forecasts made at the beginning of the year.
"It turned out to be a great year for US economic growth, which got us higher corporate profits as well," said Cameron Hinds, regional chief investment officer for Wells Fargo Private Bank.
Most strategists believe the stock market will also rise in 2015, but they expect more modest gains of between 4 per cent and 6 per cent.
There was no major catalyst for Wednesday's selling. Trading has been slow all week because of the holidays and most fund managers have closed their books for the year. However, some investors do reshuffle their portfolios in the last few days of the year for tax purposes.
Roughly 2.6 billion shares were traded on the New York Stock Exchange, compared with the 3.6 billion traded on an average day.
Energy stocks edged lower as the price of oil fell. Benchmark US crude dropped 85 cents to $53.27 a barrel in New York. Oil has plunged by half since June amid abundant supplies and weak global demand.
Oil drillers fell the most Wednesday. Diamond Offshore was the biggest decliner in the S&P 500, declining 3.6 percent. The energy component of the S&P 500 is down 10 percent this year
"I think most of the selling you're seeing today is related to the fall in oil, as well as repositioning before the end of the year," Hinds said.
US markets will be closed Thursday for New Year's Day and will reopen on a normal schedule on Friday.
On Wednesday, the Dow Jones industrial average fell 160 points, or 0.9 per cent, to 17,823.07. It ended 2014 up 7.5 per cent, lagging behind the S&P 500 and Nasdaq.
The Nasdaq lost 41.39 points, or 0.9 per cent, to 4,736.05. The Nasdaq rose 13.4 per cent in 2014.
The S&P 500 fell 21.45 points, or 1 per cent, to 2,058.90.
Prices for US government bonds rose. The yield on the 10-year Treasury note edged down to 2.17 per cent. Bonds were an unexpected strong spot for the market in 2014. The 10-year note started 2014 at around 2.99 per cent. Bond yields fall as prices rise.
Gold fell $16.30 to $1,184.10 an ounce. The precious metal barely budged in 2014, falling 0.2 per cent, compared with its drop of 28.3 per cent in 2013.
Silver fell 68 cents to $15.60 an ounce and copper fell three cents to $2.83 a pound.