New York City mayor Michael Bloomberg estimated that 30,000-40,000 houses in the city alone had been left unusable by Monday's storm.
Sandy pummelled 15 states with fierce winds and a huge tidal surge that left at least 112 dead in the United States and Canada and tens of billions of dollars of damage.
"It is starting to get cold, people are in homes that are uninhabitable," New York state governor Andrew Cuomo told a press conference. "We are going to have tens of thousands of people who need housing solutions right away."
"This is going to be a massive, massive housing problem," the governor added.
Bloomberg compared the crisis to that in New Orleans after Hurricane Katrina hit in 2005. "I don't know that anybody has ever taken this number of people and found housing overnight," the under-pressure New York mayor said.
Tens of thousands fled New Orleans because of the storm and found shelter in other cities, "in this case people are staying in New York City and it's a challenge for us," Bloomberg said.
More than 200,000 meals are already being handed out each day to the elderly and other needy victims in New York. The city is again relying on special buses and urging the homeless to go to emergency evacuation centers that remain open.
Poorer parts of the city, including the Rockaway and Staten Island districts were worst hit by the storm and Bloomberg was the target of expletive-laden rants by inhabitants when he went there on Saturday.
Bloomberg called off Sunday's New York international marathon because of protests about the diversion of resources to the event with so many suffering. Many of the 45,000 contestants descended on Central Park on Sunday to run part of the course.
The crisis remains acute in New Jersey where at least one million people were still without electricity on Sunday and fuel shortages forced governor Chris Christie to introduce rationing on Saturday.
About 730,000 people in New York state still do not have electricity nearly one week after Sandy hit, including 145,000 in New York City, the governor said.
Nassau county, one of the wealthiest parts of the United States, is now worst hit in the state however with 266,000 people still in the dark.
The US Energy Department said Saturday that 2.5 million customers remain without power across seven states. Each client could be a home with several people though.
Cuomo said that some people may have to wait two weeks to get power back and reaffirmed a vow to make sure utility companies are held "accountable".
With drivers waiting several hours in queues at gas stations to get fuel, Cuomo and Bloomberg appealed for patience and insisted that deliveries are improving.
At Coney Island people waited in line for up to six hours to get some of the free gasoline that federal authorities have sent to New York to help alleviate shortages. Temperatures in New York hovered just above freezing on Saturday night.
"I have to feed the kids, I have to clean up a terrible mess, I have to get ready for work tomorrow, but I have a generator so I have to stay in the queue," said local resident Karen Braithwaite.
Many of the few gasoline stations that are open in New York are limiting drivers to 30 dollars of fuel at a time, barely a third of a tank for many American cars.
New York subways are getting back to normal and nearly all schools are to reopen on Monday.
Severe problems remain in New Jersey however. Huge lines of cars and people on foot clutching canisters snaked back from gas stations across the northern half of the state.
Starting Saturday, drivers with license plates ending in an even number were only allowed to fill up on even-numbered dates, while those whose plates end in odd numbers had to wait for odd-numbered dates.