Martin Place in the central business district was shut down as scores of armed police surrounded the Lindt chocolate cafe, with TV pictures showing a flag -- black with white Arabic writing -- held to a window by terrified customers.
Reports said as many as 20 people were in the cafe and that there were at least two gunmen, although police had no immediate comment, only confirming that an operation was underway.
Witnesses reported hearing loud bangs that sounded like gun shots.
Patrick Byrne, a producer at Channel Seven whose newsroom is opposite the cafe, said staff at the television station watched the situation unfold.
"We raced to the window and saw the shocking and chilling sight of people putting their hands up against the panes of glass at the cafe," he told the Australian Broadcasting Corporation.
"This was just extraordinary."
Australia has been on high alert after the government raised concerns that citizens who have fought alongside jihadists in Iraq and Syria could return home radicalised and capable of carrying out attacks.
Martin Place is the finance centre of the city and houses several prominent buildings, including New South Wales state leader Mike Baird's office, the Reserve Bank of Australia, Westpac Bank and the Commonwealth Bank.
Authorities said they were also dealing with an "incident" at the nearby Sydney Opera House, although police did not say whether it was related to the ongoing siege.
"The police are responding to an incident at the Opera House," a New South Wales police spokeswoman told AFP, adding that no further details were available.
Reports said the Opera House had been evacuated.
NATIONAL SECURITY RESPONSE
"This is obviously a deeply concerning incident but all Australians should be reassured that our law enforcement and security agencies are well trained and equipped and are responding in a thorough and professional manner," he said.
The incidents came just minutes before police announced a man had been arrested in Sydney on alleged terrorism offences as part of ongoing investigations into plans for an attack on Australian soil.
They said the 25-year-old was seized as part of "continuing investigations into the planning of a terrorist attack on Australian soil and the facilitation of travel of Australian citizens to Syria to engage in armed combat".
It was not clear if the matters were related.
More than 70 Australians are currently fighting for Islamic militants in Iraq and Syria. At least 20 have died and there are mounting concerns that increasing numbers of youths are being radicalised and could mount attacks at home.
Journalist Chris Kenny, who was in the Lindt cafe just before the siege began, said he understood the automatic glass sliding doors had been disabled.
"I did speak to a couple of people who saw a bit more of this unfold than I did," he told the newspaper he works for, The Australian.
"One woman said she tried to go into the shop just after I came out with my takeaway coffee but the doors wouldn't open.
"So obviously whoever is doing this has disabled the automatic glass sliding doors to stop anyone else going in and she said immediately she could see there was a weapon.
"She mentioned it being taken out of a blue bag and people were straight away asked to put up their hands."