"Two women and one man are dead, a third person is in hospital," Interior Minister Joelle Milquet said at the scene. "We don't yet know if they were tourists or staff, they haven't been identified."
Asked whether she believed it was an anti-Semitic attack, she said it was too early to say with a police and judicial inquiry just underway, but that given the target "there are strong grounds for presuming so".
A Jewish community figure, Joel Rubinfeld, told AFP it clearly "is a terrorist act" after two men were seen driving up and double-parking outside the museum.
The gunman opened fire, allegedly shooting indiscriminately before getting away.
The area around the museum has been closed off and the security has been beefed up in places associated with the Jewish community in Belgium, Milquet said.
Rubinfeld, who heads the country's anti-Semitic League, said the act was the result of "a climate of hate."
Authorities said a search was under way around Brussels to find the suspects. Media reports had said the driver was picked up within three hours of the shooting, which took place at around 4 pm (1400 GMT), but there was no confirmation from police.
A bystander, Alain Sobotik, told AFP he saw the corpses of a young woman and a man just inside the doors of the museum.
A picture shows them lying in pools of blood.
Also at the scene shortly after the shooting was Belgian Foreign Minister Didier Reynders who told reporters that the two other victims had been shot inside the museum.
"I hope we will identify those responsible very quickly," he said.
Reynders said he had been nearby when he saw people fleeing and heard shots and rushed to help.
When he saw "bodies on the ground in pools of blood" he called the 112 emergency number and rounded up eye-witnesses to assist the police.
"I am shocked by the murders committed at the Jewish museum, I am thinking of the victims I saw there and their families," Reynders said on Twitter.
While stopping short of calling it an anti-Semitic act, Reynders said "evidently one thinks of that."
The Jewish Museum of Belgium, which was not answering calls, is located in the heart of the Sablon district which is home to the city's top antique dealers.
It is a popular weekend haunt for shoppers and tourists, hosting the city's best chocolate shops and many cafes.
Belgium Prime Minister Elio Di Rupo also expressed that he was "very shocked" by the attack.
It comes on the eve of elections in Belgium for a new federal government as well as for its regional parliaments and the European Parliament.
In 1982, a gunman opened fire at the entrance of the synagogue in Brussels, wounding four people, two with serious injuries.