One of the victims had her throat slit by the assailant, a man with a criminal record believed to be in his 30s who shouted "Allahu Akbar" (God is Greatest) at the start of his rampage.
Soldiers serving in a special force known as Sentinelle set up to guard vulnerable areas in terror-hit France responded to the stabbings and shot dead the attacker, whose identity remains unknown.
"I was on the esplanade just in front of the station," Melanie Petit, an 18-year-old student, told AFP. "I heard someone shout 'Allahu Akbar' and I saw a man who seemed to be dressed all in black."
Heavily armed police sealed off and evacuated the ornate rail terminus in the bustling heart of France's second-biggest city, stopping all train traffic on some of the country's busiest lines.
The latest deaths came with France still on high alert and under a state of emergency following a string of attacks in recent years by extremists linked to the Islamic State group or Al-Qaeda.
Since 2015, a total of 239 people have been killed in France by jihadists, according to an AFP count before Sunday's incident.
After the lunchtime attack, anti-terror prosecutors said they had opened an investigation into "killings linked to a terrorist organisation" and the "attempted killing of a public official".
Travellers around the station described "controlled panic" as security forces evacuated passengers and looked for possible accomplices, while another witness said white sheets were placed over the bodies of the victims.
"There were police everywhere," said Francois Jacquel, a retired traveller who was in a waiting room.
The incident came only days after the Islamic State group released a recording of what it said was its leader Abu Bakr al-Baghdadi urging his followers to strike their enemies in the West.
France has deployed troops and its air force to the Middle East and is a leading partner in the US-led international coalition fighting IS in Iraq and Syria, where the jihadists are being driven back.
In Nice in July 2016, 86 people were killed when an extremist drove a truck into crowds after a fireworks display on Bastille Day.
But there have also been numerous smaller attacks on police officers, soldiers or members of the public since then, sometimes carried out by people with severe psychological problems.
Since November 2015, the country has been in a state of emergency which gives the government and security forces greater powers to combat extremists and launch anti-terror raids.
Controversial security law
New centrist President Emmanuel Macron has vowed to end the state of emergency with a new and controversial security law that will make many of the provisions of the emergency regime permanent.
Despite criticism from rights groups that the law reduces judicial oversight over the actions of the police, the lower house of parliament is set to vote on a first draft of the legislation on Tuesday.
French Interior Minister Gerard Collomb posted on Twitter that he would travel to Marseille on Sunday, while Prime Minister Edouard Philippe sent his condolences and said "we will not lower our guard."
In August, a man driving a van killed one person and seriously injured another after ploughing into a bus stop in Marseille, raising fears of another terror incident.
But doctors said later that the man had severe mental problems and discounted any terror link.
The soldiers who shot the knifeman on Sunday were part of the Operation Sentinelle force composed of 7,000 troops who guard high-risk areas such as transport hubs, tourist sites and religious buildings.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.