The suspect, identified as Ichiro Kojima, attacked the victims around 10:00 pm (1300 GMT) aboard the shinkansen train which was travelling from Tokyo to a station in Osaka, police and news reports said.
Police officers stormed into the rail carriage and took the man into custody after the train, carrying some 880 passengers, made an emergency stop at Odawara station.
Police arrested the 22-year-old unemployed suspect on suspicion of attempted murder, a local police spokesman told AFP.
"A male passenger was sent to hospital and his death was confirmed later," the spokesman said.
News reports said two women who appeared to be in their twenties were wounded but their injuries were not life-threatening.
The suspect told investigators that he felt "frustrated and wanted to kill someone", public broadcaster NHK said, adding that he used at least two knives.
Witnesses said passengers fled in panic, some of them in tears.
"Everyone appeared to be feeling the strain and some of them cried," he said.
A female passenger told NHK: "Everyone fled and fell like one domino falling after another. I was scared to death."
Japan has one of the lowest rates of violent crime in the developed world and mass attacks are extremely rare.
In 2015, a man set himself ablaze on board a moving bullet train, killing himself and a woman in an unprecedented incident.
(This story has not been edited by NDTV staff and is auto-generated from a syndicated feed.)
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.