A news alert saying: "It appears North Korea launched a missile... 'Evacuate inside buildings or underground': government" was published by the NHK website and app at 6:55 pm local time (0955 GMT).
But after just five minutes, NHK admitted the alert was sent in error.
"We're very sorry," NHK said on its website, without explaining further.
It comes just days after a false cellphone alert warning of an incoming ballistic missile terrified residents in Hawaii.
Japan and North Korea have a tense relationship, with Pyongyang routinely issuing verbal threats as well as firing missiles near or above Japan.
"Don't scare me. It's good that (North Korea) didn't launch a missile," one Japanese Twitter user said.
The mistake also comes less than two weeks after a fake earthquake alarm in Tokyo, where millions of residents received an alert warning them that a strong quake was about to hit the Japanese capital.
In Hawaii, the notification was sent out on Saturday, lighting up phones with a disturbing alert urging people to "seek immediate shelter."
Emergency management officials later admitted "the wrong button was pushed" during a shift change.
But it took nearly 40 minutes for a corrected message to be issued -- with Hawaii's governor saying there was no automatic way to cancel the false alarm, meaning it had to be done manually.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections. Election results will be out on May 23.