Kim made the remarks as the pair chatted ahead of their summit in the Demilitarized Zone that divides the Koreas, the official said.
According to the official, Moon told the visitor he could "show you scenes far better than this if you come to the Blue House", with Kim responding: "Really? I will go to the Blue House any time if you invite me."
Earlier, on the Military Demarcation Line that delineates the border, Moon asked him: "While you come to the South, when can I possibly go over there?"
After stepping over the line and becoming the first North Korean leader to set foot in the South since the end of the Korean War in 1953, Kim responded: "Why don't we just cross over now?" - prompting Moon's unscripted entry into Northern territory.
Later on, Kim said: "The border is not even that high. Wouldn't it disappear if many people keep walking across it?"
Last year the North carried out a series of missile launches, many of them in the early hours, and its sixth nuclear test, but Kim promised Moon there would be no more.
"I heard that you had early morning sleep disturbed many times because you had to attend the National Security Council meetings because of us," he said.
"I will make sure that your morning sleep won't be disturbed," he added, reiterating a pledge he made when Moon's envoys visited Pyongyang earlier this year.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.