Kim made the remarks as the pair chatted ahead of their summit in the Demilitarized Zone that divides the Koreas, the official said.
According to the official, Moon told the visitor he could "show you scenes far better than this if you come to the Blue House", with Kim responding: "Really? I will go to the Blue House any time if you invite me."
Earlier, on the Military Demarcation Line that delineates the border, Moon asked him: "While you come to the South, when can I possibly go over there?"
After stepping over the line and becoming the first North Korean leader to set foot in the South since the end of the Korean War in 1953, Kim responded: "Why don't we just cross over now?" - prompting Moon's unscripted entry into Northern territory.
Later on, Kim said: "The border is not even that high. Wouldn't it disappear if many people keep walking across it?"
Last year the North carried out a series of missile launches, many of them in the early hours, and its sixth nuclear test, but Kim promised Moon there would be no more.
"I heard that you had early morning sleep disturbed many times because you had to attend the National Security Council meetings because of us," he said.
"I will make sure that your morning sleep won't be disturbed," he added, reiterating a pledge he made when Moon's envoys visited Pyongyang earlier this year.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections.