Pence first joined a working lunch with Prime Minister Shinzo Abe before meeting Deputy Prime Minister Taro Aso for economic talks that the vice president said he hoped would yield a framework for further dialogue.
Earlier, U.S. Commerce Secretary Wilbur Ross met Japan's trade minister, Hiroshige Seko. Ross, seen as more hardline on trade, told reporters Washington was eager to increase trade ties with Tokyo through a two-way agreement.
Pence landed in Tokyo from South Korea after a trip that included a visit to the heavily fortified border separating the North and South. He described the U.S.-Japan alliance as the "cornerstone" of regional security.
Pence's 10-day tour of Asia is aimed at emphasising that U.S. President Donald Trump wants to boost U.S. trade in the region even though Trump has abandoned the 12-nation Trans-Pacific Partnership (TPP) trade pact.
Advocates for the TPP, negotiated by former President Barack Obama and supported by Abe, said it would have opened markets for American exports. U.S. business groups supported the deal but U.S. labour unions argued it would hurt American workers.
Trump campaigned for office on an "America First" platform, saying he would boost U.S. manufacturing jobs and shrink the country's trade deficit with countries like Japan.
Trump also vowed to renegotiate existing regional trade deals to focus on bilateral agreements. Tokyo is wary of a two-way free trade agreement (FTA), fearing it would boost pressure to open up politically sensitive sectors such as agriculture.
"It's a little bit early to say just what forms things will take but we are certainly eager to increase our trade relationships with Japan and to do so in the form of an agreement," Ross said when asked about a possible FTA.
Japan had a $69 billion trade surplus with the United States last year, the U.S. Treasury Department said, expressing concern over what it called the "persistence" of the imbalance.
Japanese officials counter that Tokyo accounts for a much smaller chunk of America's deficit than in the past, while China's imbalance is much bigger.
Trump has complained that Japan keeps its currency artificially low, although a Treasury Department report last week did not label Japan a currency manipulator. The issue was not expected to be raised in talks on Tuesday.
Pence said in Seoul earlier on Tuesday the Trump administration would review and reform a 5-year-old free trade agreement with South Korea.
The Korea Automobile Manufacturers Association expressed concern about any review of the pact, which removed tariffs on auto trade between the two countries.
Trump and Abe agreed in February to have Pence and Aso, who is also Japan's finance minister, open an economic dialogue.
The two sides were expected to agree on principles and a process for further detailed discussions between ministries.
"Obviously, the first high-level talk is not going to have an immediate resolution of anything but I think we have a reasonable path forward," Ross said.
Aso told reporters before the talks he would not discuss any bilateral free trade deal with Pence. The White House adviser said Tuesday's talks would not prescribe a free-trade deal but talks might eventually lead to such negotiations.
Pence developed ties with Japanese business and political leaders as governor of Indiana, a state that is home to Subaru, Honda and Toyota plants, and about 260 Japanese companies employing about 60,000 residents.
"To some extent, we want to do for the United States what we did for Indiana," the White House adviser said.
Indiana is the top U.S. state for per-capita foreign direct investment from Japan, including higher-technology and better-paying manufacturing jobs, said Victor Smith, Pence's commerce secretary when he led the state.
The Trump administration wants to attract more foreign direct investment, hoping to lure some with a $1 trillion plan to rebuild U.S. roads, bridges and other infrastructure.
Pence will take that message to upcoming stops this week in Jakarta and Sydney. While no immediate announcements are expected from Tokyo, the White House expects a "handful" of investment announcements while Pence is in Australia at the weekend, the adviser said.
He is due to meet corporate leaders at each of his stops to reassure businesses that trade in the United States is worth their while.
"Part of the trip is the very big symbolism of listening," a White House adviser said. "We're not pivoting away from the region."
(Additional writing by Linda Sieg; Additional reporting by Kiyoshi Takenaka, Leika Kihara and Tetsushi Kajimoto in TOKYO, Hyunjoo Jin in SEOUL and David Lawder in WASHINGTON; Editing by Toni Reinhold and Paul Tait)
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)