The Chinese government sent tanks to quell the June 4, 1989 protests, and has never released a death toll. Estimates from human rights groups and witnesses range from several hundred to several thousand.
The Tiananmen crackdown is a taboo subject in China and 29 years later it remains a point of contention between China and many Western countries.
In a statement on Sunday the recently appointed Pompeo said he remembered "the tragic loss of innocent lives".
"As Liu Xiaobo wrote in his 2010 Nobel Peace Prize speech, delivered in absentia, 'the ghosts of June 4th have not yet been laid to rest'," Pompeo said referring to the Chinese dissident who died last year while still in custody.
"We join others in the international community in urging the Chinese government to make a full public accounting of those killed, detained or missing," Pompeo added.
In response to Pompeo's comments, China had lodged "stern representations" with the United States, foreign ministry spokeswoman Hua Chunying said on Monday.
China long ago reached a clear conclusion about the events of that era, but the United States every year issues statements of "gratuitous criticism" and interferes in its internal affairs, Hua said.
"The U.S. Secretary of State has absolutely no qualifications to demand the Chinese government do anything," she added.
Hu Xijin, editor of nationalistic tabloid the Global Times, called Pompeo's statement a "meaningless stunt" that "represents a wish of the Western world to meddle in China's political process".
Tens of thousands of people are expected to gather later in the day in Hong Kong to mark the anniversary, the only place in China where such large-scale public commemorations happen.
On Tiananmen Square, security was tight as is usual for the anniversary, with no signs of any protests or other memorial events.
Foreigners' passports were checked by Chinese police at a checkpoint nearly a kilometre from the square. A Reuters reporter was turned away and told that unapproved "interview activities" were forbidden in the square on Monday.
In their annual open letter, the Tiananmen Mothers, who represent the families of those who died, said the government was guilty of serious disrespect by ignoring their requests for redress.
"Such a powerful proletarian dictatorship apparatus is afraid of us: the old, the sick, and the weakest and most vulnerable of our society," they wrote in a letter addressed to Chinese President Xi Jinping.
In Taiwan, the democratic and self-ruled island China claims as its own, President Tsai Ing-wen said that if China could face up to what had happened it could become the bedrock for China's own democratic transformation.
"I hope that both sides of the Taiwan Strait can enjoy the universal values of freedom and democracy," she said in a statement on her Facebook page written in the simplified Chinese script used in China. Facebook is also blocked in China.
Get the latest election news, live updates and election schedule for Lok Sabha Elections 2019 on ndtv.com/elections. Like us on Facebook or follow us on Twitter and Instagram for updates from each of the 543 parliamentary seats for the 2019 Indian general elections. Election results will be out on May 23.