Aung San Suu Kyi has asked Annan to head an advisory commission to recommend how her government can narrow bitter sectarian divides in the western state.
Rakhine has suffered deadly anti-Muslim violence since 2012 and the Rohingya question remains incendiary in Buddhist-majority Myanmar.
Their plight has also seen Suu Kyi's reputation as a rights defender tarnished.
Myanmar nationalists insist the million-strong group are illegal immigrants from Bangladesh and refuse to use the word "Rohingya", instead labelling them "Bengalis".
Annan, who is due to arrive in the state capital Sittwe on Tuesday, told reporters he intends to be impartial in his peace and reconciliation bid.
But Rakhine activists object to the visit.
"Rakhine affairs are local affairs. We acknowledge Kofi Annan and his reputation but we do not accept his interference in our affairs," Aung Htay, one of the leaders of the protest group, said.
"We will protest tomorrow at Sittwe airport. We respect Kofi Annan personally but we do not think he knows Rakhine history."
The envoy is expected to visit Rohingya camps and meet Rakhine community groups and officials.
But the region's largest political group, the Arakan National Party, has already ruled out meeting the former UN secretary-general.
More than 100,000 of the Muslim group live in squalid displacement camps.
Their treatment is seen as a black spot on Myanmar's democratic progress since Suu Kyi's elected government took power in April.
Suu Kyi, Myanmar's de facto leader, said Monday Annan's nine-member commission could help "heal the wounds" of sectarian conflict.
"We cannot ignore problems.... ignoring problems will simply allow them to get worse and worse," she told reporters.
But for the first few months of her tenure in office Suu Kyi tried to avoid inflaming Buddhist nationalist sentiment.
In June she ordered officials to refer to the group only as "Muslims of Rakhine State".
But even that order sparked mass protests in Rakhine, with local Buddhists demanding the government call them "Bengalis".
Rakhine protesters have previously targeted international aid agencies in the state as well as the UN over their perceived support for the Rohingya.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.