The billboard was erected on the side of a road in Coimbatore in Tamil Nadu as part of memorials across India and the world to Mandela, who died on December 5.
But Freeman's face loomed large in the billboard over small images of rights icons Martin Luther King, Mother Teresa and Mahatma Gandhi.
"We should be proud that we were part of an era when they lived," read the Tamil-language condolence message on the board.
Cloth merchant Chandrashekhar, who paid for the board in a private capacity as a mark of respect to the former South African leader, said it was a mistake by the designer.
"We will replace it with the correct picture of Mandela," the merchant, who uses one name, told AFP by telephone from Chennai, adding that he did not know how the gaffe occurred.
Morgan Freeman played Mandela in the 2009 film "Invictus".
A photo of the billboard was being tweeted on Thursday.
India declared five days of national mourning for Mandela, who was hailed as a "true Gandhian" and a "great friend" by the country's leaders.