Hammond's comments, some of his most outspoken on the subject yet, underscore how the British government is ramping up its anti-immigration rhetoric in response to a spike in migrant attempts to reach Britain via the Channel Tunnel from France.
They are also part of a wider EU trend which has seen Alexis Tsipras say Greece cannot cope with the number of migrants fleeing instability in the Middle East and Africa and German calls for tighter immigration curbs.
"We have got to be able to resolve this problem ultimately by being able to return those who are not entitled to claim asylum back to their countries of origin," Hammond, speaking while visiting Singapore, told BBC TV.
"That's our number one priority."
Hammond said there would always be millions of Africans with "the economic motivation" to want to get to Europe and that EU laws meant migrants were "pretty confident" they could stay.
"That is not a sustainable situation because Europe can't protect itself and preserve its standard of living and social infrastructure, if it has to absorb millions of migrants from Africa," he said.
Britain's Conservative government is under pressure to show it is acting to solve what the press has dubbed "the Calais crisis" with hundreds of migrants trying nightly to scale fences around the entrance to the Channel Tunnel in France.
That has disrupted passenger and freight traffic and dominated the summer's headlines.
Some of the migrants manage to reach Britain.
On Sunday, police said they had arrested 18 suspected illegal migrants found hidden in the back of a lorry in England.
But the government's increasingly shrill tone on the issue - Cameron was criticised for referring to migrants as "a swarm" - has upset charities, churchmen and left-wing politicians.
Earlier this month, Church of England Bishop Trevor Willmott told the government not to forget its humanity.
"When we become harsh with each other and forget our humanity then we end up in these stand-off positions," he told the Observer newspaper.
Hammond said the Calais crisis was far from solved.
"So long as there are large numbers of pretty desperate migrants marauding around the area there will always be a threat to the tunnel's security," he said.