Speaking in Brussels a day before Britain triggers the two-year Brexit process, Sadiq Khan also urged Prime Minister Theresa May to give a "cast iron" guarantee of the rights of EU citizens living in Britain.
"There is no need for the EU to send a message or to instil fear by punishing the UK," Khan told an event run by the Politico Europe newspaper.
"A bad Brexit deal that hurts London would hurt the EU too," added Khan, who is meeting key EU figures including European Parliament chief Antonio Tajani while in Brussels.
Khan warned that businesses driven out of London, the world's leading financial centre, would "leave Europe altogether and go to New York and Singapore instead."
"A bad Brexit really is a lose-lose situation."
"It would be a perfect gesture of goodwill to provide a cast iron guarantee of their rights to remain in the UK," Khan said.
Khan also held a minute of silence for victims of last week's attack on the British parliament and said it was important for Britain and the EU to keep cooperating on security after Brexit.
After meeting Khan, Tajani said that he would "work hard towards achieving a fair and orderly divorce between the European Union and the United Kingdom."
MEPs will have the final vote on any Brexit deal, and are also set to vote next week in Strasbourg on a resolution on their goals for the negotiations..