The poster shows a woman dressed in a burka in front of black minarets standing on a Swiss flag.
But Zurich city council said campaign posters were protected by free speech.
The advert is being used by the far-right Swiss People's Party (SVP) ahead of next month's referendum on whether to ban the building of new minarets.
The Swiss Federal Commission Against Racism said earlier this week that the poster was "tantamount to the denigration and defamation of the peaceful Swiss Muslim population".
Some media reports have said the minarets resemble missiles.
Zurich city council said on Thursday that although it disapproved of the "negative and dangerous" poster, it had to be accepted as part of political free speech ahead of the 29 November national referendum.
The city followed the examples of Geneva, Lucerne and Winterthur, who earlier also gave the green light to the use of the SVP's advert.
But a number of Swiss cities - including Basel, Lausanne and Fribourg - have banned the advert in public spaces.
Meanwhile, an opinion poll on Thursday showed that 51% of those questioned would reject the proposed ban.
Nearly 35% of the respondents supported the ban, according to the poll in the Tages-Anzeiger newspaper.
Switzerland is home to some 300,000 Muslims, who make up about 4% of the population.
It has hundreds of mosques, but only a handful of them have minarets.
Plans to build more minarets prompted the campaign for a ban.