Communities Secretary John Denham's comments came as he announced a drive to prevent white working class people being 'exploited' by extremists.
He singled out protests being organised by the English Defence League, some of which have turned violent over recent months.
Hundreds of supporters are expected to gather in Trafalgar Square tomorrow.
Yesterday protesters from the Stop Islamisation of Europe (SIOE) group were involved in a demonstration outside a mosque in Harrow, north east London, to mark the anniversary of the September 11 attacks. Ten people were arrested.
'I think that the English Defence League and other organisations are not actually large numbers of people,' Mr Denham told the Guardian. 'They clearly, though, have among them people who know what exactly they're doing.
Mr Denham pointed to historical 'parallels' with the so-called 'Battle of Cable Street' in October 1936, when Mosley, leader of the British Union of Fascists, attempted to lead supporters through a Jewish area of the East End of London, leading to violent clashes.
'You could go back to the 1930s if you wanted to - Cable Street and all of those types of things. The tactic of trying to provoke a response in the hope of causing wider violence and mayhem is long established on the far-right and among extremist groups.'
EDL's supporters are alleged to include far-right activists and football hooligans filmed at recent protests chanting racist slogans and making Nazi salutes.
Mr Denham said there was a need for a broader strategy from government to 'undercut issues that racists try to exploit'.
In the coming weeks ministers will unveil a programme targeted at mainly white, working-class communities, he said.
'You need to be prepared to let people's real underlying fears and concerns come out, but to be able to address them frankly and openly,' he said.
He gave the example of perceptions of unfair allocation of council housing and new jobs, and said there could be changes at street level to allow local people to 'influence and shape' how resources are distributed in their area.
The clashes on Friday afternoon came after a right-wing extremist group protested against the construction of a new five-floor mosque next to the Harrow Central Mosque.
Riot police stepped in to prevent the rally from going ahead after members of the Stop Islamification of Europe group clashed with more than 1,000 anti-fascist and Muslim demonstrators who gathered to defend the building.
More than 550 police charged at the protesters, largely made up of Muslim men, and corralled them around the mosque.
During one skirmish, a white teenage youth was chased by a mob of up to 200 chanting demonstrators.
Scotland Yard said there were seven arrests for public order offences or possession of weapons while one person was arrested to prevent a breach of the peace.
Weapons confiscated include a hammer, a chisel and a bottle of bleach.
Muslim supporters - some covering their faces with bandanas - could be seen throwing rocks and bricks while one group trampled a wire fence as the two groups goaded each other.
Source: Mail Online