Al-Shabab, which is fighting to control the east African country, accused Israel of "starting to destroy" the Al Aqsa mosque, where standoffs have taken place recently between Israeli police and Palestinians.
The mosque is part of the complex that Jews call the Temple Mount and Muslims call Haram al-Sharif.
"The Jews started to destroy parts of the holy mosque of Al Aqsa and they routinely kill our Palestinian brothers, so we are committed to defend our Palestinian brothers," said Mukhtar Robow Abu Mansur, a prominent Al-Shabab commander.
His threat was part of a series of fiery sermons delivered after Friday prayers in Baidoa in southwest Somalia. Al-Shabab controls the region, which is part of a country that has been without an effective national government for nearly 20 years.
Other leaders of the group also threatened Israel, the first time the group is known to have done so.
"We will transfer and expand our fighting in the Middle East so we can defend Al Aqsa mosque from the Israelis," Al-Shabab commander Abdifatah Aweys Abu Hamza said in Mogadishu, the Somali capital.