The army operation got under way in the early hours of Sunday morning. Reports said that three hostages and four fighters were killed in the assault.
"It appears the operation is over as the rattle of gun fire has calmed down," Kamal Hyder, Al Jazeera's correspondent in Islamabad, said.
"The military is very confident they have been able to save a lot of the military personnel [held captive].
"The situation is still quite fluid and it's not quite clear how many people have been rescued, [but they say] over 20 hostages have been released," he said.
Gun fire and explosions were heard near the army headquarters just before dawn and soon afterwards reports from the Pakistani military said they had freed "most" of the hostages.
There was no immediate word on the number of people wounded, but a reporter with the Reuters news agency said that he saw three ambulances leaving the area.
Armed men, dressed as soldiers, attacked the army compound on Saturday, sparking a gunbattle that killed four fighters and six soldiers.
The fighters took hostages, after which the military surrounded the compound and the ensuing siege lasted for more than 18 hours.
It was unclear who was behind the attack, but the gunmen had been demanding the release of Taliban prisoners.
The fighter's brazen assault on a military compound in the garrison city of Rawalpindi is likely to revive fears for Pakistan's stability.
Source: Al Jazeera (English)