When it is finished the wall will be 10-11km (6-7 miles) long and will extend 18 metres below the surface.
The Egyptians are being helped by American army engineers, who the BBC understands have designed the wall.
The plan has been shrouded in secrecy, with no comment or confirmation from the Egyptian government.
The wall will take 18 months to complete.
For weeks local farmers have noticed more activity at the border where trees were being cut down, but very few of them were aware that a barrier was being built.
That is because the barrier, made of super-strength steel, has been hidden deep underground.
The BBC has been told that it was manufactured in the US, that it fits together in similar fashion to a jigsaw, and that it has been tested to ensure it is bomb proof.
It cannot be cut or melted - in short it is impenetrable.
Intelligence sources in Egypt say the barrier is being sunk close to the perimeter wall that already exists.
They claim 4km of the wall has already been completed north of the Rafah crossing, with work now beginning to the south.
The land beneath Egypt and Gaza resembles a Swiss cheese, full of holes and tunnels through which the Palestinians smuggle the everyday items they are denied by the blockade.
But the Israelis say the tunnels are also used to smuggle people, weapons, and the components of the rockets that are fired at southern Israeli towns.
The wall is not expected to stop all the smuggling, but it will force the Palestinians to go deeper and it will likely cut the hundreds of superficial tunnels closer to the surface that are used to move the bulk of the goods.