At about the same time of the Imperial Guard attack, the Prussian 5th, 14th, and 16th Brigades were starting to push through Plancenoit, in the third assault of the day. The church was by now on fire, while its graveyard—the French centre of resistance—had corpses strewn about "as if by a whirlwind". Five Guard battalions were deployed in support of the Young Guard, virtually all of which was now committed to the defence, along with remnants of Lobau's corps. The key to the Plancenoit position proved to be the Chantelet woods to the south. Pirch's II Corps had arrived with two brigades and reinforced the attack of IV Corps, advancing through the woods.

The 25th Regiment's musketeer battalions threw the 1/2e Grenadiers (Old Guard) out of the Chantelet woods, outflanking Plancenoit and forcing a retreat. The Old Guard retreated in good order until they met the mass of troops retreating in panic, and became part of that rout. The Prussian IV Corps advanced beyond Plancenoit to find masses of French retreating in disorder from British pursuit. The Prussians were unable to fire for fear of hitting Wellington's units. This was the fifth and final time that Plancenoit changed hands.

French forces not retreating with the Guard were surrounded in their positions and eliminated, neither side asking for nor offering quarter. The French Young Guard Division reported 96 per cent casualties, and two-thirds of Lobau's Corps ceased to exist.