1980 LTD 1000 Rear brake caliper alignment

I am putting everything back together after new tube, tire, sprocket, master rebuild and brake pads.  My axel bolt was oriented left to right (castle nut on the right), presumably because of the aftermarket header.  The spacers as I remember them coming out on the right side were:

1. 92022-248 plane washer 4.5 mm thick was first and fit into the wheel bearing seal.
2. 92027-1177 was next.  That is a right side collar slightly rounded on the outer side
3. Brake caliper assembly
4. chain tensioner
5. castle nut

The left side had no spacers at all between the axel bolt head and the chain tensioner

From what I'm reading online, #1 above should be last on the right side next to the castle nut.  Ideally the brake assembly would move about 3mm to center it on the brake rotor.

This would move my brake caliper towards the wheel 4.5mm  but that seems too far given the alignment I'm seeing in the pictures.  In addition where the caliper attaches to the brake torque rod is already out of alignment with that spacer on the left side.  I need to push the torque rod inward towards the wheel to make it fit.  This is just slight pressure.

Anyone know how these parts should fit together?