So are you saying you think the benefit to the Cans will outweigh any noticable difference with the speakers? Because that was my suspicion as the cans IMO tend to be more revealing.
On reflection, your tape loop should only run through a monitor switch and not through your main amp volume control. The set-up as you have it, headphone amp via tape loop, is probably the best set-up for both headphone and ‘speaker listening. Try a direct connection from your CD player to headphone amp (removing the main amp entirely) and see if any different.