Well I can't say for definite as I've not done it myself ("yet" - it's in the plans). I believe that the RCA outs are at line level, i.e. after the input switching but before the volume control, so it doesn't involve most of the pre-stage. So you're taking a relatively unmolested signal into the headphone amp to boost it to the level needed for headphones, and to control attenuation.
As I said, I'd check with Frederik because on the 250S you hold down the standby button to enable / disable the speaker outputs (and the source button to enable / disable the home theatre bypass function). On the 150 the standby button does the HT bypass as I understand, and due to the difference in controls (you have rotary input selection) I don't know for definite that the speaker output can be disabled, and if so, how to do it. None of the manuals mention this function unfortunately - I found it out by mistake but couldn't work out what I'd done, I had to call him to find out.