Bach Sonatas and partitas for solo violin - the Grumiaux set on Philips because I love the sound of his Guarneri and it's a good price;
Debussy, Ravel and Faure string quartets - Quatuor Ebene on Virgin, good intro to this repertoire, even though I often prefer other performances as the Ebenes get a bit muscular at times. A very good bargain Faure and Ravel is the Ad Libitum Quartet on Naxos;
The Tallis Scholars disc of Allegri's Miserere, Mundy's Vox Patris Caelestis and Palestrina's Missa Papa Marcelli - a good intro to renaissance polyphony;
a selection of Sibelius symphonies - I've listening to the Segerstam/Helsinki set quite a lot, good recordings;
I have to put a plug in for something English, so Boult's Vaughn Williams Symphonies 3 & 5, which are my fave VW symphonies, good performances and available cheaply.
On the Debussy piano front, I'd suggest the recent Bavouzet set on Chandos, which is all round rather excellent.
Brandenburgs: English Concert on DG if you want full "authenticist" performance and instrumentation, but for a neophyte The English Chamber Orchestra on Philips may be better, as it is a "period"-informed performance, but modern instruments.
Beethoven symphonies: 5 & 7 has to be the Carlos Kleiber on DG; don't forget 9 - the recent Vanska/Minnesota set are good.
Elgar `Cello Concerto - well, the obvious reco is Jacqueline du Pre, but if a somewaht less over-wroought performance is wanted try Tortelier.
For Schubert's Death and the Maiden, the Quartetto Italiano are good and reliable, but I tend to prefer The Lindsays, who are more volatile and really power into the introduction, or the recent Jerusalem Quartet.