Declan, over the years since I bought Derek's book, I've slowly come to the conclusion that he is slightly generous in his assessments. Still a good book, nonetheless. Almost, but not quite a bible, to be honest.
I'd estimate your 05 to be VG, your 03 as fine but battered, and your 04 as a nice straight, problem free fine. Given the rarity of the 05 (and don't forget the 03 & 04 aren't exactly common, being both sub million mintages), you weren't done by paying what you did for the Edward VII date run.
I agree with Peck's estimation of F/NVF for the LCA 05 halfcrown. I also agree that they are very conservative with their grading.