My 2 cents:

1/ Why do all our wargames rules have to cost the bomb?

They don’t. You can get pdfs from Wargames Vault for just about any period, some even cheaper than the ye olde £6 you mentioned.

You don’t like PDFs fair enough, think cheap equals dodgy quality, either reading of a screen or printing yourself isn’t everyones cup of tea.

Well Wargame magazines, especially Miniatures Wargames have there own inhouse rules for games presented in an article. They can be restricted to a particular period or campaign, but I believe they’ve often used a Version of By Crom? That costs £6 or less unless over 1 or two issues and there’s some generally still some supplemenatary support on the website.

The Osprey Blue Books are comfortably under £15 and to take one of there more successful titles Lions Rampant, you had everything you needed in that one book. Same with most of the others like Ronin, and Honours of War.

You can still get some A5 rule sets of the sort you mention from Tumbling Dice for there air war. Although there quite short but there starter sets of rules and a handful of aircraft I think are a tenner or there about.

Going 1 better then that there are the Peter Dennis & Andy Callan Paper Army Rulebooks. You get not only the rules but the participatent armies for under £20.

2/ When it happened? Well the first instance I can think of would be the September 1987 40K Rogue Trader, and Fire and Fury a year or two later also being of the moving away from A5 stock cover and B&W illustrations. Also around the time you got A4 Glossy cover and B/W interior such as Frank Chadwicks Wargames and Star Wars Miniatures Battles by West End Games. Like 40K they had toes in the RPG world, so probably not toally unrelated to how they started to produce some wargames with others following and enhanced shiny production. Also probably not unrelated to late 80s early 90s computers were becoming more of a feature in the publishing world.

Also perhaps, those rule makers were from a time they used old Hardbacks of H G Wells, Grant and Donaldson and wanted to catch a more meaty feel in there hands rulebook.

But to be competitive with the other companies, glossy and coffee table book rulesets certainly dominate, but there are still small fish in the shallows. I think It depends what you want, and what you and your friends can settle on.