You are 100% correct that this is all unproven speculation.
However, I'm not aware of any Scientists even trying to prove and/or disprove it and thus it isn't really the theory's fault that Scientific Data cannot be marshaled in support of it.
but where is the science to support that those mares stopped will produce inferior performing foals?
As far as I know this has not been studied by scientists. However, laymen have observed a significant drop-off in the SW percentage for top stallions since the era of 150+ Books began. Perhaps a less selective choosing of potential mates is entirely at fault, but the Sperm could very well also be to blame...
Are the remaining viable sperm inferior to sperm produced by the same stallion bred to fewer mares?
Perhaps yes, Perhaps no; my main point was that the less sperm = less high-quality sperm even if the Percentage of high Quality sperm remains the same (for instance, .0001% of 50,000 is 5, but .0001% of 100,000 is 10). Thus, there will be less competition for the Egg and a lower quality sperm will likely get there first, possibly resulting in a lower quality offspring.
I don't see any definitive evidence supporting the theory that more stallion covers results in fewer superior performing foals result.
Again, you are 100% correct; the idea that stallions with bigger books get far worse stats is based more on anecdotes than methodical statistical research. Hopefully someone qualified will take up the gauntlet of studying this all-too often ignored issue (just don't expect Coolmore to bankroll it!).
-llbean