But science may have just proved them right – because beautiful women are more likely to have daughters than their plainer counterparts, according to a study.
As parents tend to pass on genes that determine looks, this could result in handsome men becoming rather thin on the ground.
And it may also explain why many models have daughters who follow in their glamorous footsteps – such as Yasmin Le Bon, who is signed to the same modelling agency as daughter Amber, and Jerry Hall, whose daughters Elizabeth and Georgia Jagger have both taken to the catwalk.
Dr Satoshi Kanazawa, of the London School of Economics, analysed data from a survey of 17,000 babies born in Britain in March 1958 and tracked them throughout their lives.
At the age of seven, their attractiveness was rated by their teachers.
When they reached 45, they were asked about the gender of any children they had.
Those rated as attractive were equally likely to have a son or daughter as their first child – but the unattractive sorts were more likely to have a son.
Put another way, the beautiful women were more likely to have daughters than those who were less blessed in the looks department, the journal Reproductive Sciences will report.
A previous study of 2,000 Americans suggested that women are becoming more beautiful over the generations because attractive women have more children than plain ones – and a higher proportion of their offspring are girls.
Dr Kanazawa believes that parents tend to produce children who benefit from their own attributes.
Beauty, he says, is of more benefit to a woman than a man, and so it pays for attractive women to have daughters.
But couples blessed with strength and aggression rather than looks are better off having boys, as these characteristics are of more use to males.
However, not everyone subscribes to his theory.
Andrew Geltman, a statistician at Columbia University in the U.S., analysed People magazine’s annual 50 most beautiful people lists for 1995 to 2000.
With the featured celebrities slightly more likely to have sons that daughters, his conclusion was the opposite of Dr Kanazawa’s.
And his verdict will probably prove more popular with glamorous mothers such as Victoria Beckham – who has three sons with husband David.
This is not the first time that Kanazawa has come up with a controversial theory.
Earlier this year another of his studies made headlines when it found that men who cheat on their partners are more likely to have lower intelligence.
Dr Kanazawa explained that entering a sexually exclusive relationship is an 'evolutionarily novel' development for them.
According to his theory, intelligent people are more likely to adopt what in evolutionary terms are new practices - to become 'more evolved'.
Therefore, in the case of fidelity, men who cannot adapt and end up succumbing to temptation and cheating are likely to be more stupid.
'The theory predicts that more intelligent men are more likely to value sexual exclusivity than less intelligent men,' he explained.