Though it is largely admitted that non-cognitive skills matter for adult outcomes, little is known about how the family environment affects their formation. In this paper, we use a cohort study of children born in 2000-2001 in the U.K. (Millennium Cohort Study) to estimate the effect of family size on socio-emotional skills, measured by the Strengths and Difficulties Questionnaire. To account for the endogeneity of fertility decisions, we use a well-known instrumental approach that exploits parents’ preference for children’s gender diversity. We show that an increase in family size strongly and negatively affects the socio-emotional skills of the two first children in a persistent manner. However, we show that this negative effect is entirely driven by children below age six and by girls. We investigate two potential mechanisms to explain this gender effect: the unequal response of parents’ time investments and the unequal demand for household chores.