I don't know what to say since am not married. Am sorry for how hurt and mentally stressed his words made you feel. Motherhood is a lot of work, perhaps you can sit him down and talk about how his words make you feel?

Am sure he loves his daughter but then some men feel scared to hold a child that young, they have this impression that they might break them. Am glad he helps you out in other areas this shows that he values you and doesn't lean too much towards the I am the husband factor, and he might not know that he is hurting you. Communicate with him and from there you know how to proceed