Im sure you would like to secure your family relations in every way, and to do that, you have to be smart enough to inculcate in your child about all the positive things in his or her father. When this kind of positive manipulation is done, the child will calmly talk to his father and probably make him feel guilty in the end, this will come with time and patience but its definitely worth a try.
When your husband will see how wisely you bring up your child, he will inshAllah come to his senses and give you more permission than you ever thought of. With prayers and patience, mountains can be moved!