This couldn’t be further from the truth. In fact signing requires almost no extra time and in the long run, actually saves parents time. Once you understand the basics of signing which can take about an hour, more if you really want to master signing, it takes no much longer than learning a few new signs every other day or so. Learning a sign, by the way, only takes seconds in most cases since a lot of ASL is based on the things it is meant to replace.
There’s really no legitimate reason not to sign with your baby. While not every baby will become a master signer (100+ signs), learning 10-20 is simple and almost mandatory, in my opinion. Signing really is something that you do as you live your life, signs are just done as you speak and eventually, your baby will pick them up and use them himself.