Turns out that signing colours is a difficult thing to do. While my son understood readily the labels and could do the signs, he seemed to guess randomly when quizzed on objects that were unfamiliar to him. With time, he got better, but this might be due to knowing more colours. It seemed that when he concentrated, he would get the colours better. This sort of trial and error progressed from 16 months right through to 20 months plus.
I’ve read that children can start labeling colours at around 3 years of age, so there might be something to waiting. We never pressed our son hard with colours or made a big deal when he guessed incorrectly. I think you should be careful when you push a child past an age appropriate expectation. If he guessed wrong, we just tell him the real colour. Also, when teaching colours, you never know if your baby is colour blind or can’t see shades well in a given colour spectrum. Because of this, you don’t want to make colour labeling a “high stakes” game.