Most likely when he's actually concentrating on something that he likes to do a lot.

Or, hell, asleep and naked and clinging to your arm in your bed which he has mysteriously appeared in yet again.

I just know it's definitely not when he actively tries to be. Then, he's just sort of silly and cute, maybe kind of sweet, but definitely not sexy in the classical way of thinking about it.