Well, it could be. I'm not convinced it should make that much of a different. The faces in CelebA differ tremendously in hair color, skin color, lighting, glasses, jewelry etc, but that doesn't stop people from generating highly realistic samples. I also don't see why it would make the watercolor problem go away. When I was trying anime faces with WGAN, I prepared a face-only dataset much like OP did, with girls sourced from Danbooru, and did some cleanup by deleting the faces rated least-face-like by the discriminator, but it didn't get me any noticeable quality improvement. I suspect it is probably the labels and SRResNet (conditional GANs seem to usually improve a great deal over unsupervised ones, and residuals are better training & higher model capacity), but I can only speculate; this project would be a lot more informative if they can figure out where the quality improvements are coming from, exactly.