×
all 18 comments

[–]zzzthelastuserStudent 23 points24 points  (0 children)

Just Wow!

I can totally see this revolutionize the manga and comic industry in the future.

[–]gwern[🍰] 11 points12 points  (4 children)

I still can't believe how good Style2Paints has gotten. The first iteration, the 'watercolor' look made it obvious it was machine-generated, but these results, especially the final stage with illumination/shading, often look human or better. And the video is even more impressive - how is it even possible to have control over lighting direction etc? I'm going to have to read the paper, aren't I...

Another random thought: once the pretrained models & code are up to run locally, this could be used for data-augmentation of anime datasets, similar to how some people do regular 'style transfer' for image classification CNNs, as the classification/category doesn't (usually) change when you recolor or style transfer a photo.

EDIT: one thing I notice about the app: you waste a lot of time in the video expanding the default selection. Why not just use the whole uploaded image by default and let the user crop if necessary? It'd also be faster if the various styles were rendered in the background, or at least perhaps the most popular few styles, and shown side by side so the user can easily pick out the best rendering; but perhaps that's too expensive for the free web app. (That should be doable with a local app... In the long run, you could train an esthetics CNN to find the best set of settings - Google has shown this is doable with photos and videos. Just take some sketches, render every variant, collect pairwise comparisons, and regress.)

[–]paintstransfer[S] 5 points6 points  (3 children)

Oh thank you for these suggestions. I also think it is better to use the whole uploaded image by default. Maybe I will make some minor change to the UI. It is also a good idea to pre-render some other styles. Maybe we should add this feature in local version? emmmm. Anyway, thank you very much for your suggestions and your previous attention to style2paints.

[–]gwern[🍰] 2 points3 points  (2 children)

Another issue on the app: it doesn't handle large screen resolutions right. I run most things fullscreen on my 3840x2160px screen but then all the toolboxes and the image blow up to the point of overlapping and filling most of the screen, rendering it unusable. If I shrink the browser window to about a third of the screen, then it looks fine (just much smaller). Screenshots: https://imgur.com/a/hss5CPu

[–]paintstransfer[S] 7 points8 points  (1 child)

God. u have a really big screen lol. Thank you for reporting this! we will consider the responsive UI later.

[–]gwern[🍰] 3 points4 points  (0 children)

(It's really good for reading papers and stuff, and you can rotate it if you're playing a game or watching a movie.)

[–]paintstransfer[S] 17 points18 points  (0 children)

Hi! Authors here. About paper:

The style2Paints V3 (the previous version of style2paints) is accepted to SIGGRAPH 2018 ASIA TOKYO! You can meet our team at 2018.12.07 afternoon in Tokyo! Style2paints V3 paper is titled as “Two-stage Sketch Colorization”. In this paper we discussed how we can get rid of artifacts during automatic sketch colorization.

https://github.com/lllyasviel/style2paints/blob/master/papers/sa.pdf

[–]Reiinakano 7 points8 points  (0 children)

wtf that's awesome

[–][deleted] 6 points7 points  (0 children)

Remarkably intelligent colorization,- great work!

[–]medomomis 4 points5 points  (0 children)

awesome!

[–][deleted] 1 point2 points  (0 children)

Just saw this on r/anime. Looks incredibly good. It's probably better than most humans.

And as a noob I'm reminded that I should try breaking difficult style transfer tasks down into simpler tasks.

[–]kaido_the_dragon 1 point2 points  (1 child)

Edit:

Tested a nsfw example of Tsugumomo. You can see the results in /r/AnimeResearch

[–]paintstransfer[S] 8 points9 points  (0 children)

Thank you for test. But we will be happy if you can move these NSFW results from this official release post to somewhere else, e.g. /r/animeresearch . We will appreciate your help.

[–]Naoshikuu 0 points1 point  (0 children)

This is just amazing

[–]evan_brown 0 points1 point  (0 children)

Just want to say AMAZING!!

[–]PaleInsect 0 points1 point  (0 children)

Where can I download the program? It says "Under Construction" :(