Corona Renderer Forum

General Category => General CG discussion => Topic started by: maru on 2018-01-31, 14:35:06

Title: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-01-31, 14:35:06
Here is a couple of tests I made of the https://letsenhance.io/ tool. It basically lets you upload an image and get a 4x upscaled version after a few minutes of processing time, all thanks to the magic of neural networks.
Currently you can upload just a few images for free (3? 5?), and then the paid version kicks in. You can choose between the "boring" or "magic" upscaling presets. The "magic" one is where the real magic begins. Suddenly textures are magically recovered from the areas which are otherwise blurry (even in the original image), and more details are visible. You basically get an image generated in 4x higher resolution, or using a camera with 4x more mpx.

I think this is really interesting for upscaling photos and renders for print, especially renders, where RAM may be an issue at some point. You can get a pretty much lossless version of your image in 4x the original resolution, which can be then used for printing or display in super high res.


"ori" is a crop of the original image upscaled by 400% in Photoshop using the "automatic" supersampling option
"boring" is the result of using the "boring" preset, then cropped to show the same region
"magic" is the result of using the "magic" preset, then cropped to show the same region

Discuss. I am also curious about your tests!
Note: this tool is not related to Corona in any way, that's why I am writing in the "General CG" category.

I suggest middle mouse button clicking on the thumbnails below to open the examples in tabs one by one.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: cecofuli on 2018-01-31, 14:49:18
I'm in shock, really! O_O
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Nejc Kilar on 2018-01-31, 15:05:44
(http://images.fanpop.com/images/polls/51651_1202775788548_full.jpg)

So CSI was doing it right all along, we just didn't know about the tech at the time. Go Horatio!

Thanks for sharing, very interesting stuff!

Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-01-31, 15:16:43
With the right combination of algorithms...

Title: Re: letsenhance.io - CSI-style image upscaling
Post by: romullus on 2018-01-31, 15:46:20
Is this paid persion, that you tested with? I'm asking because i tested that exact service few weeks ago (free version) and my results was disapointing - it created many double lines and "enhanced" image with creative "details" wherever there was noise in original picture. Also in my case there was barely observable difference between "magic" and "boring" methods, but in your examples, the difference is very obvious.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-01-31, 15:55:57
Is this paid persion, that you tested with? I'm asking because i tested that exact service few weeks ago (free version) and my results was disapointing - it created many double lines and "enhanced" image with creative "details" wherever there was noise in original picture. Also in my case there was barely observable difference between "magic" and "boring" methods, but in your examples, the difference is very obvious.
I am using the free version. I don't think there is any difference in terms of quality between free and paid. Paying just gives you more slots to upload images.
They upgraded to version 2.0 today (I think), so maybe that's the difference. I also did not test with noisy images. But I also tried this much earlier and had similar results, so maybe it just depends on the image.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: pokoy on 2018-01-31, 17:25:11
Many thanks for posting this, I'll certainly need that at some point.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: romullus on 2018-01-31, 19:15:08
Let's see if i still have free slots available, maybe they indeed changed something to the better :]
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-01-31, 19:17:35
Let's see if i still have free slots available, maybe they indeed changed something to the better :]
They should give you free slots after this update.
If not, then you can contact them and I am sure they will give you something, they are pretty responsive.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Charlie Nicols on 2018-01-31, 19:43:01
My results are very mixed! Cool spot to keep an eye on though. Thanks.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: romullus on 2018-01-31, 19:57:48
I think mixed is the right word. Some parts of an image after their magic enhance looks definitely better than photoshop's bilinear upscale and some parts looks actually worse. But look what's happens when you downscale both images (bilinear and "magic") back to original. Ewww!

Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-02-01, 09:54:08
I think mixed is the right word. Some parts of an image after their magic enhance looks definitely better than photoshop's bilinear upscale and some parts looks actually worse. But look what's happens when you downscale both images (bilinear and "magic") back to original. Ewww!
I guess it's the effect of their "hallucinations" in action. I wonder what would happen if this process was repeated like 100 times. :)
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: romullus on 2018-02-01, 10:55:09
It could be that selfie of AI slowly would emerge out of your picture o_O
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-02-01, 13:11:57
It could be that selfie of AI slowly would emerge out of your picture o_O
or something like "help me, they keep me here and force me to photoshop images 24/7"
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: romullus on 2018-02-01, 15:23:46
Yeah, if only we could have such enhacing program, like they show in CSI soap operas, i bet we could read that hidden message...
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Eddoron on 2018-02-07, 14:01:23
Good find! it's really cool.

The missing features are being "hallucinated".
This thing should be trained for normal maps etc. and we don't have to bake large things anymore.
Not perfect but worked quite well in some areas of my test image.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Noah45 on 2018-02-07, 18:01:46
Often the SU models, I receive, have poor mapping images. However, it is the 'look' they want. Attached is a typical map- notice how well this magic process worked.

Daily Retail workflow.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: sebastian___ on 2018-02-07, 21:59:34
It works especially well with high quality content. Or with down-resized photos.
Works less well with noisy, grainy, low detail pics.

Original = the original resolution image
then I took this and resized it down 400% (so I can have a well defined low grain image) and I up-scaled it back up 400% with the website and with photoshop.
The website algorithm seems to be able to "make up" or invent new details, including grain and surface detail so it closely match the original image. And it's better than the new "preserve detail 2.0" photoshop algorithm. And much better compared to some sharp bicubic or lanc algorithm.

Some parts of the up-scaled image look almost the same, or even better than the original, even though it's missing like 75% or something of original image content.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-09-03, 16:18:27
Found another one if you're interested:
https://topazlabs.com/downloads#gigapixel
It offers 30-day trial with no image limit (?). From my initial tests it seems it is slightly worse than letsenhance, but offers more options (e.g. 600% upscaling).
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: sebastian___ on 2018-09-03, 19:44:30
It would be interesting if you could apply this to an image sequence (a movie).
I sent a message to the letsenhance people about it but I got no reply. It would be expensive anyway. But probably programs like these who invent new details for resizing would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: TomG on 2018-09-03, 19:47:25
would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

But could still be trippy fun! ;)
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Juraj Talcik on 2018-09-03, 22:14:40
It would be interesting if you could apply this to an image sequence (a movie).
I sent a message to the letsenhance people about it but I got no reply. It would be expensive anyway. But probably programs like these who invent new details for resizing would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

This is possible and some other services even specialize it in. Most of them are in-house and the only public one is not available for even beta test yet, it's part of Artomatix suite. ( I am beta tester but currently the focus is for texture synthetization ).
But they did showcase upscaling to 4k using this tech, it was temporally well stabilized and you couldn't tell. But the detail enhanced wasn't that great compared to others I have seen, not sure why.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: sebastian___ on 2018-09-04, 17:48:09
They are keeping this in-house or closed for the public or web based only, because they think it's something revolutionary and people will pay a lot of cash for each picture resized. Well it is something revolutionary, but I think soon the technology will leak to the public in forms of After Effects plugins and so on, like it always does eventually.

 "AI" resizing for everybody ! :)

Perhaps with only 200% resize, the artefacts will be almost invisible.
If anyone makes a video test up-size with this, or the topazlabs thing, please post here.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-09-05, 11:53:17
By the way, does some kind of AI upscaling in the Corona VFB sound interesting to you?
The idea would be: render something, press a button, wait, you get double the output size.
I guess it would be interesting for prints and fast previews that turn out good enough.

It's not something the devs are working on, just a wild fantasy, but I am curious how many people would be interested in this. :)
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: PROH on 2018-09-05, 11:56:47
I think this could be a great feature to have in the CIE! So + 1 for that :)
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Juraj Talcik on 2018-09-05, 13:06:50
By the way, does some kind of AI upscaling in the Corona VFB sound interesting to you?
The idea would be: render something, press a button, wait, you get double the output size.
I guess it would be interesting for prints and fast previews that turn out good enough.

It's not something the devs are working on, just a wild fantasy, but I am curious how many people would be interested in this. :)

In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: maru on 2018-09-05, 13:26:37
In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.
So basically you are IRing a lower-res image, which is then upscaled in almost real time?
Title: Re: letsenhance.io - CSI-style image upscaling
Post by: Juraj Talcik on 2018-09-05, 13:41:29
In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.
So basically you are IRing a lower-res image, which is then upscaled in almost real time?

Yeah :- ). In fact I could imagine this being the best order of operation:

1) Sample at half the resolution Resolution for best quality, or 1/4 for best speed.
2) Up-sample in real-time
3) Denoise the upsampled result.

Not that is to be seen if this gives better quality than sampling at true res and then denoising, but imho it could give a lot more information to the denoiser.
This is also how nVidia advises for it to be used.

Title: Re: letsenhance.io - CSI-style image upscaling
Post by: sebastian___ on 2018-09-05, 14:32:47
If the upscale would be even 70% as good as letsenhance.... And then rendering and tweaking in IR with maybe a very small 480 x 360 pixels would be superfast, and then to have that upscaled on the fly with the GPU to maybe 960 x 720 or more with minimal loss ? That sounds awesome.