Author Topic: letsenhance.io - CSI-style image upscaling  (Read 4144 times)

2018-01-31, 14:35:06

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
Here is a couple of tests I made of the https://letsenhance.io/ tool. It basically lets you upload an image and get a 4x upscaled version after a few minutes of processing time, all thanks to the magic of neural networks.
Currently you can upload just a few images for free (3? 5?), and then the paid version kicks in. You can choose between the "boring" or "magic" upscaling presets. The "magic" one is where the real magic begins. Suddenly textures are magically recovered from the areas which are otherwise blurry (even in the original image), and more details are visible. You basically get an image generated in 4x higher resolution, or using a camera with 4x more mpx.

I think this is really interesting for upscaling photos and renders for print, especially renders, where RAM may be an issue at some point. You can get a pretty much lossless version of your image in 4x the original resolution, which can be then used for printing or display in super high res.


"ori" is a crop of the original image upscaled by 400% in Photoshop using the "automatic" supersampling option
"boring" is the result of using the "boring" preset, then cropped to show the same region
"magic" is the result of using the "magic" preset, then cropped to show the same region

Discuss. I am also curious about your tests!
Note: this tool is not related to Corona in any way, that's why I am writing in the "General CG" category.

I suggest middle mouse button clicking on the thumbnails below to open the examples in tabs one by one.

2018-01-31, 14:49:18
Reply #1

cecofuli

  • Active Users
  • **
  • Posts: 1500
    • View Profile
I'm in shock, really! O_O

2018-01-31, 15:05:44
Reply #2

Nejc Kilar

  • Active Users
  • **
  • Posts: 890
    • View Profile
    • My personal website


So CSI was doing it right all along, we just didn't know about the tech at the time. Go Horatio!

Thanks for sharing, very interesting stuff!


2018-01-31, 15:16:43
Reply #3

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile

2018-01-31, 15:46:20
Reply #4

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 6371
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Is this paid persion, that you tested with? I'm asking because i tested that exact service few weeks ago (free version) and my results was disapointing - it created many double lines and "enhanced" image with creative "details" wherever there was noise in original picture. Also in my case there was barely observable difference between "magic" and "boring" methods, but in your examples, the difference is very obvious.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2018-01-31, 15:55:57
Reply #5

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
Is this paid persion, that you tested with? I'm asking because i tested that exact service few weeks ago (free version) and my results was disapointing - it created many double lines and "enhanced" image with creative "details" wherever there was noise in original picture. Also in my case there was barely observable difference between "magic" and "boring" methods, but in your examples, the difference is very obvious.
I am using the free version. I don't think there is any difference in terms of quality between free and paid. Paying just gives you more slots to upload images.
They upgraded to version 2.0 today (I think), so maybe that's the difference. I also did not test with noisy images. But I also tried this much earlier and had similar results, so maybe it just depends on the image.

2018-01-31, 17:25:11
Reply #6

pokoy

  • Active Users
  • **
  • Posts: 1489
    • View Profile
Many thanks for posting this, I'll certainly need that at some point.

2018-01-31, 19:15:08
Reply #7

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 6371
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Let's see if i still have free slots available, maybe they indeed changed something to the better :]
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2018-01-31, 19:17:35
Reply #8

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
Let's see if i still have free slots available, maybe they indeed changed something to the better :]
They should give you free slots after this update.
If not, then you can contact them and I am sure they will give you something, they are pretty responsive.

2018-01-31, 19:43:01
Reply #9

Charlie Nicols

  • Active Users
  • **
  • Posts: 83
    • View Profile
My results are very mixed! Cool spot to keep an eye on though. Thanks.

2018-01-31, 19:57:48
Reply #10

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 6371
  • Let's move this topic, shall we?
    • View Profile
    • My Models
I think mixed is the right word. Some parts of an image after their magic enhance looks definitely better than photoshop's bilinear upscale and some parts looks actually worse. But look what's happens when you downscale both images (bilinear and "magic") back to original. Ewww!

I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2018-02-01, 09:54:08
Reply #11

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
I think mixed is the right word. Some parts of an image after their magic enhance looks definitely better than photoshop's bilinear upscale and some parts looks actually worse. But look what's happens when you downscale both images (bilinear and "magic") back to original. Ewww!
I guess it's the effect of their "hallucinations" in action. I wonder what would happen if this process was repeated like 100 times. :)

2018-02-01, 10:55:09
Reply #12

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 6371
  • Let's move this topic, shall we?
    • View Profile
    • My Models
It could be that selfie of AI slowly would emerge out of your picture o_O
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2018-02-01, 13:11:57
Reply #13

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
It could be that selfie of AI slowly would emerge out of your picture o_O
or something like "help me, they keep me here and force me to photoshop images 24/7"

2018-02-01, 15:23:46
Reply #14

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 6371
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Yeah, if only we could have such enhacing program, like they show in CSI soap operas, i bet we could read that hidden message...
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2018-02-07, 14:01:23
Reply #15

Eddoron

  • Active Users
  • **
  • Posts: 552
  • Achieved Pedestrian
    • View Profile
Good find! it's really cool.

The missing features are being "hallucinated".
This thing should be trained for normal maps etc. and we don't have to bake large things anymore.
Not perfect but worked quite well in some areas of my test image.

2018-02-07, 18:01:46
Reply #16

Noah45

  • Active Users
  • **
  • Posts: 429
  • press *1
    • View Profile
Often the SU models, I receive, have poor mapping images. However, it is the 'look' they want. Attached is a typical map- notice how well this magic process worked.

Daily Retail workflow.
« Last Edit: 2018-02-08, 01:26:36 by Noah45 »
Retail Illustrator  (for ever' 80's )
3DMax 2020/Corona Version: 6DB

2018-02-07, 21:59:34
Reply #17

sebastian___

  • Active Users
  • **
  • Posts: 150
    • View Profile
It works especially well with high quality content. Or with down-resized photos.
Works less well with noisy, grainy, low detail pics.

Original = the original resolution image
then I took this and resized it down 400% (so I can have a well defined low grain image) and I up-scaled it back up 400% with the website and with photoshop.
The website algorithm seems to be able to "make up" or invent new details, including grain and surface detail so it closely match the original image. And it's better than the new "preserve detail 2.0" photoshop algorithm. And much better compared to some sharp bicubic or lanc algorithm.

Some parts of the up-scaled image look almost the same, or even better than the original, even though it's missing like 75% or something of original image content.
« Last Edit: 2018-02-07, 22:18:53 by sebastian___ »

2018-09-03, 16:18:27
Reply #18

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
Found another one if you're interested:
https://topazlabs.com/downloads#gigapixel
It offers 30-day trial with no image limit (?). From my initial tests it seems it is slightly worse than letsenhance, but offers more options (e.g. 600% upscaling).

2018-09-03, 19:44:30
Reply #19

sebastian___

  • Active Users
  • **
  • Posts: 150
    • View Profile
It would be interesting if you could apply this to an image sequence (a movie).
I sent a message to the letsenhance people about it but I got no reply. It would be expensive anyway. But probably programs like these who invent new details for resizing would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

2018-09-03, 19:47:25
Reply #20

TomG

  • Corona Team
  • Active Users
  • ****
  • Posts: 2960
    • View Profile
would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

But could still be trippy fun! ;)

2018-09-03, 22:14:40
Reply #21

Juraj Talcik

  • Active Users
  • **
  • Posts: 3977
  • Tinkering away
    • View Profile
    • studio website
It would be interesting if you could apply this to an image sequence (a movie).
I sent a message to the letsenhance people about it but I got no reply. It would be expensive anyway. But probably programs like these who invent new details for resizing would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

This is possible and some other services even specialize it in. Most of them are in-house and the only public one is not available for even beta test yet, it's part of Artomatix suite. ( I am beta tester but currently the focus is for texture synthetization ).
But they did showcase upscaling to 4k using this tech, it was temporally well stabilized and you couldn't tell. But the detail enhanced wasn't that great compared to others I have seen, not sure why.
talcikdemovicova.com  Website and blog
be.net/jurajtalcik   Our studio Behance portfolio
Instagram   Our studio Instagram, managed by Veronika

2018-09-04, 17:48:09
Reply #22

sebastian___

  • Active Users
  • **
  • Posts: 150
    • View Profile
They are keeping this in-house or closed for the public or web based only, because they think it's something revolutionary and people will pay a lot of cash for each picture resized. Well it is something revolutionary, but I think soon the technology will leak to the public in forms of After Effects plugins and so on, like it always does eventually.

 "AI" resizing for everybody ! :)

Perhaps with only 200% resize, the artefacts will be almost invisible.
If anyone makes a video test up-size with this, or the topazlabs thing, please post here.
« Last Edit: 2018-09-04, 22:04:32 by sebastian___ »

2018-09-05, 11:53:17
Reply #23

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
By the way, does some kind of AI upscaling in the Corona VFB sound interesting to you?
The idea would be: render something, press a button, wait, you get double the output size.
I guess it would be interesting for prints and fast previews that turn out good enough.

It's not something the devs are working on, just a wild fantasy, but I am curious how many people would be interested in this. :)

2018-09-05, 11:56:47
Reply #24

PROH

  • Active Users
  • **
  • Posts: 1066
    • View Profile
I think this could be a great feature to have in the CIE! So + 1 for that :)

2018-09-05, 13:06:50
Reply #25

Juraj Talcik

  • Active Users
  • **
  • Posts: 3977
  • Tinkering away
    • View Profile
    • studio website
By the way, does some kind of AI upscaling in the Corona VFB sound interesting to you?
The idea would be: render something, press a button, wait, you get double the output size.
I guess it would be interesting for prints and fast previews that turn out good enough.

It's not something the devs are working on, just a wild fantasy, but I am curious how many people would be interested in this. :)

In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.
talcikdemovicova.com  Website and blog
be.net/jurajtalcik   Our studio Behance portfolio
Instagram   Our studio Instagram, managed by Veronika

2018-09-05, 13:26:37
Reply #26

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9554
  • Marcin
    • View Profile
In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.
So basically you are IRing a lower-res image, which is then upscaled in almost real time?

2018-09-05, 13:41:29
Reply #27

Juraj Talcik

  • Active Users
  • **
  • Posts: 3977
  • Tinkering away
    • View Profile
    • studio website
In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.
So basically you are IRing a lower-res image, which is then upscaled in almost real time?

Yeah :- ). In fact I could imagine this being the best order of operation:

1) Sample at half the resolution Resolution for best quality, or 1/4 for best speed.
2) Up-sample in real-time
3) Denoise the upsampled result.

Not that is to be seen if this gives better quality than sampling at true res and then denoising, but imho it could give a lot more information to the denoiser.
This is also how nVidia advises for it to be used.

talcikdemovicova.com  Website and blog
be.net/jurajtalcik   Our studio Behance portfolio
Instagram   Our studio Instagram, managed by Veronika

2018-09-05, 14:32:47
Reply #28

sebastian___

  • Active Users
  • **
  • Posts: 150
    • View Profile
If the upscale would be even 70% as good as letsenhance.... And then rendering and tweaking in IR with maybe a very small 480 x 360 pixels would be superfast, and then to have that upscaled on the fly with the GPU to maybe 960 x 720 or more with minimal loss ? That sounds awesome.