Author Topic: AI super-resolution speedup  (Read 1409 times)

2019-04-12, 23:18:38

lolec

  • Active Users
  • **
  • Posts: 81
    • View Profile
Nvidia denoiser has made a huge impact in our workflow, allowing us to vizualize a close aproximation of the final render much faster.

With the latest set of supersampling algorithms, I think there is another oportunity to bring even more speed into the process.

My idea is to have a checkbox called SuperSampling that would essentially devide the render size by half and use AI to upscale to the expected resolution.

I'm not sure, but I believe supersampling is much faster than actually rendering 4x the pixels.

This won't be intended for final renders. But it could be awesome to speed up work.

I've done a few tests with https://topazlabs.com/gigapixel-ai/ , but having it integrated in corona would be amazing. What do you think ?

 

2019-04-13, 08:36:20
Reply #1

sebastian___

  • Active Users
  • **
  • Posts: 135
    • View Profile
Can you post your tests with Topaz AI ?  It would be cool if someone would make a video test with the AI resize.

2019-04-13, 10:25:30
Reply #2

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 5796
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Question is, would AI upscaler benefit from 3D data that renderer can provide? If not, then i think it's better to leave it for post.  Personally, i don't think that i would use such feature much, if at all, but then again, i thought the same about denoiser, before Corona had any, but now i barely can do a render without it.
I'm not Corona Team member. Everything i say, is my personal opinion only.

2019-04-13, 12:10:27
Reply #3

Juraj Talcik

  • Active Users
  • **
  • Posts: 3629
  • Tinkering away
    • View Profile
    • studio website
Question is, would AI upscaler benefit from 3D data that renderer can provide? If not, then i think it's better to leave it for post.  Personally, i don't think that i would use such feature much, if at all, but then again, i thought the same about denoiser, before Corona had any, but now i barely can do a render without it.

I believe the AI would benefit since 3D data can provide information where the edges are. So everything would be bit less smooth than post-production tool provides.
Although, judging the results from nVidia DLSS which sort of does this (although primarily as means for smooth animation AA) the results is kinda smooth.

I believe it will be necessary eventually anyway, rendering 2k animation is quite expensive and timey hassle in Corona (or offline renderer of any sort for the matter) yet we already see amazingly sharp animation in 4k from Unreal rendered basically instantly with the new cards. Offline rendering is still easier and much better quality but at some point, if you can get 8k Unreal animation within single day for nothing with single GPU, or wait two weeks for 2k Corona animation, the decision will be a lot different.

I would never personally use these hacks for still images where every loss of quality is noticeable (I barely use denoising on finals these days), but animation is concerning me.
talcikdemovicova.com  Website and blog
be.net/jurajtalcik   Our studio Behance portfolio
Instagram   Our studio Instagram, managed by Veronika

2019-04-13, 15:59:19
Reply #4

lolec

  • Active Users
  • **
  • Posts: 81
    • View Profile
I will post some images later.

I didn't imagine to be meant for final images, but for drafting... however, it would be great for final animations. In my tests, upscaling 1920 to 4k is kind of the best scenario, it's hard to see the difference. 

Upscaling 960 to 1920 does produce some artifacts, but I still think it would be useful in the same way Nvidia Denoiser is. 

2019-04-13, 20:44:51
Reply #5

sprayer

  • Active Users
  • **
  • Posts: 628
    • View Profile
why you want  this in 3ds max as it's post processing after all, and there is many upscaling algorytms in presents what you may choose without implementing it in 3ds max. I can't imagine why you need to save images with upscaling artifacts right after rendering, and how you will fix them if you rendering and save animation sequence? Again this should be postprocessing work from raw renders images

2019-04-13, 22:34:14
Reply #6

sebastian___

  • Active Users
  • **
  • Posts: 135
    • View Profile
why you want  this in 3ds max as it's post processing after all

This would suppose to be magical "AI" upscaling, and it will result in a much faster rendering while looking almost the same, so it will help you in lookdev and also in time consuming animation rendering.

Kind of like Nvidia denoising, which already does something similar, instantly showing you the results. You could have the same argument there too: Why not denoise in post-processing ? instead of instant, during rendering.

2019-04-14, 02:17:36
Reply #7

lolec

  • Active Users
  • **
  • Posts: 81
    • View Profile
AI upscaling is much better than pretty much any other upscaling algorithm out there. Opening the possibility of a feature where you would actually render at a much lower resolution and 4x faster (or even more), while looking pretty much the same as a higher resolution render.

Again, I'm not saying this feature would be useful for final renders. As even the animation scenario described above would be covered by current plugins, as you only need to set up and run once.

Maybe you work in a different way, but when I'm working on a new scene, I don't need to see 100% of the details all the time. If I can get a 95% approximation that allows me to place and adjust lights, adjust materials etc... it can speed up my workflow significantly, the same way Nvidia Denoiser did.


2019-04-14, 02:46:36
Reply #8

PROH

  • Active Users
  • **
  • Posts: 1005
    • View Profile
Hi. In this video (at 14 min) they show an example of Gigapixel used on a render (upscaled 200%):

:)


2019-04-14, 11:28:43
Reply #10

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 5796
  • Let's move this topic, shall we?
    • View Profile
    • My Models
I will post some images later.

Please do. Unless i'll see the magic with my own eyes, i have a hard time to believe those claims "nearly indistinguishable from originally high-rez". I have very little experience with AI upscalling - only used some online service demo and ocassionally waifu2x - but never managed to see astonishing results even at basic 2x upscalling.

Maybe you work in a different way, but when I'm working on a new scene, I don't need to see 100% of the details all the time. If I can get a 95% approximation that allows me to place and adjust lights, adjust materials etc... it can speed up my workflow significantly, the same way Nvidia Denoiser did.

If you're refering to IR, then there are few questions to be answered. Would IR still be interactive with AI denoiser + AI upscaler? Also you mentioned, that upscaler works best at already pretty high resolution input. Would IR benefit much from upscaler if it does not give optimal results at lower resolution?

I'm not against this request, but i hope that the team won't jump on that AI bandwagon in expense to more conventional features, which are waited by community with great anticipation for a long time.
I'm not Corona Team member. Everything i say, is my personal opinion only.

2019-04-14, 12:25:23
Reply #11

Juraj Talcik

  • Active Users
  • **
  • Posts: 3629
  • Tinkering away
    • View Profile
    • studio website
I wouldn't worry about devs jumping on bandwagon ;- ). It took half a year (or year?) to get nVidia denoiser which is amazing and less than day to get Intel one which is pure shit and no one asked for it. It's not done on basis of request intensity.

This tech doesn't look near close to ready for real-time implementation anyway, but let's watch how fast and good something like DLSS will get and it will be an absolute must eventually. 8k TVs are here, 8k monitors around the corner.
talcikdemovicova.com  Website and blog
be.net/jurajtalcik   Our studio Behance portfolio
Instagram   Our studio Instagram, managed by Veronika

2019-04-14, 16:40:55
Reply #12

sebastian___

  • Active Users
  • **
  • Posts: 135
    • View Profile
i have a hard time to believe those claims "nearly indistinguishable from originally high-rez".

Usually the results could have big differences in quality depending on the source image. Some said it can depend on what kind of  images the resize algorithm was trained with.

that upscaler works best at already pretty high resolution input. Would IR benefit much from upscaler if it does not give optimal results at lower resolution?

Also usually works best not from high resolution images, low resolution could work just as well, but for good results that low resolution image should be at a "final" quality and noise free, which would maybe conflict with the idea of drafting or lookdev usage.

2019-04-14, 23:02:09
Reply #13

burnin

  • Active Users
  • **
  • Posts: 854
    • View Profile
Been using Enhanced AI in my (RnD) pipeline for some time now... quite a good time saver.
BTW, bad quality comes from not having enough of proper training, lack of "learning" material to cover all cases. Which is most obvious with OIDN & was also observed with NVidia Ai. But better times are coming, machines are learning & we're just at the beginning... soon even standard scenes will be designed, visualized, produced, delivered & build by machines... still under human control ;)

Tho let's not get ahead of ourselves... we've already seen idiots on speed doing more damage than good. Speed kills!
« Last Edit: 2019-04-14, 23:15:56 by burnin »

2019-04-15, 05:35:57
Reply #14

SairesArt

  • Active Users
  • **
  • Posts: 678
  • Pizza | The Cheesen One
    • View Profile
    • SairesArt Portfolio
[rant]
Ever since a thread linked a product called Evotis (now their website is not public facing anymore, but there are press reports and the wayback machine), which saved samples instead of pixels to create subpixel perfect masks, I am very hopeful for a resolution independent renderer.
Instead of losing a sample's information in a pixel through the reconstruction filter, it would be interesting to save all the samples to disk. Then after rendering you could set the resolution after the fact.

Kinda like RAW allows you to set white balance, because debayering has not been done yet - saving samples would allow you to set a resolution after rendering finished, because the samples were not collapsed into pixels yet. Be it 480p or 4k. Ignoring pixel-grid alignment, 16 passes at 1080p would equate to 4 passes at 4k with no loss in sharpness, allowing you to switch back and forth.
In the world of offline rendering this would be way more useful than upscaling.
Hope to code up a prototype of this sometime this year.
[/rant]