Author Topic: Corona saving layered EXR on render end  (Read 22460 times)

2019-08-02, 04:27:07

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Hi,

We have an issue where in some scenes, and apparently for no reason, Corona starts to save out HUGE layered exr files rather than normal single image files.

The scenes haven't really changed, a couple of geometry changes, but all render settings are the same.  We are saving as half float exr.

Does Corona even support saving as multilayered exr?  I have never seen an option for it.


This is a massive issue for us, it takes 30 mins to save out the files and because it is doing this for every render element we can easily chew through a couple of hundred Gb of space overnight.

We are using Backburner, but we have the exact same issue if we manually save straight from the VFB


Cheers

2019-08-02, 10:29:30
Reply #1

jms.lwly

  • Active Users
  • **
  • Posts: 71
    • View Profile
    • jms.lwly studio
I’ve had a similar (unexplained) issue recently - rendered on two machines via backburner, one saved out multilayered single EXRs, the other saved each channel as a separate file.

Literally the same render job from Backburner, but this happened consistently over several different camera passes.

2019-08-02, 10:53:54
Reply #2

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
the other saved each channel as a separate file.

The other saved additionally each channel, right? The main output file still contained all other passes? It's a very strange behaviour. Sometimes I had the feeling that a node from which the job was submitted writes the (layererd) EXRs correctly (and only this) when included in the BB job. but I'm not sure. That mess in the output directories is really annoying :-|


Good Luck




Never underestimate the power of a well placed level one spell.

2019-08-02, 14:52:29
Reply #3

alexyork

  • Active Users
  • **
  • Posts: 521
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Same here... happened once or twice. Unfortunately no time to diagnose.
Alex York
Partner
RECENT SPACES
recentspaces.com

2019-08-02, 15:16:12
Reply #4

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
no time to diagnose.

Don't waste your time, because I fear you will find nothing. This is -again- a "since ever" issue with max :(

Does Corona even support saving as multilayered exr?  I have never seen an option for it.

If you use EXR as output and choose the "Setup" button you can activate "Automatically Add/Remove Render Elements from Render Dialog". This would put all your render elements into one EXR (Max feature, not a Corona feature). In pratice it happens often that additionally the render elements also get saved as extra files. And CXR (which is basically multilayered EXR + additional Corona specific information) has it all inclusive anyway :)


Good Luck



Never underestimate the power of a well placed level one spell.

2019-08-02, 16:11:37
Reply #5

alexyork

  • Active Users
  • **
  • Posts: 521
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Are you guys using Deadline always? Maybe related?
Alex York
Partner
RECENT SPACES
recentspaces.com

2019-08-12, 06:11:56
Reply #6

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Cheers for all the info.  Never realised there was a native layered option within the 3ds max setup dialogue. It makes me surer that this is probably just a 3ds max issue, no wonder VRay had their own save options from years ago..

We are only using Backburner here.

We constantly get this issue though (see attached), where the files aren't even layered but are all huge and identical sizes except the beauty.  I can't split them with exr-io or in Fusion/AE  and it gobbles up server space on a crazy level.  Only seems to do it on large renders, 5k and above

2019-08-12, 09:07:31
Reply #7

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
Oh, this looks like you are not using any compression. Go to "setup" while you are defining the output and use zip compression or wavelet (yes, the latter is also lossless, no worries).


Good Luck



Never underestimate the power of a well placed level one spell.

2019-08-14, 12:37:24
Reply #8

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9443
  • Marcin
    • View Profile
Is the below correct?

- You would like to save only a single EXR file with all render elements included in it
- Right now, instead of a single file, you are getting the "main" EXR + all other render elements saved separately as EXR files

---

One extra note:
If you use the "Save CXR" option and then convert your CXR to EXR, you will end up with a large file containing some layers which are normally hidden (for example information used to perform denoising). Maybe you are getting this kind of files? Could you send us some samples? The uploader is in my signature.

2019-08-14, 19:04:22
Reply #9

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
Right now, instead of a single file, you are getting the "main" EXR + all other render elements saved separately as EXR files

I guess it's again a Max issue not passing the file type properties (does not depend on output type EXR or CXR, even not on the renderer). We noticed that the node from which the job was submitted does it correctly if included in the BB job. Meaning: it saves only that one multilayered exr in this instance just like intended. All others do whatever. Completely weird, but also there "forever" in Max. Additionally I guess it's about the local settings of a node, just like JPG quality (had this issue recently while scripting).

Just tried it: seems to be as guessed :)

Use attached script as prerender script for a BB job (Render Setup -> Scripts -> Pre-Prender) and look at the max.log of the participating nodes: on nodes which write all additional passes you get 'false'. On nodes which work as intended you get 'true'. The script also sets this value to 'true' so that if you render ONE job with the script as prerender script on all nodes, the issue should be solved because the setting is locally stored on the node. Time to add this as another prerender script network wide.


Good Luck


Edit: This affects EXR only unfortunately, but not CXR.
Edit: Removed the script to avoid data loss. Reason: When using it (permanently as prerender script) and not wanting one layered EXR as output but multiple ones, additional REs are not saved at all. We always do layered EXRs so it does what it should do for us.
« Last Edit: 2019-09-19, 17:02:26 by Frood »
Never underestimate the power of a well placed level one spell.

2019-08-23, 03:34:03
Reply #10

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Oh, this looks like you are not using any compression. Go to "setup" while you are defining the output and use zip compression or wavelet (yes, the latter is also lossless, no worries).


Good Luck

That's how it is set up anyway, with ZIP compression.

Is the below correct?

- You would like to save only a single EXR file with all render elements included in it
- Right now, instead of a single file, you are getting the "main" EXR + all other render elements saved separately as EXR files

---

One extra note:
If you use the "Save CXR" option and then convert your CXR to EXR, you will end up with a large file containing some layers which are normally hidden (for example information used to perform denoising). Maybe you are getting this kind of files? Could you send us some samples? The uploader is in my signature.

No.  I want all my exr's saved out separate (which is happening).  The issue is that the files are bigger than they should be. MUCH bigger. Even if I make a mistake and have an empty render element i.e Volumetrics but I have no volumes in the scene, it will save out a massive file with no info in it.  Check the image above, there is no way that my alpha should be more than double the size of my beauty pass, and it has no extra info in there that I can access.



So after more testing, this isn't just an issue with Backburner...  and we dont get the issue with VRay, Redshift etc.  It seems to happen with only the high res files.  3k, we get the expected different file sizes per pass, jump to 6k or 7k and we get all [asses with the same file size.  This also happens with TIFFs.  And across every machine in the studio.


I genuinely think this is a Corona issue, something about saving large-resolution files seems to make it shit it's pants, the contents of which are filling up our server space

2019-08-23, 09:45:53
Reply #11

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Hi,

Does this only happen when using backburner? Or does it also happen when rendering with Corona on its own?

Secondly, would you be able to provide a scene where this is happening for you? And I will do my best to investigate the issue.

Instructions on how to send the scene to us are in my signiture.

Thanks,

Rowan
Please read this before reporting bugs:
How to report issues to us!

2019-08-25, 13:53:27
Reply #12

valwizard

  • Active Users
  • **
  • Posts: 55
  • "We get up in the morning. We do our best. Nothing
    • View Profile
It is happening when using DR. I have created a topic but perhaps it was deleted in order to not duplicate the matter
"We get up in the morning. We do our best. Nothing else matters! "

2019-09-02, 08:08:36
Reply #13

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Hi,

Does this only happen when using backburner? Or does it also happen when rendering with Corona on its own?

Secondly, would you be able to provide a scene where this is happening for you? And I will do my best to investigate the issue.

Instructions on how to send the scene to us are in my signiture.

Thanks,

Rowan

this happens with both.  The only way to get 'normal sized' render passes is to save as CXR then save out p[asses individually from that.

I'll try and get a file over to you by then end of the week

2019-09-02, 16:30:02
Reply #14

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Thanks, I'll look forward to receiving the scene.
Please read this before reporting bugs:
How to report issues to us!

2019-09-02, 19:19:23
Reply #15

denisgo22

  • Active Users
  • **
  • Posts: 681
    • View Profile
Yes this always max problem/and this always happened with corona render  and whether to use backburner or not, because what file extension will be in Common/render output of 3dmax,
the same will always be in Render Elements:)
But Vray have separate from max frame buffer options, that override max options for saving files , that allow to saving output files in different formats or not saving at all separately render
channels  //, for example channels in jpg and main max output to exr.
which, to a great inconvenience, not exist in Corona render:((
time to pay attention to this in a future release:))
« Last Edit: 2019-09-02, 19:42:59 by denisgo22 »

2019-09-11, 15:39:44
Reply #16

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
That's how it is set up anyway, with ZIP compression.

In your original post, you wrote, the huge(r) files are layered. What makes you think so? I still bet the render elements you've shown in the screenshot are just uncompressed. Max seems to remember the exr settings at creation time of a render element. You can reproduce this doing the following (no DR or BB involved and independently from render size - even works like this using scanline renderer):

  • New scene
  • Set main output to some exr without compression (press "Setup" and make sure it's switched off)
  • Add any render element, e.g. WireColor
  • Set main output to same exr but press "Setup" and enable compression of your choice this time
  • Add any render element, e.g. CMasking_Mask
  • Render 4k or larger (just to better see the differences)


-> WireColor is uncompressed, Cmasking_Mask is compressed.

So what you've shown in the screenshot here could be the result of setting up the render elements on a node/at a time where exr compression was switched off for exr output and then submitting/rendering it on another node with other exr settings; compression active that is.

All in all I think there are at least three topics mixed in this thread meanwhile? To sort it out:

  • Original issue with large render element files (I suspect just uncompressed EXRs, see above)
  • Issue having a layered EXR but also additionally all passes as extra EXR (that's how I misinterpreted it, cause: local settings)
  • The dependency between main output file type and render elements file type(s) as denisgo22 mentioned


As for the latter: It seems that even if you do not explicitly specify a file name/type for a render element (usually a bad idea), the exr settings at creation time get stored and are not adopted from the current main output settings. At least compression settings - have not tested others. Well, that's new to me :)



Good Luck




Never underestimate the power of a well placed level one spell.

2019-09-12, 01:34:09
Reply #17

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
That's how it is set up anyway, with ZIP compression.

In your original post, you wrote, the huge(r) files are layered. What makes you think so?


I opened with EXR.IO plugin in photoshop and each file was layered.  But tbh that looks like it was a very rare glitch, we haven't had the issue many times, and possibly could have been an issue with max.  I should have probably started a new thread about the other issue of exr file sizes all being the same...



So what you've shown in the screenshot here could be the result of setting up the render elements on a node/at a time where exr compression was switched off for exr output and then submitting/rendering it on another node with other exr settings; compression active that is.

All in all I think there are at least three topics mixed in this thread meanwhile? To sort it out:

  • Original issue with large render element files (I suspect just uncompressed EXRs, see above)
  • Issue having a layered EXR but also additionally all passes as extra EXR (that's how I misinterpreted it, cause: local settings)
  • The dependency between main output file type and render elements file type(s) as denisgo22 mentioned


As for the latter: It seems that even if you do not explicitly specify a file name/type for a render element (usually a bad idea), the exr settings at creation time get stored and are not adopted from the current main output settings. At least compression settings - have not tested others. Well, that's new to me :)


I genuinely think this is a Corona issue.  In our studio, we do colour drafts at 3.5k and then final images at 6k or 7k.  The 3.5k renders are always fine and render elements have different file sizes, as you would expect, they are saved out as EXR.  The settings don't change, we change the output size to 6k/7k, render, and then we have the issue that I screenshotted, where each file is large and the exact same size.  We regularly use VRay, Redshift and sometimes FStorm in the studio and never get this issue, and we never had it before Corona 4.  Everything is setup exactly the same I we have done for the past 3-4 years.

I will try some more tests as you have mentioned though.





2019-10-14, 11:08:31
Reply #18

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Hi,

We will investigate this.

Thanks,

Rowan

(Internal ID=409573480)
Please read this before reporting bugs:
How to report issues to us!

2019-10-31, 02:30:16
Reply #19

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
any news on this yet?

We are getting to a point where we have to keep clearing old renders out on a regular basis because they are eating up so much server space.  We have the same problem with TIFF files too.

Our only solution seems to be save as CXR, save out passes manually then delete cxr.

2019-10-31, 09:47:23
Reply #20

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Hi Dan

We are looking into this, but so far we have been unable to reproduce this behaviour yet.

Do you have any further reproduction steps which may help us?

Thanks,

Rowan
Please read this before reporting bugs:
How to report issues to us!

2019-10-31, 13:14:35
Reply #21

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
we have been unable to reproduce this behaviour yet.

What exactly have you tried? Remember that you have to send the scene through Backburner/farm (another node) to at least get the effect of layered EXR+extra EXR for every render element (RE) in some cases.

Here are questions for you:

1. when using CXR as output and having additonal REs defined. Should there be a) only a single, multilayered CRX as output or b) the multilayered CXR plus additional CXRs for every RE? If b), what is it for?

2. when using EXR as output and having additional REs defined AND you have ticked "Automatically Add/Remove Render Elements" in Render Output File -> "Setup". What do you get when a) rendering locally and b) sending it through Backburner? Please use the attached script as prerender script in Render setup -> common -> scripts to log the current state of the EXR output settings on the node(s) into max.log. My findings are: it depends on local settings (that's what the script reports, don't worry it does not change anything). This should never be the case.


Good Luck


Never underestimate the power of a well placed level one spell.

2019-10-31, 14:22:48
Reply #22

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
we have been unable to reproduce this behaviour yet.

What exactly have you tried? Remember that you have to send the scene through Backburner/farm (another node) to at least get the effect of layered EXR+extra EXR for every render element (RE) in some cases.

Here are questions for you:

1. when using CXR as output and having additonal REs defined. Should there be a) only a single, multilayered CRX as output or b) the multilayered CXR plus additional CXRs for every RE? If b), what is it for?

2. when using EXR as output and having additional REs defined AND you have ticked "Automatically Add/Remove Render Elements" in Render Output File -> "Setup". What do you get when a) rendering locally and b) sending it through Backburner? Please use the attached script as prerender script in Render setup -> common -> scripts to log the current state of the EXR output settings on the node(s) into max.log. My findings are: it depends on local settings (that's what the script reports, don't worry it does not change anything). This should never be the case.


Good Luck

I have done some testing this week. And for far it seems to save out correctly. i.e. I have one layered exr and then a seperate exr for each element which are the correct sizes.

Quote
1. when using CXR as output and having additonal REs defined. Should there be a) only a single, multilayered CRX as output or b) the multilayered CXR plus additional CXRs for every RE? If b), what is it for?

The answer to this is b). I get a multilayered cxr + an additional element for each render element. Honestly I don't know why it is like this and there is already a feature request logged to change this behaviour.

(Internal ID=403268985)

Quote
2. when using EXR as output and having additional REs defined AND you have ticked "Automatically Add/Remove Render Elements" in Render Output File -> "Setup". What do you get when a) rendering locally and b) sending it through Backburner? Please use the attached script as prerender script in Render setup -> common -> scripts to log the current state of the EXR output settings on the node(s) into max.log. My findings are: it depends on local settings (that's what the script reports, don't worry it does not change anything). This should never be the case.

I will get back to you on this after I have had the chance to do some more testing.

Cheers.
Please read this before reporting bugs:
How to report issues to us!

2019-11-01, 02:22:35
Reply #23

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Hi Dan

We are looking into this, but so far we have been unable to reproduce this behaviour yet.

Do you have any further reproduction steps which may help us?

Thanks,

Rowan

Well, that's annoying.  I have no idea how you're going to reproduce the issue then.

We just submit the job as normal to BB and the render is spat out a while later, there's no script or special process that we follow, we just use max's native functions.

2019-11-01, 10:40:35
Reply #24

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Well, that's annoying.  I have no idea how you're going to reproduce the issue then.

We just submit the job as normal to BB and the render is spat out a while later, there's no script or special process that we follow, we just use max's native functions.

Hi,

Which versions of Max and backburner are you using? I couldn't see this info in the thread..

Rowan
Please read this before reporting bugs:
How to report issues to us!

2019-11-06, 01:16:44
Reply #25

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Well, that's annoying.  I have no idea how you're going to reproduce the issue then.

We just submit the job as normal to BB and the render is spat out a while later, there's no script or special process that we follow, we just use max's native functions.

Hi,

Which versions of Max and backburner are you using? I couldn't see this info in the thread..


Rowan

Max 2018.4

Just spotted our backburner manager hasn't been updated from 2016 to 2018.  I'm going to update now and I'll let you know if that fixes it
« Last Edit: 2019-11-06, 01:58:33 by Dan Rodgers »

2019-11-06, 03:32:38
Reply #26

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
didn't fix it..   

so we are not running max 2018.4 and BB 2018.

2019-11-06, 10:53:57
Reply #27

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Hi Dan,

Thanks for the info. I had been using 2019. I will install the exact versions you are using and see if I can reproduce it.

Rowan
Please read this before reporting bugs:
How to report issues to us!

2019-11-07, 13:33:31
Reply #28

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Hi Dan,

Can you send me a screenshot of your file save settings for exr?

Thanks,

Rowan
Please read this before reporting bugs:
How to report issues to us!

2019-11-14, 03:15:24
Reply #29

Dan Rodgers

  • Active Users
  • **
  • Posts: 52
    • View Profile
Hi Dan,

Can you send me a screenshot of your file save settings for exr?

Thanks,

Rowan

They should just be default, the only thing I change is every now and then I use full-float

2020-01-22, 13:52:50
Reply #30

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 9443
  • Marcin
    • View Profile
I did manage to reproduce this, but only under very specific circumstances:
- You must be saving from the VFB> Save All (it does not happen when the files are automatically saved on render end)
- You must manually add all render elements to the list in the EXR save dialog
What happens then is:
- The beauty is saved in EXR format with all render elements stored inside it (can be found for example in Fusion, but not in PS)
- Each single render element is also saved in EXR format with all render elements stored inside it (again this can be found in Fusion)

So if the same behavior can be triggered when rendering via BB/DR, then I suppose this would be the culprit.
We will look into it further, and will see if the same happens with other renderers. If it only happens when using Corona's "Save all", then it would be a Corona-only issue. I can't test it right now.

2020-01-23, 13:16:19
Reply #31

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
Maru, I tested this with BB2019 and Max2020 finally. I used a simple scene with a single render element added in the RE tab, not touched afterwards and not manually added to the EXR output. Maybe the case you described is just a subset:



1. Scanline and EXR:

BB job A: Autoadd ON in output configuration when submitting the scene:
   1) BB server local setting autoadd on: 1 layered EXR written (beauty + RE)
   2) BB server local setting autoadd off: 1 layered EXR written (beauty + RE) plus 1 flat EXR (the render element)

BB job B: Autoadd OFF in output configuration when submitting the scene:
   1) BB server local setting autoadd on: 1 flat EXR (beauty only), NO render element at all
   2) BB server local setting autoadd off: 1 flat EXR written (beauty only) plus 1 flat EXR for the render element

---------------------

2. Scanline and CXR:

   4 times 1 flat CXR (Beauty only) plus 1 flat CXR (the render element). No Alpha.

---------------------

3. Corona and CXR:

   4 times 1 layered CXR written (Beauty + RE) plus 1 flat CXR (the render element)

---------------------

4. Corona and EXR:

BB job A: Autoadd ON in output configuration when submitting the scene:
   1) BB server local setting autoadd on: 1 layered EXR written (beauty + RE)
   2) BB server local setting autoadd off: 1 layered EXR written (beauty + RE) plus 1 flat EXR (the render element)

BB job B: Autoadd OFF in output configuration when submitting the scene:
   1) BB server local setting autoadd on:1 flat EXR (beauty only), NO render elements at all
   2) BB server local setting autoadd off:1 flat EXR written (beauty only) plus 1 flat EXR for the render element




You can see that Corona + EXR is exactly the same as Scanline + EXR, a weird Max issue - not bound to a specific renderer. And again you can also see the impact of local settings. Case B.1 is the reason for removing the script from here.That post is still valid for me, since I was able to reproduce it with the newest versions available. Only B.2 and A.1 are correct results, B.1 is data loss in the worst case and A.2 creates annoying additional render elements as Corona CXR always does ;)


Good Luck




Never underestimate the power of a well placed level one spell.

2020-02-03, 18:09:53
Reply #32

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
You can see that Corona + EXR is exactly the same as Scanline + EXR, a weird Max issue - not bound to a specific renderer. And again you can also see the impact of local settings. Case B.1 is the reason for removing the script from here.That post is still valid for me, since I was able to reproduce it with the newest versions available. Only B.2 and A.1 are correct results, B.1 is data loss in the worst case and A.2 creates annoying additional render elements as Corona CXR always does ;)

Good Luck

Hey Frood,

Just to confirm by A.2 and B.1 you mean:

2) BB server local setting autoadd off: 1 layered EXR written (beauty + RE) plus 1 flat EXR (the render element)
and
1) BB server local setting autoadd on: 1 flat EXR (beauty only), NO render element at all

Thanks,

Rowan
Please read this before reporting bugs:
How to report issues to us!

2020-02-04, 04:47:40
Reply #33

Frood

  • Active Users
  • **
  • Posts: 1414
    • View Profile
    • Rakete GmbH
you mean:

Yes, exactly.

Good Luck


Never underestimate the power of a well placed level one spell.

2020-02-06, 13:39:38
Reply #34

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 603
    • View Profile
Hi,

Okay so it looks like we can finally reproduce this in one way or another. It seems that there are some issues with saving EXR files.

With the help of Frood, we discovered that this is because 3ds Max saves the EXR saving settings locally on the machine and not in the scene.

This can lead to settings getting mixed up when rendering over a network with backburner. I have sent this over to Autodesk and I'm awaiting their response.

In the mean time, it might be best to set a pre-render script in the scene which sets the EXR file saving settings before each render commences. I hope this can be a suitable workaround for the time being.

I hope this information is useful.

Thanks,

Rowan
Please read this before reporting bugs:
How to report issues to us!