Understanding CDPR's rendering pipeline

Lighting

Lighting in REDengine is complex. Since the conception of it, REDengine has been using a probe-based global illumination system, which requires a single bake of lighting for all times of day. This method is inferior to baked RT, but is suitable for systems with dynamic time of day and weather.

However, this method has its limitation, several, and we see them being corrected with the initial implementation of fullly dynamic semi-global illumination in the form of Psycho RT. Differing from other quality levels, Psycho implements a pseudo-global illumination tech, instead of other RT levels simply tracing lighting for global light sources, such as the sun and the moon.

After patch 1.62, and after the creation of ReSTIR GI, we see now that CDPR has implemented a fully-realized RT pipeline. Each step of the way is handled by rays rather than normal approximations.

The benefits, other than the obvious ability for fully realized global illumination and direct illumination, are numerous. Each light can now cast shadows, use material parameters for nearly-perfect subsurface scattering, ideal light blending and intensity.

I will go further into detail on the new possibilities with PT in another entry, today, we will focus on...

Color

Recently, I have been working on recreating a cinema-accurate and professional-quality workflow for color grading in Cyberpunk, and generally, trying to retrace the steps CDPR took to get such a unique, albeit, polarizing look to Cyberpunk.

A history lesson

Before 2020, while the game was in heavy development, CDPR used a different, more parametrizable method for handling HDR assets and lighting. They would simply running through a TMO (tone mapping operator) to get back to 8-bit range and afterwards apply color grading.

This, while convenient, did make handling HDR display quite difficult. How could you apply a color grade made for SDR on top of HDR without losing precision or missing data? This required a full restructuring of the method.

Eventually, CDPR landed on the method I have been able to reproduce. They used a normal, neutral LUT texture and ran it through ACES (Academy Color Encoding System) transforms to go to a neutral color space, ACES AP1. I do not know the gamma they used (could be ACEScct or ACEScc, maybe even the linear ACEScg), but I do know that eventually, the HDR data was ran through the RRT (Reference Rendering Transform) as a tone mapper, then through the ODT to go to either HDR or SDR.

Unfortunately, I have not been able to single out a method to generate HDR LUTs from scratch, however, I have been able to create a workflow to create tone mapped LUTs for SDR.

Tools used/required

• DaVinci Resolve Does not matter if it is studio, but studio will be required for the eventual HDR workflow.

• Photoshop

• WolvenKit

• NVTT Exporter and the respective Photoshop plugin. Can be found on NVIDIA developer, does require an account to download.

Intermission: Troubleshooting DaVinci

Right here is where I almost gave up.

DaVinci refused to open. I tried all I could, searched the internet, tried every damn solution. Nothing worked.

BUT, looking at the log files, I was able to deduce the culprit.

SyManager.DeckLink failed to respond due to some USB issue, I even looked that up and well, no dice.

Eventually, I came to think that it may be an issue with the control panels utility for Resolve, as Blackmagic sells some cool decks and accessories for colorists to work faster. But, I'm not rich enough for those, so, I figured, why not just uninstall the entire framework for them.

Boom, DaVinci opened right after.

Setting up your project

Some of you who are more familiar with DaVinci may be already going ahead and setting up ACES in the DaVinci color settings, changing and messing with color science and management, etc. Stop. We're gonna be using nodes.

Below are 2 sample pictures captured with the method, we will be getting a grasp with non-destructive color grading this way.

Insert these pictures into your timeline and head into the Color tab. Then, select your clips and add them to a group, the group name doesn't really matter. We'll only be doing this to use pre-clips and post-clips.

Head into the pre-clip level and apply an ACES transform node. Set your input transform to SRGB and output transform to ACEScct (ACEScct behaves better than ACEScc at times, use your preference) and don't forget to turn on the reference gamut compress.

Afterwards, go to the post-clip level and add another ACES transform node. Now set your input transform to ACEScct (or ACEScc) and set the output transform to SRGB.

Your clip level now has full range and fidelity. You can color grade to your heart's content in the clip level, but I won't be going into detail in how to do this. Or you could proceed onwards, if you do so, then ignore anything about importing your color graded LUT.

So, how do we put your grade into the game? Glad you asked. After you're done grading, turn off everything but your effects in the clip level. Generate a LUT from here. It doesn't matter if the image you see doesn't match up with the one you graded with, we'll be correcting for it further on when we generate the LUT.

Generating a finished CUBE LUT

It's now good to mention that Cyberpunk outputs data with an ARRI LogC3 encoding, but SRGB color space. ACES, on the other hand, expects ARRI LogC3 encoding but ARRI Wide Gamut 3 color space as well. At the end, Cyberpunk expects SRGB data with SRGB gamma too, so you need a TMO to handle this conversion from HDR data encoded in a Log curve to fit in SDR range, but decoded and then put into the display gamut and dynamic range.

ACES is apt for the job.

Take the LUT texture above and add it to your timeline as well, and assign it to another group.

Go to the pre-clip level for your LUT clip and add a Color Space Transform node. Specify the input color space and gamma as SRGB, but output color space as ARRI Wide Gamut and output gamma of SRGB as well.

Subsequently after the CST (Color Space Transform) add your IDT (Input Device Transform) using an ACES transform node. Set the input transform to ARRI LogC3 with an exposure index of 800 and output transform of ACEScct (or ACEScc depending on what your color grade was made in/for). Make sure you enable color gamut compression, or else you will get horrible clipping in higher exposures.

At the post-clip level, add your ODT (Output Device Transform). Set your input transform to ACEScct/cc and output transform to SRGB. Once again, turn on reference gamut compress.

Now, in your clip level, add a corrector node and then drag your generated color grading CUBE LUT atop the corrector node (skip this if you didn't grade)

Once you have done all of this, right click and select "Generate LUT" and export a CUBE.

Creating the LUT texture

Congrats! You have now reached the homestretch in the reverse-engineered LUT workflow. This part could be confusing for some, but it's easy enough if you're familiar with visual modding or handling XBM textures on a very high level.

Once your CUBE is done exporting, open the lut.png file you downloaded before in Photoshop. Here, you need to add a "Color Lookup" layer. Select the 3D LUT file to be your generated LUT from when you finished the CUBE LUT in the last step. Once it is applied, apply a channel mixer correction layer and flip the blue and green channels.

After this is done, export using the NVIDIA Texture Tools plugin as a 32x4f DDS with no mipmaps as well as highest quality compression.

Once you are done, put the texture into WolvenKit and import using these settings:

Afterwards, set the depth, height, and with properties to 32, then go to renderTextureResource -> renderResourceBlobPC -> header -> sizeInfo and set all of the properties there to 32 also. Don't forget to change the texture type to 3D in textureInfo!

There it is. Your tonemapped LUT from scratch.

Footnotes

1. This initial revision is messy, very messy. Ideally I would've loved to make a video showing off the process in detail, being able to go on tangents and show you around exactly how I stumbled and developed upon the system, reasoning behind a few of the decisions I made in this entry, but I lack the resources right now (notably a microphone)

2. Normally I'd try to make the entire thing seem more accessible in general, but really, this is an extremely complex topic that would take some experts press their eyes in frustration. I know it happened to me a ton.

3. If you're looking for pre-made examples, I have a few right here:

What is a LUT?

The term LUT has its origins in mathematics, where a Look-Up Table would be able to shorten certain math operations by containing readily-calculated values for any input value, or an operation for an input value for a desired outcome. This shortcut was quite efficient and sped up many operations, and was normally designed as a matrix of different numbers and variables. Soon enough, this improvement in speed would be applied in many other fields, and eventually reached computer graphics and film production.

LUTs are essentially a texture or text file containing every single possible color that can be displayed, and the transformation of an input color into an output color through it.

Cyberpunk's use of LUTs

There are many different types of color-grading-related LUTs, such as .CUBE files and .3DL files, but video games often use a texture.

Initially, Cyberpunk took an approach like many other games, and encoding the LUT as an unwrapped 3D texture:

Afterwards, Cyberpunk would repackage LUT textures with the blue and green channels inverted to get higher quality, as the human eye values green more than other colors and is more efficient to get all gradients laid on top of green instead of blue. Here we can safely assume is where the switch to using true 3D textures was also made, but we will not show a 3D texture due to redundancy and inability to do so, so just assume from now on all textures are actually just a 3D cube.

To get even more odd, Cyberpunk would then put the image tone mapping, the process in which HDR colors are processed into SDR, inside of the LUT! This came with the conversion from using a normal sRGB transform, which is industry standard, to using an input of ARRI LogC1 and outputting a linear color which is afterwards processed into normal sRGB gamma by the linear tonemapping.

Editing base-game LUTs

With all that backstory done--how do we start?!

You'll need WolvenKit, NVIDIA Texture Tools Exporter + its Photoshop plugin, and we also require Adobe Photoshop CS4 or later. Photoshop CC is preferred.

Okay, now we can get started!

Create a new WolvenKit project and call it whatever you want.

After that, import the file base\weather\24h_basic\luts\cp2077_gen_lut_nge_v017.xbm into your project.

Then open it, and change depth to 1, height to 32, and width to 1024. After that, navigate to renderTextureResource/renderResourceBlobPC/header/sizeInfo and apply the same edits as before.

Export the file into a DDS...

Open it up in Photoshop and... voila!

We now successfuly have the 32-bit HDR LUT imported into Photoshop. You can do any color correction here, or do some ReShade edits in-game and then put that into a LUT image and then use a Photoshop plugin to import it and apply it ontop of this LUT. Make sure you DO NOT simply replace this image with your exported LUT from ReShade or some other editing app, as you will completely ruin the tone mapping the base-game does. I recommend DaVinci Resolve as well and then exporting as as .CUBE file from inside of it.

When you edit your LUT, make sure you flip the vertical axis (make it upside down) inside of Photoshop!

Re-importing your edited LUT.

Let's say I just did some curves tweaking and contrast, etc., etc. Maybe make the green saturation lower and other saturation higher to get rid of the green tint. Now what?!

Save back your DDS using NVTT and apply these settings:

• Format: 32x4f RGBA 128bpp | floating-point

• Generate Mipmaps: OFF

• Image Options - Compression Quality: Highest

If you forgot to flip your image inside of Photoshop, turn on the Image Options - Flip Vertically option inside of NVTT.

Wolvenkit:

• Open the Wolvenkit project where you have the original LUT of the game in .xbm

• Take your newly made LUT and name it the same as the original LUT, so: cp2077_gen_lut_nge_v017.dds.

• Move it to the folder where you exported the original LUT from the game in .dds format, and overwrite that one with yours.

• The folder in question is: yourwolvenkitprojectname\source\raw\base\weather\24h_basic\luts.

• You can also open the exact location from the 'Project explorer' in Wolvenkit. Under 'raw' folder dropdown you should see the exported .dds file. If you hover over it, it will show a yellow folder button that you can click to open its location.

• Now to import it, hover over 'Tools' at the top left in Wolvenkit, and select 'Import Tool'.

• Select the LUT and make sure the import settings are correct

Changing the settings back:

• When your LUT imports, double click on the .xbm file to open it in an editor.

• Then change: depth and width in both sections that we modified, back to the default of 32.

• Under renderTextureResource>renderResourceBlobPC>Header>mipMapInfo>0>layout change 'rowPitch' to '512'.

• Under renderTextureResource>renderResourceBlobPC>Header>textureInfo change 'type' to 'TEXTYPE_3D'.

That's it, now either use 'Pack mod' to pack the mod and have .archive file in the Wolvenkit project folder, or 'Install Mod' to preview it in game!

This will replace the sdr LUT for the base game (Night City).

If you want the LUT to also apply to Dogtown, or HDR: cp2077_gen_lut_nge_hdr_v017 is HDR lut. cp2077_gen_lut_ep1_sdr_v02b and(?) cp2077_gen_lut_ep1_sdr_v002is Dogtown.

If you want to change these as well, before importing you have to have the paths for these LUTs correct, along with the names.

To achieve this with ease, I suggest exporting the LUTs you want replaced, as they will create paths and give you the names by default.

Then you can go over to their locations, copy your LUT .dds file, copy the name of the LUT you want to replace, delete the original and overwrite it with yours while making sure it retains the name from the original. Rinse and repeat. Process for importing and adjusting the settings afterwards will be the same as before.

Results!

What is a LUT, really?

It's imperative to understand what LUT really is before starting. Often, beginners have a misconception caused by other software which often uses different formats for LUTs and different ways to apply them, such as ReShade.

Cyberpunk LUTs are distinctly different from ReShade LUTs due to two things:

1. They are 3D textures, not 2D textures with tiles describing different parts of the RGB cube, no, Cyberpunk LUTs are full-on 3D textures sampled with trilinear filtering.

2. They are applied with an input of ARRI LogC3 data with an sRGB color space. This means you can't go and use any ReShade LUTs, as they often do not expect LogC3 output. Along with this, many tools expect ARRI LogC3 data with an ARRI Wide Gamut 3 color space when generating your LUT. If you do not convert to it, you will end up with clipping colors.

But, aside from that, they are similar. Any LUT expects an input of a color, with an output of a new color. That's it. Cyberpunk's implementation is as simple as extracting the pixel out of the LUT texture that matches the input color, and if there isn't an exact match, it interpolates linearly to get a near-perfect approximation of the true output color.

If you're still confused, I recommend looking at this.

Prerequisites

You only need to complete this section once.

Workflow

WolvenKit

Create a new project in WolvenKit and import the file base\weather\24h_basic\luts\cp2077_gen_lut_nge_v017.xbm and export to a DDS, then rename it, removing the xbm file extension. You should have a dummy name with a file path containing that same file in the "raw" folder. You can delete the file in there, we only need the folder set up correctly.

DaVinci Resolve

Create a Resolve project, and head straight over to the Fusion tab, without importing any media. Add a LUT Cube Creator node, change the type to vertical and size to 32, 48, or 64. Remember this number, as you will use it later. Select for the generated Media Out node to output to both sides of the preview by enabling both of the circles below it. On one of the sides, select Views>3D Histogram. You should now have a LUT cube present. If you'd like more accuracy, right click and go to 3D Histogram and select solid with 1:1 sampling.

Right after the LUT Cube Creator node, add a Color Space Transform node, and matching the fact that tools expect ARRI Wide Gamut 3 with ARRI LogC3 data, but the game outputs sRGB with ARRI LogC3 gamma, we need to go from sRGB color space with ARRI LogC3 gamma to an output color space of ARRI Wide Gamut 3 with output gamma of ARRI LogC3

After this, we need to go from this LogC3 data to sRGB, which is what our display expects.

ACES method (recommended)

This is the method used by the original game to go from the input LogC3 to sRGB output, so it's what's recommended.

Add an ACES Transform node after the Color Space Transform node. Set the input transform to ARRI LogC3 with an output transform of sRGB and use ACES reference gamut compression.

ARRI LogC3 method

This method uses LUTs from ARRI themselves, and as such gives us two options: LogC3 and LogC3 "classic". I encourage you to use the normal LogC3 LUT, but experiment with Classic and see if you like it. Though, this method requires a change in the settings that are described later.

In the link above, select LogC wide gamut as the source format and destination format as video. Select your preference of ARRI Classic 709 and ARRI 709 in the conversion parameter. The file type can be either Blackmagic Fusion or Blackmagic DaVinci Resolve, it really doesn't matter.

Add a serial File LUT node and point to your downloaded file.

ARRI LogC4 method

It may share a name with the LogC3 method, but is distinct from it as it uses a more recent revision of the color science compared to the LogC3 method, but we will need to change things before and after this step if you decide to use this method.

Follow the same link described in the #arri-logc3-method section until you reach this part. Download the "ARRI LogC4 LUT Package".

Once you have downloaded it, extract the Rec.709 Gamma 2.4 65-point .CUBE file from the zip file. Add a serial node after the Color Space Transform called File LUT and point the LUT file to your extracted .CUBE file.

We have to make some changes to the Color Space Transform node, change the output color space to ARRI Wide Gamut 4 (instead of 3) and then the output gamma to ARRI LogC4. Your LUT should now look correctly exposed.

You can mess with the Tone Mapping Method, but just select None as a good option.

We need to change an option afterward that will be described later for both the LogC3/4 methods.

Resolve Color Managed method

This method uses DaVinci's integrated tone mappers to go from the LogC3 input to sRGB output. It produces a neutral and accurate output. It's also very minimal.

For this method to work, we just need to change the settings in the Color Space Transform.

Instead of having an output color space of ARRI Wide Gamut 3, set the color space to sRGB and gamma to sRGB as well. Then, change the tone mapping method to DaVinci, or some other method if you know what they mean.

This method creates a color cube that is very "unbiased"--it has fidelity in all directions, which, while creating a good representation of true color, can result in unnatural saturation. To account for this, the saturation compression gamut compression method can be selected.

After you have used your preferred method, you need to apply gamma correction. If you do not do this, you end up with incorrect gamma, which can mainly be seen on skin tones.

For the ACES method, you need to add another Color Space Transform node, with input color space and gamma of sRGB, with the output color space of sRGB but output gamma of linear.

For both ARRI methods, nearly the exact same steps are taken as the ACES method, but, instead of sRGB input color space and gamma, we use an input color space of Rec.709, but input gamma of Gamma 2.4. Keep the output color space at sRGB and output gamma at linear.

For the Resolve Color Managed method, simply change the output gamma in the color space transform to linear, but turn on Apply Forward OOTF.

This is the final stage. We now just need to output to an image, and use the right format. To do this, use a Resolve FX Transform>Transform node (not any other transform node) and switch on "Flip Vertical" to correctly output.

Right click on the LUT picture and save your image as a TGA or PNG file.

NVIDIA Texture Tools

Drag on your exported file onto the NVTT window and tick off "Generate Mipmaps" and select format as 32x4f. Tick off KTX2 compression and tick on DXT10 header as well turn on Highest compression quality.

Turn on "Extract From Atlas" and set "Depth of Volume" to the exact size of your LUT that you set up when first creating it. To verify you got it right, you should see a Z slider on the top right of the NVTT interface. Drag it and see if there is any vertical shift in the texture. If there is not, you got it right.

Save as a DDS file (only DDS) and put it in the folder that was generated after you exported the XBM file at the start of the article.

WolvenKit again

You should now see it in the import tool in WolvenKit. Change its TextureGroup property to TEXG_Generic_LUT, turn off IsGamma, VFlip, GenerateMipMaps, IsStreamable, and PremultiplyAlpha. Set compression to TCM_None. If your file doesn't have RawFormat as TRF_HDRFloat, then something in the DDS importing went wrong, and you need to re-set the format as 32x4f in NVTT.

Import the DDS. We need to tell the file it is actually a 3D texture in the header (even if the 3D data is already there), so go to renderTextureResource>renderResourceBlobPC>header>textureInfo>type and change it from TEXTYPE_2D to TEXTYPE_3D.

Make a copy of the XBM file and rename it to cp2077_gen_lut_nge_v017. Our dummy file there is for being able to quickly copy the filename, so, select your LUT XBM, C-c, C-v, select the dummy file, F2, C-c, select your LUT copy XBM, F2, C-v.

Your LUT is now correctly set up. You can install and launch now!

Results

All of these articles are out-of-date. Only use if you really know what you're doing.