1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

IMPROVEMENT [color correction] Add 3D LUT correction

Discussion in 'Feature Requests' started by lildadou, 15 December 2017.

  1. lildadou

    lildadou New Member

    Messages:
    5
    Hardware:
    32/64bit, +Arduino
    Hello,
    Currently, the color correction proposed by hyperion is a 1D correction by matrix transformation.
    Matrix transformation is ideal for transforming a linear space into another linear space. For example, to transform a video from the Rec. 2020 space (4K) to the Rec. 709 space (HD).
    However, the matrix transformation for color correction is really not suitable:
    - when the gamma must be corrected (this is a curve)
    - when the RGB channels are not independent (eg. asking for red will also lower the blue)
    For these reasons, lookup tables (LUT) are used to correct screens and projectors. LEDs have the same problems... worse.

    I have just coded this type of correction for FastLED. The result is quite satisfactory (15ms to correct 300leds by an arduino nano) but it's still quite expensive for such a small machine. That is why I think it would be interesting to propose this type of correction in hyperion.

    Autodesk 3DLUT to C 3D array (html)
    3DLUT color correction on GitHub:
    Code:
    /// 3D array index
    struct LUTCoord {
        union {
            struct {
                uint8_t redIndex;
                uint8_t greenIndex;
                uint8_t blueIndex;
            };
            uint8_t raw[3];
        };
    
        inline LUTCoord() __attribute__((always_inline))
        {
        }
    
        inline LUTCoord( uint8_t ir, uint8_t ig, uint8_t ib)  __attribute__((always_inline))
            : redIndex(ir), greenIndex(ig), blueIndex(ib)
        {
        }
    };
    
    
    /// This 3D LUT can be used provided that the space between entries is the same for all dimensions.
    template<uint8_t SIZE>
    class Array3DLUT {
    public:
        CRGB (*cube)[SIZE][SIZE];
    
        Array3DLUT(CRGB entries[SIZE][SIZE][SIZE]);
        virtual ~Array3DLUT();
    
        /// Similar to lookup but instead of a coordinate in the LUT,
        /// we pass a point and the result will be interpolated.
        /// @param input - RGB coordinates of a point in the LUT
        /// @returns Interpolated value in the LUT
        struct CRGB lookup3DTetrahedral(CRGB &input);
    };
    
    
    
    
    template<uint8_t SIZE>
    Array3DLUT<SIZE>::Array3DLUT(CRGB (*entries)[SIZE][SIZE]) {
        this->cube = entries;
    }
    
    template<uint8_t SIZE>
    Array3DLUT<SIZE>::~Array3DLUT() {
        // TODO Auto-generated destructor stub
    }
    
    // Ref implem: https://github.com/ampas/CLF/blob/master/python/aces/clf/Array.py
    template<uint8_t SIZE>
    struct CRGB Array3DLUT<SIZE>::lookup3DTetrahedral(CRGB &input) {
        LUTCoord subcubeDarkestVertice; // Store the subcube's origin coordinates
        uint8_t darkSpotDistances[3]; // Distance input <-> subcube's darkest vertice
    
        #define STEP (255/(SIZE-1))
        #define MAX_INDEX SIZE-2
        for (int i = 0; i < 3; i++) { // for each rgb channel
            subcubeDarkestVertice.raw[i] = input.raw[i] / STEP;
            darkSpotDistances[i] = input.raw[i] % STEP;
    
            if (subcubeDarkestVertice.raw[i] > MAX_INDEX) {
                // But, if we are on an upper border then we must take the
                // inner border or the subcube will not exist.
                subcubeDarkestVertice.raw[i] = MAX_INDEX;
                darkSpotDistances[i] = 255;
            }
        }
    
        // Rebind for consistency with Truelight paper
        #define fx darkSpotDistances[0]
        #define fy darkSpotDistances[1]
        #define fz darkSpotDistances[2]
        #define N000 this->cube[subcubeDarkestVertice.redIndex  ][subcubeDarkestVertice.greenIndex  ][subcubeDarkestVertice.blueIndex  ]
        #define N001 this->cube[subcubeDarkestVertice.redIndex  ][subcubeDarkestVertice.greenIndex  ][subcubeDarkestVertice.blueIndex+1]
        #define N010 this->cube[subcubeDarkestVertice.redIndex  ][subcubeDarkestVertice.greenIndex+1][subcubeDarkestVertice.blueIndex  ]
        #define N011 this->cube[subcubeDarkestVertice.redIndex  ][subcubeDarkestVertice.greenIndex+1][subcubeDarkestVertice.blueIndex+1]
        #define N100 this->cube[subcubeDarkestVertice.redIndex+1][subcubeDarkestVertice.greenIndex  ][subcubeDarkestVertice.blueIndex  ]
        #define N101 this->cube[subcubeDarkestVertice.redIndex+1][subcubeDarkestVertice.greenIndex  ][subcubeDarkestVertice.blueIndex+1]
        #define N110 this->cube[subcubeDarkestVertice.redIndex+1][subcubeDarkestVertice.greenIndex+1][subcubeDarkestVertice.blueIndex  ]
        #define N111 this->cube[subcubeDarkestVertice.redIndex+1][subcubeDarkestVertice.greenIndex+1][subcubeDarkestVertice.blueIndex+1]
    
        struct CRGB result;
        if (fx > fy) {
            if (fy > fz) {
                for (int i=0; i < 3; i++) {
                    result.raw[i] = (
                            (uint16_t)(255-fx) * N000.raw[i] +
                            (uint16_t)(fx-fy)  * N100.raw[i] +
                            (uint16_t)(fy-fz)  * N110.raw[i] +
                            (uint16_t)(fz)     * N111.raw[i] ) >> 8;
                }
            } else {
                if (fx > fz) {
                    for (int i=0; i < 3; i++) {
                        result.raw[i] = (
                                (uint16_t)(255-fx) * N000.raw[i] +
                                (uint16_t)(fx-fz)  * N100.raw[i] +
                                (uint16_t)(fz-fy)  * N101.raw[i] +
                                (uint16_t)(fy)     * N111.raw[i] ) >> 8;
                    }
                } else {
                    for (int i=0; i < 3; i++) {
                        result.raw[i] = (
                                (uint16_t)(255-fz) * N000.raw[i] +
                                (uint16_t)(fz-fx)  * N001.raw[i] +
                                (uint16_t)(fx-fy)  * N101.raw[i] +
                                (uint16_t)(fy)     * N111.raw[i] ) >> 8;
                    }
                }
            }
        } else {
            if (fz > fy) {
                for (int i=0; i < 3; i++) {
                    result.raw[i] = (
                            (uint16_t)(255-fz) * N000.raw[i] +
                            (uint16_t)(fz-fy)  * N001.raw[i] +
                            (uint16_t)(fy-fx)  * N011.raw[i] +
                            (uint16_t)(fx)     * N111.raw[i] ) >> 8;
                }
            } else {
                if (fz > fx) {
                    for (int i=0; i < 3; i++) {
                        result.raw[i] = (
                                (uint16_t)(255-fy) * N000.raw[i] +
                                (uint16_t)(fy-fz)  * N010.raw[i] +
                                (uint16_t)(fz-fx)  * N011.raw[i] +
                                (uint16_t)(fx)     * N111.raw[i] ) >> 8;
                    }
                } else {
                    for (int i=0; i < 3; i++) {
                        result.raw[i] = (
                                (uint16_t)(255-fy) * N000.raw[i] +
                                (uint16_t)(fy-fx)  * N010.raw[i] +
                                (uint16_t)(fx-fz)  * N011.raw[i] +
                                (uint16_t)(fz)     * N111.raw[i] ) >> 8;
                    }
                }
            }
        }
    
        return result;
    }
    
     
  2. Brindosch

    Brindosch Administrator Administrator

    Messages:
    679
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
  3. lildadou

    lildadou New Member

    Messages:
    5
    Hardware:
    32/64bit, +Arduino
    Okay, I'll see what I can do.
     
  4. Brindosch

    Brindosch Administrator Administrator

    Messages:
    679
    Hardware:
    RPi1/Zero, RPi2, RPi3, +nodeMCU/ESP8266
    Amazing!
    Thank you

    Be aware that i linked to the .ng repository. As offered just ask if you have any questions regarding the hyperion code.

    brindosch
     
  5. lildadou

    lildadou New Member

    Messages:
    5
    Hardware:
    32/64bit, +Arduino
    I finally have a little time to look at the dev. I'm not usualy work on cpp projects so I just finished preparing my dev environment (vm linux, IDE etc). I have some questions about RgbTransform before begin:
    • Can you explain the differences between each variable/transformation brightness?
    • Can you tell me in what order the different transformations are applied?
    • I believe that some transformations (backlight, minBrightness and maxBrightness) can be applied to your 1D LUT (_mappingR/G/B) rather than being calculated on the fly. What do you think?
    • Can we consider a 'Chain of responsibility' pattern for RgbTransform. it will allow to separate well each type of transformation and to add new ones more easily (hue, saturation, etc)?
     
  6. Tocinillo

    Tocinillo New Member

    Messages:
    8
    Hardware:
    RPi3
  7. lildadou

    lildadou New Member

    Messages:
    5
    Hardware:
    32/64bit, +Arduino
    @Tocinillo if you are using FastLED for led driving then you can use my fork.
    Have you a colormeter?
     
  8. lildadou

    lildadou New Member

    Messages:
    5
    Hardware:
    32/64bit, +Arduino
    Brindosch
    Hello and sorry I haven't heard from you in a long time.
    I just wanted to let you know that I haven't given up even though I can't free up any time right now and take stock.

    I decided to create an external library. This is motivated by:
    - the problem is not specific to hyperion. for example, FastLED users will probably be interested in color correction for other projects.
    - the classic benefits of modularization: easier to test, develop, reuse, maintain, replace etc.

    My design choices are based on a paper from spectralcal (which is a specialist in color correction): http://www.spectracal.com/Documents/White Papers/Best Practices - 3D LUT Implementation for Manufacturers.pdf

    I decided to create a library of color processing components that can be arbitrarily chained. The code is available on my spike branch
    I spent most of my time on gtests, CMake and Travis (quality is paramount to me) but the general idea still emerges.

    I hope I can get back to work on it.
     
  9. mosiris

    mosiris New Member

    Messages:
    5
    Hardware:
    RPi1/Zero, RPi3, +Arduino, +nodeMCU/ESP8266
    Is the FASTLed fork updated for esp32?