Color palette extraction

The extraction of colors from images is a well-studied field called color quantization, and most tools on the web use some variation these algorithms to group colors together.

The main problem with this is that these techniques are optimized for preserving the perceptual qualities of the original image, whereas we're more concerned with the final color palette. Also they present only one set of colors as the most representative, but building a color palette is subjective in nature - there may be many different ways to look at a single image.

Human-designed color palettes typically have some high-level structure - a gradient running left to right, similar hues grouped together etc, and have some minimum amount of contrast between each color. Automatically created palettes looks more haphazard, with colors distributed according to how they were used in the original image.

I think this could be solvable with yet another GAN, but for now I’ve settled on a more brute-force technique that I call generative-MMCQ. It roughly mirrors the generator-discriminator dual of a GAN but is a lot more random.

The classifier acts as a gatekeeper of sorts, the ultimate judge of what a "good looking" color palette looks like. It would be elegant to use the discriminator half of the Colormind GAN for this, which is what I initially tried. It turns out that the discriminator is very good at telling the difference between the training set and the generator, which is different from "good" vs "bad" palettes. The discriminator also tends to look at high frequency details at pixel-level whereas we care more about broad strokes.

In the end I built a self-contained classifier and trained it on a hand-picked list of examples. Good color palettes generally have good color contrast and an overarching theme, and bad ones look random and/or has bad inter-color contrast.

The final result is determined by RNG, so for any given image there could be many different palettes that score highly. This is most striking in images with a diverse palette.

You can give this a try through the image upload feature.