Color palette extraction

The extraction of colors from images is a well-studied field called color quantization, and most tools on the web use some variation of color quantization algorithm to group colors together.

The main problem with these techniques is that they are optimized for preserving the perceptual qualities of the original image, whereas we're more concerned with the final color palette. Moreover they present only one set of colors as the most accurate, when the space of possible color palettes that can be extracted is virtually infinite.

Here's an example of a color palette generated through the MMCQ algorithm:

Human-designed color palettes typically have some high-level structure - a gradient running left to right, similar hues grouped together etc, and have some minimum amount of contrast between each color. Automatically created palettes looks more haphazard, with colors distributed according to how they were used in the original image.

When trained on this type of color palette, the output from Colormind tends to resemble the input - rather random and unintuitive.

I think this could be solvable with yet another GAN, but for now I’ve settled on a more brute-force technique that I call generative-MMCQ. It roughly mirrors the generator-discriminator dual of a GAN but is a lot more random.

The classifier acts as a gatekeeper of sorts, the ultimate judge of what a "good looking" color palette looks like. It would be elegant to use the discriminator half of the Colormind GAN for this, which is what I initially tried. It turns out that the discriminator is very good at telling the difference between the training set and the generator, which is different from "good" vs "bad" palettes. The discriminator also tends to look at high frequency details at pixel-level whereas we care more about broad strokes.

In the end I built a self-contained classifier and trained it on a hand-picked list of examples. Good color palettes generally have good color contrast and an overarching theme, and bad ones look random and/or has bad inter-color contrast.

The possible downside to this approach is that the final result is determined by RNG, so for any given image there could be many different palettes that score highly.

You can give this a try through the image upload feature.