AI to Simplify

There are two main types of image files: Raster and Vector. Raster images are more common in general such as jpg, gif, png, and are widely used on the web. Vector graphics are common for images that will be applied to a physical product. 

When using a raster program you paint an image and it’s similar to dipping a brush in paint and painting. You can blend colors to soften the transition from one color to another.

When using a vector program you draw the outline of shapes: e.g. an eye shape, a nose shape, a lip shape. These shapes display one single color each.


A lot of images can be made with either raster or vector program and look exactly the same on both programs. Images with a subtle gradation of one color to another are the images that will look most different since vector programs need to create a separate shape for each shade of color.


Some vector programs do have the ability to create color gradients within one single shape, but these are actually raster effects. A vector graphic with gradients contains both vector and raster elements and won’t be suitable for a process that requires 100% vector or true vector art. 

Why are we not using vector graphics everywhere?

1) The smoother the color transitions and the more detail we have in our images, the quicker the advantage of vectorization evaporates. 

2) Image tracing procedure is not trivial and heavily depends on vectorization algorithm in terms of output quality. It is often unclear in advance whether it will work well on a given input. So now we know everything we need to know to be able to understand and appreciate this amazing piece of work. The input is a rough sketch, that is a raster image, and the output is a simplified, cleaned-up and vectorized version of it. We’re not only doing vectorization but simplification as well. This is a game changer, because this way, we can learn on the additional knowledge that these input raster images are sketches, hand-drawn images, therefore there is a lot of extra fluff in them that would be undesirable to retain in the vectorized output, therefore the name, sketch simplification. 


In each of these case, it is very impressive how well it works. The next question is obviously, how does this wizardry happen? It happens by using a classic deep learning technique, a convolutional neural network.


A convolutional neural network is trained on a large number of input and output pairs. However, this is no ordinary convolutional neural network! This particular variant differs from the standard well-known architecture as it is augmented with a series of upsampling convolution steps. Intuitively, the algorithm learns a sparse and concise representation of these input sketches, this means that it focuses on the most defining features and throws away all the unneeded fluff. And the upsampling convolution steps make it able to not only understand, but synthesize new, simplified, and high-resolution images that we can easily vectorize using standard algorithms. It is fully automatic and requires no user intervention.

Sketch Simplification researchers performed a user study and compared against top vectorization tools that work directly on raster images, Users preferred this approach over 97% of the time with respect to either of the two tools.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s