A while back I was super fed up with the complexity of creating 3D assets for games. Normal maps, occlusion baking, retopologizing, UV layout, lighting; I wanted to toss it all out the window.
Then I saw what artists like Chelsea Saunders were doing with solid blocks of color on low-poly models, and I saw how ugly (in my opinion) the WiiU Wind Waker remake was, and I started getting into flat-shaded vertex painting. It's changed pretty much everything about how I work in 3D; letting me make art spontaneously and creatively without being weighed down by a complicated pipeline. I rarely even need to open Photoshop. Also it helped me get a sweet job, check out Guildlings.com.
This document is my attempt at a quick intro to help people get started. It's written for Autodesk Maya, though I'm sure the basic concepts would apply to other 3D software as well, and assumes a basic familiarity with the program. Let me know if you have comments and/or suggestions, and thanks for reading.
A 3d mesh is comprised of a bunch of points (vertices) in space, with flat surfaces drawn between them. Each vertex can store an array of numbers that define its position in space, normal vector, and texture coordinates. A vertex can also store numbers to represent color information (Red, Green, Blue, Alpha). These numbers can be used to color a model instead of applying textures. By using Maya's Paint Vertex Colors tool we can add color directly to our models instead of applying textures.
Vertex colors are typically put to use storing additional information to be used in a game engine for blending materials together. A game artist might paint in a value where they want snow to build up, or bake in ambient occlusion to multiply on top of your diffuse material.
Textures take up a lot of memory and storage. Keeping texture use to a minimum drastically reduces the download size of a game, (especially useful on mobile) and allows a game to have more tame hardware requirements. Unlit shaders work really well with vertex colors, and also let you get away with a lot of visual tricks and cheats.
Textures are limited by their resolution and compression settings, but geometry scales and looks good at any screen resolution. It's future-proof. Boundaries between colors can be super sharp in a way that'd normally require very high-resolution textures.
More isn't always better. Working in vertex colors forces you to accept limitations and improvise. Flat-shaded 3D often has a timeless graphic or illustrative look, and I think the result can be really vibrant and fun.
It’s much faster to not have to worry about UV layouts, texture files, and switching between programs to texture and model. A single artist can create way more content in a low-poly flat-shaded style than if they have to worry about spending weeks in Zbrush modeling out the pores on a dog's nostril.
Complex textures or patterns can be difficult or impossible to pull off (unless you become a boss at modeling, which you should).
Poly count suffers; something that could have been accomplished with a texture now has to be created by slicing up a mesh.
It’s not a common workflow. There isn't as much information on it out there, and software isn’t necessarily designed for it so requires some customization.
Start by creating a cube.
￼Select your cube and activate the Paint Vertex Color Tool (Modeling: Mesh Display>Paint Vertex Color Tool). You’ll probably want to set this to a hotkey (I use shift+v).
You should now see a paintbrush cursor with a red circle representing your brush size. Hold 'b' and left-click in the viewport to change the brush size.
￼Make sure your Tool Settings are open (Windows>Settings/Preferences>Tool Settings, alternatively: double-click the Paint Vertex Colors icon in the Tool Box).
In tool settings, make sure the paint mode is set to “vertex”. Double-click the color swatch to the right of “Color value” and select a bright color.
Click on a vertex of your cube and it’ll turn the color you chose. Try again with different colors, paying attention to how Maya interpolates between vertices.
In tool settings, set the paint mode to “face”. This restricts the color you’re applying to the inside of a face, creating hard lines along edges. Test this out a bit.
Once you get the hang of that, try “vertex face” mode. This restricts color to the inside of a face, but only applies to a single vertex rather than filling the face.
Ok, great, so we can color a cube. It's a start! The tricky part of vertex painting is using the topology of a model to help define the vertex colors. This means you've got to understand how colors blend and how the geometry of a mesh can support what you want to create.
Let's say we want to create a square of blue that fades into an outer border of green. We might start with this 3x3 plane, thinking we can paint the inner four vertices blue and the outer ring green.
But whoops, there's some weird business going on with the left and right faces. They're not blending how we might expect.
This is because secretly all these 4-sided polygons are actually two 3-sided polygons. Maya's automatically splitting them behind the scenes. Use "toggle hidden triangles" to view them.
If we use Maya's Multi-Cut tool to split the faces in the direction we want them to blend, we get a proper gradient in the direction we want.
Ok, this is the worst part of any tutorial: you gotta practice. Start small, pick a simple self-contained prop or some familiar object and try to model it out. See how much character you can cram in there with as few vertices as possible. See how much you can push simple geometry to create appealing, complex shapes.
Become amazing at using Maya's modeling toolkit. A big part of vertex painting is learning when and where to make small cuts in your geometry. This workflow requires a lot of experimentation and knowledge of Maya's modeling tools, but it's also a great way to learn them. Have fun!
Once you get started with this stuff, you'll probably realize that Maya's not really set up for vertex painting in this way. You're going to want to do a bit of customizing to make things work for you.