A short tutorial about Vertex Paint in 3dsmax

Some useful tricks you can do with Vertex Paint in 3dsmax

Hi and welcome to a short tutorial on Vertex Paint. These are just some simple tricks that I thought might be useful for anyone using vertex colours. I won’t touch so much on the basic functionality of the Vertex Paint modifier since most of it is pretty straight forward. If you have a tablet, you can use pressure sensitivity to control the opacity and/or size of your brush just like in Photoshop. Those controls together with a lot of useful stuff can be found in the Brush Options.

By selecting sub-objects acts as a masked selection in Photoshop. Selecting vertices will give you a feathered edge, blending the colours between the selected vertices and the closest neighbouring vertices. However, you can use Soft Selection to get an effect similar to Feather in Photoshop where the area you paint will be larger than the area you select and a nice feathered edge. Using the Falloff value is similar to setting the Radius in Photoshop and will cause your painting to be gradually faded out to the edge of the Soft Selection. If you select polygons it will give you a hard edged transition and you can use Soft Selection to modify is so it fades out instead. See images below for example

Bucket fill on a vertex selection, bucket fill on a polygon selection

Bucket fill with Soft Selection on vertices and polygons

You can of course use the other tools on selections such as Blur and Erase. Just as with your regular colour, you can use a brush to blur/erase or the bucket. This, together with controls for Strength, Opacity and Size, gives you a lot of control over your painting.

Example of Erase and then Blur

You also have layers with different blending modes such as Multiply, Screen etc, pretty much like layers in Photoshop

Multiply and Hard Light

You also have a little button that says “Adjust Color”. This will let you adjust the Hue, Saturation, Value,RGB, Contrast and Levels of your Vertex Paint layer/Modifier.

On top of this you can also use Vertex Paint to bake your lighting. What this does is basically take a Vertex Paint modifier and colour the mesh according to the light in your scene. You can use Radiosity if you want or just a few regular lights. The beauty of this is that you can keep painting on top of your lighting and if you change your lighting, you can just bake it again and re-arrange the layers as you see fit by moving the modifiers up or down in the stack.

I added a plane and some random geometry to the scene, plopped in a main light and a skylight then ran the radiosity solution with the default settings. To bake it I opened the Utilities tab and picked “Assign Vertex Colors”. You can also do this by selecting all your objects and applying a Vertex Paint modifier which is basically what the utility version does. You can see the settings I used in the image.

Assign Vertex Colors settings

Below is the result.

Baked light

The cool thing is that now all the selected objects has a Vertex Paint modifier applied containing the lighting information. You can lower the opacity of it, change the blending mode to whatever you feel like (Overlay maybe?) and paint on it. Or you can add a new layer and paint more light into the scene if you feel like it. As you can see in my scene, you do need to think of the poly distribution when you bake the light, it can easily look less than stellar so bear that in mind when you model your scene. The good news is that you can edit the mesh as well as the painting so you can fix any lighting issues and re-bake the lighting. I decided to fiddle some more with this scene and below is the result of some quick adjustments. The arch-thing looks a bit better and I did some edge turning to smooth the lighting a bit as well as add more geometry where the hemisphere and the arches meets the plane but these are just minor edits to show what you might have to do to make the light bake work for you instead of against you.

Post Vertex Paint and mesh edits

I hope this has helped somewhat in showing the versatility of Vertex Paint together with baking your lights.

Now go forth and experiment!