DigitalEro Offline

Normal Map to Sculptable Mesh

Sat, 11 Jan 2014 06:12:25


One of FigureSculptor's contributions from the old forum.
"FigureSculptor" said ...
I'm not sure how many people here use Blender. Seems like 3DS is the more common package for creating and modding characters, but I figured out something rather useful today that I thought I'd share. In Blender 2.5+, I figured out that it's possible to take a normal map for an extracted game character and recreate the high resolution sculpt data that was originally used to create the normal map. The amount of detail depends, of course, on the size of the normal map in relation to the size of the mesh. Once you have this sculpt data, you can use Blender's sculpt mode to make changes to the higher-resolution mesh, which you can either render directly from it or you can use it to generate new normal maps. Let's say, for example, you have a model ripped from a game, like this one: (If you know all about normal maps, you can skip ahead a little here) Game models are generally pretty low resolution since they have to be rendered many times per second. You can see just how low resolution by going into edit mode, like I've done for the dress in the next shot: Thanks to the normal map, when rendered, the dress will look like it has a much higher density mesh: Normals are used to do light calculations and a normal map simply stores the normal data from a much higher density mesh into an image, and than can be used to "fool the eye". A normal map adds very little overhead to the rendering calculations, whereas, increasing the density of a mesh adds a LOT of overhead. As a result, this is a great technique for doing runtime graphics like games because it looks pretty convincing without adding a lot of processing overhead. But, if you want to make any changes to the detail in the normal map of a game character, it's a pain. It can be done, but it's imperfect and tedious and usually involves working directly with the normal map data. Being able to re-sculpt the model would be much easier. Here is how you do it: The first step you need to do is to convert the mesh from triangles to quads. Game models use only triangles for performance reasons, however quads are much better for sculpting and subdivision modeling. Depending on the model you're using, this might be an easy conversion, or it might be a really painful and tedious one. Go into edit mode (tab button) and select all (press A key once or twice if any vertices are selected already). Try the easy way first by hitting Alt-J. This will attempt to combine pairs of triangles into quads. With my example model, it actually gave nearly a perfect result: That won't always be true, however. If you choose Select->By Number of Verts from Blender's 3D view menu and then select 3 in the T-panel (on the left side of the 3D view), it will highlight any remaining triangles. If you're on < 2.63, instead choose Select->Tris. You can join triangles into quads by selecting two adjoining triangles and hitting the F key. If there are lots of triangles that aren't next to each other, you may need to manually convert from quads to tris, which means going into Face mode and selecting pairs of triangles and pressing the F key for each pair, or selecting small groups of triangles and trying Alt-J to see if the algorithm can handle the smaller area. It usually does well with square-shaped areas without any oblique triangles. It also seems to do a better job with smaller areas in general, so before trying to do every quad manually, try Alt-J on subsections of the mesh. Once you have your mesh of quads (an occasional triangle may not be a problem, especially if they're in out-of-the-way places), go back to Object mode (tab key), then go to the properties panel and select the Modifier icon - the one that looks like a wrench. It should be pretty empty, with just a single control for adding new modifiers: Click the Add Modifier control and from the popup, choose Multiresolution. The new modifier will look like this: Press the Subdivide button. Every press of the this button will increase the density of the mesh. The higher the density, the more detail it can hold, but the more memory and processing power it will require to work with. I suggest pressing once or twice to start with, and then later, subdividing more if you need more detail. I'd also recommend hitting the Save External button and specifying a .btx file location. This will move the high-resolution mesh data out of your .blend file into its own file and will also allow Blender to remove that high-resolution mesh data from memory when it's not needed. Next, hit the Add Modifier control again, and this time, choose Displace. It will look like this: To configure, first find the Texture: dropdown, select the normal map you want to use. Leave the Direction: control set to Normal. Leave the Vertex Group: control empty, and under Texture Coordinates, select UV. Next, select the model's UV map from the UV Map: control. Don't panic, it'll probably look kind of crazy right now, sorta like this: That's because the default value displaces WAY too much. Take the Strength: control value down until it looks right. The exact scale will depend on the model, but I find .05 to be a good starting point. You shouldn't need to adjust the "Midlevel:" value for most models, but feel free to play with it to see if you like the result better. Midlevel identifies what value in the normal map means no displacement (not away from nor towards the camera) and 0.5 is usually right. Now, we're getting somewhere: Not quite done, but we're getting there. Now it's time to figure out how much detail you want (or your computer can handle). Subdivide until you're happy. If you go to far, use the left arrow on the Preview: control to go to a lower number, then hit the Delete Higher button. You may want to further tweak the Strength: control on the Displace modifier as you move to a higher density mesh. Once you're happy with what you have, you have to press the Apply button on the Displace modifier (but NOT on the Multires modifier). Currently, Blender Sculpt mode can't work with Displace modifier data, so it has to be applied before you can sculpt. And that's it. You now have a higher density mesh displaced according to the normal map data, so go into sculpt mode and knock yourself out. If you want to create a new Normal map from your changed sculpt mesh so your changes can be used in GMod or another game engine, just follow one of the may tutorials on the web for baking normal maps in Blender. Note: you'll want to disable or remove the normal map texture from your material once you've gone through the process above, otherwise your renders will include both the normal map and sculpt data. Here's a comparison showing a render using the normal map and one using the derived sculpt data (looking at this, I realize I probably should have bumped the Strength: up just a little more): The derived sculpt data will sometimes be a little blocky, especially if the normal maps are smaller (e.g. 512x512). You can improve the quality by importing the normal map into Photoshop or Gimp and scaling it up 2x it's original size and using that instead of the original normal map for the Displace modifier Even though we're just interpolating the image data up, it still results in smoother, less blocky sculpt data. Note 2: This same technique can be used with a Subdivision Surface modifier instead of a Multiresolution modifier, however doing so is going to require a lot more processing power. The Sculpt mode is optimized for working with very high density meshes; the edit mesh system is not.