Jan 2016
1 / 3
Jan 2016
Feb 2016
user_deleted
Jan '16

Hi -
I'm running into a very weird behavior with normal map baking. See the image below.
First the hipoly, lowpoly and tangeant space normal map so you can see what I'm working with.
And below that you can see the unexpected result I get when applying my normal map. Broken shading circled in red.
What are these shading errors and how to I resolve this?

I started from a hipoly zbrush mesh. Decimating in zbrush with decimation master. Calculating normal map in xnormal.
I've done everything I know to do for technically correct baking - hardening normals at UV seams, working with a projection cage, etc.
The low poly mesh and the normal map seem fine. But circled in red below you can see weird "inversions" I'm getting on certain faces. Those inversions are certainly not visible in the normal map.
There's no sign of them in the untextured shading of the low poly either. 
What could possibly be going on?

j7gfp3.jpg512x861

I've been looking into everything I can think of. Honestly at this point I am just guessing and grasping at straws!! I've done a fair amount of normal baking before but running into this artifacting on several objects and I think I'm running into the limits of my knowledge.

I'm using maya 2013 x64, on win7 x64.
My GPU is a Geforce GTX 680.
Zbrush 4R7
Xnormal 3.18.4.37652

  • created

    Jan '16

  • last reply

    Feb '16
  • 2

    replies

  • 3.0k

    views

  • 1

    user

user_deleted
Jan '16

In case this helps spark a thought for anybody:

I'm wondering if this has to do with some kind of flipped normal or tangeant direction. 
So I checked my face normals and vertex normals. They all seem OK and are all pointing in the same direction. 
When I display tangeants in maya however, tangeants seem to be rather unruly. I'm not sure if this matters. See this image:

You can see the red tangeant on two of the vertices of the faces that have a shading problem, are pointing in the opposite direction as all the others. Could this be a clue?
If I view them as vertex face tangeants, some of them seem to be pointing all over the place...

I've read that tangeants are dependent on UV space. As you can see from the normal map screen grab, the UVs are continguous, no funny business there.

Is this perhaps a vertex ordering issue?? Something in the OBJ export from Zbrush to Maya?

To compound the problem, I'm really struggling to troubleshoot this because of various things that feel like GPU bugs: 
1. whenever I select a vertex, the shading in the viewport changes a little bit, as if the normals were getting edited (even if I don't move any vertex)
2. if I resize the object by typing a new scale value into the channel box, the viewport shading completely breaks
3. In either of this case I have to close the scene and re-open it and it displays properly again.

By the way, I had read that you can't do technically correct bake from xnormal to maya without using Handplane, so I got handplane and went through all the steps, but this made no difference. It feels like the error is something more fundamental and inside maya.
Please help -- I feel like I've run up against the limits of my knowledge here on what is ACTUALLY going on with a face's normals! :slight_smile:

8 days later
chris-giusti
Feb '16

Maya being maya.. I had this problem once before too and I don't remember what resolved it... maya just has so many switches that it's likely it's just some one thing that causes it that is buried in menus. (I know this doesn't help, but all I can do without the file is ask the same questions that would maybe cause things like this: Do you have double sided on? Are the verts all welded? Is all the history cleared?)

For an example though, if you take it into another renderer and apply the normal map, does the graphic still break? I remember one time I was having issues in maya but the normal map and object worked completely fine outside of it... like you, I never found the cause.