it has nothing to do with the normal map (that would help a little but its not the only reason). the reason is that the tangent is calculated using the uv, and seams mess that up.