The issue turned out not to be the wall, but instead a high detail model in front of the camera when looking at the wall. The TGA and wall side and distance objects had zero effect, however if this detailed object was close to the camera and covering a large part of the screen, frame rates died.
