Yes, thats certainly right. It would make sense not to remove important obstacles, but only remove unimportant eye candy objects when the physics is not hardware accelerated.