Okay, let's talk about your camera code. That used to be my specialty, anyhow.

You set camera.x to the player's position (only not caring about z - probably NOT a wise idea). Then, you calculate another camera position that lies 30 quants above the player. Note that if your player's tilt (and roll) is 0 (as it is in most games), the vec_rotate line will have no effect (apart from making the game slower).
Then, you lerp between those position. The net effect is that you lerp between two coordinates, one taking care of x/y, and the other of z. That's an unusual behaviour.
If you want a first-person perspective, you probably want to go:
vec_set(camera.x,my.x);
vec_set(camera.pan,my.pan);
camera.z += SOMETHINGSOMETHING; //for instance, "0.9*my.max_z", or 30 in your example
(at least for the simple case of tilt=roll=0).
If you want to smooth it, then simply smooth it!

vec_set(cam_pos.x,my.x);
cam_pos.z += 30;
vec_diff(temp,cam_pos,camera.x);
vec_scale(temp,clamp(1-0.5*time_step,0,1)); //0.5 being camera speed
//ALTERNATIVELY, if you prefer a constant speed:
//vec_normalize(temp,minv(vec_length(temp),10*time_step)); //10 being camera speed
vec_add(camera.x,temp);
//and something similar with the angles:
camera.pan -= ang(camera.pan-my.pan)*minv(1,0.5*time_step);
camera.tilt = 0;
camera.roll = 0;
I've assumed that your tilt and roll is always zero.
EDIT: I realized later with a creeping horror I didn't talk much about what caused the problem in the first place, and my disappointing take on it is "it's complicated".

It's a bit hard to name the one detail that caused it in your game, but your way of handling camera movement still seems a bit strange to me, it's two different ways to do it put together. Since I don#t know the exact placement of the camera code, I'd rather guess that what you see is a relatively accurate "jerking" of the z-coordinate that can come about when you do several micro-adjustments, that tend to change in value, rather strongly so in relative terms. But that's a wild guess, and probably a bad one at that. What kind of "jerking" are we talkin' here?
~
Sidenote:
With the line
camera.z -= (camera.z - cam_pos.z) * time_step;
Remember that for very LOW framerates (<16), time_step is actually bigger than one, and you end up overshooting. So you'll want to correct that, for instance by using minv(1,time_step) - but that's just an aside.