Well yeah way too glossy (except the stonewall, that one looks awesome). Adjust your specularmaps and you should be set
Physics are great indeed.
Oh and btw are you using dynamic shadowmaps? At 0:54 there are some shadowgliches that look like the typical problems of a shadowmappingshader. Nothing bad about it, just curious