The real speed comes when you have to do math with 64-bit numbers.

1+1 is the same speed with both 32 and 64-bit but
18,446,744,073,709,551,615 + 18,446,744,073,709,551,615 is almost twice as fast using 64-bits (2 reads vs. 4 reads).