I don't want to be nitpicking, but you're misunderstanding the concept of potential and actual infinity. This concept was introduced by Aristotle, and has nothing to do with whether an infinite quantity has a "beginning" or not.
An example of potential infinity is the sequence of real numbers. An example of actual infinity is the cardinal number of the set of all real numbers. Another actual infinity is the famous number "Omega", the smallest number that is greater than any finite number.
Infinities in nature, like space and time, are usually treated as actual infinities. Space and time are similar in modern physics - general relativity makes no basic distinction between them. If you had some starting point for space, or for time, they'd still be actual infinities, as long as there's no ending point.
http://en.wikipedia.org/wiki/Infinityhttp://en.wikipedia.org/wiki/Georg_CantorAnd, just FYI: negative sizes are used in physics all the time. The time-space metric becomes negative along certain curves, for instance movement faster than light.
