Yes but think how dim some stars look, and these are very close by; in our own galaxy. Other galaxies themselves are not visible with the naked eye usually.

The term "infinite" is often used when we really mean "indefinite". The universe is certainly indefinite.

Think about this: the Earth is seveal billions of years old, most galaxies are likely more than this many light years away.. so light from the majority of universe has simply not reached yet anyway.

Just assuming that the sky should be brighter if the univere was infinite is a flawed approach for many reasons. In any case, the sky isnt so dim.,. have you looked at the night sky on clear night in country, or the mountains? There are a heck of lot of stars out there...not mentions all the other forms of non-visible radiation. The sky is not very dark at all.