24,000+ portals is very, very high for a level and, as you are finding out, will take ages to build at a full build. Most of my levels are between 3,000 - 6,000+ portals. The result is that they build fairly quickly on a P4 (and the levels I am talking about are very detailed).
There are a few things you can do to speed things up depending on what you are after. I'll state the obvious ones as well

:
Remove truly unneeded geometryLook through your level and consider all that you have put in it. Ask yourself what the purpose of each area is and what the player will be doing there. Then ask if the player will really notice that detail or the other. If not, then cut it. If the player, for instance, is to be running through a hall and fighting off monsters, then his attention will not be on the fine details there. Cut them from the level.
Some details can be textures and not geometryIf you are modeling every little lightswitch, then don't. Create a texture or a sprite for it and use that instead. Take a look through your level and see what can be done just as well in a texture or with a sprite and use them where you can.
Use map entitiesYou can divide up your level into smaller parts, build those parts as map entities and then assemble the whole thing together again as a whole map. This will greatly decrease your build times overall. You need to be careful about a few things (like where lighting crosses over from one map entity to the next) but this is a great method to build complex levels and you can even use LOD and other tricks to speed up the level while playing it.
Split up the levelOf course you can always split up the level into more than one level and use C-Script to program a level change.
There are other options and perhaps someone will share more, but these should get you started.
I hope this helps!