An increase in the number of unique meshes will increase load time. This means the more/bigger mdl files you have in your level the longer it will take. If you have many objects all using the same mdl file and the same textures the load time will not increase.

If your map is not made up of all models, but uses the map compiler then you either have to reduce the lightmap size or the amount of polygons in the level mesh. Level meshes are all individually stored in memory. Even if you duplicate a block and leave it unchanged, there will be two of those blocks stored in memory.