Hi EvilSOB, and thanks for your advice!

Joining the PSUs together seems like a bad idea to me. While I did what I already tried I acidentally kinda did already join them together once, while only one of them was running:
The GTX 570 came with adapters from two 4-pin connectors (like for IDE HDDs) to a 6-pin PICe connector. I connected only one of the 4-pin connectors to the new PSU while the other one was still connected to the old one. I turned the external PSU on while it was like this and a case fan began running, making me aware that thereby I connected the 4-pin connector of one PSU to the one of the other. I turned the external PSU off immediately. After that, I tried to turn my computer on, but it didn't react. Only after I kept the normal PSU unplugged for like ten seconds and turned it back on, it worked again. It seems like there was no permanent damage, but it scared the hell out of me.
(Also, this kinda reminds me of old Märklin (a model railroad manufacturer) transformers which had the warning not to connect them EVER. Because apparently, the transformers also worked backwards, so if you'd unplug only one of them, you'd have 230V on the plug! And when I say old, I mean that I bought one of these in 2003 and they were not obsolete...)

About the keeping a harddisk connected idea:
That sounds like a good idea. I'm going to try that. Propably not today anymore, but tomorrow. Thanks!



EDIT: Btw, I don't think it's because some of the power from the external PSU gets into the mainboard, because it behaves the same when I don't turn it on before I start my computer.