Although the rapid development of deep generative models (DGM) enables diverse applications of content creation, increasing illegal uses of the technologies also severely threaten the privacy and security of personal information, especially for faces. Several previous works have been proposed to leverage adversarial attacks to fight against these malicious manipulations by adding an imperceptible perturbation to each input image to disrupt the output. In addition, to improve its scalability, a sequential cross-model universal perturbation attack has been proposed to learn a common adversarial perturbation to defend the images from the manipulation of multiple DGMs. However, we find that the order of DGMs for the adversarial perturbation generation does matter and influence the final defense performance. To address this issue, we propose to generate the universal perturbation through joint optimization of multiple DGMs. From the extensive experimental results, we find that the universal perturbation generated by the proposed method can successfully disrupt the output faces of multiple DGMs at the same time and achieves higher attack success rates than the previous state-of-the-art method based on the sequential generation, even under the situations where the model robustness of DGMs are enhanced by random perturbations.