The success of Deep Neural Networks with image classification prompted researchers to explore the applications of Deep Learning in Medical Imaging and Medical Image Analysis (MIA). Deep Neural Networks have sufficiently demonstrated their capabilities of performing MIA tasks tirelessly and with fewer errors as opposed to their human counterpart. However the challenge of training neural networks using sensitive medical data, without violating the privacy of patients remains an active field of research. Many solutions exist to address this concern, however a systematic review and analysis of these techniques is yet to be conducted. This paper attempts to conduct the first systematic review of privacy-preserving techniques to train deep learning models. Emphasis is especially put on the performance and privacy analysis of the techniques. In addition, the communication and runtime costs, the ability of the solutions to scale, tolerance to faults and the level of security against threats and attacks are also studied.