Convolutional Neural Network (CNN) has been widely proposed for different tasks of heart sound analysis. This paper presents the results of a novel study on the performance of a conventional CNN in comparison to the different architectures of recurrent neural networks combined with CNN for the classification task of abnormal-normal heart sounds. The study considers various combinations of parallel and cascaded integration of CNN with Gated Recurrent Network (GRN) as well as Long- Short Term Memory (LSTM) and explores the accuracy and sensitivity of each integration independently, using the Physionet dataset of heart sound recordings. The accuracy of the parallel architecture of LSTM-CNN reached 98.0% outperforming all the combined architectures, with a sensitivity of 87.2%. The conventional CNN offered sensitivity/accuracy of 95.9%/97.3% with far less complexity. Results show that a conventional CNN can appropriately perform and solely employed for the classification of heart sound signals.