This paper proposes a novel voice activity detector (VAD) based on singular value decomposition (SVD). The spectro-temporal characteristics of background noise region can be easily analyzed by SVD. The proposed method naturally drops hangover algorithm from VAD. Moreover, it adaptively changes the decision threshold by employing the most dominant singular value of the observation matrix in the noise region. According to simulation results, the proposed VAD shows significantly better performance than the conventional statistical model-based method and is less sensitive to the environmental changes. In addition, the proposed algorithm requires very low computational cost compared with other algorithms.