Label imbalance and data missing are two major challenges in the problem of credit card fraud detection. However, existing matrix completion algorithms are generally difficult and cannot be easily applied to real-world credit card fraud detection since the scale of the normally used dataset is oversized. In this paper, we develop a spectral regularization algorithm to complete the large-scale sparse matrices, and further utilize an over-sampling algorithm to tackle the problem of the imbalance between positive and negative samples. Experimental results on a real-world dataset demonstrate that our model can outperform the state-of-the-art baseline methods. The proposed method could also be extended to other large-scale scenarios where data is missing or labels are imbalanced.