Objective To investigate the longitudinal characteristics of the brain gray matter atrophy of mild cognitive impairment (MCI) patients in comparison with the normal control (NC) group over time, and further explore the differences of the brain gray matter atrophy between NC, stable MCI (SMCI) and progressive MCI (PMCI), for providing relevant imaging parameters for diagnosis and treatment evaluation. Methods Firstly, NC, SMCI, PMCI 5-year follow-up magnetic resonance T1-weighted image data obtained from the Alzheimer disease neuroimaging initiative (ADNI) were analyzed based on the improved VBM-DARTEL method. Then, the processed longitudinal follow-up data of the three groups were analyzed by ANOVA, and paired t-test which was relative to the baseline time point separately to explore brain structure changes of each group at different time points. Results With the time changing, gray matter atrophy appeared within the global brain gray matter in NC, PMCI and SMCI, and the atrophy regions were gradually expanding. The speed of atrophy in PMCI was the fastest, and then was SMCI. The atrophy regions mainly located in the temporal lobe, hippocampus, occipital lobe, cingulate gyrus, etc. Conclusions Our results showed that the brain gray matter changes over time were more obvious in MCI. Compared to SMCI (low probability of converting to AD), more obvious atrophy appeared in certain regions of gray matter in PMCI (high probability of converting to AD). Thus the atrophy differences between SMCI and PMCI could be the evidence to identify PMCI, and was helpful to clinical diagnosis, intervention and treatment of early AD.