In this article, we aimed to assess the degree to which measurement invariance (MI) is tested and indeed holds in psychological studies that aim to compare scale mean scores across (experimental) groups or time. Our goals were to evaluate how often MI is tested, the quality of reporting on MI testing, and how often MI holds in a sample of psychological studies. To this end, we sampled 426 articles with open data from two psychology journals and considered 918 scale mean comparisons (SMC) reported in these articles.