ONVAR: A simultaneous component analysis approach for disentangling outlying and non-outlying variables

Multivariate column-coupled multi-block data are collected in many fields of science. Such data consist of multiple object-by-variable blocks, which all share the variable mode. To summarize the main information in such data, principal component analysis per block (separate PCAs) and simultaneous component analysis (SCA) across blocks are highly popular. The main difference is that, with separate PCAs, the loadings can differ from block to block, whereas SCA restricts the loadings to be the same across the blocks. However, one often sees that most variables have similar correlation patterns across the blocks, whereas a few variables behave differently. We consider those variables as 'non-outlying' and 'outlying' respectively. For various research goals, it makes sense to model the data in blocks simultaneously but to disentangle the outlying and non-outlying variables and impose different restrictions on them. To this end, we present the new Outlying and Non-outlying Variable (ONVar) method. ONVar models all variables and all blocks simultaneously but allows to capture for which variables the loadings are the same and for which they differ. In a simulation study, we show that ONVar flags the outlying variables correctly most of the time and outperforms existing outlying variable detection heuristics. Finally, we illustrate ONVar by applying it to sensory ratings of multiple bread samples.