Does the Choice of a Segmentation Algorithm Affect the Performance of Text Classifiers?

Text segmentation, a pre-processing step present in most
NLP pipelines, is a rich topic that tends to get little attention in most
NLP studies. We provide a brief empirical account on whether the choice
of a particular segmentation algorithm (BPE, SentencePiece, WordPiece,
Unigram, Morfessor) has an effect on the performance on downstream
tasks, without transfer learning: natural language inference (SNLI), and
sentiment classification (SST2). In contrast to previous studies which
have compared BPE to Unigram segmentations on 4 downstream tasks,
or language models built with BPE and morphologically aligned segmen-
tations, this paper provides a comparison between 5 common segmen-
tation algorithms and runs experiments with the algorithms fitted on
20 random subsets of the training corpus. In all experiments, we use a
simple LSTM-based artificial neural network (in a Siamese setup for the
SNLI) in either binary, or three-way classification set-up. The resulting
accuracy data across the segmentation algorithms do not conclusively fa-
vor any particular one of them, in contrast to what the related literature
suggests.