deepQuest: A Framework for Neural-based Quality Estimation

Julia Ive, Frédéric Blain, Lucia Specia

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Predicting Machine Translation (MT) quality can help in many practical tasks such as MT post-editing. The performance of Quality Estimation (QE) methods has drastically improved recently with the introduction of neural approaches to the problem. However, thus far neural approaches have only been designed for word and sentence-level prediction. We present a neural framework that is able to accommodate neural QE approaches at these fine-grained levels and generalize them to the level of documents. We test the framework with two sentence-level neural QE approaches: a state of the art approach that requires extensive pre-training, and a new light-weight approach that we propose, which employs basic encoders. Our approach is significantly faster and yields performance improvements for a range of document-level quality estimation tasks. To our knowledge, this is the first neural architecture for document-level QE. In addition, for the first time we apply QE models to the output of both statistical and neural MT systems for a series of European languages and highlight the new challenges resulting from the use of neural MT.
Original languageEnglish
Title of host publicationProceedings of the 27th International Conference on Computational Linguistics
EditorsEmily M. Bender, Leon Derczynski, Pierre Isabelle
Place of PublicationSanta Fe, New Mexico, USA
PublisherAssociation for Computational Linguistics
Number of pages12
ISBN (Print)978-1-948087-50-6
Publication statusPublished - 1 Aug 2018
Externally publishedYes
EventProceedings of the 27th International Conference on Computational Linguistics - Santa Fe, United States
Duration: 20 Aug 201826 Aug 2023


ConferenceProceedings of the 27th International Conference on Computational Linguistics
Abbreviated titleCOLING 2018
Country/TerritoryUnited States
CitySanta Fe


  • Quality estimation
  • Machine Translation
  • Neural approaches


Dive into the research topics of 'deepQuest: A Framework for Neural-based Quality Estimation'. Together they form a unique fingerprint.

Cite this