Integrating source-language context into phrase-based statistical machine translation

R. Haque, S.K. Naskar, A. van den Bosch, A. Way

    Research output: Contribution to journalArticleScientificpeer-review

    Abstract

    The translation features typically used in Phrase-Based Statistical Machine Translation (PB-SMT) model dependencies between the source and target phrases, but not among the phrases in the source language themselves. A swathe of research has demonstrated that integrating source context modelling directly into log-linear PB-SMT can positively influence the weighting and selection of target phrases, and thus improve translation quality. In this contribution we present a revised, extended account of our previous work on using a range of contextual features, including lexical features of neighbouring words, supertags, and dependency information. We add a number of novel aspects, including the use of semantic roles as new contextual features in PB-SMT, adding new language pairs, and examining the scalability of our research to larger amounts of training data. While our results are mixed across feature selections, classifier hyperparameters, language pairs, and learning curves, we observe that including contextual features of the source sentence in general produces improvements. The most significant improvements involve the integration of long-distance contextual features, such as dependency relations in combination with part-of-speech tags in Dutch-to-English subtitle translation, the combination of dependency parse and semantic role information in English-to-Dutch parliamentary debate translation, or supertag features in English-to-Chinese translation.
    Original languageEnglish
    Pages (from-to)239-285
    Number of pages47
    JournalMachine Translation
    Volume23
    Issue number3
    DOIs
    Publication statusPublished - 2011

    Fingerprint

    Semantics
    language
    Scalability
    Feature extraction
    Classifiers
    semantics
    parliamentary debate
    Source Language
    Statistical Machine Translation
    weighting
    Contextual
    Semantic Roles
    Language
    learning

    Cite this

    Haque, R. ; Naskar, S.K. ; van den Bosch, A. ; Way, A. / Integrating source-language context into phrase-based statistical machine translation. In: Machine Translation. 2011 ; Vol. 23, No. 3. pp. 239-285.
    @article{95937fe45f9442f1b805fcbd4c148f90,
    title = "Integrating source-language context into phrase-based statistical machine translation",
    abstract = "The translation features typically used in Phrase-Based Statistical Machine Translation (PB-SMT) model dependencies between the source and target phrases, but not among the phrases in the source language themselves. A swathe of research has demonstrated that integrating source context modelling directly into log-linear PB-SMT can positively influence the weighting and selection of target phrases, and thus improve translation quality. In this contribution we present a revised, extended account of our previous work on using a range of contextual features, including lexical features of neighbouring words, supertags, and dependency information. We add a number of novel aspects, including the use of semantic roles as new contextual features in PB-SMT, adding new language pairs, and examining the scalability of our research to larger amounts of training data. While our results are mixed across feature selections, classifier hyperparameters, language pairs, and learning curves, we observe that including contextual features of the source sentence in general produces improvements. The most significant improvements involve the integration of long-distance contextual features, such as dependency relations in combination with part-of-speech tags in Dutch-to-English subtitle translation, the combination of dependency parse and semantic role information in English-to-Dutch parliamentary debate translation, or supertag features in English-to-Chinese translation.",
    author = "R. Haque and S.K. Naskar and {van den Bosch}, A. and A. Way",
    note = "Integrating source-language context into phrase-based statistical machine translation Pagination: 47",
    year = "2011",
    doi = "10.1007/s10590-011-9100-2",
    language = "English",
    volume = "23",
    pages = "239--285",
    journal = "Machine Translation",
    issn = "0922-6567",
    publisher = "Springer Netherlands",
    number = "3",

    }

    Integrating source-language context into phrase-based statistical machine translation. / Haque, R.; Naskar, S.K.; van den Bosch, A.; Way, A.

    In: Machine Translation, Vol. 23, No. 3, 2011, p. 239-285.

    Research output: Contribution to journalArticleScientificpeer-review

    TY - JOUR

    T1 - Integrating source-language context into phrase-based statistical machine translation

    AU - Haque, R.

    AU - Naskar, S.K.

    AU - van den Bosch, A.

    AU - Way, A.

    N1 - Integrating source-language context into phrase-based statistical machine translation Pagination: 47

    PY - 2011

    Y1 - 2011

    N2 - The translation features typically used in Phrase-Based Statistical Machine Translation (PB-SMT) model dependencies between the source and target phrases, but not among the phrases in the source language themselves. A swathe of research has demonstrated that integrating source context modelling directly into log-linear PB-SMT can positively influence the weighting and selection of target phrases, and thus improve translation quality. In this contribution we present a revised, extended account of our previous work on using a range of contextual features, including lexical features of neighbouring words, supertags, and dependency information. We add a number of novel aspects, including the use of semantic roles as new contextual features in PB-SMT, adding new language pairs, and examining the scalability of our research to larger amounts of training data. While our results are mixed across feature selections, classifier hyperparameters, language pairs, and learning curves, we observe that including contextual features of the source sentence in general produces improvements. The most significant improvements involve the integration of long-distance contextual features, such as dependency relations in combination with part-of-speech tags in Dutch-to-English subtitle translation, the combination of dependency parse and semantic role information in English-to-Dutch parliamentary debate translation, or supertag features in English-to-Chinese translation.

    AB - The translation features typically used in Phrase-Based Statistical Machine Translation (PB-SMT) model dependencies between the source and target phrases, but not among the phrases in the source language themselves. A swathe of research has demonstrated that integrating source context modelling directly into log-linear PB-SMT can positively influence the weighting and selection of target phrases, and thus improve translation quality. In this contribution we present a revised, extended account of our previous work on using a range of contextual features, including lexical features of neighbouring words, supertags, and dependency information. We add a number of novel aspects, including the use of semantic roles as new contextual features in PB-SMT, adding new language pairs, and examining the scalability of our research to larger amounts of training data. While our results are mixed across feature selections, classifier hyperparameters, language pairs, and learning curves, we observe that including contextual features of the source sentence in general produces improvements. The most significant improvements involve the integration of long-distance contextual features, such as dependency relations in combination with part-of-speech tags in Dutch-to-English subtitle translation, the combination of dependency parse and semantic role information in English-to-Dutch parliamentary debate translation, or supertag features in English-to-Chinese translation.

    U2 - 10.1007/s10590-011-9100-2

    DO - 10.1007/s10590-011-9100-2

    M3 - Article

    VL - 23

    SP - 239

    EP - 285

    JO - Machine Translation

    JF - Machine Translation

    SN - 0922-6567

    IS - 3

    ER -