Text segmentation with character-level text embeddings

    Research output: Contribution to conferencePaperOther research output

    Abstract

    Learning word representations has recently seen much success in computational linguistics. However, assuming sequences of word tokens as input to linguistic analysis is often unjustified. For many languages word segmentation is a non-trivial task and naturally occurring text is sometimes a mixture of natural language strings and other character data. We propose to learn text representations directly from raw character sequences by training a Simple recurrent Network to predict the next character in text. The network uses its hidden layer to evolve abstract representations of the character sequences it sees. To demonstrate the usefulness of the learned text embeddings, we use them as features in a supervised character level text segmentation and labeling task: recognizing spans of text containing programming language code. By using the embeddings as features we are able to substantially improve over a baseline which uses only surface character n-grams.
    Original languageEnglish
    Publication statusPublished - 18 Sep 2013
    EventWorkshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013 - Atlanta, United States
    Duration: 16 Jun 2013 → …

    Workshop

    WorkshopWorkshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013
    CountryUnited States
    CityAtlanta
    Period16/06/13 → …

    Fingerprint

    Computational linguistics
    Linguistics
    Computer programming languages
    Labeling

    Cite this

    Chrupała, G. (2013). Text segmentation with character-level text embeddings. Paper presented at Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013, Atlanta, United States.
    Chrupała, Grzegorz. / Text segmentation with character-level text embeddings. Paper presented at Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013, Atlanta, United States.
    @conference{371727ab8c894b2a928c80d9d6f16ff6,
    title = "Text segmentation with character-level text embeddings",
    abstract = "Learning word representations has recently seen much success in computational linguistics. However, assuming sequences of word tokens as input to linguistic analysis is often unjustified. For many languages word segmentation is a non-trivial task and naturally occurring text is sometimes a mixture of natural language strings and other character data. We propose to learn text representations directly from raw character sequences by training a Simple recurrent Network to predict the next character in text. The network uses its hidden layer to evolve abstract representations of the character sequences it sees. To demonstrate the usefulness of the learned text embeddings, we use them as features in a supervised character level text segmentation and labeling task: recognizing spans of text containing programming language code. By using the embeddings as features we are able to substantially improve over a baseline which uses only surface character n-grams.",
    author = "Grzegorz Chrupała",
    note = "Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013; Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013 ; Conference date: 16-06-2013",
    year = "2013",
    month = "9",
    day = "18",
    language = "English",

    }

    Chrupała, G 2013, 'Text segmentation with character-level text embeddings' Paper presented at Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013, Atlanta, United States, 16/06/13, .

    Text segmentation with character-level text embeddings. / Chrupała, Grzegorz.

    2013. Paper presented at Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013, Atlanta, United States.

    Research output: Contribution to conferencePaperOther research output

    TY - CONF

    T1 - Text segmentation with character-level text embeddings

    AU - Chrupała, Grzegorz

    N1 - Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013

    PY - 2013/9/18

    Y1 - 2013/9/18

    N2 - Learning word representations has recently seen much success in computational linguistics. However, assuming sequences of word tokens as input to linguistic analysis is often unjustified. For many languages word segmentation is a non-trivial task and naturally occurring text is sometimes a mixture of natural language strings and other character data. We propose to learn text representations directly from raw character sequences by training a Simple recurrent Network to predict the next character in text. The network uses its hidden layer to evolve abstract representations of the character sequences it sees. To demonstrate the usefulness of the learned text embeddings, we use them as features in a supervised character level text segmentation and labeling task: recognizing spans of text containing programming language code. By using the embeddings as features we are able to substantially improve over a baseline which uses only surface character n-grams.

    AB - Learning word representations has recently seen much success in computational linguistics. However, assuming sequences of word tokens as input to linguistic analysis is often unjustified. For many languages word segmentation is a non-trivial task and naturally occurring text is sometimes a mixture of natural language strings and other character data. We propose to learn text representations directly from raw character sequences by training a Simple recurrent Network to predict the next character in text. The network uses its hidden layer to evolve abstract representations of the character sequences it sees. To demonstrate the usefulness of the learned text embeddings, we use them as features in a supervised character level text segmentation and labeling task: recognizing spans of text containing programming language code. By using the embeddings as features we are able to substantially improve over a baseline which uses only surface character n-grams.

    M3 - Paper

    ER -

    Chrupała G. Text segmentation with character-level text embeddings. 2013. Paper presented at Workshop on Deep Learning for Audio, Speech and Language Processing, ICML 2013, Atlanta, United States.