Semantic tone of research ‘environment’ submissions in the UK’s Research Evaluation Framework 2014

Andy Thorpe, Russell Craig, Glenn Hadikin, S. Batistič

Research output: Contribution to journalArticleScientificpeer-review

Abstract

This article applies DICTION computer-assisted text analysis software to evaluate the tone of research ‘Environment’ submissions by Business and Management Studies schools in the UK’s 2014 Research Evaluation Framework. We find that submissions contain distinctive differences in semantic tone between high-ranked and low-ranked universities, particularly in terms of DICTION’s master variable, ACTIVITY. The language of high-ranked institutions has a tone of low ACTIVITY, whereas the language of low-ranked institutions has a tone of high ACTIVITY. More adjectives are used than expected: by high-ranked universities to bolster strong public reputations, and by low-ranked universities to atone for weaknesses. High-ranked universities are advantaged because they are more likely to be represented on assessing panels and be better-attuned to reader expectations. The results suggest that low-ranked universities could have achieved higher scores by reflecting on particular areas of word choice and the potential effects of those choices on assessors.
Original languageEnglish
Pages (from-to)53–62
JournalResearch Evaluation
Volume27
Issue number2
DOIs
Publication statusPublished - 2018

Fingerprint

evaluation research
semantics
university
text analysis
language
reputation
management
school

Keywords

  • RESEARCH QUALITY
  • UNIVERSITY
  • environment
  • evaluation
  • research
  • submissions
  • text analysis
  • tone

Cite this

@article{22ca0bd1c56749a8a8b39e25e685068d,
title = "Semantic tone of research ‘environment’ submissions in the UK’s Research Evaluation Framework 2014",
abstract = "This article applies DICTION computer-assisted text analysis software to evaluate the tone of research ‘Environment’ submissions by Business and Management Studies schools in the UK’s 2014 Research Evaluation Framework. We find that submissions contain distinctive differences in semantic tone between high-ranked and low-ranked universities, particularly in terms of DICTION’s master variable, ACTIVITY. The language of high-ranked institutions has a tone of low ACTIVITY, whereas the language of low-ranked institutions has a tone of high ACTIVITY. More adjectives are used than expected: by high-ranked universities to bolster strong public reputations, and by low-ranked universities to atone for weaknesses. High-ranked universities are advantaged because they are more likely to be represented on assessing panels and be better-attuned to reader expectations. The results suggest that low-ranked universities could have achieved higher scores by reflecting on particular areas of word choice and the potential effects of those choices on assessors.",
keywords = "RESEARCH QUALITY, UNIVERSITY, environment, evaluation, research, submissions, text analysis, tone",
author = "Andy Thorpe and Russell Craig and Glenn Hadikin and S. Batistič",
year = "2018",
doi = "10.1093/reseval/rvx039",
language = "English",
volume = "27",
pages = "53–62",
journal = "Research Evaluation",
issn = "0958-2029",
publisher = "Oxford University Press",
number = "2",

}

Semantic tone of research ‘environment’ submissions in the UK’s Research Evaluation Framework 2014. / Thorpe, Andy ; Craig, Russell; Hadikin, Glenn; Batistič, S.

In: Research Evaluation, Vol. 27, No. 2, 2018, p. 53–62.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Semantic tone of research ‘environment’ submissions in the UK’s Research Evaluation Framework 2014

AU - Thorpe, Andy

AU - Craig, Russell

AU - Hadikin, Glenn

AU - Batistič, S.

PY - 2018

Y1 - 2018

N2 - This article applies DICTION computer-assisted text analysis software to evaluate the tone of research ‘Environment’ submissions by Business and Management Studies schools in the UK’s 2014 Research Evaluation Framework. We find that submissions contain distinctive differences in semantic tone between high-ranked and low-ranked universities, particularly in terms of DICTION’s master variable, ACTIVITY. The language of high-ranked institutions has a tone of low ACTIVITY, whereas the language of low-ranked institutions has a tone of high ACTIVITY. More adjectives are used than expected: by high-ranked universities to bolster strong public reputations, and by low-ranked universities to atone for weaknesses. High-ranked universities are advantaged because they are more likely to be represented on assessing panels and be better-attuned to reader expectations. The results suggest that low-ranked universities could have achieved higher scores by reflecting on particular areas of word choice and the potential effects of those choices on assessors.

AB - This article applies DICTION computer-assisted text analysis software to evaluate the tone of research ‘Environment’ submissions by Business and Management Studies schools in the UK’s 2014 Research Evaluation Framework. We find that submissions contain distinctive differences in semantic tone between high-ranked and low-ranked universities, particularly in terms of DICTION’s master variable, ACTIVITY. The language of high-ranked institutions has a tone of low ACTIVITY, whereas the language of low-ranked institutions has a tone of high ACTIVITY. More adjectives are used than expected: by high-ranked universities to bolster strong public reputations, and by low-ranked universities to atone for weaknesses. High-ranked universities are advantaged because they are more likely to be represented on assessing panels and be better-attuned to reader expectations. The results suggest that low-ranked universities could have achieved higher scores by reflecting on particular areas of word choice and the potential effects of those choices on assessors.

KW - RESEARCH QUALITY

KW - UNIVERSITY

KW - environment

KW - evaluation

KW - research

KW - submissions

KW - text analysis

KW - tone

U2 - 10.1093/reseval/rvx039

DO - 10.1093/reseval/rvx039

M3 - Article

VL - 27

SP - 53

EP - 62

JO - Research Evaluation

JF - Research Evaluation

SN - 0958-2029

IS - 2

ER -