Can human judgement be machine-sourced? An approach to measure the perceptual dimensions embedded in software

Poonacha Medappa*, Shirish C. Srivastava

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Considering the challenges of measuring the perceptual dimensions of the IT artifact, we propose a computational method for developing latent concepts through the lines of code that make up a software artifact. The proposed “machine-sourcing human judgement” approach, combines a novel technique of extracting the semantic properties (meanings) of the code from the software engineering literature with the machine-learning techniques used in the IS literature. Using the illustration of open source software (OSS), we demonstrate that the ‘contextual’ and ‘economic’ limitations of evaluating the creativity of OSS code contributions can be overcome through our approach. The performance of the proposed approach is tested by using a labelled dataset of code contributions created by two experienced OSS developers. We find that our approach of using semantic properties from the “software code” matches in performance to evaluating “textual descriptions” of the code. Potential methodological improvements and future research opportunities are also discussed.
Original languageEnglish
Title of host publicationProceedings of the ICIS 2020
Subtitle of host publicationMaking Digital Inclusive: Blending the Local and the Global
PublisherAssociation for Information Systems
Number of pages9
Publication statusPublished - 14 Dec 2020
Event41st International Conference on Information Systems - online, India
Duration: 13 Dec 202016 Dec 2020


Conference41st International Conference on Information Systems
Abbreviated titleICIS 2020


  • Computational methods
  • machine learning
  • open source software
  • software artifact
  • creativity of code


Dive into the research topics of 'Can human judgement be machine-sourced? An approach to measure the perceptual dimensions embedded in software'. Together they form a unique fingerprint.

Cite this