Semantic theory of survey response
Semantic Theory of Survey Response
Acronym
STSR
Alternate name(s)
Semantic Theory of Survey Responses; Manifest Validity
Main dependent construct(s)/factor(s)
None
Main independent construct(s)/factor(s)
None
Concise description of theory
The Semantic Theory of Survey Response (STSR) is a psychological theory about how people respond to survey questionnaires with so-called Likert-scale response categories.
Background
The basic tenet of STSR is that people will respond to a question by judging how similar it is to other questions in the survey. This theory is contrary to other theories of responses to survey questionnaires. Originally, Rensis Likert (1932) introduced the now well-known technique of making people respond to the questions by using a five-point response category of the degree to which they agree or disagree with the proposition in the survey item. Such responses can readily be changed to numerical values on scales typically from 1 through 5 or 1 to 7 and subsequently be treated statistically. The most prevalent statistical methods will treat the ensuing numbers as an expression of the respondents’ attitudes to the topics in the items, so-called «attitude strength». Modern social science has developed a variety of very advanced statistical methods to explore the systematic patterns in people’s attitudes. Examples of areas where such modelling occurs is in social psychology, management, marketing, or information systems adoption.
In contrast to the original views on survey responses, STSR proposes that when people respond to a questionnaire, they first need to understand the meaning of the presented items. When the data from the responses are treated statistically, the information about attitude strength is lost and is instead replaced by information about how similar the respondents perceive the items to be. This is determined by their semantic overlap, hence the name of the theory. This matters because the semantic patterns are given a priori – the statistics are principally possible to foresee prior to having anyone respond. The statistical information is basically redundant if it is obtainable without the survey responses from actual human subjects. STSR was empirically supported for the first time in 2008 under the name Manifest Validity (Larsen et al. 2008), when digital semantic algorithms were found to predict the actual response patterns in a series of well-known surveys in the Information Systems discipline. The major theoretical arguments for STSR were made in 2014 by Arnulf et al. That study found support for STSR in surveys concerning leadership, motivation and other topics in organizational psychology. However, the same study found little or no support for the predictability of a personality test based on the five-factor theory. These findings were later independently supported by Nimon et al. (2015). A recent study by Gefen and Larsen (2017) in JAIS showed how to integrate STSR evidence with Structural Equation Models, and showed that the Technology Acceptance Model is almost entirely explainable through STSR.
Methodological Background
The possibility to test STSR in practice occurred through the development of digital text analysis as e.g., Latent Semantic Analysis (LSA) [4]. These are computerized techniques for establishing quantitative estimates of the similarity in meaning between pairs of texts, as in survey items. A simple form of LSA is available to the public by the University of Colorado [1]. An exposition on the algorithms likely to have implications for STSR may be found in Larsen and Bong (2016) [2].
Practical Consequences
The theory has a number of practical consequences. For example, it can be shown that the concept of «leaders» and «heroes» have so much in common semantically that questionnaires about leadership seem prone to capture exaggerated and omnipotent ideas about leaders (Arnulf and Larsen 2015). Further, the theory offers a powerful way of predicting responses that have not yet happened, allowing quite precise estimates of missing responses where up to 75% of the responses are missing (Arnulf et al. 2018). For such reasons, the theory raises doubt about whether statistical techniques for the modelling of survey data are actually dealing with attitude strength. It seems instead that statistical models based on correlations and co-variances may describe cognitive processes involved in language parsing.[7]
Precursors to STSR
Feldman & Lynch (1988) found that respondents frequently calculate attitudes to which they had no previous conceptions from other, similar topics involved in the questionnaire. Coombs and Kao (1960) developed the so-called "unfolding theory", showing that statistical patterns from surveys are dependent on the mutual logical «unfolding» of responses relative to the intial response to one question. To express reliably different attitudes to the same topic, the respondents need to agree on the meaning of the items. Based on this theory, Michell (1994) could show that survey responses are distributed precisely as expected, based on purely semantic criteria.
At the end of the 19th century, the German philosopher and logician Gottlob Frege (1884) used formal logics to show how to sentences may have overlapping meaning despite containing no similar words. An exposition on Frege's work exists in more accessible form in Blanchette (2012). The human brain is effortlessly recognizing whether words and sentences in surveys share overlapping meaning. It remains to be seen if and in which areas STSR can claim a position as the most likely explanation for statistical patterns in data from survey items. This is of importance to social science because statistical modelling of survey data is one of the most important quantitative methods in contemporary social science.
Originating author(s)
J. Ketil Arnulf and Kai R. Larsen
Seminal articles
Larsen, Kai R., Dorit Nevo, and Eliot Rich (2008), Exploring the Semantic Validity of Questionnaire Scales, Proceedings of the 41st Hawaii International Conference on System Sciences, Waikoloa, Hawaii, January 7-10, pp. 1-10.
Arnulf, J.K., Kai R. Larsen, Øyvind Lund Martinsen, Chih How Bong (2014), Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour, PLOS One, 9(9)
David Gefen and Kai R. Larsen (2017), Controlling for Lexical Closeness in Survey Research: A Demonstration on the Technology Acceptance Model, Journal of the Association for Information Systems, 18(10), pp. 727-757. (ISSN 1536-9323)
Originating area
Information Systems; Management
Level of analysis
From individual to theory.
Articles that Use or Extend the Theory
Nimon, K., Shuck, B., & Zigarmi, D. (2015). Construct Overlap Between Employee Engagement and Job Satisfaction: A Function of Semantic Equivalence? Journal of Happiness Studies, 1-23. doi:10.1007/s10902-015-9636-6.
Arnulf, J. K., & Larsen, K. R. (2015). Overlapping semantics of leadership and heroism: Expectations of omnipotence, identification with ideal leaders and disappointment in real managers. Scandinavian Psychologist, 2(e3).
Arnulf, J.K., Larsen, K.R., Martinsen, Ø.L. (2018). Respondent Robotics: Simulating responses to Likert-scale survey items. Sage Open. [3]
References
Feldman, J. M. and J. G. J. Lynch (1988). "Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior." Journal of Applied Psychology 73(3): 421-435.
Coombs, C. H. and R. C. Kao (1960). "On a connection between factor analysis and multidimensional unfolding." Psychometrika 25: 219-231.
Michell, J. (1994). Measuring dimensions of belief by unidimensional unfolding. Journal of mathematical psychology, 38(2), 244-273.
Frege, G. (1884). Die Grundlagen der Arithmetik: eine logisch-mathematische Untersuchung über den Begriff der Zahl. Breslau, W. Koebner.
Blanchette, P. A. (2012). Frege's conception of logic. Oxford, Oxford University Press.
Larsen, Kai R. and Bong C.H. (2016): A Tool for addressing Construct Identity in Literature Reviews and Meta-Analyses. MIS Quarterly, 40(3), pp. 529-551; A1-A21. (ISSN 2162-9730).
Links from this theory to other theories
Adaptive structuration theory, Organizational knowledge creation
External links
Internomological Network (INN) -- System by Kai R. Larsen and Chih How Bong at [4]
TheoryOn -- System by Jingjing Li and Kai R. Larsen at [5]
Please feel free to make modifications to this site. In order to do so, you must register.
Return to Theories Used in IS Research