CC BY-ND
7
National Research University Higher School of Economics Journal of Language & Education Volume 6, Issue 3, 2020
Morozova, I.S., Chusovlyankin, A.A., Smolianina, E.A., & Permyakova, T.M. (2020). The Language of Praise in Russian Students' Evaluation of Teaching. Journal of Language and Education, 6(3), 94-107. https://doi.org/10.17323/ jle.2020.9833
The Language of Praise in Russian Students' Evaluation of Teaching
Irina S. Morozova, Aleksey A. Chusovlyankin, Elena A. Smolianina,
Tatyana M. Permyakova
National Research University Higher School of Economics
Correspondence concerning this article should be addressed to Irina Morozova, National Research University Higher School of Economics, ul. Makarenko, 18-17, Perm, 614017, Russian Federation. E-mail: [email protected]
Recent decades have seen a dramatic rise in student evaluation of teaching (SET). However, they have overwhelmingly focused on quantitative ratings, neglecting students' written feedback. This study addresses the lack of qualitative research on SET by applying a semantic theory and computational methods for analysing the language of positive feedback comments provided by students of the Higher School of Economics (HSE) in Russia. Analysing a corpus of student commentary about teaching also contributes to the theory of pragmatics as the approach to analysing qualitative evaluations of teaching is based on the premise that students' positive feedback can be treated as a sort of the compliment/praise speech act reflecting cultural specificity. Our findings showed that quantitatively the most common semantic pattern used by HSE students is ACTOR + (AUGMENTOR) EVALUATOR + PHYSICAL/MENTAL ACTION PERFORMED BY THE ACTOR + (AUGMENTOR) EVALUATOR. Thus, HSE students tend to praise the teacher more often than the other components of the teaching process and the teacher's behaviour, thoughts, and feelings are viewed as more important than skills and speech.
Keywords: positive evaluation, students' feedback about teaching, semantic patterns, the Russian language, text analytics
Introduction
Student evaluation of teaching (SET) is a common assessment tool used to measure teaching effectiveness and evaluate courses at colleges and universities. Studies of SET in higher education have dramatically risen over the past several decades; however they focus on quantitative ratings, assessing the reliability and validity of these measures, or improving their design (Cheng & Marsh, 2010; Richardson, Slater, & Wilson, 2007; Spooren, Brockx, & Mortelmanset, 2013). Apart from measurement scales in SET, students are invited to write open comments. Nonetheless, they are generally not taken into account, owing to the lack and challenges of systematic analysis and processing techniques for subjective commentary despite the fact that open-ended questions are more appropriate for formative evaluation (Alhija & Fresko 2009; Huxham et al., 2008). Thus, in spite of the growing body of literature on SET, student qualitative evaluations have been under-researched and there is a lack of theoretically grounded analysis of the language of SET. Besides, in the literature, more attention is given to negative commentary or criticism and their effects on teachers while students' positive comments are particularly more frequent than negative ones and generously praise teachers (Alhija & Fresko 2009). Students seem to regard qualitative feedback as more meaningful than quantitative feedback (Davison & Price 2009).
As Grainger (2018) showed, Brown and Levinson's theory of politeness (Brown & Levinson, 1987) and Gricean pragmatics (Grice, 1975) can be useful for the analysis of different types of interaction. We suggest that the study of student positive commentary about teaching may also benefit from these theories using some concepts of traditional pragmatics (Austin, 1962; Leech, 1983; Searle, 1975). We base this on the premise that students' positive evaluation of teaching can be treated as praise or compliment speech acts (Holmes, 1988) reflecting cultural values (Herbert, 1986; Manes, 1983). It is claimed that despite being semantic universals, positive student feedback varies from one language to another, from culture to culture, and from society to society (Wierzbicka, 1991; Wolfson & Manes, 1980). Thus, the study of the language of praise in student evaluation of teaching may fill a gap in the speech act theory.
Research Articles
This article is published under the Creative Commons Attribution 4.0 International License.
Despite extensive research of compliments across cultures (Herbert, 1989; Holmes, 1988; Lin, Woodfield, & Ren, 2012; Mafz-Arevalo, 2012; Seifoori & Emadi, 2015; Tang & Zhang, 2009) and in the foreign/second language classroom (Allami & Montazeri, 2012; Bai, 2015; Holmes & Brown, 1987; Huth, 2006), there has been little attention paid to complimenting and praise between teachers and students. A few studies that investigated this issue either suggested guidelines for effective teacher praise (Brophy, 1981) or focused on praise used by teachers to soften criticisms (Hyland & Hyland, 2001). Burnett (2002) also looked into the relationships between teacher praise and students' perceptions of the classroom. Suketi (2014) analyzed students' response to the compliment given in consultation. No literature on student complimenting or praising behaviors toward teachers was found. In this study, we investigate student complimenting and praising behaviour in the context of teacher evaluation.
Students' evaluation of teachers, particularly in higher education, has become a widely discussed subject. There were attempts to determine the relationships between SET scores and such factors as students' gender (Amin, 1994; Centra & Gaubatz, 2000; Thawabieh, 2017), expected grades (Chen, Wang, & Yang, 2017; Sarwar, Dildar, & Hussain, 2017) or overall satisfaction with the course (Ahmad, 2015). Yet very few studies have systematically investigated the content of students' written feedback while students were found to be generally willing to provide such feedback (Spencer & Schmelkin, 2002), and it be both negative and demotivating as well as positive and encouraging (Light & Cox, 2001).
There are a small number of studies related to the content of student feedback (Blair & Noel, 2014; Steyn, Davies, & Sambo, 2018), but these works restrict themselves to conventional thematic analysis eliciting factors and themes of student qualitative feedback. "Attention to the ... language used has potential to reveal insight into students' relationships with teachers and the institution" (Stewart, 2015, p. 1-2). Nevertheless, only scant research exists on the language of students' feedback. For instance, Zaitseva, Milsom, and Stewart (2013) interpreted the qualitative data from student satisfaction surveys utilizing concept mapping software. Stewart
(2015) analysed the extent to which students managed language through intensification or moderation of views and highlighted the asymmetry between positive and negative feedback. Rajput, Rajput, Haider, and Ghani
(2016) presented a lexicon-based sentiment analysis of students' open-ended textual feedback using word clouds visualization techniques. Our study seeks to expand the linguistic perspective of SET as, being a window into the mind, language is one of the best ways to provide insight into underlying concepts and inform on academic and interpersonal experience.
Natural Semantic Metalanguage Theory
Linguists point out that languages are culture-specific (Humboldt, 1999; Sapir, 1929; Whorf, 1982). However, it is important to find common ground for making reliable comparison between languages, the so-called "true tertium comparationis" (Wierzbicka, 1996, p. 16). Wierzbicka's (1996) natural semantic metalanguage theory has become a noteworthy approach with considerable longevity and "a substantial output within the field of contemporary linguistics" (Goddard, 2008, p. 1).
In Wierzbicka's understanding, there are indefinable elements or absolute particles of meaning in all languages, "that are indefinable not because they are considered elementary in a particular language but in any kind of human language system" (Dziedziul, 2017, p. 256). Wierzbicka (1996) and Goddard (2008) suggest using this semantic cognitive core as a metalanguage that will allow us to compare different languages and cultures. Nevertheless, Wierzbicka (1996) points out that the natural semantic metalanguage does not imply that all languages are equivalent. The theory of elementary semantic units remains culture sensitive. It assumes the presence of words in a language, that have no direct equivalents in others, and that the superficial unique elements can be compared by extracting the elementary particles of meaning shared by all languages. "In plain words: it is not the meaning that is unique, but the sequence of universal elements that stand behind the word" (Dziedziul, 2017, p. 257)
The palette of the elementary units of meaning has been changing. However, despite wild fluctuations, their number has not exceeded 100 items (Tabakowska, 2001). Wierzbicka (1996) outlined the following elementary units of meaning (See Table 1).
Table 1
Elementary Units of Meaning
Elementary Unit
Word
Substantives
Determiners
Quantifiers
Mental predicates
Non-mental predicates
Space
Speech
Actions and events Evaluators Descriptors Time
Partonomy/ taxonomy Metapredicates Intercausal links Imagination/possibility Words
I, you, someone, something, people, etc.
this, the same, other, etc.
one, two, many, much, all, etc.
think, know, want, feel, see, etc.
move, there is, be alive, etc.
far, inside, here, where, under, etc.
Say
do, happen, etc.
Good
Big
after, along, before, now, when, etc.
part of, kind of, etc.
can, not, very, etc.
because, if, like, etc.
maybe, would, etc.
Say
The present study aims to address the shortcomings of qualitative research on SET by applying a semantic theory and corpus-driven methodology. The research question to be answered is: What semantic patterns are commonly used by HSE students in their positive written comments about teaching?
This study describes semantic patterns used by students of the Higher School of Economics in Russia when presenting praise in an institutional satisfaction survey.
In our study, the data analysed were drawn from Teaching Quality Assessment (TOA), a tool available on the Higher School of Economics (HSE) website. TQA is conducted at the end of each module in order to improve the planning and organisation of the teaching process. Undergraduate students across Year 1, 2, 3, and 4 are offered the opportunity to assess courses in which they had an examination in the current module or received a grade as part of an interim assessment. The name of the course and the teacher who taught it are reflected in the evaluation form. Students are invited to assess different aspects of the courses and the teachers' performance on a 5-point scale as well as to leave free-form written comments in a separate field, with the direct wording of the instruction: "Your comments:__".
We obtained ethical approval from the HSE's ethics committee with the stipulation that we not use any direct quotes from the respondents. We retrieved and analyzed 363 anonymous positive comments about teaching from first through fourth-year students across six Bachelor's programmes (Business Informatics, Economics, History, Law, Management, and Software Engineering) for the academic years 2015/2016 and 2016/2017. We differentiated between positive and negative feedback using a teacher's overall score. If a teacher received a positive score (4 and 5), the following comment was qualified as positive while the comments following the teacher rating of 3 or below were negative. It is acknowledged from the outset that the dataset investigated here may not be substantial enough to generalize findings beyond the sample. However, as this was not the required purpose, the dataset is useful for exploring semantic models of the positive feedback language used by HSE students in Perm.
Methodology
Participants
Assessments and Measures
The analysis of students' positive feedback about teaching was carried out in several steps: data preprocessing (tokenization, punctuation and stop word removal, and lemmatization), part-of-speech tagging (POS tagging), counting the number of occurrences of different parts of speech, calculating word occurrence and frequency, extracting semantic concepts from the notional words, and building semantic patterns of HSE students' positive feedback about teaching (See Figure 1). We refer to the sequences of semantic concepts that represent entities appearing in SET as semantic patterns for simplicity.
Figure 1
Student positive feedback analysis
For the data analysis we applied Pymorphy2, a morphological analyser and generator for the Russian and Ukrainian languages. Pymorphy2 was implemented as cross-platform software in the Python programming language and is widely used in academic research. Pymorphy2 uses morphological word features and relies on large lexicons for the analysis of common words. For the Russian language, Pymorphy2 uses the OpenCorpora dictionary, available at OpenCorpora.org (~ 5 * 106 word forms, ~ 0.39 * 106 lemmas) and converted from OpenCorpora XML6 format to a compact representation optimized for morphological analysis and generation tasks. Pymorphy2 provides users with methods to analyze, inflect, lemmatize, or tag words as well as with data about the word gender, number, part of speech, and root. For data processing, it is necessary to choose the correct analysis from a list of possible analyses. It is generally required to take the word context in account.
The retrieved comments were categorised as unstructured and noisy owing to the informal writing style, grammatical and spelling mistakes, use of abbreviated shortcuts, and incomplete sentences. Therefore, we preprocessed the data removing unwanted and noisy data. This stage of analysis comprised tokenization (breaking a stream of text into a list of words), punctuation and stop words removal, and lemmatization (grouping together the inflected forms of a word and analysing them as a single item). The tokenizer split the
input text into 5448 words. After that, we filtered function words and lemmatised the remaining 4193 notional words. The resulting data set was a list of 1294 lemmas serving as input for further processing steps.
Next was POS tagging. We marked up each lemma as corresponding to a particular part of speech and counted the number of occurrences for each part of speech elucidating their usage patterns. In our research, we also analysed word frequency by counting how many times each word was used in students' positive feedback comments. Finally, there was semantic analysis dividing the notional words into semantic categories and yielding semantic patterns of students' positive evaluation of teaching.
Results
As the first step of data processing, the morphological analyzer Pymorphy2 performed POS tagging marking up all the notional words as a noun, pronoun, adjective, verb, adverb, or numeral (See Figure 2).
Figure 2
The word 'хороший' in Pymorthy2
In [5]: import pymorphy2
morph = pymorphy2.Morphanalyzer()
morph. parse( орший )[ ]
Out [5]: Рагее(иой='хорший', tag-OpencorporaTag( ADJF,Qual masc,sing,nomn'),
normal_form='хорший', score=0.8, methods_stack=((<DictionaryAnalyzer>, 'хорший', 2998, @),))
As is illustrated in Figure 2, parsing the word 'хороший' ('good') returns the following tags: ADJF, Oual masc, sing, nomn meaning 'хороший' is a masculine singular adjective.
Next, we computed the number of occurrences for each part of speech (See Table 2). Table 2
The Number of Occurrences of Different Parts of Speech
Part of speech Total Number of Occurrences (7098) Percentage (100%)
Noun 1704 24
Adjective 837 12
Verb 3860 54
Adverb 492 7
Pronoun 176 2.5
Numeral 29 0.5
As shown in Table 2, students' positive commentary about teaching was dominated by verbs, accounting for more than a half of all words in the dataset. Next, frequent were nouns (24%) and adjectives (12%), with adverbs, pronouns, and numerals lagging behind.
Further investigation narrowed down to the computation of word frequencies for each part of speech (See Tables 3-7).
Table 3
Occurrence of Notional Parts of Speech in Positive SET
Nouns
Occurrence Word
250 a proper name, e.g. Olga, Vladimir, Ivan
189 prepodavatel'
68 kurs
57 material
55 lektsia
47 seminar
36 student
35 predmet
27 znanie
24 rabota
22 vremya
20 vopros
19 distsiplina, para
17 primer
16 praktika
15 zanyatie
13 auditoria, chelovek
12 yazyk, zadanie
11 kontakt
9 god, podhod, informatsia, vozmozhnost', tema
8 interes, raz, podacha
7 forma, budustchee, yumor, novizna, shkola, ball, vyshka
6 delo, igra, blagodarnost', teoria
5 konsultatsia, diskussia, oblast', uroven', protsess + 10 words
4 kommentarii, kommunikatsia, opisanie, gruppa + 15 words
3 sovet, caryera, svyaz', reshenie, fakt, drug, rech, format + 20 words
2 primenenie, vospriyatie, formirovanie, upravlenie, metod, chast' + 55 words
1 raznoobrazie, nagl'adnost', instrument + 225 words
Total: 1704 Total: 476
Adjective
58 interesny
52 horoshy
38 kotory, zamechatel'ny
34 samy
28 takoi
24 svoi
21 bol'shoi
20 polezny
18 otlichny
17 prekrasny
15 kazhdy
12 real'ny
11 anglijsky
10 praktichesky
9 slozhny, lyubimy, uchebny
8 krutoi, ogromny, super, lyuboi
7 vysoky, danny, polozhitel'ny
6 razlichny, otzyvchivy
5 sovremenny, pon'atny, trebovatel'ny + 5 words
4 dostupny, yarky, neobhodimy, teoretichesky + 5 words
3 aktivny, competentny, nuzhny, dobry, glavny, + 15 words
2 umny, gramotny, erudirovanny + 45 words
1 velikolepny, original'ny, ideal'ny, bestsenny, prevoshodny, neobychny,nezauryadny + 110 words
Total: 837 Total: 230
Verb
280 byt'
155 dat'/davat'
140 ob'asnyat'/ob'asnit'
135 hotet'sya
100 provodit'/vesti
95 rasskazat'
90 (po)nravit'sya
80 delat'
75 pomoch/ pomogat'
70 moch
65 slushat', ponimat' poluchat'/poluchit'
60 prohodit'
55 otvetit'/otvechat', rabotat', znat'
50 najti
40 umet', zainteresovat', potryasat',
35 prepodavat'
30 idti, prigodit'sya, (na)pisat', razbirat',
25 skazat, vstrechat, prihodit', tsenit' + 5 words
20 starat'sya', primenyat' + 10 words
15 videt, pokazyvat, hvalit, poluchat', motivirovat', reshat' +15 words
10 diskutirovat', sprosit', reshat', vybrat' + 35 words
5 organizovat',vnikat', vystupat', razobrzt' + 185 words
Total: 3860 Total: 244
Adverb
144 ochen'
35 vsegda
29 interesno, mnogo
14 ponyatno
8 horosho, dostupno, bolee
7 dejstvitel'no, priyatno
6 yasno, bistro, nikogda
5 chetko, kruto, prekrasno, dostatochno, uzhe, legko, nastol'ko
4 posledovatel'no, chasto
3 otlichno, srazu, krajne
2 polezno, slozhno, tochno + 18 words
1 naglyadno, konkretno, udobno + 88 words
Total: 492 Total: 138
Pronoun
41 ya
31 on
25 ona
24 my
11 eto
8 sebya
6 oni
5 vy
4 kto, nichto
3 ty, chto, chto-to, kto-to
2 nikto, mnogoe
1 mnogie
Total: 176 Total: 18
Next was semantic categorization of the notional words in positive SET. For this the semantic metalanguage particles, which either coincided with or helped create compound concepts of student positive comments, were used. According to the natural semantic metalanguage theory, substantives are universally divided into persons (Who is it?) and things (What is it?). Furthermore, there are categories describing those who act (Actor), the means by which they act (Tool), the objects they make or use (Artifact), and the properties of actors, tools, and artifacts (Property). Table 4 shows the semantic categories of the nouns used in students' positive feedback comments.
Table 4
Semantic Categories of Nouns in Positive SET
Semantic category Occurrence Word
Person Actor 583 prepodavatel'
Teaching Tools 590 kurs, lektsia, podhod
Thing Educational Artifacts Material Mental 329 186 plan, prezentatsia znanie, informatsia
Property 16 adekvatnost', gumannost'
Example 1
Prepodavatel' who? a person teaches
does the teaching action
acts
actor
Prepodavatel' is an ACTOR
Example 2
Lektsia what? a thing a means of teaching something used by a teacher something used by an actor Lektsia is a TOOL
Example 3
Presentatsia what? a thing
a talk made by a teacher/student something made by someone
an artifact
can be seen or heard
physical
relates to something physical material
Presentatsia is a MATERIAL ARTIFACT
The adjectives were classified into two big groups: those attributing people and things. Each of the two groups consists of three semantic categories: determiners serving to express the reference of a noun, descriptors describing or identifying a person or thing, and evaluators giving an opinion about a person or thing. Table 5 shows different trends for attributing animate and inanimate nouns in positive SET. Nouns denoting people are overwhelmingly attributed by evaluators while those for things are more commonly used with descriptors.
Table 5
Semantic Categories of Adjectives in Positive SET
Category Adjective Occurrence Person Thing - Word
Descriptor 24 221 uchebny, praktichesky, anglijsky
Determiner 69 87 danny, lyuboi, nekotory
Evaluator 305 131 interesny, horoshy, zamechatel'ny, otlichny, prekrasny
Example 4
Danny
used for a person/object
is referred to now
the nearest in space/time
used to differentiate between people/things
Danny is a DETERMINER
Example 5
Prekrasny
someone thinks about something that something is very good Prekrasny is an EVALUATOR
Example 6
Uchebny
someone says about something that something is used for learning someone says how something is used someone describes the purpose of something Uchebny is a DESCRIPTOR
Table 6 presents the semantic types of the verbs used in students' positive comments about teaching.
Table 6
Semantic Categories of Verbs in Positive SET
Semantic category Occurrence Word
Action/Event 1110 delat', napisat', rabotat'
Mental Predicate 1080 videt', dumat', schitat'
Metapredicate 110 moch, umet'
Non-mental Predicate 1000 byt', udavat'sya, obladat'
Speech 560 govorit', izlagat', kommentirovat'
Example 7
Rabotat' do a job do something cause something to do Rabotat' is an ACTION
Example 8
Schitat'
to think about something that something is good/bad to count/calculate
to judge about the number/value of something to have information about something to know something about someone or something Schitat' is a MENTAL predicate
Example 9
Kommentirovat'
to give an opinion about something to say your opinion about something to talk about something Kommentirovat' is SPEECH
The next table (Table 7) shows the semantic categories of the adverbs used by students to praise teachers. As seen in Table 7, students tended to intensify their positive opinion about teaching using augmentors.
Table 7
Semantic Categories of Adverbs in Positive SET
Semantic category Occurrence Word
Augmentor 240 krajne, naibolee, ochen'
Time 71 vovremya, inogda, obychno
Evaluator 133 interesno, kruto, horosho
Descriptor 40 bystro, lakonichno, medlenno
Space 8 vezde, vyshe, zdes'
Discussion
Our main finding is that students praised the teacher more often than the other components of the teaching process (e.g. means of teaching, teaching results). It aligns with Stewart's (2015) research and proves that an effective teacher plays a very important role in student achievement.
With reference to Wolfson (1980), our findings revealed that students tended to use more verbs than other notional parts of speech to compliment and praise teachers. To summarize the parts-of-speech patterns in Table 3, five semantic groups of verbs were found in students' positive feedback comments: actions/events, mental predicates, non-mental predicates, metapredicates, and speech verbs. Interestingly, the figures for actions, mental, and non-mental predicates were almost equal and considerably exceeded those for speech and metapredicate groups. This means that students value teachers for what they do, what and how they think, and how well they conduct a seminar or lecture rather than how they speak or what their personal expertise and skills are.
Both animate and inanimate nouns were attributed by adjectives of three semantic types: determiner, descriptor, and evaluator. The analysis showed that evaluators tended to attribute nouns denoting people while descriptors
overwhelmingly referred to things. Evaluators were also numerous among adverbs. Quantitatively, the most common semantic models of students' positive evaluation of teachers are as follows:
Actor + Evaluator + Action/Mental predicate + Augmentor/ Evaluator Tool/ Artifact + Descriptor + Non-mental predicate + Augmentor/Space/Time
Although compliments are thought to be formulaic and lack originality in choice of lexical items and semantic structures (Manes & Wolfson, 1981; Wolfson, 1981), our results showed some unique characteristics of Russian compliments in the higher education context. One major difference was noted when examining the use of parts of speech. As mentioned above, we found that student positive evaluations of teachers in Russian were dominated by verbs, whilst previous research pointed out that the majority of Russian compliments were adjectival (Nguyen, 2013; Solodka, Sukhomlynsky, & Perea, 2018). Another difference that was noted in complimenting behaviour in the higher education context concerns thematic imperatives or topics chosen by Russian students to compliment teachers. Russian students tend to praise teachers for their actions (physical, mental, and non-mental) rather than their appearance, abilities, and personality traits occurring most in non-educational contexts (Issers, 2006; Kachevskaia, 2007; Shcheboleva & Sun', 2016).
The study of verbal representations of the complimenting speech act used by students of the Higher School of Economics in Russia offers insights into the system of values of the Russian student community. The prevalence of mental and non-mental verbs, expressing one's state (to be, to have, to like, to think, to understand, to want, etc.), represents Russian students' orientation value of "being" and "feeling". The orientation value of "doing" represented by action/event verbs (to do, to get, to give, to help, etc.) is less important for Russian students. Surprisingly, teaching delivery appears to be ranked the lowest in the verb classes. Russian students' complimenting behaviour is most often directed at interest, which manifests itself in the use of such lexemes as to be interested, interesting, and interestingly/with interest. These results compliment the study on the linkage between the teachers' emotional characteristics and personality type and students' estimation of the teachers' qualities (Busygina & Busygina, 2013).
However, this research has several limitations. It is based on feedback provided by students in just one campus of the Higher School of Economics at one point in time. So, no claims can be made that the obtained semantic patterns will be generalizable to all students' positive feedback, which was never the intention of this research. Different findings could be expected between different kinds of institutions and different cultures and languages, especially in weights of semantic groups in models. Nevertheless, addressing this limitation in our methodology may be an objective of future research. The study could also be extended to word collocations or sentence structures in student feedback and include age, gender, and nationality in the research. Moreover, the limited sample could not include negative comments, which might also contain the language of praise. It means that further studies should incorporate negative evaluations, to strengthen the validity of the results. Overall, the validity of quantitative ratings in SET and the improved design of evaluation measures can be tested in combination with the current findings and potential comparisons with negative comments.
Conclusion
Our study offers a new perspective on the theory of speech acts and reveals some patterns of interpersonal positioning in a single aggregated sample of student positive commentary. The findings provide information that concerns how expressive illocutionary acts are performed in the higher education context as well as how their implicit cultural meanings can be interpreted. It is expected that some aspects of students' evaluative language identified in this research may be universal.
The semantic models and their features can be applied in automatic reading of subjective commentaries to identify positive feedback and 'good' teachers. This application can fill the gap caused by the lack of systematic analysis and processing techniques for approaching open commentaries in students' evaluations. Because this study is on text analytics in semantic research, i.e. the application of the Python programming language and the OpenCorpora dictionary, its main contribution is processing a large number of user-generated texts as well as building semantic patterns of student positive commentary in the national language.
We also anticipate that a better understanding of the semantic structure and proposition of positive textual evaluation will allow educators to interpret students' subjective commentary more fully and improve both student learning and instructor performance, mainly through using them as guidance for the choice of instructors, knowing factors for faculty loads or rating biases, and students' expectations (Otto, Sanford, & Ross, 2008; Wongsurawat, 2011). Satisfaction with a teacher is assumed to facilitate learning and to positively impact learning outcomes. Particular recommendations for educational managers might include a better match of a teacher for a course and/or improvements in course selection by students.
Conflict of Interest
The authors declare that they have no conflict of interest.
References
Ahmad, S. Z. (2015). Evaluating student satisfaction of quality at international branch campuses. Assessment &
Evaluation in Higher Education, 40(4), 488-507. http://dx.doi.org/10.1080/02602938.2014.925082 Alhija, F. N. A., & Fresko, B. (2009). Student evaluation of instruction: What can be learned from students' written comments? Studies in Educational Evaluation, 35(1), 37-44. http://dx.doi.org/10.10Wj.stueduc.2009.01.002 Allami, H., & Montazeri, M. (2012). Iranian EFL learners' compliment responses. System, 40(4), 466-482. http://
dx.doi.org/10.10Wj.system.2012.10.010 Amin, M. (1994). Gender as a discriminating factor in the evaluation of teaching. Assessment & Evaluation in
Higher Education, 19(2), 135-143. http://dx.doi.org/10.1080/0260293940190206 Austin, J. L. (1962). How to do things with words. Harvard University Press.
Bai, G. (2015). On the pragmatic patterns of Chinese college English learners' compliment responses. International
journal of English linguistics, 5(4), 106-119. http://dx.doi.org/10.5539/ijel.v5n4p106 Blair, E., & Noel, K. V. (2014). Improving higher education practice through student evaluation systems: Is the student voice being heard? Assessment & Evaluation in Higher Education, 39(7), 879-894. https://doi.org/10. 1080/02602938.2013.875984 Brophy, J. (1981). Teacher praise: A functional analysis. Review of Educational Research, 51(1), 5-32. http://
dx.doi/10.3102/00346543051001005 Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage (Studies in Interactional
Sociolinguistics). Cambridge University Press. Burnett, P. C. (2002). Teacher praise and feedback and students' perceptions of the classroom environment.
Educational Psychology, 22(1), 51-6. https://doi.org/10.1080/01443410120101215 Busygina, A. L., & Busygina, T. A. (2013). Influence of the teacher's emotionality on the students' assessment of
their pedagogical image. Samara Journal of Science, 4(5), 44-45. Centra, J., & Gaubatz, N. (2000). Is there gender bias in student evaluations of teaching? Journal of Higher
Education, 71(1), 17-33. https://doi.org/10.2307/2649280 Chen, Ch. Y., Wang, Sh. Y., & Yang, Y. F. (2017). A study of the correlation of the improvement of teaching evaluation scores based on student performance grades. International Journal of Higher Education, 6(2), 162168. https://doi.org/10.5430/ijhe.v6n2p162 Cheng, J., & Marsh, H. W. (2010). National student survey: Are differences between universities and courses reliable and meaningful? Oxford Review of Education, 36(6), 693-712. https://doi.org/10.1080/03054985.201 0.491179
Davison, E., & Price, J. (2009). How do we rate? An evaluation of online student evaluations. Assessment &
Evaluation in Higher Education, 34(1), 51-65. https://doi.org/10.1080/02602930801895695 Dziedziul, P. (2017). The universal categories of praxeology in light of natural semantic metalanguage theory.
Libertarian Papers, 9(2), 249-276. https://doi.org/10.2307/409588 Goddard, C. (2008). Natural semantic metalanguage: The state of the art. In Goddard, C. (Ed.), Cross-Linguistic semantics. Studies in language companion series (vol. 102, pp.1-34). John Benjamins Publishing Company. https://doi.org/10.1075/slcs.102 Grainger, K. (2018). "We're not in a club now": A neo-Brown and Levinson approach to analyzing courtroom data.
Journal of Politeness Research, 14(1), 19-38. https://doi.org/10.1515/pr-2017-0039 Grice, H. P. (1975). Logic and conversation. In Cole, P. & Morgan, J. L. (Eds.), Studies in syntax and semantics:
Speech acts (vol. 3, pp. 41-58). Academic Press. Herbert, R. K. (1986). Say "thank you" - or something. American Speech, 61(1), 76-88. https://doi. org/10.2307/454710
Herbert, R. K. (1989). The ethnography of English compliments and compliment responses. In Oleksy, W. (Ed.), ContrastivePragmatics (pp.3-37). John Benjamins Publishing Company. https://doi.org/10.1075/pbns.3.05her Holmes, J. (1988). Compliments and compliment responses in New Zealand. Anthropological Linguistics, 28(4), 485-508.
Holmes, J., & Brown, D.F. (1987). Teachers and students learning about compliments. TESOL Quarterly, 21(3),
523-546. https://doi.org/10.2307/3586501 Humboldt, W. (1999). On language: On the diversity of human language construction and its influence on the mental
development of the human species. Cambridge University Press. Huth, Th. (2006). Negotiating structure and culture: L2 learners' realization of L2 compliment-response sequences in talk-in-interaction. Journal of Pragmatics, 38(12), 2025-2050. https://doi.org/10.1016/j. pragma.2006.04.010
Huxham, M., Laybourn, Ph., Cairncross, S., Gray, M., Brown, N., Goldfinch, J., & Earl, Sh. (2008). Collecting student feedback: A comparison of questionnaire and other methods. Assessment & Evaluation in Higher Education, 33(6), 675-686. https://doi.org/10.1080/02602930701773000 Hyland, F., & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written feedback. Journal of Second
Language Writing, 10(3), 185-212. https://doi.org/10.1016/S1060-3743(01)00038-8 Issers, O. S. (2006). Kommunikativnye strategii i taktiki russkoi rechi [Communicative strategies and tactics of Russian speech]. URSS.
Kachevskaia, A. A. (2007). Kompliment kak verbal'noie sredstvo obshchenia [Compliment as a verbal means of
communication]. MSLU. Leech, G. N. (1983). Principles of pragmatics. Longman.
Light, G., & Cox, R. (2001). Learning and teaching in higher education: The reflective professional. Paul Chapman. Lin, Ch. Y., Woodfield, H., & Ren, W. (2012). Compliments in Taiwan and Mainland Chinese: The influence of region and compliment topic. Journal of Pragmatics, 44(11), 1486-1502. https://doi.org/10.1016/j. pragma.2012.06.012
Maiz-Arevalo, C. (2012). ''Was that a compliment?'' Implicit compliments in English and Spanish. Journal of
Pragmatics, 44(8), 980-996. https://doi.org/10.1016/j.pragma.2012.04.004 Manes, J. (1983). Compliments: A mirror of cultural values. In N. Wolfson & E. Judd (Eds.), Sociolinguistics and
language acquisition (pp. 96-102). Newbury House. Manes, J., & Wolfson, N. (1981). The compliment formula. In F. Coulmas (Ed.), Conversational routine (pp.115132). Mouton.
Nguyen Thu Huong. (2013). K voprosu o leksicheskih sredstvah vyrazheniia komplimenta v sovremennoi russkoi i vietnamskoi kul'ture [Lexical means of the compliment in Russian and Vietnamese cultures]. Global Scientific Potential. Philology, 12(33), 74-78. Otto, J., Sanford, D. A., & Ross, D. N. (2008). Does ratemyprofessor.com really rate my professor? Assessment &
Evaluation in Higher Education, 33(4), 354-368. https://doi.org/10.1080/02602930701293405 Rajput, Q., Haider, S., & Ghani, S. (2016). Lexicon-based sentiment analysis of teachers' evaluation. Applied Computational Intelligence and Soft Computing, 2016, Article ID 2385429. http://dx.doi. org/10.1155/2016/2385429
Richardson, J. T. E., Slater, J. B., & Wilson, J. (2007). The national student survey: Development, findings and
implications. Studies in Higher Education, 32(24), 557-580. https://doi.org/10.1080/03075070701573757 Saif, H., He, Y., Fernandez, M., & Alani, H. (2014). Semantic patterns for sentiment analysis of Twitter. In Mika, P., Tudorache, T., Bernstein, A., Welty, Ch. et al. (Eds.), Lecture notes in computer science (vol. 8797, pp. 324340). Springer. https://doi.org/10.1007/978-3-319-11915-1_21 Sapir, E. (1929). The status of Linguistics as a science. Language, 5(4), 207-214. https://doi.org/10.2307/409588 Sarwar, M., Dildar, M., Shah, A. A., & Hussain, S. (2017). Relationship among students' academic achievement,
students' evaluation of teacher and students' evaluation of course. The Dialogue, 12(1), 50-58. Searle, J. R. (1969). Speech acts: An essay in the philosophy of language. Cambridge University Press. Searle, J. R. (1975). A taxonomy of illocutionary acts. University of Minnesota Press.
Seifoori, Z., & Emadi, S. E. (2015). A Contrastive study of American and Persian compliment speech act. Modern Journal of Language Teaching Methods, 5(3), 543-549.
Shcheboleva, I. B., & Sun', M. (2016). Lingvokul'turologicheskie osobennosti komplimenta v russkom i kitaiskom jazykah [Linguocultural features of compliment in Russian and Chinese]. Problemy prepodavania philologicheskih distsyplin inostrannym uchashchimsya, 4, 449-455. Solodka, A., Sukhomlynsky, V. O., & Perea, L. (2018). The speech act of complimenting as part of the Ukrainian, Russian and English-Speaking Communities: Ukraine and the USA. Arab World English Journal, 9(4), 39 -55. https://dx.doi.org/10.24093/awej/vol9no4.3 Spencer, K. J., & Schmelkin, L. P. (2002). Student perspectives on teaching and its evaluation. Assessment &
Evaluation in Higher Education, 27(5), 397-409. http://dx.doi.org/10.1080/0260293022000009285 Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of
the art. Review of Educational Research, 83(4), 598-642. https://doi.org/10.3102/0034654313496870 Stewart, M. (2015). The language of praise and criticism in a student evaluation survey. Studies in Educational
Evaluation, 45, 1-9. https://doi.org/10.10Wj.stueduc.2015.01.004 Steyn, C., Davies, C., & Sambo, A. (2018). Eliciting student feedback for course development: The application of a qualitative course evaluation tool among business research students. Assessment and Evaluation in Higher Education, 44(1), 11-24. https://doi.org/10.1080/02602938.2018.1466266 Suketi, T. M. (2014). Analysis of compliment response strategies in consultation between students and teacher
[Undergraduate thesis]. Universitas Kristen Satya Wacana. Tabakowska, E. (2001). Kognitywne podstawy jgzyka i j^zykoznawstwa. Towarzystwo Autorów i Wydawców Prac Naukowych University.
Tang, Ch. H., & Zhang, G. 0., (2009). A contrastive study of compliment responses among Australian English and Mandrian Chinese speakers. Journal of Pragmatics, 41(2), 325-345. https://doi.org/10.10Wj. pragma.2008.05.019
Thawabieh, A. M. (2017). A comparison between students' self-assessment and teachers' assessment. Journal of
Curriculum and Teaching, 6(1), 14-20. https://doi.org/10.5430/jct.v6n1p14 Wierzbicka, A. (1980). Lingua mentalis: The semantics of natural language. Academic Press Australia. Wierzbicka, A. (1991). Cross-cultural pragmatics: The semantics of human interaction. Mouton de Gruyter. Wierzbicka, A. (1996). Semantics: Primes and universals. Oxford University Press. Whorf, B. L. (1982). Language, thought and reality: Selected writings of Benjamin Lee Whorf. MIT Press. Wolfson, N. (1981). Invitations, compliments and the competence of the native speaker. International Journal of
Psycholinguistics, 8(4), 7-22. Wolfson, N. (1983). An empirically based analysis of complimenting in American English. In N. Wolfson & E.
Judd (Eds.), Sociolinguistics and Language Acquisition (pp. 82-95). Newbury House. Wolfson, N., & Manes, J. (1980). The compliment as a social strategy. Papers in Linguistics: International Journal
of Human Communication, 13(3), 410-451. https://doi.org/10.1080/08351818009370503 Wongsurawat, W. (2011). What's a comment worth? How to better understand student evaluations of teaching.
Quality Assurance in Education, 19(1), 67-83. https://doi.org/10.1108/09684881111107762 Zaitseva, E., Milsom, C., & Stewart, M. (2013). Connecting the dots: Using concept maps for interpreting student satisfaction. Quality in Higher Education, 19(2), 225-247. https://doi.org/10.1080/13538322.2013.802576
