21REVISITING PREDICTIVE POWER ANALYSES OF UZBEKISTAN MOD FOREIGN LANGUAGE APTITUDE TEST IN TERMS OF INTENSIVE ENGLISH LANGUAGE TRAINING SUCCESS
Ilya Sergeevich Zverev
Partnership for Peace Training Center of the Armed Forces of the Republic of
Uzbekistan, zverev.elij ah@gmail .com
ABSTRACT
Foreign language teaching and foreign language learning have been an integral part of civilizational development since the very dawn of humanity. There is, however, a widely attested difference in the amount of effort required by language learners being taught in identical conditions for the actual progress in their foreign language studies. The difference is question is usually conceptualized as the difference in learners' foreign language aptitude. Uzbekistan Ministry of Defense measures foreign language aptitude for the purposes of candidate selection for its intensive foreign language courses by means of a specialized test battery. In one of our previous studies, we have demonstrated the lack of predictive power for the results obtained in three of the four subtests comprising the test battery based on the data from 39 intensive English language course students. The principal aim of the present study is to consider another variable, the total score obtained by a candidate in Uzbekistan MoD Test battery, in terms of its predictive power for intensive English language training success. The pool of the study participants has also been increased up to 64 individuals.
Keywords: Uzbekistan Ministry of Defense, foreign language aptitude, foreign language aptitude test battery, predictive power
INTRODUCTION
The Ministry of Defense of the Republic of Uzbekistan (Uzbekistan MoD) provides intensive foreign language training for its service members in order to guarantee their successful participation in international military cooperation events conducted both in the Republic of Uzbekistan and abroad.
The principal body within the structure of Uzbekistan MoD tasked with organization and conduct of the said training in accordance with international standards and following generally accepted practices is Partnership for Peace
Training Center of the Armed Forces of the Republic of Uzbekistan (Uzbekistan PfP). The languages taught in intensive format include English, German, Russian, Turkish, Pashto, Dari and Urdu. A typical course duration does not exceed 6 months.
Intensive English language training course at Uzbekistan PfP is taught via American Language Course (ALC) materials developed by the USA Defence Language Institute English Language Center (DLI ELC) and widely used in training centers functioning within NATO "Partnership for Peace" cooperation program worldwide.
For each of the twenty-four weeks of intensive English language training, there
are
- an ALC Student Text (comprising 5 units including 1 review unit);
- an ALC Instructor Text (providing all the necessary details on the presentation/practice/production techniques to be utilized);
- an ALC Language Laboratory Activities Text (for the trainees' work at a linguistic laboratory);
- a set of ALC flash cards (for ALC Student Texts 1 - 12 only).
The trainees typically spend 5 academic hours daily (Monday - Friday) working with their instructor in the classroom, 1 hour working in the linguistic laboratory and 3 hours studying independently under the guidance of their instructor. The 8 hours of English language studies, consequently, do not include the time for students' self-study (additionally up to 3 hours daily). Thus, there are significant cognitive demands imposed upon the trainees in terms of the amount of information they have to process in order to successfully pass both interim and final examinations.
The aforementioned cognitive demands coupled with limited duration of the course necessitate the employment of a specialized selection mechanism that could guarantee that the limited resources at Uzbekistan MoD disposal are spent training only those actually capable of achieving the results required.
This mechanism is a specialized test battery developed by specialists of Uzbekistan MoD in 2011 and comprising four Subtests. For the purposes of consistency preservation among various pieces of research published, hereinafter it will be referred to it as "Uzbekistan Ministry of Defense Foreign Language Test Aptitude Test Battery" (Uzbekistan MoD FLA TB).
Based on the results of correlational and multiple regression analyses of the data obtained from 39 intensive English course trainees, there has already been
demonstrated the lack of statistically significant predictive power of three out of four Uzbekistan MoD FLA TB (Zverev, 2020a).
The principal aim of the present study is to establish the predictive validity of the sum total of all the Subtest scores obtained by an intensive English language course trainees.
The procedures we follow in the present study (correlational and regression analyses) will be identical to those we have employed previously. However, we have expanded the pool of the trainees whose data we analyzed from 39 in the previous study to 64 in the current one.
LITERATURE REVIEW AND RESEARCH METHODOLOGY
In all of our previous publications on the issue of foreign language aptitude (Zverev, 2019, 2020a, 2020b), this collection of various cognitive skills/abilities considered to be important/conducive to foreign language learning has been viewed through the prism of its being a psychological construct, i.e. an attribute whose existence is postulated but cannot be verified through immediate observation.
There is no distinctive list of all the skills/abilities in question and, depending on the particular construct conceptualization approach, those might include phonetic coding ability, grammatical sensitivity, inductive language learning ability, rote memory ability (Skehan, 2002, p. 70, 2012, p. 381; Wen et al., 2017, p. 3), grammar sensitivity, native language vocabulary range (Grigorenko, 2002, p. 97), native language skills (Sparks & Ganschow, 1991), attentional control, working memory, language analysis ability, retrieval memory (Skehan, 2012, p. 386), perceptual speed, pattern recognition (Robinson, 2012, p. 67), etc.
A detailed discussion of the most important foreign language aptitude construct conceptualization approaches is provided in (Zverev, 2019), yet for the purposes of the present study we have to reemphasize the fact of the foreign language learning aptitude's generally being considered to be a multicomponential rather than a monolith construct, which is reflected not only in all the conceptualization approaches, but also in all its measurement instruments.
Uzbekistan MoD FLA TB comprises four distinct Subtests, each of which is supposed to measure a distinct individual ability considered by the battery developers to be important for foreign language teaching and learning. Uzbekistan MoD FLA TB is administered to every candidate for intensive foreign language training at Uzbekistan PfP, the exact language he or she will be studying throughout the course notwithstanding. The Battery, therefore, is not language-specific.
A correct response on each of the Uzbekistan MoD FLA TB subtests is worth one point and no penalty is imposed for an incorrect answer. Based on his or her results demonstrated in each of the subtests, the candidate is allocated in one of the four Professional Fitness Groups (PFG).
It is the membership in a particular PFG (1 - 4) that is taken as an indication of whether or not a candidate would be capable of achieving satisfactory foreign language learning results (PFG-4 candidates are not considered for intensive foreign language training, whereas PFG-1 candidates are deemed to be the best ones for it). The exact procedure for the allocation process proper is beyond the scope of the present study, since all its participants eventually made the cut and were enrolled in the intensive English language training course.
The four abilities purportedly assessed and measured by the four Subtests are: the ability to determine the underlying relationship between pairs of lexical items (Subtest 1 - "Lexical Analogies"); the spacial reasoning ability (Subtest 2 - "Shape Selection"); the inductive language learning ability (Subtest 3 - "Linguistic Decoding"); rote memory ability (Subtest 4 - "Narration Summary").
The testing procedure based on the usage of Uzbekistan MoD FLA TB is typically organized and run by a group of specifically designated invigilators (service members of Uzbekistan MoD). Having been granted access to the results obtained by the study participants upon their completion of an intensive 6-month English language course at Uzbekistan PfP, we did not participate in the test delivery proper.
Throughout the six months of their intensive English language training, all the study participants underwent weekly formative testing getting an ALC BQ score for each of the twenty-four ALC books. The testing was conducted in a computer linguistic laboratory (equipped with headsets) by a group of test invigilators specifically set the task in question on rotation basis. The results were calculated by senior department instructors and the data entered into the students' personal files.
All in all, with the completion of an intensive English language training course at Uzbekistan PfP, we managed to obtain Uzbekistan MoD FLA TB total scores, ALC BQ mean scores and ALCPT scores for 26 more students in addition to the data we utilized for the purposes of the previous study, whose results have been published in the European Journal of Research and Reflection in Educational Sciences (Zverev, 2020a). In the present study, we analyzed three pieces of data from 64 male service members of Uzbekistan MoD enrolled at some point during the last three years in an intensive English language course provided by Uzbekistan PfP: the sum total of their Uzbekistan Mod FLA TB Subtests, ALC BQ mean score and ALCPT score.
The data has been derived from 64 intensive English language course students who managed to successfully graduate after 6 months of studies. All of them were male service members of Uzbekistan MoD with non-commissioned officers comprising approximately 14% of the total number. A detailed information on the ranks of the students, whose data we utilized within the framework of the present study, is provided in Table 1.
Table 1
Rank-Based Distribution of the Study Participants (N = 64)
Rank f Rel. f Cf Percentile
Lieutenant Colonel 1 0,02 64 100,00
Major 11 0,17 63 98,44
Captain 16 0,25 52 81,25
Senior Lieutenant 12 0,19 36 56,25
Lieutenant 15 0,23 24 37,50
Sergeant 2 0,03 9 14,06
Junior Sergeant 2 0,03 7 10,94
Private 5 0,08 5 7,81
The minimum score obtained by intensive English course trainee in his Uzbekistan MoD FLA TB was 7 (and the person was still enrolled in the course) and the maximum equaled 58. The mean Uzbekistan MoD FLA total score was 43.84. The data was negatively skewed (-1.521).
In terms of the ALC BQ mean score, after six months of intensive English language studies, the minimum value for this variable was 36.38 (out of 50) and the maximum equaled 49.46 (out of 50). The data was negatively skewed (- 0.061).
Last, but not least, ALCPT minimum score in the data pool selected for consideration in this study was 60 points (out of 100), whilst the maximum score equaled 89. Mean ALCPT score equaled 72. The data was positively skewed (0.517).
RESULTS
Correlational analysis
The correlational analysis was carried out by means of IBM SPSS software package (version 26). For the purposes of the analysis, we have opted for the usage of Spearman's rho correlation coefficient, defined as "a standardized measure of the strength of relationship between two variables that does not rely on the assumptions of a parametric test" (Field, 2009, p. 794), rather than Pearson's correlation coefficient. There were two main factors that lay at the foundation of this decision of ours:
- it was Spearman's rho that was reported as the correlation coefficient by original Uzbekistan MoD FLA TB developers (Akhrorov & Rakhimmirzaev, 2011);
- there were insufficient number of study participants to meet the normality distribution criterion (Field, 2009, p. 133).
Based on the results of the analysis conducted, there has been established the existence of statistically significant positive correlations between Uzbekistan MoD FLA TB total score and ALC BQ mean score (rs = .341, p (two-tailed) = .006), as well as between ALC BQ mean score and ALCPT score (rs = .518, p (two-tailed) < .001). We have failed to find a significant correlation between Uzbekistan MoD FLA TB total score and the ALCPT score (rs = .133, p (two-tailed) = .294).
Simple regression analysis
In order to measure the predictive power of Uzbekistan MoD FLA TB total score in terms of intensive English language course success, we have conducted a simple regression analysis following the procedure outlined in Field (2009, pp. 205209). The predictor variable in the analysis conducted was Uzbekistan MoD FLA TB total score and the outcome variable was ALC BQ mean score. The results of the analysis can be found in Table 2 below.
There was no regression analysis run with ALCPT score as the outcome variable due to our having failed to demonstrate the existence of any statistically significant correlation between this particular variable and Uzbekistan MoD FLA TB total score (see above).
Table 2
Simple regression analysis summary with ALC BQ mean score as the outcome variable
Model B B SE B CI LB UB ß
1 Constant 39.405 1.589 36.229 42.580
FLA Test Total .080 .035 .009 .151 .275
Note: R2 = .076, adjusted R2 = .061.
DISCUSSION
here are a number of points that must be discussed based on the results of the correlational and simple regression analyses conducted.
First, despite the fact that only Subtest 3 score when considered separately was found to correlate positively with both ALC BQ mean score and ALCPT score within the framework of our previous study (Zverev, 2020a, p. 39), in this study we have demonstrated that the sum total of all the four Subtest scores does correlate positively
with ALC BQ mean score. Therefore, we would have to correct our previous claims about the predictive power Uzbekistan MoD FLA TB taken as a whole.
Second and most interestingly, the fact that we have failed to establish the existence of statistically significant positive correlation between Uzbekistan MoD FLA TB total score and the principal intensive English language course summative assessment instrument (ALCPT) score requires some further investigation. A possible explanation that we can offer are additional efforts invested by the course students in their preparations for the final test that include instructor-guided test taking strategy development, practice test and independent pre-test cramming. This explanation, however, is a tentative one.
CONCLUSION
The principal aim of the present study was to establish whether or not an additional variable, Uzbekistan MoD FLA TB sum total score, possesses any predictive validity in terms of intensive English language training course success.
Having conducted two basic statistical analyses (correlation and simple regression), we have managed to demonstrate that the variable in question does correlate positively with one of the language course success measures chosen for the purposes of the study (ALC BQ mean), but does not correlate with the other (ALCPT score).
In terms of the predictive validity of Uzbekistan MoD FLA TB sum total score, we have managed to demonstrate that it accounts for approximately 6% of the variation in ALC BQ mean score only, which is rather disappointing.
Our finding pertaining to the lack of significant positive correlation between Uzbekistan MoD FLA TB sum total score and ALCPT score can serve as a foundation for an additional study.
REFERENCES
1. Akhrorov, S., & Rakhimmirzaev, S. (2011). The validation results of psychologic assessment of candidates for Uzbekistan PfP intensive foreign language training courses (p. 16).
2. Field, A. (2009). Discovering Statistics using SPSS Statistics. In SAGE Publications. SAGE Publications Ltd. http://www.amazon.com/Discovering-Statistics-using-IBM- SPSS/dp/1446249182
3. Grigorenko, E. L. (2002). Foreign language acquisition and language-based learning disabilities. In P. Robinson (Ed.), Individual Differences and Instructed
Language Learning (pp. 95-112). John Benjamins Publishing Company. https://doi.org/10.1075/lllt2.07gri
4. Robinson, P. (2012). Individual Differences, Aptitude Complexes, SLA Processes, and Aptitude Test Development. In M. Pawlak (Ed.), New Perspectives on Individual Differences in Language Learning and Teaching (pp. 57-75). Springer International Publishing AG. https://doi.org/10.1007/978-3-642-20850-8_4
5. Skehan, P. (2002). Theorising and updating aptitude. In P. Robinson (Ed.), Individual Differences and Instructed Language Learning (pp. 69-93). John Benjamins Publishing Company.
6. Skehan, P. (2012). Language Aptitude. In S. M. Gass & A. Mackey (Eds.), The Routledge Handbook of Second Language Acquisition (pp. 381-395). Routledge.
7. Sparks, R. L., & Ganschow, L. (1991). Foreign language learning difficulties: affective or native language aptitude differences? Modern Language Journal, 75(1), 3-16.
8. Wen, Z., Biedron, A., & Skehan, P. (2017). Foreign language aptitude theory: Yesterday, today and tomorrow. Language Teaching, 50(1), 1-31. https://doi.org/10.1017/S0261444816000276
9. Zverev, I. (2019). Uzbekistan MoD Foreign Language Aptitude Test: A Critical Evaluation. Filologiya Masalalari, 29(2), 138-151.
10. Zverev, I. (2020a). Uzbekistan MoD Foreign Language Aptitude Test Battery Predictive Power Analysis. European Journal of Research and Reflection in Educational Sciences, 8(7), 35-44. https://doi.org/10.36078/987654354
11. Zverev, I. (2020b). Computer-based foreign language learning deficienty diagnostics using LLAMA test battery. Chet Tillarni O 'qitishda Xorijiy Tajribalardan Foydalanish, 72-74.
