CC BY-ND
17
National Research University Higher School of Economics Aiko, M. (2018). The Relationships Between the Accuracy of Self-Evaluation,
Journal of Language & Education Volume 4, Issue 2, 2018 Kanji Proficiency and the Learning Environment for Adolescent Japanese
Heritage Language Learners. Journal of Language and Education, 4(2), 6-23. doi:10.17323/2411-7390-2018-4-2-6-23
The Relationships Between the Accuracy of Self-Evaluation, Kanji Proficiency and the Learning Environment for Adolescent Japanese Heritage Language Learners
Mizue Aiko
RMIT University
Correspondence concerning this article should be addressed to Mizue Aiko, RMIT University, GPO Box 2476, Melbourne VIC 3001 Australia. E-mail: mizue.aiko@rmit.edu.au
This paper focuses on Japanese heritage language (JHL) learners in an Australian context. The paper reports on a research project in a hoshuu-koo institution, a Japanese supplementary school, and explores the experiences of a group of Year 7 students. This study was initiated by identifying to what extent JHL learners can recognise their own skills, especially in proficiency in kanji, one of the Japanese scripts. It was predicted that several elements could relate to the accuracy of self-evaluation. By exploring levels of self-evaluation skills and the elements concerned in Japanese learning, the aim of the research was to help develop differentiated curriculum in the future. Data were based on student performance on kanji tests and answers to questionnaires, and the Excel Correl Function was used to calculate correlation coefficients. Graphs were also used to analyse the data. It was found that students who had relatively high kanji proficiency, especially in higher year levels, recognised their own skills but an overall overestimation was found amongst other students. Specific areas of kanji learning, such as okurigana and radicals, were identified as areas that need to be enhanced for appropriate self-evaluation for most of the students. Learning environment related to evaluation skills was also identified. Concluding comments centre on implications for further teaching approaches and research on the enhancement of kanji self-evaluation skills.
Keywords: Japanese heritage language (JHL) learners, hoshuu-koo, kanji, self-evaluation, home environment, tests
This study explores kanji (one of the Japanese scripts) proficiency, the self-evaluation skills of Japanese Heritage Language (JHL) learners, and the factors that relate to self-evaluation skills. A heritage language is the first language that children learn to speak fluently; for example, as children grow they will come to use the local language (e.g., English) more frequently and skilfully and thus their first language (e.g., Japanese) becomes their "heritage language" (Nakajima, 1998). It has been reported by researchers that the kanji proficiency of JHL learners has not been satisfactorily developed (Kataoka & Shibata, 2011) and it is suspected that learners who lack self-assurance in kanji miss the opportunities to develop their skills as they hesitate to learn. This attitude tends to increase as the learners become older, especially for adolescents
and adults. It is assumed that recognising one's own proficiency properly helps in gaining confidence and devising one's own learning methods. Therefore, it is important to identify how accurately each JHL learner recognises their own kanji proficiency and to find out if there are any factors relating to self-recognition skills so that further learning methods can be explored to enhance self-evaluation ability.
A hoshuu-koo in Australia was selected for this research. Hoshuu-koo are supplementary Japanese schools outside of Japan and are organised by the Japanese Ministry ofEducation, Culture, Sports, Science and Technology (MEXT) to provide Japanese education for first grade to ninth grade students who go to a local school during the week. Hoshuu-koo were originally designed for children who would eventually return to
This article is published under the Creative Commons Attribution 4.0 International License.
Japan and were expected to provide these children with access to part of the Japanese compulsory education curriculum (Doerr & Lee, 2010). As well as teaching Japanese language to the designated original cohort of students, many supplementary schools also teach Japanese as a heritage language (JHL) (Yamaguchi, 2008). At the hoshuu-koo where this research was conducted, almost 80% of students learn Japanese as a heritage language. It was observed that students were struggling to acquire kanji and often stopped learning Japanese when they finished Year 6, i.e., at the end of elementary school. Many of the students who do not have confidence in kanji showed passive and negative attitudes towards learning kanji. When students have a negative attitude towards learning kanji, it is hard to acquire kanji; therefore it is crucial to make kanji learning attractive by introducing culture and allowing students to understand the usefulness of kanji (Shimizu & Green, 2002). As hoshuu-koo is the main institution for JHL learners, a solution for this situation is necessary to encourage learning JHL.
Researchers insist that heritage language education is important socially, culturally, and economically as well as for maintaining a positive sense of identity (Douglas & Chinen, 2014; Wang & Green, 2001). Additionally, heritage language speakers have pragmatic advantages, such as gaining a qualification and better employment opportunities as well as expecting to obtain good marks on exams (Doerra & Leeb, 2009; Willoughby, 2006). Despite the recognition of its importance, difficulties in maintaining learning and acquiring age-appropriate proficiency have been reported (Douglas, 2008; Oguro & Moloney, 2012), especially as learners become older, and this may influence learning motivation. Age-appropriate proficiency in JHL education is regarded as knowledge of vocabulary, script, grammar, and the skills to use them that students learn at school in each grade based on the curriculum designed by MEXT. Difficulty in learning and teaching kanji at hoshuu-koo as well as at tertiary institutions is widely acknowledged, although difficulties in other areas of JHL learning have also been recognised. For example, an imbalance between the four language skills of reading, writing, speaking, and listening (Douglas, 2010; Nishimura, 2012), the use of wrong registers (Calder, 2008), and restricted knowledge and competence in morphology and vocabulary (Calder, 2008; Kataoka, Koshiyama, & Shibata, 2008). There could be several factors influencing learners' proficiency, and Aiko (2017) indicates that the increasing use of a dominant language and lack of opportunities to hear or use formal language registers could be example factors.
The complicated system of kanji and the number of kanji characters could be reasons for the difficulty of its acquisition. There are three types of Japanese
script: hiragana, katakana, and kanji. Hiragana and katakana are phonetic scripts; each character represents one sound and does not represent any particular meaning. Ootsuki (2010) commented that Japanese is not very different from other languages in terms of the difficulty in acquiring the system, despite its unique grammar; however, the most difficult part of Japanese learning is the script system. Douglas (2010) also found many JHL students insist that kanji is the hardest part of learning Japanese. According to a large-scale international investigation of the experience of hoshuu-koo teachers, findings indicated that many JHL learners do not reach age-appropriate levels of kanji proficiency (Kataoka & Shibata, 2011). Nakajima (2003) also found that JHL learners' kanji proficiency does not necessarily progress as they grow older. Findings from research conducted by Douglas (2008), reported that JHL students' reading skills are very similar to the skill levels of second language learners at university. Moreover, Douglas (2010) found that university JHL students have problems in writing and reading combination kanji words amongst kanji components, although they can read kanji within context relatively well. JHL learners usually have insufficient opportunities to be exposed to spoken and written texts or enough opportunities to use the language in their daily lives, which affects the amount of acquired age-appropriate vocabulary and kanji (Kataoka & Shibata, 2011). Unlike second language learners, the amount of kanji that is expected to be remembered is large and students may not be able to grasp how much kanji they maintain. Thus, it is suspected that students have insufficient recognition of their own kanji proficiency.
Blanche and Merino (1989) mentioned that self-evaluation is a source of information about abilities and progress provided by learners themselves. This information also includes learners' expectations, needs, and worries (Harris & McCann, 1994). It has been said that students can become aware of their progress by being given the opportunities to assess themselves and by being made responsible for their own learning (Dickinson, 1987; Harris, 1997). As a result, self-assessment helps students become active participants in their education (Joo, 2016; Sloan, 1996) . Blanche and Merino (1989) also insisted that metacognitive skills, as well as student proficiency, can be developed by adopting this approach. Thus, it can be said that the effect of a self-evaluation system in language education and how students then self-rate their proficiency could be a key point in making the system valuable. Research has found several tendencies which affect the accuracy of self-evaluation, such as learning environment and negative relations with family and school environment (Konaszewski & Sosnowski, 2017). The level of proficiency and length of learning
are reported as affective factors. Gertsen's (2006) research proved that students who studied over a longer period can evaluate their skills more accurately. Moreover, the higher that language proficiency is, the more accurate the self-rating (Davidson & Henning, 1985; Heilenman, 1990). Komori and Fujisawa (2004) conducted research on the same students over a span of two years and found that students' self-evaluation abilities increased with improvements in proficiency; however, the increasing rate of self-evaluation was different depending on the proficiency. Students who obtained high scores in tests believed in their improvement, whereas students who did not perform well on the tests rated themselves lower than their actual improvement. Although a correlation between accuracy of self-evaluation and proficiency was found, this does not mean that high proficiency always leads to accurate self-assessment. Yoshizawa (2009) found a correlation between self-assessment and the difficulty of reading texts but no significant relation with listening. Therefore, the fields of learning relating to self-evaluation and actual proficiency vary.
This research investigated whether the relationship between self-evaluation skills and kanji proficiency of adolescents could be identified. The importance of self-evaluation skills and the relationship with language proficiency have been discussed in prior research (Bandura, 1997); however, not enough research has been conducted in the JHL area, especially for adolescents whose meta-cognitive skills are developed more than elementary school students and thus are at the age where enhancing one's own self-evaluating skills and organising one's own learning style are expected. In this research, a focus on the kanji learning of adolescents, the appropriateness of self-evaluation, and the elements which may correlate with it are explored in the expectation of improving kanji proficiency.
The context for the study was a hoshuu-koo in Australia. The participants were students in Year 7 and the researcher was a participant-observer. It was found through the researcher's observations of students during class, and the results of kanji tests, that not many students recognised their own kanji proficiency and gave up memorising kanji. It is assumed that students are able to improve their kanji proficiency and participate in learning actively by establishing and developing their learning styles if they evaluate their own proficiency appropriately and recognise which areas of kanji they are competent in, as kanji has several elements such as compounds words and okurigana. Okurigana are kana suffixes following kanji that show grammatical functions of the word. In other words, appropriate self-evaluation is one of the ways to improve kanji proficiency and helps in setting up
appropriate goals. Moreover, finding the elements that influence the appropriateness of self-evaluation skills and characteristics of evaluation skills would be a cue for improving both teaching methods for educators and organising the environment at home as well as to enhance evaluation competence for students.
Based on the hypothesis mentioned above, this study will explore: How does self-evaluation of kanji proficiency contribute to improving kanji proficiency? To address this question, the research aimed to explore:
• How well does each student understand their own proficiency? Is there any relationship between kanji proficiency and self-evaluation skills?
• In which components of kanji do students recognise their own skills appropriately?
• Are there any elements that influence kanji self-evaluation?
Methodology and methods
Participants
Year 7 students studying Japanese at an Australian hoshuu-koo in 2017 were approached to participate in this research. There was no control group and in total 31 students agreed to join this research, with consent forms obtained from the participants, their parents, and the school. Most of the students were born in Australia and started studying Japanese at this hoshuu-koo when they were in Year 1. Pseudonyms have been used throughout the study.
Research design
Quantitative approaches to data collection and analysis were used in this research. Quantitative research helps to build a theory and a theoretical frame that reflects a reality and observes a phenomenon in detail. The data were collected when the participants entered Year 7 after graduating from elementary school (Year 6) and the contents of the test included areas introduced before the end of Year 6. The data were analysed to determine how accurately JHL students analysed their kanji skills and to identify what specific elements influenced the accuracy of their self-evaluation. The data from the questionnaires and kanji tests was assessed and the results used to detect emerging patterns in student self-evaluation skills and learning environment. The Excel Correl Function was used to calculate correlation coefficients. Graphs were used to analyse the data and trends of correlation amongst each element were explored.
Materials
Materials used in this research comprised tests and questionnaires.
Tests:
Kanji that was introduced during elementary school was tested. Ten words that used kanji from each year level were selected for reading and writing, respectively, and okurigana and the use of kanji in context were also tested and each student's performance analysed. Words in the test included both single kanji words and compound kanji words. Besides this, the five most frequently used radicals were selected and the students were required to write words that included kanji with each radical.
Questionnaires:
The following items were asked in the questionnaires:
• Confidence level in kanji and radicals for each year level
• Confidence rating for areas within kanji learning (reading, writing, compound words, radicals, meaning, and okurigana)
• Details of the learning environment, including:
- Number of family members and friends who speak Japanese
- Ratio of the use of Japanese at home
- Items in Japanese that students have access to, such as books, websites, and movies
- The number of Japanese books that students possess
Procedure:
An examination was made of the following points:
1. Differences between actual scores in the reading kanji test and students' self-confidence
2. Differences between actual scores in the writing kanji test and students' self-confidence
3. Relationship between reading and writing kanji scores, radical scores and differences with students' confidence of radicals
4. Relationship between confidence in reading kanji and actual performance in kanji components for each year level of kanji
5. Relationship between students' confidence in writing kanji and their actual performance in kanji components for each year level of kanji
6. Relationship between students' individual confidence in the components of kanji and their actual performances for each component
7. Correlation between self-evaluation and learning environments
Results
Using the results of the kanji tests and questionnaires, differences between actual proficiency and self-evaluation were calculated for each area of kanji. Correlation coefficients between the results and the learning environment were calculated using the Excel Correl Function to see if any relationships were apparent.
Correlation between self-evaluation and kanji reading, writing, and radical test results
Procedure 1
First, the correlation coefficients between reading test results and differences with self-evaluation were calculated. The correlation coefficients were 0.768; therefore, a strong positive linear relationship was found. This indicates that when students have higher proficiency in reading kanji, differences between actual proficiency and self-evaluation are smaller.
Figure 1 below shows the differences between self-evaluation and actual performance based on students' individual reading scores. The graph is sorted by score. It was found that students who obtained more than 40 out of 100 in reading tests had fewer differences between actual test scores and their self-evaluation and most of the students who obtained more than 80 showed an underestimated self-evaluation. This means that the higher the kanji reading skills that students had, the more accurate their self-evaluation was.
Next, the individual accuracy of self-evaluation was graphed by each year level of kanji and compared to reading scores (Figure 2). Overall, bigger differences between self-evaluation and actual performance were found in Years 2, 3, and 4 levels of kanji than others. Most of the students who obtained less than 40 in the tests overestimated their skills when average differences were focused on; however, some students underestimated their skills more in Year 4 levels. Besides this, some of the students who achieved 80 in the kanji reading tests showed a degree of overestimation in the lower level years of kanji learning.
Procedure 2
Kanji writing test scores and the differences
Differences in self-evaluation and reading score
50
-50
| I I I I I I I.......
-100
Accuracy Reading score
Figure 1. Accuracy of self-evaluation in reading kanji and individual reading scores.
between self-evaluation and actual scores are displayed in Graph 3. This graph is sorted by score; the correlation coefficient between the accuracy of self-evaluation and test scores was 0.507. A moderate positive relationship was found and this is lower than the reading tests. Most of the students overestimated considerably and under 20% accuracy rates were found amongst students who obtained more than 60 out of 100 in the kanji writing tests. However, differences varied amongst the students whose scores were less than 60. Moreover, few students underestimated, indicating that the students who achieved high-level scores in kanji writing recognised their skills quite accurately or only slightly underestimated them.
Individual performance in each year level for kanji and writing scores are graphed below (Figure 4). A relatively high overestimation was found in Year 2 and Year 3 level kanji amongst the students who obtained less than 40 out of 100 in the tests. Overestimation in Year 4 level was found in most of the students, regardless of the scores. Overestimation in Year 5 and
Year 6 levels was mainly found amongst the students who achieved more than 20 in the tests. The number of the students who underestimated was smaller than in the reading test but this was found mainly amongst the students who achieved between 20 and 30 in Year 1 and Year 2. Similar to the reading tests, the students who attained higher scores tended to recognise their writing skills more accurately than others.
Procedure 3
The graphs below show the relations between the accuracy of self-evaluation skills of radicals and kanji reading and writing scores. The graphs are sorted by scores.
Figure 5 indicates that the majority of the students whose reading score was higher than 40 underestimated their radical skills, while other students varied regardless of their score.
In Figure 6, sorted by writing score, it was found that the higher scores that students achieved, the more underestimation they had, although there were some
Accuracy of self-evaluation in reading for each year level of kanji and reading scores
100
50
-50
-100
Jl
rfffifrf'fTriiri^
11 j Jl J ! .
1 2 3 4 5 6 Reading score
Figure 2. Individual accuracy of self-evaluation for each year level of kanji reading.
0
0
Accuracy of self-evaluation and writing score
100 50 0 -50 -100
pTlll'|l'l''||ll'lllllll""
Qü2'e ° a o tu Qü^'e S 'P S «'Cia'C a, tu -G 3'P
E-^ g 2 5 S u z J ¡3 4g | I K £
E-h u_
Accuracy
* 3 1=
■Writing score
o o «
a -C o to
a I = I * I I
Figure 3. Accuracy of self-evaluation in writing kanji and individual writing scores.
Accuracy of self-evaluation in writing for each year level of kanji and writing scores
100
50
-50
II I l I
-100
f[f|Ff'MI""|f|MIIM|>
T"I ifl 'i
1 2 3 4 5 6 Writing score
Figure 4. Individual accuracy of self-evaluation for each year level of kanji writing.
Accuracy of self-evaluation in radicals and reading scores
120 100 80 60 40
jJ-l-rHtl.fhllillL
Accuracy Reading score
Figure 5. Accuracy of self-evaluation in radicals and reading scores.
0
0
variations. This means that the students whose scores were high in the writing tests tended to underestimate their radical skills, and this was similar to the trend in the reading scores.
Moreover, the accuracy rates as sorted by radicals scores below (Figure 7) show more than 50% underestimation, mainly amongst the students who achieved relatively high scores in radicals tests.
2. Analysis by kanji components
How individual students recognised each component of kanji skills was analysed. Confidence in reading and writing kanji for each year level and the actual scores for single kanji, compound kanji words, the use of words and okurigana in context were graphed. The students were required to make sentences using specific words that were provided in order to enable an examination of how much students understood the meaning of the words and if the students could use them in context. The graphs were
sorted by confidence levels. Figure 8 to Fugure 13 and Figure 14 to 19 represent the confidence levels and the performance in each kanji component for each year level of kanji reading and writing, respectively.
Procedure 4
First, the relationship between the self-evaluation in kanji reading and the performance of each kanji component was analysed.
Year 1 level of kanji reading
Most of the students answered they were 100% confident and their performance in reading single kanji was almost 100%. Some overestimated reading compound kanji words but the students who had higher confidence obtained better scores in compound kanji words. In contrast, the scores for the use of kanji in context varied regardless of confidence levels and actual reading kanji scores and it was found that some students could use the words even if they could not read the kanji correctly.
120 100 80 60 40
20
Accuracy of self-evaluation in radicals and writing scores
lL^Ui i rtHt4fl
rttTil
-20
Accuracy Writing score
Figure 6. Accuracy of self-evaluation in radicals and writing scores.
0
Accuracy of self-evaluation in radicals and radicals scores
Accuracy Radical score
Figure 7. Radicals self-evaluation results and actual scores.
Single kanji Compound kanji words Use of kanji Confidence
Noriko
Fumiya Chie NeneToki
Figure 8. Accuracy of self-evaluation for each kanji component: Year 1 kanji reading.
Year 2 level of kanji reading Almost half of the students had 100% confidence in reading Year 2 level kanji. Most performed perfectly in reading single kanji, similar to Year 1 levels; however, variations in reading compound kanji words were also found. Even among the students who had 100% confidence, not all achieved 100% in reading compound kanji words, although quite a big variation was found amongst the students who had lower confidence. Regarding the use of kanji in context, the line in Graph 9 shows almost the same trend as reading compound kanji words and this indicates that the students who could read kanji usually understood the use of words in context.
Reading Year 2
Single kanji Compound kanji words Use of kanji Confidence
Noriko
Figure 9. Accuracy of self-evaluation in each kanji component: Year 2 kanji reading.
Year 3 level of kanji reading
At YYear 3 level, the number of the students who showed 100% confidence was lower than Year 1 and Year 2 levels and the differences between confidence and actual performance levels became smaller. It
was found that overall, the students who had higher confidence performed better in reading kanji for all components. Most of the students whose confidence was under 90% performed better than they believed in reading single kanji, while the majority of them could not reach their confidence levels when reading compound kanji words. Similar trends were observed in the use of kanji in reading.
Reading Year 3
Single kanji Compound kanji words Use of kanji — Confidence
Noriko
Figure 10. Accuracy of self-evaluation in each kanji component: Year 3 kanji reading.
Year 4 level of kanji reading
Around one-quarter of the students answered with 100% confidence in Year 4 level kanji reading, and nine students answered they had approximately 50% confidence. Some students performed almost the same as they believed and most of the others performed better in single kanji reading; however, few students obtained better scores than they recognised in compound kanji words. The students who had more than 80% confidence performed similarly in compound kanji words and the use of words in context but a big variation in the use of words was found amongst students who did not reach 80% confidence level.
Year 5 level of kanji reading
Almost half of the students answered that they had less than 50% confidence in reading Year 5 level kanji. The level of confidence and the performance level of both single and compound kanji words showed similar trends amongst the students whose confidence level was more than 80%. However, a performance level that was lower than the reported confidence level for the use of kanji in context and compound words was found amongst the students who had less than 80% confidence.
Year 6 level of kanji reading
For Year 6 level, the trend in the confidence level was similar to the Year 5 level kanji reading. The students who had more than 80% confidence performed almost to the same level in single and compound kanji words.
■ Single kanji
»Compound kanji words
jiKaz400
Use of kanji
Nuka
Terumi
Tsurumi
Kohji Kurumi
Eàmiko FumiSachi
Mao Naoki
Figure 11. Accuracy of self-evaluation for each kanji component: Year 4 kanji reading
•Single kanji
Reading Year 5 ■Compound kanji words «
■Use of kanji
were rarely found for the single and compound kanji words and large variations were observed in the use of kanji amongst the students whose confidence was under 70%.
Procedure 5
Next, the self-evaluation skills in kanji writing will be analysed. Single kanji, compound kanji words, and okurigana were included in the kanji writing analysis.
Year 1 level of kanji writing
The number of the students who had 100% confidence in kanji writing was smaller compared to kanji reading. Most of the students' confidence levels were distributed between 70% and 100% and some performed better than their confidence levels. Actual performance in single kanji writing shows almost the same trends as confidence levels, with some variations. The accuracy of okurigana varied regardless of confidence levels but actual scores in compound kanji words and okurigana did not correspond with confidence levels
Kaz400
Hiro
Tsurumi Terumi Nuka
Harumi Hokuto Heiri Nene Mao Niko Toki
-Single kanji
Reading Year 6
■Compound kanji words Noriko
Kazuinn
■Use of kanji
■Single kanji "
Writing Year 1
■Compound kanji words
Okurigana
Nuka
Seijloo Naoki
1100 Chie
Fumif>oki
Figure 12. Accuracy of self-evaluation in each kanji component: Year 5 kanji reading.
Otoha Ichiroh
Fumij^
Figure 13. Accuracy of self-evaluation in each kanji component: Year 6 kanji reading.
Higher performance levels than confidence levels
Figure 14. Accuracy of self-evaluation in each kanji component: Year 1 kanji writing.
Year 2 level of kanji writing
Approximately half of the students had more than 80% confidence. However, few students reached their confidence levels in both single and compound kanji words. The students whose confidence levels were more than 80% obtained 60% or more in actual tests but only a few other students obtained high scores. The performance in okurigana varied regardless of confidence levels.
Year 3 level of kanji writing
At Year the 3 level, the number of students who had more than 90% confidence was similar to the Year 2 level; however, the confidence levels of other students were obviously lower than Year 1 and Year 2 levels. For single and compound kanji words or okurigana, no clear relations with confidence levels could be
»Single kanji
Writing Year 2
■Compound kanji words —
Noriko
Okurigana
Single kanji
Writing Year 4
■Compound kanji words — Noriko
■Okurigana «
Kazu100 80
Kohji Sumiko
Otohaimiya
Terumi
Tsurumi
Heiri Mao Harumi Toki Nene Niko Chie Sachi
E&miko FumiyN^oki
Hokuto Nuka
Figure 15. Accuracy of self-evaluation in each of the Figure 17. Accuracy of self-evaluation for each kanji kanji components: Year 2 kanji writing. component: Year 4 kanji writing.
observed and the majority of students did not reach the confidence level in actual scores. It was also found that even the students who could correctly write single kanji did not perform well in okurigana.
»Single kanji
Writing Year 3 ■ Compound kanji words —
»Okurigana «
Takeo Kazu Ichiroh
Noriko Akiraioo___Ts urumi
Terumi Hiro
Heiri Nuka Mao Harumi Toki Chie Nene Niko
Fumiya Naoki
KurumSuimiko
Figure 16. Accuracy of self-evaluation for each kanji component: Year 3 kanji writing.
Year 4 level of kanji writing
Similar trends in confidence at the Year 3 level were found in the Year 4 level of kanji writing, although the scores and confidence levels were smaller than the Year 3 level. Similar to the Year 3 level, it was rare that actual scores were higher than confidence levels. The shapes of single kanji, compound kanji words and okurigana lines for each student were very similar. This means that the performance in single, compound kanji words and okurigana were almost the same; therefore, the students who reached certain levels in single kanji could reach a similar level in other components but no clear correlation was recognised with confidence levels.
Year 5 level of kanji writing
Different trends were found in Year 5 level kanji writing compared to Year 1 to 4 levels. Twelve students had more than 70% confidence and big variations were found in the performance in the tests amongst these students, although most of them performed better in single kanji and okurigana than in compound kanji words. The students whose confidence was under 70% did not write any kanji or wrote incorrect kanji.
Writing Year 5
Single kanji Compound kanji words Okurigana Confidence
Haru mi
Seiji Kazui _______Hiro Noriko
Ichiroh Terumi
80
Takeo Tsurumi
Akira 60 Heiri
Umi \/ Mao
Keisuke Toki
Shiroh — A Nene
Kurumi Nuka
Kohji Chie
Otoha Niko
Sora Sachi
Kimio Hokuto
SumikFoumiya NaokiEri
Figure 18. Accuracy of self-evaluation for each kanji component: Year 5 kanji writing.
Year 6 level of kanji writing
Overall, the figures for Year 6 were the same as the Year 5 level. Students whose confidence was more than 80% reached certain scores in single kanji, compound kanji words, and okurigana with some fluctuations. More students performed better in okurigana compared to other year levels of kanji; however, other students barely obtained scores in any component and they mostly underestimated their skills.
Single kanji Compound kanji words Okurigana Confidence Harumi
Otoha Niko
Sora Sachi
Kimio Hokuto
SumiFWiya Naokfri
Figure 19. Accuracy of self-evaluation for each kanji component: Year 6 kanji writing.
3. Individual analysis of each kanji component.
In the previous section, the confidence levels of students in reading and writing kanji and their actual performance levels in each of the components of kanji were compared. This section will examine whether or not individuals' confidence in the components of kanji and their actual performances matched.
Procedure 6
In the questionnaires, the students were asked to rate their proficiency in each area of kanji learning. The actual results of the tests were then compared with the self-ratings. The areas included kanji reading, writing, compound kanji words, radicals, the use of kanji words in context, and okurigana. The graph below (Figure 20) shows the individual confidence order of kanji components and is sorted by the total scores of kanji reading and writing tests.
Confidence order
reading writing compound
radical use of kanji okurigana
Hokuto
Figure 20. Individual confidence - order and actual performance.
It was found that reading kanji was recognised by the largest number of students as being the easiest part of kanji learning, followed by okurigana. Most students chose compound kanji words as their least or second least confident area. It was also found that the students who achieved higher scores in the tests answered that reading was their most confident area, while amongst the students whose scores were not high, the most confident areas varied.
Next, the individual order of confident areas and actual scores in each area were compared. The most confident area in the questionnaires was marked as "6" and the least area as "1". The areas in which each student performed the best and worst in the test were marked as "6" and "1", respectively. The differences between each students' confidence order and actual competent areas were calculated and graphed below (Graph 21). Positive numbers in the graph indicate that the student performed better in the test than he/ she believed they would in that specific area, and the negative numbers indicate that his/her performance in other areas was worse than they predicted. The graph was sorted by the totals of reading and writing test scores. On the right-hand side is the student who achieved the highest score in the tests.
It was found that figures for radicals were high overall. This indicates that most of the students considered radicals as their least confident area but they could actually use radicals properly in making kanji. In contrast, the line for okurigana shows mainly negative figures. This indicates that most of the students recognised they were competent in okurigana but they could not actually perform well in the tests. The line for kanji writing shows both positive and negative figures. Students' recognition of writing skills varies; some recognised they could perform in writing other elements and others diagnosed the opposite, regardless of their actual performance skills. Especially, the figures for the students who achieved high scores were mostly negative. This means that the students who reached high scores in the total of reading and writing kanji recognised writing kanji as the hardest area, although they could perform relatively well in writing compared to other skills. In contrast, relatively small differences between confidence orders and actual test results were found in reading kanji for most of the students. Most students recognised reading kanji as their most competent area and they actually performed best in this area. Regarding compound kanji words, more positive figures were found than negative figures. This means that more students recognised it as a difficult area compared to other areas but their performance in compound kanji words was actually better than other areas, more or less. However, the figures remained between +2 and -2, indicating that there were no big differences between their recognition and actual skills in compound kanji
words.
4. Correlation between self-evaluation and learning environments
Finally, whether the learning environment at home related to the accuracy of self-evaluation was explored.
Procedure 7
The data from the questionnaires were used to determine each students' learning environment at home. Five types of information were gathered: access to types of Japanese media at home; those people who speak in Japanese; the ratio of the use of Japanese at home; the number of friends who speak in Japanese; and the number of books written in Japanese that they have at home. Japanese media included books, television, websites, games, letters, and magazines. The types of people who speak in Japanese included parents, relatives who live in Japan, siblings, and friends. The graphs were sorted by the accuracy of self-evaluation in reading and writing scores. The left-hand side of the graphs shows the students who overestimated their
own skills the most; reading and writing kanji scores were graphed separately. The correlation coefficients between the learning environment and reading and writing scores are summarised in Table 1.
Table 1
Correlation coefficients between learning environment and accuracy of reading and writing self-evaluation
Reading Writing
Use of Japanese at home 0.54 0.28
Number of books written in Japanese 0.49 0.005
Access to Japanese media at home 0.19 0.02
People who use Japanese with the students 0.17 0.38
Number of friends the students speak to in 0.13 0.14
Japanese
Figure 22 below shows the accuracy of self-evaluation in reading kanji, the ratio of the use of Japanese at home and the number of books written in Japanese. Regarding the use of Japanese at home, the students on the left-hand side of the graph used Japanese less than 60% at home, and most of the others
Confidence order and rest results
6
Reading Writing Compound kanji Radical use of kanji words Okurigana
Reading differences and learning enviounment 1
Reading differences Ratio of use of Japanese Number of books
Figure 22. Self-evaluation accuracy in reading kanji and the learning environment 1.
Figure 21. The difference between confidence order and actual test performance order in kanji components.
used it more than 60%. This means that the students who used Japanese at home had a better understanding of their own kanji reading skills. Similar trends can be seen for the number possessing books written in Japanese, including manga. Differences between self-evaluation and actual scores were less than 30% for most of the students who had more than 200 books, although variations were observed. Correlation coefficients also indicated a moderate relationship between the possession of books in Japanese and accuracy of self-evaluation in reading kanji.
The next graph (Figure 23) shows the accuracy of self-evaluation in reading kanji, the types of access to Japanese at home, those people the students speak to in Japanese, and the number of friends who speak in Japanese. Twelve students had more than twenty friends who used Japanese when they spoke but the line in the graph has big fluctuations; some students self-evaluated their kanji reading skills relatively accurately while others did not. Correlation coefficients also prove quite weak relations. Also, access to types of media in Japanese at home and people who use Japanese at home did not influence
self-evaluation skills.
Prior research has recognised higher Japanese proficiency amongst students who use Japanese with their siblings (Nakajima, 1998). The differences between self-evaluation and actual scores in kanji reading tests and the use of Japanese among siblings were examined to see if the same could be said for the accuracy of self-evaluation skills. The scores "1" and "0" on the right-hand side of Graph 24 indicate whether students used Japanese with siblings or not, respectively. The graph indicates a weak relationship between the use of Japanese amongst siblings and the accuracy of self-evaluation in reading.
Figure 25, below, shows the accuracy of self-evaluation in writing kanji, the ratio of the use of Japanese at home, and the possession of the number of books written in Japanese. Another graph (Graph 26) shows the accuracy of self-evaluation in reading kanji, types of access to Japanese at home, people that the students speak to in Japanese, and the number of friends who speak in Japanese. Regarding the ratio of the use of Japanese at home, students who used more Japanese at home recorded slightly higher
Figure 23. Self-evaluation accuracy in reading kanji and the learning environment 2.
Reading differences and use of Japanese within siblings
......Y&MMJXM
-80 0
Reading average siblings
Figure 24. Self-evaluation accuracy in reading kanji and the use of Japanese amongst siblings.
accuracy levels in their self-evaluation than others. In contrast, the opposite trend can be seen in Figure 26 showing the people that students speak to in Japanese. Correlation coefficients for these items indicate they were slightly related. The types of media in Japanese that students have access to at home and the number of friends that students speak to in Japanese did not show any clear relationship in terms of the accuracy of self-evaluation skills. Conversation with siblings also showed no clear relationship with the accuracy of self-evaluation in kanji writing (Figure 27).
Discussion
In this section, the research questions will be discussed based on the results of the analysis. It was predicted that the more accurately students evaluate their competence, the higher their kanji proficiency. It was also predicted that the learning environment was another condition that influenced the accuracy of self-evaluation. The research questions below were established based on the predictions above.
Writing diffences and learning environment 1
600 - cnn 40 20
/inn . 1 1 n 0
4UU 300 ........................... -20
200 -40
100 ■ ,____\ ___ -60
0 writing average Ratio of use of Japanese Number of books -80
Figure 25. Self-evaluation accuracy in writing kanji and the learning environment 1.
Writing differences and learning environment 2
25 20 15 10 5 0
[jMjmMMjn^^
40 20 0 -20 -40 -60 -80
cp' " \<> ^ % N ^ ^
■ writing average Type of access to Japanese
Number/type of people
Number of friends
Figure 26. Self-evaluation accuracy in writing kanji and the learning environment 2.
Writing differences and use of Japanese within siblings
30 - 1,2
20 -
writing average siblings
Figure 27. Self-evaluation accuracy in writing kanji and use of Japanese amongst siblings.
How does self-evaluation of kanji proficiency contribute to improving kanji proficiency?
1. How well does each student understand their own proficiency?
In order to explore the relationship between self-evaluation skills and kanji proficiency, how each student recognises his/her kanji skills and performance in kanji reading and writing tests was examined. Overall, it was proven that students who could self-evaluate appropriately performed better than others.
Different tendencies in accuracy of self-evaluation were found between kanji reading and writing. First, it can be said that students who had high proficiency tended to understand their own skills relatively accurately compared to other students, which is similar to what Davidson and Henning (1985) insisted (See Procedure 1 in the Results section). However, it seems harder for students to recognise their own proficiency in writing than reading overall, and most of the students overestimated here. That means that the students believed they could write kanji correctly but actually they did not perform as well as they expected.
It is also assumed that students tend to judge their own proficiency based on their kanji reading skills in terms of the accuracy of their self-evaluation in reading was higher than in writing and the difference in the accuracy of their self-evaluation was bigger in writing than in reading. Although Maehr and Stallings (1972) insisted that learners tend to judge their skills more accurately in difficult tasks than easier tasks, this was not observed in the kanji writing tasks; big differences between self-evaluation and actual scores in the tests for higher year levels of kanji were found amongst the students who scored in the middle range. Moreover, some students who recorded in the middle range or lower in the scores for the lower year levels of kanji often overestimated their kanji proficiency. This may be because they believed they understood the use of lower year level kanji, although they sometimes read and wrote them incorrectly. When students feel confident with those kanji, they might not pay enough attention to writing or reading the kanji. However, some students in the middle range underestimated for lower year level kanji. Therefore, quite big variations in the accuracy of their self-evaluation were identified amongst the students in the middle range compared to other students; thus, it would be necessary to devise learning methods for these students (Procedure 2).
2. In which components of kanji do students recognise their own skills appropriately?
Accuracy in the self-evaluation in the components of kanji varied depending on the difficulty of the kanji. Regarding the accuracy of self-evaluation for each year level, different tendencies between reading and
writing were found.
Although lower confidence was found as the year levels increased in reading, it can be said that most of the students reached the levels they believed that they were at for single kanji in the lower year levels but variations were found in compound kanji words and the use of kanji in context regardless of the level of confidence. In the higher year levels of kanji, in contrast, performance levels of single kanji, compound kanji words, and the use of kanji in context drew closer to the confidence levels with some variations amongst the students who performed well in the test. Moreover, it was found that these students could usually read compound kanji words if they could read single kanji, even at higher levels of kanji. In summary, it can be said that the students who had high confidence performed almost the same as their confidence levels for all components of kanji; in contrast, the students whose confidence was relatively low did not reach their confidence levels at any year levels of kanji, except for single kanji. Therefore, it can also be assumed that students recognised their kanji reading skills based on their understanding of single kanji rather than other components (Procedure 4 & 6).
Regarding kanji writing, little correlation was found between confidence and actual performance in the lower year levels of kanji. However, as the year levels became higher, several trends started to be observed: confidence levels became lower and the number of students who reached higher scores than they estimated became smaller, and no clear relations were found between performance and confidence level, even for single kanji. Incorrect okurigana were often found even if single kanji were written correctly in the lower year levels; however, the use of correct okurigana, regardless of accuracy of single kanji, were observed in higher year levels of kanji amongst the students who had relatively high confidence. An imbalance between single kanji and compound kanji words was found in writing kanji for all year levels although some similar performance was found in lower year levels for reading kanji. Confidence in reading corresponded with single kanji reading for all year levels but confidence levels were not achieved in writing for either single or compound kanji words, especially at higher year levels of kanji. Besides this, fewer students did not reach their confidence levels as the difficulty of kanji increased, but differences between performance of single kanji, compound words, and okurigana became smaller amongst the students who had certain levels of confidence (Procedure 5 & 6).
Regarding the self-evaluation of radicals, most of the students underestimated regardless of the actual test scores in radicals and kanji reading and writing tests. Especially, students who obtained relatively high scores in kanji reading and writing tests showed
a high level of underestimation regardless of the scores in radicals tests. This may indicate that most of the students did not have a positive attitude toward learning radicals. Therefore, enabling them to recognise their actual knowledge of radicals would be important to elicit positive perspectives (Procedure 3 & 6).
Thus, it can be said that differences between confidence and actual performance can be identified depending on the components of kanji and different year levels and that the differences are especially big amongst the students who are in the middle range or lower. Regardless of performance in the tests, an underestimation was found in radicals. Therefore, focusing on compound kanji words and okurigana practice for this range of students, and radicals recognition for all students, would be necessary.
3. Are there any elements that influence kanji self-evaluation?
It was predicted that the Japanese language environment at home related to the accuracy of self-evaluation of kanji proficiency. It was found that the number of books written in Japanese that students possessed and the ratio of the use of spoken Japanese at home were positively related to the appropriateness of self-recognition of kanji proficiency in reading, although a clear relationship with the types of people who speak in Japanese, such as parents or siblings, and access to Japanese media such as videos and magazines was not found. No correlation between kanji writing and the learning environment at home in the use of Japanese was found in this research. It is supposed that the students have more opportunities to come across unknown words or be corrected for the wrong use of Japanese when they use more Japanese at home and this may help them in understanding their own proficiency more appropriately. Prior research has proven that the use of Japanese among siblings positively influences Japanese proficiency (Nakajima, 1998); however, this cannot be said for appropriateness of self-evaluation. Regardless, it is expected that the recognition of kanji reading proficiency and the amount of vocabulary would be improved if students had more opportunities to be exposed to Japanese script and communication (Procedure 7).
Conclusion
Baased on the analysis of the results and discussion, implications for further teaching and study will be explored. As it was proven that students who had high kanji proficiency were able to identify their own proficiency more accurately than other students,
it would be necessary to investigate an effective approach to enhance self-evaluation skills, especially focusing on students who are in the middle level.
It was predicted that students tend to have an optimistic view or identify their own kanji proficiency ambiguously and that seems to make students overestimate their own skills. It would be efficient to enable students to develop a habit of using dictionaries all the time, especially in writing composition. The use of okurigana should be carefully checked by teachers or peers. Moreover, adapting regular mini quizzes into PowerPoint slides (for example) as a quick check of the reading and writing skills of each year level of kanji could enhance students' recognition of their own skills and increase confidence. Mini quizzes on slides might give students less pressure compared to formal paper tests.
Moreover, focusing on recognition skills for reading kanji would be helpful to enhance overall self-evaluation skills as students tend to judge their own skills based on reading kanji skills. Therefore, it is expected that reading kanji materials aloud may help students improve their self-evaluation skills. At the same time, letting students write furigana (kanji reading) would be important as minor errors, such as missing double consonants and long vowels, are rarely found by reading aloud. In this way, it is expected that students can identify what they do not understand correctly.
To improve the underestimation on radicals skills, giving students more awareness of radicals when introducing and reviewing kanji regularly in class would be necessary. Providing positive feedback on radicals could be expected to allow students to be more confident.
As it was also found that conversation at home and materials written in Japanese were effective for enhancing kanji reading self-evaluation skills, collaborating with students' families would be also necessary.
Employing the approaches discussed so far could be expected to improve students' self-evaluation skills. This research was conducted based on data collected at the time when students had just finished Year 6 study and it would be valuable if data could be collected one year after this research to see if the approach is effective. A continuous research period over the longer term for the same students would be necessary.
Finally, the amount of kanji for each year level in the tests was limited. It is expected that more accurate data could be obtained if the number of kanji in the tests is increased; however, this could be too much of a burden for students. More accurate figures would be available if data was taken from writing passage tasks and the reading of textbooks throughout the year.
References
Aiko, M. (2017). Learning Japanese as a heritage language : The home school environment. European Journal of Foreign Language Teaching 2(3), 103-130.
Bandura, A. (1997). Self-efficacy: The exercise of control. NY: W.H. Freeman.
Blanche, P., & Merino, B. J. (1989). Self assessment of foreign-language skills: Implications for teachers and researchers. Language learning, 39, 313-338.
Calder, T. (2008). Hoshuukoo ni okeru bogoshien: Princeton community Japanese langauge school no jissen kara. [Hoshuukoo ni okeru bogoshien: Princeton community Japanese langauge school no jissen kara]. Paper presented at the Bairiteraru Baikarucharu no Ikusei o Mezashite Jissen to Kadai, Obirin University. Retrieved from http://harmonica-cld. com/wp/wp-contentuploads/2014/07/0e59c8c4ee5 acced9054666cc29e0b42.pdf
Davidson, F., & Henning, G. (1985). A self-rating scale of English difficulty: Rasch scalar analysis of items and rating categories. Language Testing, 2, 164-179.
Dickinson, L. (1987). Self-instruction in language learning. Cambridge, UK: Cambridge University Press.
Doerr, N., & Lee, K. (2010). Inheriting "Japanese-ness" diversely: Heritage practice at a weekend Japanese language school in the United States. Critical Asian Studies, 42(2), 191-216.
Doerra, N. M., & Leeb, K. (2009). Contesting heritage: language, legitimacy, and schooling at a weekend Japanese-language school in the United States. Language and Education, 23(5), 425-441.
Douglas, M. (2008). Curriculum design for young learners of Japanese as heritage language. In K. Kondo-Brown. & J. D. Brown. (Eds.), Teaching Chinese, Japanese and Korean Heritage Language Students: Curriculum Needs, Materials, and Assessment (pp. 237-270). NY: Lawrence Erlbaum.
Douglas, M. (2010). Analysis of kanji ability of heritage learners of Japanese. Japanese Heritage Language Education, 3, 1-24.
Douglas, M., & Chinen, K. (2014). Keishou nihongo kyouiku. [Heritage Japanese langauge education]. In Y. Tosaku (Ed.), Japanese Language Education in the U.S. --Past, Present, and Future. Broadway, NY: American Association of Teachers of Japanese.
Gertsen, C. (2006). Self evaluation in the Dutch Language portfolio: Utrecht University Repository.
Harris, M. (1997). Self-assessment of language learning in formal settings. ELT Journal, 51(1), 12-20.
Harris, M., & McCann, P. (1994). Assessment. Oxford, UK: Heinemann.
Heilenman, L. K. (1990). Self-assessment of second language ability. The role of response effects.
Language Testing, 7, 174-201.
Joo, S. H. (2016). Self- and peer-assessment of speaking. Applied Linguistics and Teaching English to Speakers of Other Languages, 16(2), 68-83.
Kataoka, H., Koshiyama, Y., & Shibata, S. (2008). Japanese and English language ability of students at supplementary Japanese schools in the United States. In K. Kondo-Brown. & J. Brown (Eds.), Teaching Chinese, Japanese and Korean Heritage Language Students: Curriculum Needs, Materials, and Assessment (pp. 47-76). NY: Lawrence Erlbaum.
Kataoka, H., & Shibata, S. (2011). Japanese language proficiency and home language use among children of international marriages: Breaking free from common assumptions. Online Heritage Joural, 4, 1-40.
Komori, M., & Fujisawa, Y. (2004). The correlation between the rate of improvement on the achievement test and students' self-assessment in second language learning. Osaka Shoin Women's University Faculty of Liberal Arts collected essays, 41, 9-17.
Konaszewski, K., & Sosnowski, T. (2017). Factors affecting self-esteem among juveniles from youth educational centers Resocjalizacja Polska(1), 165180.
Maehr, M., & Stallings, R. (1972). Freedom from external evaluation. Child development, 43, 177-185.
Nakajima, K. (1998). Gengo to kyouiku. [Language and education]. Tokyo: Kaigaishienzaidanhoujin.
Nakajima, K. (2003). Keishou nihongo gakushuusha no kanji shuutoku to kokugo kyoukasho. [Kanji acquisition of heritage Japanese language and Japanese textbooks]. Obirin Synergy, 1, 1-21.
Nishimura, P. Y. (2012). Keishougo to shite no Nihongo koosu. [Japanese course as a heritage language]. Journal for Children Crossing Borders, 3, 129-134.
Oguro, S., & Moloney, R. (2012). Misplaced heritage language learners of Japanese in secondary schools more. Heritage Language Journal, 9(2), 207-221.
Ootsuki, M. (2010). Katakana to hiragana. Paper presented at the Oochou Jidai no Kotoba to Moji (Language and Script in DynastyEera), Kyoto University Tokyo office. Retrieved from http:// www.kyoto-u.ac.jp/static/ja/news_data/h/h1/ news7/2010/documents/101201_2/03.pdf
Shimizu, H., & Green, K. E. (2002). Japanese language educators' strategies for and attitudes toward teaching kanji. The Modern Language Journal, 86(2), 227-241.
Sloan, M. (1996). I love this piece because.... Instructor, 105(7), 30-32.
Wang, S. C., & Green, N. (2001). Heritage language students in the K-12 educaton system. In J. K. Peyton., D. A. Ramard., & S. McGinnis. (Eds.), Heritage Languages in America: Preserving a
National Resource (pp. 167-196). Washington, DC & McHenry, IL: Center for Applied Linguistics & Delta Systems.
Willoughby, L. (2006). Heritage LOTE at VCE level: Student perspectives on current programs. Monash University Linguistics Papers 5(1), 3-15.
Yamaguchi, Y. (2008). Kanji instruction at Japanese supplementary school in the U.S. (Master's
thesis), University of Hawaii at Manoa, Retrieved from https ://scholarspace.manoa.hawaii.edu/ bitstream/10125/20517/1/M.A.CB5.H3_3511_r.pdf Yoshizawa, K. (2009). To what extent can self-assessment of language skills predict language proficiency of EFL learners in school context in Japan? Foreign Language Education Research Bulletin, 17(1), 65-82.
