CC BY
87Copyright © 2018 by Academic Publishing House Researcher s.r.o.
W TT
I
Published in the Slovak Republic
International Journal of Media and Information Literacy
Has been issued since 2016.
E-ISSN: 2500-106X
2018, 3(1): 11-17
DOI: 10.13187/ijmil.2018.1.11 www.ejournal46.com
Assessing Information Literacy Skills of Undergraduate Freshmen: A Case Study from Hong Kong
William Ko-Wai Tang a , *
a The Open University of Hong Kong, China
Abstract
University students use information from the Internet for solving their own problems or assignments. However, there has been a little discussion on the use of internet-based information literacy skills of freshmen in Hong Kong. The present study assessed the internet-based information literacy skills of undergraduate freshmen from author's university. This study included four information literacy areas: (1) identify information needs and internet sources; (2) locate information from the internet; (3) evaluate information from the Internet; (4) synthesize information. These areas are important learning skills for survival in the information age. This study measured the perception of information literacy skills by self-rated survey and actual information literacy skills by using multiple-choice knowledge test and task-based information problem from sixty-one undergraduate freshmen. The findings indicate that the participants have difficulties in all information literacy areas. These results signify that further support on information literacy skills should be provided.
Keywords: information literacy, information skills, information problem solving, freshmen.
1. Introduction
In the 21st century, students use information from the Internet for solving their problems or searching some information for their assignments and projects (Kolikant, 2009; Van Deursen, 2013). Information is easily available on the Internet. According to Netcraft Web Server Survey, there were 1,734,290,608 websites on the World Wide Web in December 2017. If students have difficulties in identifying information needs, locating, evaluating and organizing information from the Internet, they cannot solve their information problem effectively and efficiently.
However, there is little care in formal school curriculum. Teachers believe that students should develop their information skills without any instructional support (Walraven, 2008; Van Deursen, 2013). Researches have shown that both secondary school and first year undergraduate students have serious difficulties in handling information skills from the Internet (Argelagos, 2012; Foo, 2014; Fain, 2011). The skills of information literacy have become a main concern for academics and librarians. In this information-rich environment, information literacy is a critical skill to survive. Ramamurthy (Ramamurthy, 2015) pointed out that insufficient information literacy skills is a global problem. Such skills are important to undergraduate students, they should be able to use information literacy skills effectively and efficiency at the beginning of their university life (Yager, 2013).
* Corresponding author
E-mail addresses: wtang@ouhk.edu.hk (W. Ko-Wai Tang)
2. Materials and methods
The study included a group of sixty-one first year undergraduate students. All of them enrolled in the first year first semester foundation level information technology module. All participants were assessed by all assessment tools on the first and second week of the module.
Survey is the most simple and common way to assess the information literacy skills (Walsh, 2009). We aimed at evaluating the perception of the information literacy skills of university freshmen. The questionnaire consisted of 23 Likert-type questions. It was adopted by OuYang (OuYang, 2007)' and Serap Kurbanoglu's instrument for measuring the perception of information literacy. Each item had a 7-point Likert type scale (1 refers to strongly disagree and 7 refers to strongly agree) (Serap Kurbanoglu, 2006).
The self-rated survey valuate the actual performance. We used multiple-choice knowledge test and information task to check their actual information literacy skills. We developed 18 multiple choice questions on evaluating the information literacy skills. In addition, we designed an information task to measure their information problem solving skills. The information problem and guided sub-questions are shown in Table 1.All participants required to respond to the information problem and all sub-questions within 90 minutes.
Table 1. Problem and sub-questions of information task
Information problem: What is Web 2.0? Do you think our IT course should make use of Web 2.0 tools for learning? How advantageous would this model of learning be?
Sub-questions
Area Questions
Identify the information needs and sources 1. Think about the information that you need to know or search. 2. List all relevant information sources from Internet. 3. Determine best information source. Why?
Locate information 4. Write down all relevant keywords. 5. Write down search statements with three most relevant web articles or websites.
Evaluate information 6. Evaluate the quality of web articles or websites.
Synthesize information 7. Write down THREE most relevant articles or websites by using APA format. 8. Prepare a PowerPoint presentation to respond to the information problem.
We analyzed the performance on each sub-question by using Diller and Phelp's (Diller, Phelp, 2008) categorization of the performance of information literacy. On each assessment item, the maximum score was 7. One mark should be awarded if the respondent demonstrated limited recognition of the skills, whereas a maximum of seven marks should be awarded if the respondent demonstrated full understanding of a specific skill. To improve the inter-scorer reliability, each script was assessed by two markers and we used the average score of two markers.
To validate all instruments in this research, all questionnaires, multiple choice questions and information problem were sent to experts in Information Literacy for comments.
3. Discussion
Different professional organizations have developed several framework (ACRL, SCONUL) and model of Information Literacy (Big 6 model) for higher education. In 2016, the Hong Kong Education Bureau introduced new Information Literacy for Hong Kong students. It presented eight literacy areas in three categories. It includes (1) effective and ethical use of information from lifelong learning; (2) Generic IL (identify, define, locate, access, evaluate and organize information); (3) Information World (Education Bureau, 2016). This new Information Literacy provides some idea for schools to develop students' knowledge, skills and attitude to use information. With refer to above frameworks and model, this study focuses on solving problems by
using information from the Internet media. We investigated the current situation of internet-based information literacy skills of undergraduate freshmen in author's university. The findings of this study may help educators to develop information literacy course at undergraduate level.
The assessment data can be categorized as perception and evidence-based data (Abdullan, 2010). The perception data can be collected from survey while evidence-based data can be collected from their actual performance. McCulley (McCulley, 2009) identified three major tools for information literacy assessment, including self-rated survey, multiple-choice knowledge test and information task.
Self-rated surveys are used to collect how people feel about their current performance. In terms of information literacy, it assesses the level of confidence of information literacy skills. Serap Kurbanoglu (Serap Kurbanoglu, 2006) developed a 17-item information literacy scale with three main components, which were basic, intermediate and advanced information literacy skills. Ou Yang (Ou Yang, 2007) developed an instrument to assess the developmental and confidence level of pre-service teachers' information problem solving skills on Internet resources. It provided 34 survey items in six categories, including define information problem, search information, process information, organize and present information and regulation.
Multiple-choice knowledge tests include a list of multiple choice questions to measure the actual skills of information literacy. McCulley (McCulley, 2009) pointed out that the knowledge tests provided a starting point to know more about your students' information literacy skills. A number of large scale information literacy knowledge test have been developed, such as Tool for Real-time Assessment of Information Literacy (TRAILS) and Project Standardized Assessment of Information Literacy Skills (SAILS) but they were not able to track changes in information literacy skills of individual students (Fain, 2011). Belie (Belie, 2009) developed the multiple choice test questions for pre-service teachers. This question set included 22 multiple choice questions based on the Association of College & Research Libraries (ACRL) Information Literacy Competency Standards for Higher Education and International Society for Technology in Education (ISTE) National Educational Technology Standards for Teachers. In a recent study, Boh Podgornik (Boh Podgornik, 2016) developed a new information literacy test for all study programs in all scientific disciplines.
Information tasks provide a comprehensive assessment of individual information literacy skills. McCulley (McCulley, 2009) stated that task performance could assess students how to integrate, how they have learnt and how they have solved the problem. The problem can be any topic. Brand-Gruwel, Wopereis & Walraven (Brand-Gruwel et al., 2009) designed a neutral topic for students to solve. For example, how to handle out-of-date food. Argelagos & Pifarre (Argelagos, Pifarre, 2011) designed some activities on solving information problem with specific topics in different discipline. On each activity, some sub-problems with guidelines were provided.
In order to provide the comprehensive assessment of internet-based information literacy, this study measured skills by using self-rated survey, multiple-choice knowledge test and information task.
4. Results
This section discusses the results of each instrument on each Information Literacy area. Table 2 shows the results of first information literacy area - identify information needs and sources. The participants perceived that they had difficulties in defining information needs (Mean=4.24), but they had relatively confidence on identifying (Mean=4.63) and determining the best information sources (Mean=4.89). On multiple choice knowledge test, more than 60 % of participants got correct answers on this area. On information task, it ranged from 3.96 to 4.24. Results indicated that the participants demonstrated limited skills in identifying information needs and sources. Most participants could write at least two information sources. Over 85 % of participants listed Google as information sources but only 36 % of participants listed academic journal database as information sources. Refer to information task, participants were not able to define the information needs, some low-ability participants wrote down "Web 2.o"onlywhereas high-ability participants could provide more information like "definition of Web 2.0, how to use Web 2.0 tools to enhance teaching and learning, how to communicate with others by using Web 2.0".
Table 2. Results - identify information needs and sources
Item Survey MC correct % Task
Mean SD Mean SD
Define information needs 4.24 1-43 61.9% 3.96 1.22
Identify potential sources of information on Internet 4-63 1.26 69.8% 4.22 1.41
Determine best information sources 4.89 1.09 62.7% 4.24 1.63
Table 3 shows the results of second information literacy area - locate information. The participants had confidence in locating information, especially on limiting search strategy by using different keywords (Mean=5.03) and using advanced Google search (Mean=5.29). However, the participants demonstrated limited knowledge based on the results of multiple choice knowledge test and information task. Refer to the results of multiple choice knowledge test, only 23.8 % and 12.7 % of the participants got the correct answer on limiting search strategy by using different keywords and limiting search strategy by using advanced Google search respectively. It aligned with the results of information task. Most participants did not know how to revise search statement based on initial result. The average number of keywords was 4.37 but many of them used same words on the information problem like "Web 2.0" and "advantages". It is similar to the results on information needs. It reflects that they have poor performance on locating information but they believe that they can locate information.
Table 3. Results - locate information
Item Survey MC correct % Task
Mean SD Mean SD
Limit search strategy by using different keywords 5.03 1.09 23.8% 4.22 1.25
Use of search statement N/A 4.33 1.59
Limit search strategy by using Boolean Logic 4.30 1.16 49.2% N/A
Limit search strategy by using advanced Google search 5.29 1.18 12.7%
Revise search strategy to retrieve more results 4.86 1.16 27.0%
Revise search strategy to retrieve fewer results 4.52 1.19 30.2%
Table 4 shows the results of third information literacy area - evaluate information. The participants perceived that they had confidence in determining the information sources by using different evaluation criteria. The mean score ranged from 4.68 to 4.89. Refer to the results of multiple choice knowledge test, 74.6 % of the participants got the correct answer on determining the accuracy of information sources and around 50 % of participants got the correct answer on other criteria. In addition, the results of information task show that all participants could use at least one evaluation criteria. Most participants used authority and currency as web evaluation criteria. They believed that the professional author with updated information was the most important factor to evaluate the quality of website. In general, the perceived evaluation skills is aligned with the actual evaluation skills. They can evaluate the information sources based on some evaluation criteria.
Table 4. Results - evaluate information
Item Survey MC correct % Task
Mean SD Mean SD
Use of evaluation criteria N/A 4.65 0.81
Determine authority of information sources 4.81 1.23 41.3% N/A
Determine currency of information sources 4.83 1.19 55.6%
Determine reliability of information sources 4.89 1.12 55.6%
Determine objectivity of information sources 4.68 1.10 47.6%
Determine accuracy of information sources 4.76 1.06 74.6%
Table 5 shows the results of last information literacy area - synthesize information. The participants perceived confidence in synthesizing information. The mean score of this area ranged from 4.81 to 5.00. Refer to the results of multiple choice knowledge test, they had difficulties in citing information. Only 31.7 % of participants could make correct citation on all multiple-choice questions. It reflects that they have poor performance on citing information. They had difficulties in understanding the differences between journal title and article title as well as the differences between volume and issues of a journal. In addition, some participants (around 39.3 %) had not used italic font on the journal title. Some of them (4 participants) wrote down the URL only. They had no idea on citing documents by using APA format. Such results contradict the results with the survey. They do not know how to make citation but they believe that they can make correct citation. It reveals that they probably have wrong concepts on citing documents but they think that it is a correct format.
Table 5. Results - synthesize information
Item Survey MC correct % Task
Mean SD Mean SD
Determine whether the information retrieved is relevant and sufficient for solving the information problem 4.81 1.15 54.0% N/A
Make citations and use quotations within the text 4.83 1.11 31.7% 3.39 1.82
Summarize information obtained from the Internet 5.00 1.06 N/A 5.14 1.01
5. Conclusions
This study assesses the information literacy skills of first year undergraduate students in Hong Kong. Based on the results of different assessment tools, it shows that participants have limited knowledge of information literacy. On the area of identifying information needs and sources, they had difficulties in identifying information needs. On the area of locating information, they believed they had good locating skills but they had difficulties in using advanced search and how to revise the search statement based on the initial search result. On the area of evaluating information, they were weak at determining the authority and objectivity of information sources. On the area of synthesizing information, they had difficulties in citing information. However, we have a small sample size in one module at one university, there is a limitation on the generalizability of the research results. Nevertheless, the result signifies that further support in embedding information literacy skills into a formal university curriculum is essential and important.
Refer to the research results, we should provide more training on each Information Literacy area. On the area of identify the information needs and resources, we can provide more brainstorming techniques to help our undergraduate students to identify the information needs. For example, students can discuss the nature of task and the tasks to do by using online brainstorming tool. In terms of information sources, results showed that most participants identify Google as information source only, we should provide all types of information sources to our students. It includes but not limited to encyclopedia, newspaper database, journal database, Google
Scholar and e-book in university library. In addition, we should provide more training on the electronics database. Faculty members can collaborate with university library. On the area of locating information, results showed that they were weak at formulating keywords and research statements. Students should learn how to formulate keywords other than using the wordings from original problem. For example, students may use online dictionary and thesaurus to define terms and look up the similar or related terms on a topic. In addition, students should learn how to use Boolean logic and how to formulate advanced search statements. In order to help them learn better, instructors should demonstrate the benefits of using appropriate keywords, Boolean search and advanced search strategies. On the area of evaluating information, students should learn how to evaluate the website by using different evaluation criteria and instructors can provide some checklists of web evaluation. In addition, instructors should remind them to link it with the information problem. On the area of synthesizing information, we should introduce the citation format in different type of information sources. The instructor should provide some common errors in citing information. In addition, instructors should ask students to reflect their effectiveness of the whole information problem solving strategy, so that they can understand how to enhance the information problem solving process next time.
References
Abdullah, 2010 - Abdullah, S. (2010). Measuring the outcomes of information literacy: Perception vs evidence-based data. The International Information and Library Review, 42: 98-104.
Argelagós, Pifarré, 2012 - Argelagós, E., & Pifarré, M. (2012). Improving information problem solving skills in secondary education through embedded instruction. Computers in Human Behavior, 28(2): 515-526.
Beile, 2009 - Beile, P. (2009). Measuring information literacy skills in education: Development and validation of a standards-based assessment scale. Saarbrucken: VDM Verlag Dr. Muller Aktiengesellschaft & Co. KG.
Boh et al., 2016 - Boh Podgornik, B, Dolnicar, D., Sorgo, A., Bartol, T. (2016). Development, testing, and validation of an information literacy test (ILT) for higher education. Journal of the Association for Information Science and Technology, 67(10): 2420-2436.
Brand-Gruwel et al., 2009 - Brand-Gruwel, S, Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers & Education, 52: 1207-1217.
Diller, Phelps, 2008 - Diller, K. R., & Phelps, S. F. (2008). Learning outcomes, portfolios, and rubrics, oh my! authentic assessment of an information literacy program, portal: Libraries and the Academy, 8(1): 75-89.
Education Bureau, 2016 - Education Bureau (2016). Information Literacy for Hong Kong Students 2016 (Draft). Retrieved 23 Jan 2017, from http://www.edb.gov.hk/attachment/en/edu-system/primary-secondary/applicable-to-primary-secondary/it-in-edu/IL_for_HK_Student_2016(Draft)_Eng_2016m6.pdf
Fain, 2011 - Fain, M. (2011). Assessing information literacy skills development in first year students: a multi-year study. The journal of academic librarianship, 37(2): 109-119.
Foo et al., 2014 - Foo, S., Majid, S., Azura Mokhtar, I., Zhang, X., Chang, Y. K., Luyt, B., Theng, Y. L. (2014). Information literacy skills of secondary school students in Singapore. Aslib journal of information management, 66(1): 54-76.
Kolikant, 2009 - Kolikant, Y.B.D. (2009). Students' perceptions of the appropriateness and usefulness of the Internet for schoolwork and the value of school. Journal of Educational Computing Research, 41: 407-429.
Mcculley, 2009 - Mcculley, C. (2009). Mixing and matching: assessing information literacy. Communications in Information Literacy, 3: 171-180.
Ou Yang, 2007 - OuYang, Y. (2007). Factors impacting the Information Problem-Solving Skills of Internet resources on pre-service teachers. Educational Review, 28: 225-249.
Ramamurthy et al., 2015 - Ramamurthy, P., Siridevi, E. & Ramu, M. (2015). Information literacy search skills of students in five selected engineering colleges in chittoor district, Andhra Pradesh: A Persepective. International Research: Journal of Library and Information Science, 5(1): 107-121.
Serap Kurbanoglu et al., 2006 - Serap Kurbanoglu, S., Akkoyunlu, B., Umay, A. (2006). Developing the information literacy self-efficacy scale. Journal of Documentation, 62(6): 730-743.
Van Deursen, Van Diepen, 2013 - Van Deursen, A.JA.M, Van Diepen, S. (2013). Information and strategic Internet skills of secondary students: A performance test. Computers & Education, 63: 218-226.
Walsh, 2009 - Walsh, A. (2009). Information literacy assessment: Where do we start? Journal of Librarianship and Information Science, 41(1): 19-28.
Walraven, 2008 - Walraven, A. (2008). Becoming a critical websearcher: Effects of instruction to foster transfer. Doctoral Dissertation. Netherlands: Open University of the Netherlands.
Yager et al., 2013 - Yager, Z., Salisbury, F., & Kirkman, L. (2013). Assessment of information literacy skills among first year students. The International Journal of the First Year in Higher Education, 4(1): 59-71.
