CC BY-ND
174
https://doi.org/10.17323/jle.2023.17379
Writing with AI: University Students' Use of ChatGPT
Nikola Creek ®, Jakob Patekar
RIT Croatia, Zagreb, Croatia
ABSTRACT
Background: ChatGPT, a chatbot based on a large language model, captured global attention toward the end of 2022. With its potential to generate comprehensive texts of a variety of genres based on a string of straightforward prompts, it was soon perceived as a threat by many in various fields, including - and in particular - education. Schools across the world began banning its use as instructors started to receive suspiciously well-written essays and assignments from their students.
Purpose: This study aimed to investigate the prevalence of use of ChatGPT among university students for written assignments, explore the ways students utilize the tool, and examine students' perspectives on the ethical aspects of its use.
Method: An online questionnaire was designed to collect data from 201 students from private and public universities in Croatia.
Results: The results show that more than half of the participants use ChatGPT for written assignments, that most use it to generate ideas, while many use it to summarize, paraphrase, proofread, but also to write a part of the assignment for them. According to the participants, the most ethically acceptable use of ChatGPT is for generating ideas, while other uses are perceived by many as being unethical; this, however, has not prevented some students from engaging in behaviors they deem unethical.
Conclusion: We conclude that universities and instructors need to take a decisive stand on artificial intelligence in education and provide clear guidelines to students regarding the ethical use of ChatGPT and emerging technologies.
KEYWORDS
ChatGPT, academic honesty, academic integrity, plagiarism, ethics, artificial intelligence
Citation: Creek N., & Patekar J. (2023). Writing with AI: University students' use of ChatGPT. Journal of Language and Education, 9(4), 128-138. https://doi.org/10.17323/jle.2023.17379
Correspondence:
Jakob Patekar,
e-mail: [email protected]
Received: June 13, 2023 Accepted: December 15, 2023 Published: December 30, 2023
INTRODUCTION
ChatGPT, a chatbot based on a large language model, was developed by the company OpenAI and was released to the public in November 2022. It demonstrated unparalleled potential to write full texts of a variety of genres in a way a human would, based on a string of straightforward prompts. Since then, numerous authors in the media have written about the dangers of ChatGPT. For example, there are privacy concerns
because a large language model that ChatGPT was built on was trained on hundreds of billions of words from the internet, including personal information and copyrighted material, obtained without consent or compensation1. Privacy is not a concern solely from the aspect of what ChatGPT is built on, but from the aspect of the users' input as well; for example, in March 2023, Italy temporarily banned ChatGPT because of concerns regarding the collection of data from Italian ChatGPT users2. Other authors wrote
Gal, U. (2023, February 10). ChatGPT is a data privacy nightmare. If you've ever posted online, you ought to be concerned. The Conversation. https://theconversation.com/chatgpt-is-a-da-ta-privacy-nightmare-if-youve-ever-posted-online-you-ought-to-be-concerned-199283 Kim, J. (2023, March 31). ChatGPT is temporarily banned in Italy amid an investigation into data collection. NPR. https://www.npr.org/2023/03/31/1167491843/chatgpt-italy-ban-ope-nai-data-collection-ai
of ChatGPT "causing untold chaos"3, expressed concerns over its "dark side", urging businesses to prepare for the AI takeover4, and warned of the "dark risk" of large language models and the consequences for people when Al-written text becomes indistinguishable from the human-written text5. Furthermore, there are those who saw the introduction of Al-powered tools as a threat to the human creative process, considering that a text or a picture can be created through the use of a prompt rather than the laborious yet rewarding process of writing and painting6. Nature's editors (Tools such as ChatGPT..., 2023) pointed out that "ChatGPT can write presentable student essays, summarize research papers, answer questions well enough to pass medical exams and generate helpful computer code". Indeed, some authors have gone so far as to proclaim the college essay dead because of students' use of Al-powered technology to produce essays7.
However, other authors remained confident that ChatGPT will not significantly harm education. There are those who are optimistic that ChatGPT will change education for the better, although they find panic, which resulted in schools across the world banning ChatGPT, understandable as an initial reaction of the education sector because "teachers have been thrown into a radical new experiment"8. Therefore, authors call for a shift in focus - from teaching writing to developing critical reading and editing skills9. One instructor is
convinced that ChatGPT cannot replace teachers of writing who teach their students that writing is about establishing relationships, which is something ChatGPT cannot do10 (we would add, yet). Finally, an author who tested GPT-3 before it was widely available as ChatGPT, concluded that educators will need to rethink teaching and assessment rather than accelerate "an ongoing arms race between increasingly sophisticated fraudsters and fraud detectors"11. Indeed, the widespread panic in the education sector resulted in several companies offering or upgrading their software to detect Al-written text, such as the well-known Turnitin, Ze-roGPT, and the OpenAI's own AI Text Classifier. However, none of these solutions are considered a hundred percent accurate, and false positives may occur12, even though both Turnitin and ZeroGPT claim 98% accuracy13' 14. Our anecdotal evidence shows that the percentage must be significantly lower as Turnitin failed to detect any Al-produced text in six student papers written entirely by ChatGPT in April 2023.
While optimism regarding ChatGPT might be warranted, at this point it needs to be highlighted that ChatGPT (v3.5) has certain faults that impact the quality of the work it produces. In small print on the bottom of the tool's interface, it reads, "ChatGPT may produce inaccurate information about people, places, or facts"15. Indeed, ChatGPT is known to fabricate sources (or as it is known in the jargon, "hallucinate") when asked to produce in-text citations and references as it in-
3 Nolan, B. (2023, January 28). ChatGPT has only been around for 2 months and is causing untold chaos. Business Insider. https://www. businessinsider.com/chatgpt-ai-chaos-openia-google-creatives-academics-2023-1
4 Angel, B. (2023, April 1). The dark side of ChatGPT: Employees & businesses need to prepare now. Entrepreneur. https://www.entrepre-neur.com/science-technology/the-dark-side-of-chatgpt-employees-businesses-need-to/444225
5 Marcus, G. (2022, December 29). The dark risk of large language models. Wired. https://www.wired.com/story/large-language-mod-els-artificial-intelligence/
6 Eisikovits, N., & Stubbs, A. (2023, January 12). ChatGPT, DALL-E 2 and the collapse of the creative process. The Conversation. https:// theconversation.com/chatgpt-dall-e-2-and-the-collapse-of-the-creative-process-196461
7 Marche, S. (2022, December 6). The college essay is dead. The Atlantic. https://www.theatlantic.com/technology/archive/2022/12/ chatgpt-ai-writing-college-student-essays/672371/
8 Heaven, W. D. (2023, April 6). ChatGPT is going to change education, not destroy it. Technology Review. https://www.technologyreview. com/2023/04/06/1071059/chatgpt-change-not-destroy-education-openai/
9 Rigolino, R. E. (2023, January 31). With ChatGPT, we're all editors now. Inside Higher Ed. https://www.insidehighered.com/ views/2023/01/31/chatgpt-we-must-teach-students-be-editors-opinion
10 Malesic, J. (2023, February 9). What ChatGPT can't teach my writing students. The Atlantic. https://www.theatlantic.com/ideas/ar-chive/2023/02/writing-education-language-empathy-ai-chatgpt-age/672999/
11 Sharples, M. (2022, May 17). New AI tools that can write student essays require educators to rethink teaching and assessment. London School of Economics and Political Science. https://blogs.lse.ac.uk/impactofsocialsciences/2022/05/17/new-ai-tools-that-can-write-stu-dent-essays-require-educators-to-rethink-teaching-and-assessment
12 Fowler, G. A. (2023, April 3). We tested a new ChatGPT-detector for teachers. It flagged an innocent student. The Washington Post. https://www.washingtonpost.com/technology/2023/04/01/chatgpt-cheating-detection-turnitin/
13 AI Text Classifier. (n.d.). OpenAI. https://platform.openai.com/ai-text-classifier
14 Alimardani, A., & Jane, E. A. (2023, February 19). We pitted ChatGPT against tools for detecting AI-written text, and the results are troubling. The Conversation. https://theconversation.com/we-pitted-chatgpt-against-tools-for-detecting-ai-written-text-and-the-results-are-troubling-199774
15 ChatGPT. (n.d.). OpenAI. https://chat.openai.com/
vents authors and article titles16, 17, which one of the authors experienced as well when they prompted ChatGPT to produce a text on a niche as well as a very familiar topic. Upon scrutiny, it turned out the authors did not exist, nor did the articles. Obviously, an uncritical or uninformed user might misuse ChatGPT by failing to detect "inaccurate information about people, places, or facts", which makes it a less-than-ideal tool for university students and their needs.
ChatGPT is certainly not the only tool students have at their disposal if they want to take a shortcut in producing written work for college. QuillBot, which advertises as an online paraphrasing tool, has been around for some time. "Traditional" forms of academic dishonesty remain popular: students engage in contract cheating and buy papers from paper mills (e.g., Hill et al., 2021; Rigby et al., 2015; Walker & Town-ley, 2012) and use other people's work without crediting the original authors. However, none of these have shown to be as disruptive to education, and academic honesty in particular, as ChatGPT. Therefore, our aim was to investigate the use of ChatGPT among university students in Croatia in order to find answers to the following research questions:
RQ1. How prevalent is the use of ChatGPT among university students?
RQ2. How do university students use ChatGPT?
RQ3. What are the perceptions of university students regarding the ethical aspects of using ChatGPT?
LITERATURE REVIEW
Considering the recency of the topic, little is yet known about university students' use of ChatGPT. Nonetheless, several studies have been published, and we summarize their findings below as they relate to our research. However, since unethical use of the AI-powered chatbot in university can surely be viewed as academic dishonesty, we first address the findings from studies on university students' academic dishonesty, in particular in the Croatian context, as this will certainly aid us in interpreting some of the results from our research carried out on Croatian university students.
Academic Dishonesty
More than thirty years ago, Davis et al. (1992) noted that academic dishonesty has been "a perennial problem in higher education" and that "scholarly reports of academ-
ic dishonesty have appeared for more than 60 years", with the research effort intensifying in the past twenty years (p. 16). Their research on a sample of six thousand students showed that 76% have admitted to cheating, while 90% of those same participants indicated that they believe it is wrong to cheat. The authors concluded that participants demonstrated "a diminishing sense of academic integrity" (p. 19). This age-old problem that spans countries and contexts, for example, Australia (Brimble & Stevenson-Clarke, 2005), Canada (Eaton & Christensen Huges, 2022), India (Anitha & Sundaram, 2021), Romania (Ives et al., 2017), Russia, Ukraine, Lithuania, Armenia (Denisova-Schmidt, 2020), has not spared Croatia either. On the contrary, cheating appears to be a widespread, and even more worryingly, a widely acceptable phenomenon in Croatia. While academic honesty is not an unknown concept in Croatia, it exists only at a proclaimed, ethereal level, without practical applications or actual implications. Indeed, an international survey of educators and university students18 found that Croatian students and instructors believe cheating is a part of Croatian culture and is thus difficult to change. According to the same report, Croatia ranked 19th among 33 participating European countries in view of academic integrity maturity. The report also cites cases of prominent Croatian politicians who were caught plagiarizing yet experienced no consequences for their dishonest acts. That Croatia is a fertile ground for academic dishonesty was confirmed in several studies carried out in the past twenty years. What these studies (e.g., Bilic-Zulle et al., 2005; Dukic, 2022; Kukolja Taradi et al., 2016; Majstorovic, 2016; Petrak & Bartolac, 2014; Pupovac et al., 2010; Stambuk et al., 2015) in the Croatian context show is that Croatian university students understand that academic dishonesty, including plagiarism, is wrong, yet many of them do not consider it an important matter, and many of them, between 58% and 97%, engaged in some form of academic dishonesty. The studies also show that Croatian university instructors are aware of the problem yet seem to be rather passive about it. It is against this backdrop that we investigated the use of ChatGPT among university students in Croatia.
University Students' Use of ChatGPT
Although there is a scarcity of research on students' use of ChatGPT, several studies on the topic, carried out in a variety of contexts, have been published recently.
For example, Singh et al. (2023) carried out a study using a 12-item questionnaire on 430 university students of computer science in the United Kingdom. The results showed that
16 Hillier, M. (2023, February 20). Why does ChatGPT generate fake references? TECHE. https://teche.mq.edu.au/2023/02/ why-does-chatgpt-generate-fake-references/
17 Welborn, A. (2023, March 9). ChatGPT and fake citations. Duke University Library. https://blogs.library.duke.edu/blog/2023/03/09/ chatgpt-and-fake-citations/
18 What these studies ETINED. (2017). South East European Project on Policies for Academic Integrity. Final Report. http://www.plagiarism.cz/ seeppai/Final-report_SEEPPAI.pdf
although familiar with the tool, students reported not using it on a regular basis in their learning activities, possibly because they were concerned about potential misuse, but also because at the time of the study, students did not possess an advanced knowledge of the tool. The authors concluded that guidelines are needed that would steer students toward using ChatGPT "positively".
Furthermore, Jowarder (2023) used semi-structured interviews to gain insight into the levels of "awareness, adoption, perceived usefulness, and impact of ChatGPT" on 200 undergraduate social science students from the United States. The author found that most students were aware of the tool and that most had used it in their studies. Students recognized its usefulness, in particular because it is easy to use and because it helps them to clarify difficult concepts, find relevant study materials, and do research for their coursework. Apart from usefulness as a factor that affected students' use, the study found that the participants' use of ChatGPT was influenced by social factors, such as peers' recommendations to use the tool. The author noted that many students had reported using the tool to submit written assignments but were not caught, primarily because plagiarism detection tools at the time had not been able to detect AI-generated texts. Hence, the author concluded that the tool cannot be a substitute for critical thinking and independent learning as relying solely on ChatGPT for academic work could harm students' intellectual growth.
Ngo (2023) surveyed 200 Vietnamese students via an online questionnaire and interviewed 30 of the students to examine their perceptions of ChatGPT. The study found that students were positive about using ChatGPT in their learning, highlighting its simplicity of use and convenience. The tool helped students to save time, obtain information, and receive feedback. However, the participants showed to also be aware of the negative aspects of ChatGPT, in particular the questionable reliability of information and sources. The author concluded that guidelines on ChatGPT use are needed and that academic integrity should be promoted among students to "ensure ethical uses of ChatGPT in academic context" (p. 15).
Finally, Yilmaz & Karaoglan Yilmaz (2023) employed a questionnaire to examine the perspectives of 41 undergraduate students of computer science in Turkey in view of their use of ChatGPT in programming. The participants noted several advantages of using the tool, such as obtaining fast and mostly accurate responses, enhancing thinking skills, receiving help with debugging, and having a positive impact on self-confidence. However, several disadvantages were observed as well. Students felt that using ChatGPT could lead to them getting used to laziness. In addition, they were concerned about the tool not being able to answer certain prompts and about receiving incomplete or incorrect information. Nonetheless, the authors concluded that incorporating ChatGPT in programming courses is the right
approach due to the predominantly positive influence; however, not without considering "the ethical appropriateness of results generated by generative AI tools like ChatGPT".
The studies above found that university students in different countries and contexts use ChatGPT in their education and are largely aware of both the advantages and disadvantages. The authors of the studies, based on their findings, called for clear guidelines regarding ChatGPT use in the academic context. Since none of the studies above have thoroughly addressed the variety of ways students use ChatGPT or the ethical aspects of its use, we hope our findings will contribute to addressing the current gap in research.
METHOD
We opted to conduct a quantitative study on university students using an online questionnaire as the primary instrument. This choice was driven by several factors. Firstly, an online survey offered a practical and efficient means of data collection, and the structured format of the questionnaire ensured standardized responses, enhancing the reliability and comparability of the gathered data. The adoption of a non-probability convenience sampling method was driven by practical considerations and the nature of the research objectives. Convenience sampling allowed for the inclusion of participants readily accessible through online platforms and in-class settings within the specified time frame. The distribution strategy, encompassing both social media channels and in-class dissemination over a four-week period, was designed to maximize participation and capture a broad range of perspectives. The predetermined closure of the survey upon reaching a certain number of responses ensured a manageable dataset for analysis within the study's scope. The chosen distribution channels and timeline were thus guided by a balance between reach, practicality, and the study's exploratory nature.
Instrument
The online questionnaire was developed in Google Forms and consisted of eighteen items across four sections. The first two sections had one item each, with the first one asking the participants to confirm they were university students. Guided by our research questions on ChatGPT use among university students, we wanted to make sure only students participate in the survey. Hence, in case of a positive answer, the participants could proceed to the next section and next item, which required them to state whether they have heard about ChatGPT. Considering the research questions, we determined that students who had not heard about ChatGPT could not contribute to our survey, so we decided to stop such students from participating. Thus, in case of a positive answer, they could proceed to the remainder of the survey. If the participants provided a negative answer on either of the two items above, they were taken to the end of the sur-
vey without an option to see the other sections and thus complete the survey.
The third section contained eight items and was aimed at collecting general, background information about the participants. Here, we wanted to note the participants' gender, year level, institution type (private or public), academic performance, familiarity with the academic honesty policy at their institution, experience with plagiarism (a person or software writing an assignment for them), experience with other types of academic dishonesty, and whether they had ever been accused of academic dishonesty.
The final, fourth section, included nine items aimed at exploring the participants' experience with ChatGPT. In this part, we wanted to find out whether their instructors mentioned ChatGPT in class, what the instructors' general attitudes toward ChatGPT are, whether participants used ChatGPT for a written assignment, whether they shared that information with their instructors, whether instructors had detected ChatGPT use, what the participants' opinion on the helpfulness of ChatGPT is, and what their opinion on ethical uses of ChatGPT is. Regarding the latter item that contains six potential uses of ChatGPT (e.g., generating ideas, writing parts of the assignment, paraphrasing, etc.), they were developed based on researchers' discussions with non-participating students on their actual use of ChatGPT.
The last item in the category was an optional open-ended question to provide further comment.
In designing all of the items, we were primarily guided by our three research questions.
Sample
We used a non-probability convenience sampling method to reach potential participants of the survey. While there
were 202 responses to the survey, one participant was not a university student, and another 31 stated that they have not heard about ChatGPT, which left 170 participants who completed the entire survey.
There were 54.7% female and 42.9% male participants, one non-binary participant, and three who preferred not to state their gender, which makes for a balanced sample gender-wise. The majority of participants were in their second and first year of studies (Figure 1).
Most participants came from private (82.9%) as opposed to public (17.1%) universities. Considering the sensitivity of the topic, we wanted to ensure as greater a level of anonymity to the participants as possible, which is why we did not require them to state their specific institutions.
Almost half of the participants reported doing well academically by stating that they generally receive As and Bs (Figure 2). None of the participants selected that they generally receive Ds and Fs.
Procedure
The questionnaire was distributed via social media and in classes during four weeks in March 2023. Upon reaching 202 responses, the survey was closed, and data analysis commenced.
RESULTS
To better understand our participants' background, we inquired about their familiarity with academic honesty policies at their universities. As the results show, a vast number of participants are either familiar or quite familiar with the academic honesty policy at their university (Figure 3).
Figure 1
Participants' Year Level
Third year 19
Fifth year 1
Fourth year 32
Second year 74
Figure 2
Participants' Self-Reported Academic Performance
Cs and Ds 25
Bs and Cs 70
First year 44
Despite this, in the next several items, 35.9% of participants stated that they had a person or software write an assignment for them in university, and 54.7% reported that they had engaged in academic dishonesty in university, such as copying answers on an exam, using the phone to cheat, plagiarism, etc. Only 11.8% of participants reported being accused of academic dishonesty in university.
tive stance toward ChatGPT use, close to a half of the participants (44.7%) reported using ChatGPT in university. Of those who had used it, only three students shared that information with the instructor. In addition, when the participants had used ChatGPT for the assignment and had not disclosed that to the instructor, only four students reported that the instructor noticed.
We were also interested to find out what students perceive are their instructors' stances toward ChatGPT use. According to the participants' responses, it appears that in most cases some or most instructors mentioned ChatGPT in their classes (Figure 4).
However, it seems instructors take a varied approach to students' ChatGPT use, as seen in Figure 5. Around a quarter of instructors forbid students to use it, while more than a third discourage its use. There are students who simply do not know where their instructors stand (14%) as well as those who state that ChatGPT use has not been discussed in class (12%).
In the next item, we inquired into whether the participants have ever used ChatGPT for a written assignment in university. In spite of the instructors' predominantly nega-
Figure 3
Participants' Familiarity with Their University's Academic Honesty Policy
Quite familiar 112
I know mwuniversity has such a policy, but I am notfamili...
Next, in line with the research questions posed, we wanted to investigate how students use ChatGPT for their written assignments. As seen in Figure 6, of the 94 participants who use ChatGPT, most (81.9%) do so to generate ideas. Among other frequent uses are for paraphrasing (46.8%), summarizing (44.6%), proofreading (31.9%), but also for writing a part of the assignment (36.1%). Seventeen students (18%) reported that they had used ChatGPT to produce the entire assignment. The participants were also able to enter their answer to this item, and one response was received for each of the following: to find out if something sounds good, to translate, to see the structure, and to write a reference list.
Considering that students have evidently discovered several good uses of the tool, it is not surprising that more than half of the participants find ChatGPT helpful or very helpful. Only
Figure 5
Instructors' Attitudes toward ChatGPT Use (as Perceived by the Participants)
Students are discouraged from using it
Students are foribidden to u
Using ChatGPT has not been discussed with students J
do not know 24
\_ Students are allowed to use it for some things
I am not aware my university has such a policy _
^ Barely familiar 12
Figure 4
Instructors Mentioning ChatGPT in Class (as Perceived by the Participants)
Yes, only one or two instructors 541
Yes, some instructot
_ Yes, most instructors
Figure 6
Participants' Use of ChatGPT
Figure 7
Participants' Opinion about the Helpfulness of ChatGPT
nine students have a negative stance as they find it unhelpful or very unhelpful (Figure 7).
Second to the last item in the survey targeted the participants' views on the ethical aspects of using ChatGPT. It appears that the most ethically acceptable way to use ChatGPT is for getting ideas (Figure 8). Interestingly, 54.1% of participants would delegate the decision of the ethics of using the tool to their instructors, by opting for the statement that it is ethical "when the instructor says it is okay". While 17 students reported using ChatGPT for writing the whole assignment (previous Figure 6), only 11 stated that this was ethical. A similar number of participants (34) who had used it to write parts of the assignments stated that this was ethical (35). Only 10 students found the use of ChatGPT unethical in all of the cases provided.
In the last, optional open-ended item, students were able to provide their comments on the topic. Regarding the ethical aspects, one student noted that ChatGPT "is not harming anyone or any law or dishonesty policy" and, in a similar vein, another noted that if "using it is considered dishonest, universities should regulate it". A student observed that "it's obviously non-ethical when it writes instead of you, but it has some purposes where it can be used for good and very beneficial". Accordingly, a participant praised ChatGPT, writing that "it's really good for helping to express yourself when you aren't sure how to word something". Related to its potential, a participant finds it "is a very useful tool to help students with their assignments when professors don't have time for helping them or when they didn't help them a lot, so they need additional help." Another participant wisely noted, "Utilizing it shouldn't be prohibited, but encouraged in a smart way. It is the future regardless if people don't use it while it is still in its development stage." In view of future directions, a participant stated that "students should be taught about how to use it and evolve their ideas easier, not use it to create ideas but rather to do boring work, and it's basically the future". One student, however, was cautious, stating, "Writing is both a creative and cognitive challenging
Figure 8
Participants' Views on the Ethical Aspects of ChatGPT Use
For writing parts of the assignment _ I" none of these cases is it ethical
35 \ /10
process. Using ChatGPT in my opinion deprives an individual of mental engagement. Being innovative and disruptive, the true impact of such technology on human mental and cognitive agility remains to be seen in the decades (generations) to come."
DISCUSSION
With this study we aimed to find answers to the following questions:
RQ1. How prevalent is the use of ChatGPT among university students?
RQ2. How do university students use ChatGPT?
RQ3. What are the perceptions of university students regarding the ethical aspects of using ChatGPT?
Before addressing the research questions specifically, let us discuss the sample for a better interpretation of the results. We had predominantly second and first-year students from private universities, although there were some students from higher year levels as well as public universities in Croatia. Many participants reported performing well academically. Furthermore, most were familiar or quite familiar with their university's academic honesty policy. However, familiarity did not prevent them from engaging in academic dishonesty in university, as 54.7% reported doing this, 35.9% reported having a person or software write an assignment for them in university, and 11.8% reported having been accused of academic dishonesty in university. Compared to the findings from previous studies carried out in the Croatian context and summarized in literature review, the first two figures seem modest; however, this might be due to a strict implementation of the academic honesty policy at one of the institutions surveyed. As for the participants' information about their instructors, it appears that in at least half of the sample some or most instructors mentioned ChatGPT
in class, and that a quarter of the instructors forbid using it while slightly more than a third discourage students from using it. It is with this information in mind that we are to interpret the data.
How Prevalent is the Use of ChatGPT Among University Students?
Regarding the first question of the prevalence of use, when asked directly whether they had used ChatGPT for their written assignments in university, 44.7% responded that they have, but when asked to select different ways that they have used it, it turns out more than half of the participants (55.2%) used it. While this inconsistency in answers is interesting, it is certainly expected and understandable considering how sensitive the topic is as it explores students' possibly unethical behaviors and thus affects their image of self. Of those who reported having used ChatGPT, only four students shared that they were caught, that is, that the instructor noticed that they had used ChatGPT for their written assignment. The result regarding the prevalence of use is in line with that of Singh et al. (2023) who found that although UK students were generally familiar with the tool, they reported not using it on a regular basis. However, these findings contradict those of Jowarder (2023) who found that most US students in his study were familiar with ChatGPT and had indeed used it in their education. This could be explained by the difference in the time when these studies were carried out. With each passing month, students, generally quick to adopt technological innovations, probably learned more about ChatGPT and experimented more, leading to an increase in the number of students across the world using the tool.
How do University Students Use ChatGPT?
Regarding the second question of how they use ChatGPT, most students who use it, do so to generate ideas (81.9%), which is followed by paraphrasing (46.8%), summarizing (44.6%), writing a part of the assignment (36.1%), and proofreading (31.9%). Seventeen participants (18%) reported using it to write the whole assignment for them, which is certainly a cause for concern. We found these categories of uses to be quite comprehensive as the participants had an option to enter their answer, and only a few participants used this opportunity; one response was recorded for the following participant-added uses: to find out if something sounds good, to translate, to see the structure, to write a reference list. Other studies have found that students use ChatGPT to clarify difficult concepts, find relevant study materials, do research for their coursework (Jowarder, 2023) and obtain information and receive feedback (Ngo, 2023). With time, we believe students will discover other ways to use ChatGPT for their written work beyond these examples. This is especially true if they receive guidance and advice from their instructors on how they can ethically use ChatGPT to help their writing process. For this to occur, of course, the
instructors need to be very familiar with the possibilities of emerging technologies.
Undoubtedly, students in our research, at the time it was carried out, were using ChatGPT for their written assignments and were perhaps still in the stage of experimenting with its features. In the last, open-ended question in the survey, the students highlighted that ChatGPT is a great help when one is uncertain how to word something, but also when the instructor had not provided sufficient guidance or assistance (due to a lack of time or other reasons). Interestingly, only three students reported disclosing the information that they had used ChatGPT to their instructor. This indicates that the ethical aspects of ChatGPT use were not clear to the students at the time, which was probably exacerbated by instructors not taking a unanimous and clear stand on the matter. Nonetheless, most participants find ChatGPT to be a helpful or even very helpful tool (57%), while only nine students have a negative stance as they find it unhelpful or even very unhelpful. These results are in line with the findings of other studies, such as Ngo (2023) and Yilmaz & Karaoglan Yilmaz (2023), whose participants had a predominantly positive outlook on using ChatGPT in their education. Interestingly, 28% of participants in our study stated they did not know whether it was helpful, which might be because they had not tried it yet or they had not yet experienced its advantages - or disadvantages.
What are the Perceptions of University Students Regarding the Ethical Aspects of Using ChatGPT?
As for the third research question on the ethical aspects of ChatGPT use, whether they have used it or not, 85.2% of participants think it is ethical to use it to generate ideas. Around half of the participants, curiously enough, would rely on the instructors' say on the matter of what is or is not ethical use. This indicates that many students expect guidance from their instructors, which is not surprising. However, at the time of the study, the instructors had not yet taken a common stand, or had predominantly taken a negative stand on ChatGPT use. While the tool is still relatively new, and a hotly debated topic, instructors' opinions are key to setting the foundations for AI usage in classrooms. If there are no guidelines on what for and how to use ChatGPT, it cannot be expected of students to figure out the ethical usage and to solve the issue of academic honesty by themselves.
Indeed, other researchers on the topic (e.g., Jowarder, 2023; Ngo, 2023; Singh et al., 2023; Yilmaz & Karaoglan Yilmaz, 2023) have also highlighted the importance of providing guidelines to students.
In line with previous research referenced in literature review, being aware of what is unethical and engaging in unethical behavior are two different things. Thus, while 17 partic-
ipants stated they had used ChatGPT to write an entire assignment for them, 11 found this to be ethical. In addition, there were 34 participants who reported using ChatGPT for writing parts of the assignment, and there were 35 who stated that this indeed is ethical. Jowarder (2023) also found that many students used the fact that plagiarism detection software had not been able to detect AI-generated writing and submitted written assignments generated by ChatGPT without getting caught.
The fact that there are students who understand that cheating by using ChatGPT to write an assignment is unethical, yet (would) do it nonetheless, points to an under-researched aspect, and that is the role personality traits play in using AI tools for cheating. In a study on 283 university students from Austria, Greitemeyer and Kastenmuller (2003) found that "individuals who prioritize fairness over their own interests" (Honesty-Humility) were least likely to cheat using ChatGPT, as were those with a strong work ethic (Conscientiousness) and those who "prefer to tackle challenges with their own original ideas" (Openness to Experience). On the other hand, students who are manipulative and strategic (Machiavellianism), self-focused (narcissism), or unemotional (psychopathy) were more inclined to submit AI-generated texts. Similarly, Malesky et al. (2021), who carried out a study on 361 undergraduate students from the United States on the effect of peer influence, honor codes, and personality traits on academic dishonesty, found that participants who scored high on the Openness to Experience trait were less likely to cheat. Their main finding, however, is that peer influence had a significant effect on engaging in academic dishonesty. This is in line with the results from a study on 164 university students from the United States by O'Rourke et al. (2010). The authors found that witnessing other students cheat increases cheating behavior in the observers. Hence, these findings on the influence of personality traits, honor codes, and peer influence should be considered when developing guidelines and strategies for academic integrity in the age of AI tools.
While some students certainly know the boundary, such as the participant from our study who stated that the use of ChatGPT is unethical when one asks it to write instead of oneself, as another participant observed, it is the universities that need to regulate ChatGPT if they consider it unethical. Indeed, universities and instructors must lead the change (e.g., Cotton et al. 2023; Crawford et al., 2023; Currie, 2023; Sweeney, 2023; Vaccino-Salvadore, 2023) if they want to turn chaos into growth.
Students recognize the potential of the tool and are aware of its disruptive power to education. For example, a participant explained that AI-powered tools are the future regardless of whether one uses them or not, so their use should be encouraged rather than prohibited. Another participant suggested that students should be taught how to use it for "boring work". However, it is understandable that some stu-
dents are more cautious regarding AI in writing. A participant warned that ChatGPT deprives people of mental engagement as writing is not only a cognitive but a creative endeavor as well.
At this point, let us return to the sample. It should be noted that the 31 university students who did not complete the survey because they stated they had not heard about ChatGPT is still a valuable piece of information. This can be explained in two ways. It might be that the students did not feel they had the right level of anonymity to state that they had indeed heard about ChatGPT. In such circumstances, stating this might appear to the participants as their "admission of guilt", in particular because in the early months of 2023 we could observe the backlash of academia against ChatGPT, which was later substituted, or rather, supplemented, with the exploration of ways it could be used in classes. However, we are more inclined to believe that most of these participants had not in fact heard of ChatGPT at the time they were taking the survey. We believe this number would have been significantly lower just a month or two later.
Limitations
Finally, we would like to address the limitations of our study. Using a questionnaire as a data-collection instrument meant that we were relying on students self-reporting their potentially unethical behavior. Even though the questionnaire was anonymous and a number of steps were taken to ensure students feel safe to provide truthful responses, we cannot disregard the possibility that some of them were not honest when giving answers. This is related to a very human trait of trying to maintain an image of one's ideal self by providing responses one believes are more socially acceptable, even when being anonymous. In further studies, other sources of data could be used, such as instructors' input, academic dishonesty reports, or student work. In addition, our sample predominantly included students from private as opposed to public universities, and these students were at lower year levels. Further research should aim toward a more balanced sample that would be more representative of the general student population. Despite the limitations listed above, we are confident that our study provides valuable insights into the matter.
CONCLUSION
The study has provided valuable insights into the prevalence of ChatGPT use among Croatian university students and has shed light on the ethical considerations surrounding its application. It has also highlighted the dire need for guidance coming from universities and instructors. The introduction of ChatGPT in a country with a historical and cultural propensity for academic dishonesty has resulted in chaos, with both students and instructors experimenting with the AI-powered tool, testing the limits of its ethical use and ap-
plication in the classroom. While this is understandable considering the novelty of the innovation, it cannot and should not be the way forward.
We firmly believe that universities and instructors must take a proactive stance, not through prohibition, but by embracing ChatGPT and other AI-powered tools while providing clear guidelines to students. With proper guidance, these tools can be utilized in ethical and creative ways, ultimately - and hopefully - leading to a reduction in academic dishonesty offenses. The responsibility lies with educational institutions to lead the way in shaping the ethical use and application of ChatGPT, fostering a culture of integrity and responsible technology use among students.
By acknowledging the potential pitfalls and challenges associated with ChatGPT, and by actively addressing them through comprehensive guidelines and ongoing support, universities can harness the power of AI to enhance education while maintaining academic integrity. This approach will pave the way for a more ethical and productive integra-
REFERENCES
tion of AI technologies in the learning environment, benefiting both students and instructors alike.
DECLARATION OF COMPETITING INTEREST
None declared.
AUTHORS' CONTRIBUTION
Nikola Creek: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Resources; Writing -original draft; Writing - review & editing.
Jakob Patekar: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Project administration; Resources; Supervision; Visualization; Writing - original draft; Writing - review & editing.
Anitha, P., & Sundaram, S. (2021). Prevalence, types and reasons for academic dishonesty among college students. Journal of Studies in Social Sciences and Humanities, 7(1). 1-14.
Bilic-Zulle, L., Frkovic, V., Turk, T., Azman, J., & Petrovecki, M. (2005). Prevalence of plagiarism among medical students. Croatian Medical Journal, 46(1), 126-131.
Brimble, M., & Stevenson-Clarke, P. (2005). Perceptions of the prevalence and seriousness of academic dishonesty in Australian universities. Australian Educational Researcher, 32(3), 19-44. https://doi.org/10.1007/BF03216825
Cotton, D. R. E., Cotton, P. A., & Reuben Shipway, J. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2190148
Crawford, J., Cowling, M., & Allen, K.-A. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching and Learning Practice, 20(3). https://doi.org/10.53761/1.20.3.02
Currie, G. M. (2023). Academic integrity and artificial intelligence: Is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5), 719-730. https://doi.org/10.1053/j.semnuclmed.2023.04.008
Davis, S. F., Grover, C. A., Becker, A. H., & McGregor, L. N. (1992). Academic dishonesty: Prevalence, determinants, techniques, and punishments. Teaching of Psychology, 19(1), 16-20. https://doi.org/10.1207/s15328023top1901_3
Denisova-Schmidt, E. (Ed.). (2020). Corruption in higher education: Global challenges and responses. BRILL.
Dukic, D. (2022). Plagiranje u digitalnome dobu: znanja i ponasanja studenata Sveucilista J. J. Strossmayera u Osijeku. Vjesnik bibliotekara Hrvatske, 65(1), 251-272. https://doi.org/10.30754/vbh.65.1.927
Eaton, S. E., & Christensen Huges, J. (Eds.). (2022). Academic integrity in Canada: An enduring and essential challenge. Springer International Publishing. https://doi.org/10.1007/978-3-030-83255-1
Greitemeyer, T., & Kastenmuller, A. (2023). HEXACO, the Dark Triad, and Chat GPT: Who is willing to commit academic cheating? Heliyon, 9, 1-9. https://doi.org/10.1016/j.heliyon.2023.e19909
Hill, G., Mason, J., & Dunn, A. (2021). Contract cheating: an increasing challenge for global academic community arising from COVID-19. Research and Practice in Technology Enhanced Learning, 16. https://doi.org/10.1186/s41039-021-00166-8
Ives, B., Alama, M., Mosora, L. C., Mosora, M., Grosu-Radulescu, L., Clinciu, A. I., Cazan, A.-M., Badescu, G., Tufis, C., Diaconu, M., & Dutu, A. (2017). Patterns and predictors of academic dishonesty in Romanian university students. Higher Education, 74(5), 815-831. https://doi.org/10.1007/s10734-016-0079-8
Jowarder, M. I. (2023). The influence of ChatGPT on social science students: Insights drawn from undergraduate students in the United States. Indonesian Journal of Innovation and Applied Sciences, 3(2), 194-200. https://doi.org/10.47540/ijias.v3i2.878
Majstorovic, D. (2016). Stavovi studenata korisnika Nacionalne i sveucilisne knjiznice u Zagrebu o plagiranju i javnoj objavi ocjenskih radova. Vjesnik bibliotekara Hrvatske, 59(3/4), 131-152. https://hrcak.srce.hr/187613
Malesky, A., Grist, C., Poovey, K., & Dennis, N. (2021). The Effects of peer influence, honor codes, and personality traits on cheating behavior in a University Setting. Ethics & Behavior, 22(1). https://doi.org/10.1080/10508422.2020.1869006
Ngo, T. T. A. (2023). The perception by university students of the use of ChatGPT in education. International Journal of Emerging Technologies in Learning, 78(17), 4-19. https://doi.org/10.3991/ijet.v18i17.39019
O'Rourke, J., Barnes, J., Deaton, A., Fulks, K., Ryan, K., & Rettinger, D. A. (2010). Imitation is the sincerest form of cheating: The influence of direct knowledge and attitudes on academic dishonesty. Ethics & Behavior, 20. 47-64. https://doi.org/10.1080/10508420903482616
Petrak, O., & Bartolac, A. (2014). Academic honesty amongst students of health studies. Croatian Journal of Education, 76(1), 81-117. https://hrcak.srce.hr/120169
Pupovac, V., Bilic-Zulle, L., Mavrinac, M., & Petrovecki, M. (2010). Attitudes toward plagiarism among pharmacy and medical biochemistry students - cross-sectional survey study. Biochemia Medica, 20(2), 307-313.
Rigby, D., Burton, M., Balcombe, K., Bateman, I., & Mulatu, A. (2015). Contract cheating & the market in essays. Journal of Economic Behavior & Organization, 777. 23-37. https://doi.org/10.1016/jjebo.2014.12.019
Singh, H., Tayarani-Najaran, M.-H., & Yaqoob, M. (2023). Exploring computer science students' perception of ChatGPT in higher education: A descriptive and correlation study. Education Science, 73(9), 1-23. https://doi.org/10.3390/educsci13090924
Sweeney, S. (2023). Who wrote this? Essay mills and assessment - Considerations regarding contract cheating and AI in higher education. International Journal of Management Education, 27(2). https://doi.org/10.1016/j.ijme.2023.100818
Stambuk, M., Maricic, A., & Hanzec, I. (2015). Cheating is unacceptable, but... Teachers' perceptions of and reactions to students' cheating at schools and universities. Croatian Journal of Education, 77(4), 259-288. https://hrcak.srce.hr/153248
Kukolja Taradi, S., Taradi, M., & Dogas, Z. (2012). Croatian medical students see academic dishonesty as an acceptable behaviour: a cross-sectional multicampus study. Journal of Medical Ethics, 38(6), 376-379. https://doi.org/10.1136/medethics-2011-100015
Vaccino-Salvadore, S. (2023). Exploring the ethical dimensions of using ChatGPT in language learning and beyond. Languages, 8(3). https://doi.org/10.3390/languages8030191
Walker, M., & Townley, C. (2012). Contract cheating: A new challenge for academic honesty? Journal of Academic Ethics, 70(1), 27-44. https://doi.org/10.1007/s10805-012-9150-y
Yilmaz, R., Karaoglan Yilmaz, F. G. (2023). Augmented intelligence in programming learning: Examining student views on the use of ChatGPT for programming learning. Computers in Human Behavior: Artificial Humans, 7(2). https://doi.org/10.1016/jxhbah.2023.100005
