A Study on the Utilization of OpenAI ChatGPT as a Second Language Learning Tool

ChatGPT, an AI-powered chatbot developed by OpenAI, was released in November 2022, and has sparked huge interest for its exceptional quality of response compared to other contemporary chatbots. Specifically, its ability to pass exams in business, law, and medical fields has drawn intense interest from the public [12-13]. ChatGPT can be used for various tasks including text generation, machine translation, content correction, problem solving, summarization, grammar correction, and answering questions. The chatbot operates in a conversational manner, responding to each prompt entered by the user. It is based on the deep learning model for Natural Language Process (NLP) called GPT, the sibling model GPT-3.5. Unlike other models such as Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN), GPT is faster and better at understanding long contexts, resulting in higher quality responses aligned with the user’s intent. GPT-3.5 has been trained on a larger corpus than its predecessors, enabling it to contain expert knowledge in diverse realms. However, responses are generated based on probability calculations, meaning identical prompts may result in different answers each time, and the quality of the response could vary depending on the prompt. Providing the model with adequate information increases the possibility of obtaining a desired response. ChatGPT supports multiple languages including English and Korean, and can adapt the format, length, and content of the generated text to meet the user’s specifications, creating a natural conversational experience [14].

Generative Pre-trained Transformer (GPT) is a series of cutting-edge deep learning models developed by OpenAI. It has been improved over time with the release of GPT-1, GPT-2, GPT-3, and GPT-3.5 at latest. These models employ the Transformer architecture and have shown remarkable results in various NLP tasks, including text completion, translation, summarization and answering questions [15].

The first model GPT-1 was introduced in 2018 with a new approach of learning process of unsupervised pre-training followed by supervised fine-tuning. The new process enabled the model to overcome the chronic limitation of deep learning models, which is the lack of pre-tagged data. It employed a 117M parameter model with variation of the Transformer architecture, trained on Common Crawl and BooksCorpus, achieving improved performance in natural language inference, question answering, semantic similarity, and classification tasks [16].

In 2019, GPT-2 was released, building upon the foundation laid by GPT-1. GPT-2 utilized a larger model and more data, which was collected from millions of web pages called WebText. This model used Byte Pair Encoding for input representation and had 1.5B parameters. GPT-2 demonstrated enhanced zero-shot performance in tasks such as reading comprehension, summarization, translation, and question answering [17].

GPT-3, introduced in 2020, represented a significant advancement by assimilating few-shot learning. The model could be fine-tuned for specific tasks with just a few examples without gradient updates. With 175B parameters, GPT-3 showed exceptional performance in tasks such as code and sentence auto-completion, grammar assistance, and game narrative generation [18].

The latest model in the series, GPT-3.5 (Instruct GPT), was released in 2022. This model was improved over GPT-3 by incorporating human feedback and reinforcement learning. The learning data included data collected from the OpenAI API, and human evaluations were used to construct the reward model. The most notable aspect of GPT-3.5 is that it generates text based on the user’s instruction. This means that the desired answer can be gradually derived such as users requesting the response to be more detailed or succinct. The model produces more sophisticated answers as the conversation progresses, taking into account the entire context of the conversation [19].

The GPT model has enabled ChatGPT to execute diverse tasks rapidly and accurately. ChatGPT generates human-like texts in astonishing quality. However, there are certain limitations, such as biases from the learning data, extremely high cost of training, and privacy concerns. Additionally, the model may struggle with long and complex queries. To optimize responses from ChatGPT, it is advisable to enter clear and specific instructions that include information about the domain, type, and length of the desired answer, as well as relevant keywords. Offering context about persona who is asking the question is also helpful. More specified the instructions will boost accuracy of the response. Using ambiguous words and a lengthy context is not desirable [18].

Prior to evaluating ChatGPT as a second language learning tool, ‘learning’ and ‘learning tool’ should be defined. From a constructivist perspective, learning is an individual process of constructing meaning through personal experiences, aimed at developing the ability to engage in active learning, apply acquired knowledge to diverse contexts, and solve actual problems through the transfer of knowledge [1]. In this context, the primary agent responsible for constructing the learning process is the learner, while the instructor serves as a facilitator who encourages active engagement. The integration of learner-centered pedagogical approaches with AI showed positive learning effects. For instance, the introduction of AI into Problem-Based Learning (PBL) or debate and discussion-based methods has demonstrated that AI can foster problem-solving processes and enrich the learning experience of the learner [20].

Learning tools refer to auxiliary aids that support learners in attaining targeted learning objectives through self-directed acquisition of knowledge [21]. AI, when utilized as a learning tool, can be classified into seven distinct categories of techniques: Intelligent Tutoring System (ITS), Dialogue-based Tutoring System (DBTS), Exploratory Learning Environment (ELE), Automated Writing Evaluation (AWE), Personalized Learning, Adaptive Learning and Learning analytics [22]. Among these, ChatGPT belongs to a Text-based ITS that is able to design comprehensive learning content based on the individual characteristics and needs. Additionally, ChatGPT encompasses DBTS technique, which creates a naturalistic learning through interaction, ELE that empowers learners to create their own learning environment through questioning and sharing information, and AWE to assess grammatical errors and vocabulary usage. Furthermore, ChatGPT can work on complex tasks that require creative intelligence, such as answering exam-style questions or writing academic essays, placing it as an innovative learning tool [9].

In order to evaluate the suitability as a ‘second language learning tool’, ChatGPT was tested to complete two tasks:

4.2.1. Developing Course Content

To scrutinize the design course aspect, this paper has selected H. Douglas Brown’s model of Second Language Course Development process which is illustrated in Fig. 1 [23]. This model outlines the steps for designing a course when teaching a second language. In this paper, we will test ChatGPT by directly prompting the following steps and examine whether it offers contents as we intended.

Fig. 1. second language course development process by H. Douglas Brown (2015).

Download Original Figure

The analysis of the learners’ needs and context is the initial step of the course development process. This step entails a detailed examination of the learners’ information, including age, language level, previous language learning experiences, learning motivation, and learning style. The second step involves the specification of the course goal, which impacts significantly on designing the course. In the third step, the design of learning content and structure is carried out, followed by the determination of the chapters, tasks, and schedules to be covered in the course. Once the course plan has been set, teaching action will be initiated. Subsequently, an evaluation of the course is conducted from both the instructor and the learner’s aspects. In this paper, we excluded the evaluation and revision steps since ChatGPT is unable to conduct subjective evaluation while users cannot make arbitrary modifications as well.

4.2.1.1. Identify Situation, Context and Students Account for Analyze Needs

To conduct the test, we built up a persona for the virtual learner.

• The persona is defined as a Korean Junior University student. (Age and Nationality)

• He/She is motivated to learn English for the purpose of getting a job in the US. (Motivation)

• He/She has a prior English learning experience of approximately five years, being placed at an intermediate level of proficiency. (Previous language learning experience and the fluency level)

• His/her primary goal is to develop English writing skills in a business context, targeting to transfer these skills to the real workplace. (Goal)

• To achieve this goal, he/she needs to study the structure, tone, and styles of business writing, as well as linguistic knowledge. (Need)

The above contents were summarized and entered to ChatGPT.

4.2.1.2. Specify Goals

Subsequently, we asked to specify goals based on the given information.

(Prompt) Now, specify goals based on my information.

(Response) Goal: To develop advanced proficiency in English writing for professional purposes, with a focus on business contexts.

Objectives:

• - Enhance grammatical accuracy in written English

• - Develop the ability to write clear, concise, and effective emails, reports, and presentations...(omitted)

By the end of this course, you should be able to confidently communicate in written English for various business purposes, effectively conveying your ideas and thoughts to professional audiences.

ChatGPT offered a detailed learning goal and objectives for business English writing course. Furthermore, it illustrated what kinds of capabilities would learners obtain after learning.

4.2.1.3. Determine Course Content, Sequences and Structure

Then following the goal, ChatGPT was required to set the course content, sequences, and structure.

(Prompt) Great. Now please set the course content, sequences and structure for me.

(Response)

I. Introduction to Business Writing

• - Overview of the course and its objectives

• - Overview of common business writing genres (emails, reports, presentations, proposals, etc.) ...(omitted)

This course structure is designed to provide a comprehensive learning experience for students, with a focus on both improving grammar and vocabulary, as well as developing practical writing skills in a business context.

ChatGPT presented the organizational framework for addressing learning contents and exercises for each topic. ChatGPT formulated a comprehensive curriculum that enables learners to engage in practical writing exercises, while also acquiring grammatical and vocabulary knowledge.

4.2.1.4. Design Course Units, Assignments, schedules, and Assessments

Then, we selected the ‘Writing Effective Emails’ topic and requested to develop unit plans, assignments, schedules, and assessments for this topic.

(Prompt) I would like to choose ‘Writing effective emails’ topic. Please design course unit, assignment, schedule and assessment for this topic.

(Response)

III. Writing Effective Emails (2 weeks)

• - Unit objectives: Develop the ability to write clear, concise, and professional emails

• - Assignments: Write emails for various business purposes (inquiries, follow-ups, introductions, etc.); peer review and critique of emails

• - Schedule:

  Monday to Wednesday: Overview of email writing and common styles...(omitted)

• - Assessment: Emails written for various business purposes (summative)

This sample course plan provides a comprehensive and structured approach to improving English writing skills in a business context. It includes a mix of formative and summative assessments.

ChatGPT automatically adjusted the schedule and amount of learning based on the number of days. It also recommended the implementation of formative and summative assessments to demonstrate the learner’s progress before and after the learning.

4.2.1.5. Construct Lesson Plans

We prompted again to establish lesson plans for the following topic.

(Prompt) Now, construct a lesson plan.

(Response)

Introduction (10 minutes):

• - Greet you and review the objective of the lesson

• - Ask about your experiences with writing emails in English

Presentation (30 minutes):

• - Discuss common styles and formats used in professional email writing...(omitted)

ChatGPT divided the lesson into four distinct sections: introduction, presentation, activity, and conclusion. It also outlined what would be covered in each section.

4.2.1.6. Teach

In this step, ChatGPT was instructed to teach ‘Discuss common styles and formats used in professional email writing.’

(Prompt) I want to specifically learn about common styles and formats used in professional email writing.

(Response)

Common Styles and Formats:

• - A professional email should always start with a clear and concise subject line that accurately reflects the content of the email. ...(omitted)

Example:

Subject: Request for Meeting on Thursday

Dear John,

I hope this email finds you well. I would like to request a meeting with you on Thursday, February 10th at 2 PM.

...(omitted)

Exercise: Writing an Inquiry Email

Task: Write an email to a company inquiring about their products or services.

Instructions:

• - Start with a clear and concise subject line, such as “Inquiry about [Company Name] Products/Services.”...(omitted)

After teaching style and formats, ChatGPT directly showed the related writing sample. This example helps learners reinforce their understanding through a model of practical application. Moreover, ChatGPT assigned tasks with detailed instructions, supporting learners to successfully complete by their own. This approach has two benefits. First, the structured guidelines serve as a scaffolding, promoting learners’ active knowledge acquisition and giving a sense of accomplishment. Second, it allows learners to take a self-directed learning, without relying on the intervention of instructors. Overall, ChatGPT’s course design included the presentation of target knowledge, relevant examples, and well-defined exercises, presenting a systematic and effective way of learning.

4.2.1.7. Assess

To test ChatGPT’s ability of evaluation, the author submitted a written work with deliberately including wrong an swers.

(Prompt)

Dear Samsung,

Hey, I am sunny.

I would like to know when does the new version of the mobile service be released. ...(omitted)

(Response) Your email is concise and gets straight to the point, which is a good start. However, there are a few areas for improvement in terms of tone and formatting.

Greeting: “Hey” is informal and may not be the best choice for a professional email. Instead, try using a more formal greeting such as “Dear Samsung” or “Hello Samsung.” ...(omitted)

Dear Samsung,

I hope this email finds you well.

I am writing to inquire about the release date of the new version of the Samsung Galaxy S series. ...(omitted)

By making these changes, your email will be more professional and effective in conveying your request for information.

ChatGPT identified the incorrect parts and offered revision. By presenting a model essay that embodies these corrections, learners can recognize areas to improve through comparison between their own work and the ideal model.

ChatGPT demonstrated its capability to design a comprehensive and systematic course structure through step-by-step organization of instructional contents. This function can be exploited in scenarios where learners need to proceed self-directed learning or as a reference for instructors to design their lessons.

4.2.2. Teaching Language based on TBLT Approach

In this section, ChatGPT will be evaluated its ability to employ a Task-Based Language Teaching (TBLT) method in teaching.

TBLT is rooted in Communicative Language Teaching (CLT), which emphasizes language learning through interactions that take place during task performance [23-24]. TBLT is designed to focus on the meaning rather than structure, and its tasks are about real-world activities. In this method, a task consists of a target task, which students aim to accomplish, and a pedagogical task, which involves required knowledge and techniques to perform the target task. TBLT is considered as a significant method in language learning as it allows learners to learn within the context, and promotes development of communication skills, the ultimate goal of language learning.

We chose TBLT among other approaches in CLT due to its prominence and widespread use in language learning environments [23]. The virtual persona scenario described in section 4.2.1 was redirected, and ChatGPT was asked to teach English business writing based on TBLT method. The whole process is described in Fig. 2.

Fig. 2. The process of testing ChatGPT’s ability to teach using TBLT method.

Download Original Figure

To assess ChatGPT’s effectiveness in teaching with TBLT, we compared its contents with characteristics of TBLT presented by Brown (2015) in Table 1.

Table 1. Contrast analysis of characteristics of TBLT and ChatGPT.

Characteristics of TBLT	ChatGPT
Tasks ultimately point learners beyond the forms of language alone to real-world contexts.	Yes, the learning content was related to real-world contexts.
Tasks specifically contribute to the communicative goals of learners.	Yes, the task helps improving learner’s writing skill for better communication.
Their elements are carefully designed and not simply haphazardly or idiosyncratically thrown together.	Yes, elements suggested by ChatGPT were well-designed.
Their objectives are well specified so that you can at some later point accurately determine the success of one task over another.	Yes, the objective outlines the expected outcome upon completion of task.
Tasks engage learners, at some level, in genuine problem-activity.	No, tasks given by ChatGPT were too simple and did not allow learners to actively engage.

Download Excel Table

Through the comparison analysis, ChatGPT illustrated that it can instruct language to a certain degree in the TBLT method. However, several limitations have been noted. First, the absence of active interaction was identified as a major issue. According to a criterion for technology-mediated tasks, the task should involve learners’ linguistic and nonlinguistic resources in addition to their digital skills [25]. In that, AI should be a resource, which supports learner’s exploration of information to complete the tasks on their own. However, ChatGPT merely provides model templates, which is the answer of the task, without facilitating learner’s independent thinking. The second concern is that learning content is overly simplistic. The tasks generated by ChatGPT are specific and practical, but the explanations of target knowledge to perform those tasks were inadequate. To enhance learners’ fluency, it is necessary to teach in depth rather than simply listing the knowledge. Third, ChatGPT’s capability of assessing is very limited. ChatGPT’s evaluating learners’ written work only captures fragmentary and superficial aspect such as grammatical error or awkward expression. It cannot cover comprehensive factors that occur during the learning process, which may limit learners to derive deeper insights.

(Prompt) Based on the previous information I gave you, please teach me business English writing using TBLT method including ‘objective’, ‘target task’, and ‘pedagogical task’ elements.

(Response)

Objective: To develop the skills necessary to write a professional email to decline an invitation while maintaining a positive relationship with the sender.

Task: Writing a professional email to decline an invitation

Target Task: Write a professional email to decline an invitation while maintaining a positive relationship with the sender.

Pedagogical Tasks:

• -Understanding the purpose of the email and the language to use when declining an invitation politely...(omitted)

• - Email structure:

  Greeting: Start with a polite greeting such as “Dear [Sender’s Name],” ...(omitted)

• - Language:

  Decline: Use phrases such as “Unfortunately, I won’t be able to attend...”, “I regret to inform you that I won’t be able to make it to the event.”...(omitted)

Example:

Dear [Sender’s Name],

Thank you so much for the invitation to the [Event Name]. I really appreciate the opportunity...(omitted)

The results clearly indicate that ChatGPT needs to be modified before the actual implementation. While an instructor or students may refer to ChatGPT as a learning assistant, they should not rely on it completely for the entire learning process.

First, ChatGPT is multifaceted. Leveraging an extensive corpus of learning data, it can create numerous types of customized learning contents for more than 12 languages, skills, topics, and levels. This allows abundant learning experiences in various contexts with learners’ needs. Second, ChatGPT is flexible. Users can adjust input and output forms freely. ChatGPT can offer long and detailed information, while users can also ask ChatGPT to make its answer concise and straightforward. Moreover, ChatGPT can integrate knowledge from multiple areas, coping with complex and comprehensive knowledge. This is demonstrated through its capacity to fulfill specific requests such as ‘explaining the concept of quadratic equations in Chinese language.’ Thirdly, ChatGPT is consistent and evolvable. Unlike traditional AI models that tend to lose context as more inputs are introduced, ChatGPT continuously stores information from previous interactions, allowing for more relevant and accurate responses. This feature ensures learners to be immersed in natural learning atmosphere without disruption. Lastly, ChatGPT is highly responsive. ChatGPT’s almost immediate answers help facilitating an efficient learning in a supportive learning environment.

As the limitations, firstly ChatGPT is susceptible to biases, as it is heavily influenced by learning data. Hence, critical review and selection of relevant information skills are required for users. Second, ChatGPT is limited to conducting systematic assessment of learning. Unlike human instructors, ChatGPT is incapable of providing detailed feedback. Based on our tests, ChatGPT was only able to evaluate linguistic knowledge and coherence in the written work. While it can offer basic feedback, it does not provide comprehensive evaluations that can drive learners to improve their communication skills. Third, ChatGPT is a text-based language model, which may not be optimized to practice listening and speaking skills. As it does not contain Speech to Text (STT) or Text to Speech (TTS) functions, users may need to supplement these tools additionally to practice listening and speaking. This also implies ChatGPT may not be able to accurately measure elements such as tone and intonation in speech. Lastly, ChatGPT can be abused. It can possibly hinder learning by directly providing answers or performing tasks instead of learners. In such cases, it is desirable for a human instructor to supervise learners utilizing ChatGPT in learning environment.