Spoken Language Intelligence of Large Language Models for Language Learning

by   Linkai Peng, et al.

People have long hoped for a conversational system that can assist in real-life situations, and recent progress on large language models (LLMs) is bringing this idea closer to reality. While LLMs are often impressive in performance, their efficacy in real-world scenarios that demand expert knowledge remains unclear. LLMs are believed to hold the most potential and value in education, especially in the development of Artificial intelligence (AI) based virtual teachers capable of facilitating language learning. Our focus is centered on evaluating the efficacy of LLMs in the realm of education, specifically in the areas of spoken language learning which encompass phonetics, phonology, and second language acquisition. We introduce a new multiple-choice question dataset to evaluate the effectiveness of LLMs in the aforementioned scenarios, including understanding and application of spoken language knowledge. In addition, we investigate the influence of various prompting techniques such as zero- and few-shot method (prepending the question with question-answer exemplars), chain-of-thought (CoT, think step-by-step), in-domain exampler and external tools (Google, Wikipedia). We conducted large-scale evaluation on popular LLMs (20 distinct models) using these methods. We achieved significant performance improvements compared to the zero-shot baseline in the practical questions reasoning (GPT-3.5, 49.1 63.1 sizes have good understanding of concepts in phonetics, phonology, and second language acquisition, but show limitations in reasoning for real-world problems. Additionally, we also explore preliminary findings on conversational communication.


page 4

page 5

page 7

page 8

page 9

page 21

page 22


Large Language Models as Zero-Shot Conversational Recommenders

In this paper, we present empirical studies on conversational recommenda...

EmotionPrompt: Leveraging Psychology for Large Language Models Enhancement via Emotional Stimulus

Large language models (LLMs) have achieved significant performance in ma...

Can large language models reason about medical questions?

Although large language models (LLMs) often produce impressive outputs, ...

Potential Benefits of Employing Large Language Models in Research in Moral Education and Development

Recently, computer scientists have developed large language models (LLMs...

Large language models are not zero-shot communicators

Despite widespread use of LLMs as conversational agents, evaluations of ...

Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models

Incident management for cloud services is a complex process involving se...

AGenT Zero: Zero-shot Automatic Multiple-Choice Question Generation for Skill Assessments

Multiple-choice questions (MCQs) offer the most promising avenue for ski...

Please sign up or login with your details

Forgot password? Click here to reset