DeepAI
Log In Sign Up

Future of Smart Classroom in the Era of Wearable Neurotechnology

10/21/2021
by   Mojtaba Taherisadr, et al.
0

Interdisciplinary research among engineering, computer science, and neuroscience to understand and utilize the human brain signals resulted in advances and widespread applicability of wearable neurotechnology in adaptive human-in-the-loop smart systems. Considering these advances, we envision that future education will exploit the advances in wearable neurotechnology and move toward more personalized smart classrooms where instructions and interactions are tailored towards. students' individual strengths and needs. In this paper, we discuss the future of smart classrooms and how advances in neuroscience, machine learning, and embedded systems as key enablers will provide the infrastructure for envisioned smart classrooms and personalized education along with open challenges that are required to be addressed.

READ FULL TEXT VIEW PDF
07/07/2020

Computer-Aided Personalized Education

The shortage of people trained in STEM fields is becoming acute, and uni...
02/19/2018

Human and Smart Machine Co-Learning with Brain Computer Interface

Machine learning has become a very popular approach for cybernetics syst...
05/07/2017

AirDraw: Leveraging Smart Watch Motion Sensors for Mobile Human Computer Interactions

Wearable computing is one of the fastest growing technologies today. Sma...
11/14/2021

Learning Enhancement in Higher Education with Wearable Technology

Wearable technologies have traditionally been used to measure and monito...
07/06/2022

Smart Education: Higher Education Instruction and the Internet of Things (IoT)

The Internet of Things (IoT) has many applications in our daily lives. O...
05/17/2022

Embedded Systems Education in the 2020s: Challenges, Reflections, and Future Directions

Embedded computing systems are pervasive in our everyday lives, impartin...
05/23/2022

Visual and unplugged coding lessons with smart toys

Our Computer science k-12 education research group and the educational t...

I Introduction

The disruption of face-to-face exchanges and the urgent digital transformation of our everyday life activities, abide the COVID-19 hit, have spurred us to shift many working paradigms to incorporate the new reality of the online and remote collaborative environment. In particular, many sectors including the education and the healthcare systems had to integrate new technology to assist the continuation of their services even with remote interactions [30, 20]. Nevertheless, this forced digital transformation of many sectors is expected to stay even in the post-COVID-19 era. In particular, in the education sector, the e-learning market worldwide is forecast to surpass 243 billion U.S. dollars by 2022 [29].

While the education sector has already taken technological leaps over the last decades with the introduction of smart classroom concept [16, 28, 6], efficient teacher-student interaction has to be envisioned especially with the possibility of remote education. In particular, we envision that the future of education using smart classrooms should move from a one-size-fits-all approach to a personalized process in which instructions and interactions are tailored to students’ individual needs. In this paper, we argue that advances in bio-sciences, machine learning, and embedded systems can play a significant role in realizing the vision of personalized and smart education.

A cornerstone in the envisioned personalized learning is to replace the traditional measures of education (e.g., quizzes, scores in exams, and teacher evaluations) with real-time measurements of the student’s state of mind. Such real-time signals can be then used to adapt the instructed materials to maximize the student’s performance. The student’s stress level, drowsiness level, readiness to learn new knowledge, cognitive load, and learning rate are just examples of such real-time signals that can be used to adapt the instructed materials to enhance the learning performance. Indeed, there is a tremendous opportunity in attaining this vision. In particular, advances in neuroscience and wearable sensors technology have widen the horizon to reveal the fundamental processes of various human mental states [4, 15]. These advancements resulted in tremendous research efforts among interdisciplinary fields from engineering researchers, computer scientists, and healthcare (neurologists, clinicians) to integrate the brain state (drowsiness, sleep quality, readiness potential, and learning activity) as an additional sensor modality to design adaptive human-in-the-loop smart systems [9]. Using such advances, we envision the future smart classroom as shown in Figure 1. Future smart classroom will be extended to include physical and remote students. Students’ state will be used to adapt the education system through feedback and personalized recommendation to enhance the student’s performance.

In what follows, we highlight two scientific and technological advances that are key enablers for the envisioned personalized and smart classrooms along with open challenges that need to be addressed.

Fig. 1: Future smart classroom in the era of wearable neurotechnology. Student’s mental state inferred from wearable sensors is used to provide feedback and recommendation to the student and the teacher for both physical and remote classes.

Ii Advancement in Neuroscience to understand cognitive processing

As discussed above, a fundamental challenge to achieve the aforementioned vision is to measure the student’s mental state, in real-time. Indeed, recent advances in neuroscience have opened the gate to unveil fundamental processes in the human brain, such as the ability to generate emotions, memories, and actions [33, 3]. These research efforts have become possible by the ability to record and stimulate the human brain activity in the clinical setup by neurologists with a very high accuracy [25]

. Optimal cognitive processing is central to all aspects of human activities, and recent advances in neuroscience have provided critical insights on how the brain accomplishes cognitive processing, spanning single neuron to neural population level resolution 

[25].

In particular, recent studies using event-related functional magnetic resonance imaging (fMRI) showed that there exists a high correlation between the magnitudes of focal activation in the right prefrontal cortex and the bilateral parahippocampal cortex during visual learning with the memory processing [32]. In addition, recent findings in literature demonstrating that brain electrophysiological activity observed before stimulus or new concepts presentation profoundly influences subsequent behavior on fear conditioning [27], motor responses [21], attention [19], and memory tasks [1].

While these studies gave us the fundamentals to measure the ability to memorize visual content, and the specific parts in the brain that are responsible for cognitive processing, the form factor of the utilized machinery prevent the widespread use of this technology in classroom settings. The next section reviews some technological advances that can provide a solution to such a challenge.

Iii Advancement in wearable neurotechnology

With the development of efficient, intelligent sensing technology, detecting small changes in electric signals becomes plausible. This technology paved the way for developing and producing devices that are capable of monitoring the brain’s electrical activity non-invasively with enough resolution and low cost compared to the clinical setups. Although the traditional brain activity monitoring systems suffer from motion artifacts, long preparation time due to conductive gels or pastes, and large equipment (metal electrodes, long wires), the advances in sensor technology allow a fully portable, wireless, long-term, flexible scalp electronic system, incorporating a set of dry electrodes [18]. Moreover, the importance of wearable brain activity monitoring devices is that they are feasible for envisioning a real-time human-in-the-loop education system thanks to their portability, comfortability, and wireless data transfer system.

A study by Dikker et al. [8] proves that using a portable electroencephalogram (EEG) device for recording brain activity from a class of 12 high school students over a semester during regular activities, analyzing the group-based neural coherence is possible where the brain activity is synchronized across students in both student class engagement and social dynamics.

Another study by Babini et al.[2] comparatively measured the learning of the students in a virtual reality (VR) environment for using a wearable electroencephalogram (EEG) [14]. In particular, they compared the brain engagement level of a group of students while learning through a VR environment compared to the regular two-dimensional environment. Their study showed that the higher the brain engagement, the higher the attention level of the students which led to better learning performance.

Moreover, the developments in wearable neurotechnology devices can enable complex applications to be integrated to understand and predict cognitive processing. For example, sleep has a pivotal role in cognitive functions, and their relationship has been a topic of interest for over a century. Extensive research on sleep studies has shown that better sleep is associated with a myriad of superior cognitive functions in healthy adults [31, 17], including better learning and memory [24]. Although the exact mechanisms behind the relationship between sleep, memory, and learning are still research topics, the general agreement is those specific synaptic connections that were active during awake periods are strengthened during sleep, allowing for the consolidation of memory, and inactive synaptic connections are weakened [23]. Consequently, sleep provides an essential function for memory consolidation; in other words, it will enable us to remember the topics that have been studied before[23]. It is now well established from a variety of studies that sleep monitoring can be used to associate with the students’ performances. Moreover, sleep monitoring should be long-term rather than a single night or particular event recordings. Therefore, the recent developments on low-power wearable devices have crucial importance in monitoring students’ sleep or emotional situations in the long term to create the future smart classroom environments [7].

Iv Personalized feedback for the future smart classroom

When these advancements in neuroscience and sensor technology combined with the current evolution in machine learning and signal processing, classification, and analysis of the massive amount of data streaming from wearable devices have become practicable. The resourcefulness of machine learning models provides the ability to analyze the complex recurring patterns of neural activity and detect hidden patterns. However, there are several challenges that need to be addressed to use these advancements to provide personalized feedback to the students and adapt the teaching environment for future smart classrooms.

  • Challenge 1: Decision making in the face of system variability:

    Even if we are able to decode the brain activity in real-time to estimate the cognitive processing, addressing the students’ individual needs to achieve personalized feedback has a lot of challenges. In particular, there are intrinsic variations that humans exhibit that can complicate the idea of a personalized smart classroom. As discussed in one of our recent works, the same human’s preferences and responses may change over time. Even for a small period of time, the same human may produce different responses based on unmodeled external effects (intra-human variability). Similarly, different humans may have different responses under similar conditions (inter-human variability) 

    [12]. For example, adapting the pace of the teaching material or the time of a quiz based on the cognitive load and the sleep quality of the student may not be the same for a different student and even for the same student across time. Moreover, when multiple humans interact in the same environment, their preferences differ based on the number of people they interact with and the type of this interaction. Moreover, when the education system adapts to one student in a multi-student classroom (e.g., change the pace of the presented material), it has a direct effect on the other students in the same classroom. In addition to that, multiple students may require different feedback adaptation actions based on their different preferences (multi-human variability) which can raise a question of fairness of adaptation in future smart classrooms [13, 11]. Hence, these human variations have to be considered as an integral part of the design process of the adaptation algorithms for the future of the smart classroom.

  • Challenge 2: Restructuring the pedagogical materials: Prior to applying the vision of personalized feedback in the future smart classroom, the pedagogical materials have to be reconstructed to be adaptable. For example, teaching materials have to be redesigned into small modules that are self-contained to enable quizzes and breaks at any time. Moreover, the same teaching materials themselves have to be designed into different modalities, such as project-based learning with a small group of peers, independent work to complete complex tasks, using immersive technology (VR) to increase the engagement of some students, and even recordings of the material to be watched at a different pace. Quizzes have to redesigned be automated based on the time they are taken and the student’s strengths. Projects and group assignments have to take into account the group coherency and the average learning performance of the group members. Even teachers have to be trained to keep an up-to-date record from real-time data that provides a deep understanding of each student’s individual strengths, needs, motivations, progress and goals [22].

  • Challenge 3: Internet-of-Things (IoT) design and robustness to errors: Infusing technology into pedagogy to attain the vision of the future smart classroom requires careful design of the enabling technology to ensure high accuracy of real-time data and robustness to errors. Albeit the wide-ranging adaptation of the Internet-of-Things (IoT) technology in different application domains, low power and low energy design of wearable embedded systems are some of the most critical challenges to realize this new paradigm of future smart classrooms and to ensure its feasibility. Moreover, the need to process EEG data at the edge level with high accuracy (popularly termed as machine learning at the edge) due to privacy, security, and communication latency require new hardware-software design techniques to achieve low cost for accurate real-time local processing [7]. Current research in the area of battery-aware task scheduling for wearables [10], post CMOS technology, and Neuromorphic computing systems [5, 26] will play a significant role in developing such low power, low energy, and high accuracy efficient IoT design for future smart classrooms.

V Conclusion

This is an opportune time for smart classrooms. The increased demands of new affordable yet accurate solutions for human-in-the-loop systems lead to rich research areas, especially in human sensing. With the advancement in neuroscience and wearable neurotechnology, new possibilities for improving the quality of the education system by merging both fields have emerged. However, new challenges are yet to be solved in all development phases of future smart classrooms, from more neuroscience insights into the brain circuitry to design phases of accurate sensing to efficient adaptation algorithms. This article highlights the most prominent opportunities for future smart classrooms and recent approaches to tackle them. Nevertheless, this is not an exhaustive list leaving much to be discovered and done.

References

  • [1] R. J. Addante, A. J. Watrous, A. P. Yonelinas, A. D. Ekstrom, and C. Ranganath (2011) Prestimulus theta activity predicts correct source memory retrieval. Proceedings of the National Academy of Sciences 108 (26), pp. 10702–10707. Cited by: §II.
  • [2] M. H. Babini, V. V. Kulish, and H. Namazi (2020) Physiological state and learning ability of students in normal and virtual reality conditions: complexity-based analysis. Journal of Medical Internet Research 22 (6), pp. e17945. Cited by: §III.
  • [3] I. Brain (2021) IEEE brain. Note: https://brain.ieee.orgLast accessed January 2021 Cited by: §II.
  • [4] C. Chang and J. E. Chen (2021) Multimodal eeg-fmri: advancing insight into large-scale human brain dynamics. Current Opinion in Biomedical Engineering, pp. 100279. Cited by: §I.
  • [5] C. N. Computing (2015) Neuromorphic computing: from materials to systems architecture. Cited by: 3rd item.
  • [6] Y. Dai, Z. Liu, and D. Luo (2020) Research on innovation and application of multi-dimensional interactive teaching mode in smart classroom: taking junior physics applied research as an example. In Journal of Physics: Conference Series, Vol. 1575, pp. 012211. Cited by: §I.
  • [7] B. U. Demirel, I. Skelin, H. Zhang, J. J. Lin, and M. A. A. Faruque (2021) Single-channel eeg based arousal level estimation using multitaper spectrum estimation at low-power wearable devices. External Links: 2108.00216 Cited by: §III, 3rd item.
  • [8] S. Dikker, L. Wan, I. Davidesco, L. Kaggen, M. Oostrik, J. McClintock, J. Rowland, G. Michalareas, J. J. Van Bavel, M. Ding, and D. Poeppel (2017) Brain-to-brain synchrony tracks real-world dynamic group interactions in the classroom. Current Biology 27 (9), pp. 1375–1380. External Links: ISSN 0960-9822, Document, Link Cited by: §III.
  • [9] S. Elmalaki, B. U. Demirel, M. Taherisadr, S. Stern-Nezer, J. J. Lin, and M. A. Al Faruque (2021) Towards internet-of-things for wearable neurotechnology. In 2021 22nd International Symposium on Quality Electronic Design (ISQED), pp. 559–565. Cited by: §I.
  • [10] S. Elmalaki, M. Gottscho, P. Gupta, and M. Srivastava (2014) A case for battery charging-aware power management and deferrable task scheduling in smartphones. In 6th Workshop on Power-Aware Computing and Systems (HotPower 14), Cited by: 3rd item.
  • [11] S. Elmalaki, Y. Shoukry, and M. Srivastava (2018) Internet of personalized and autonomous things (iopat) smart homes case study. In Proceedings of the 1st ACM International Workshop on Smart Cities and Fog Computing, pp. 35–40. Cited by: 1st item.
  • [12] S. Elmalaki, H. Tsai, and M. Srivastava (2018)

    Sentio: driver-in-the-loop forward collision warning using multisample reinforcement learning

    .
    In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 28–40. Cited by: 1st item.
  • [13] S. Elmalaki (2021) FaiR-iot: fairness-aware human-in-the-loop reinforcement learning for harnessing human variability in personalized iot. In Proceedings of the International Conference on Internet-of-Things Design and Implementation, pp. 119–132. Cited by: 1st item.
  • [14] EMOTIV (2020) EMOTIV epoc+ 14 channel mobile brainwear. Note: https://www.emotiv.com/product/emotiv-epoc-14-channel-mobile-eeg/Last accessed June 2020 Cited by: §III.
  • [15] X. Gu, Z. Cao, A. Jolfaei, P. Xu, D. Wu, T. Jung, and C. Lin (2021) EEG-based brain-computer interfaces (bcis): a survey of recent studies on signal sensing technologies and computational intelligence approaches and their applications. IEEE/ACM transactions on computational biology and bioinformatics. Cited by: §I.
  • [16] M. Kwet and P. Prinsloo (2020) The ‘smart’classroom: a new frontier in the age of the smart university. Teaching in Higher Education 25 (4), pp. 510–526. Cited by: §I.
  • [17] J. Lim and D. F. Dinges (2010-05) A meta-analysis of the impact of short-term sleep deprivation on cognitive variables.. Psychological Bulletin 136 (3), pp. 375–389 (en). External Links: ISSN 1939-1455, 0033-2909, Link, Document Cited by: §III.
  • [18] M. Mahmood, D. Mzurikwao, Y. S. Kim, Y. Lee, S. Mishra, R. Herbert, A. Duarte, C. S. Ang, and W. H. Yeo (2019-09)

    Fully portable and wireless universal brain–machine interfaces enabled by flexible scalp electronics and deep learning algorithm

    .
    Nature Machine Intelligence 1 (9), pp. 412–422 (en). External Links: ISSN 2522-5839, Document Cited by: §III.
  • [19] S. Makeig (2002) Response: event-related brain dynamics–unifying brain electrophysiology. Trends in neurosciences 25 (8), pp. 390. Cited by: §II.
  • [20] S. Mantena and S. Keshavjee (2021) Strengthening healthcare delivery with remote patient monitoring in the time of covid-19. BMJ Health & Care Informatics 28 (1). Cited by: §I.
  • [21] A. Mazaheri, I. L. Nieuwenhuis, H. Van Dijk, and O. Jensen (2009) Prestimulus alpha and mu activity predicts failure to inhibit motor responses. Human brain mapping 30 (6), pp. 1791–1800. Cited by: §II.
  • [22] A. Morin (2014) The everything parent’s guide to special education: a complete step-by-step guide to advocating for your child with special needs. Simon and Schuster. Cited by: 2nd item.
  • [23] K. Okano, J. R. Kaczmarzyk, N. Dave, J. D. E. Gabrieli, and J. C. Grossman (2019-12) Sleep quality, duration, and consistency are associated with better academic performance in college students. npj Science of Learning 4 (1), pp. 16 (en). External Links: ISSN 2056-7936, Link, Document Cited by: §III.
  • [24] J. F. G. PhD (2010) The prevalence of sleep disorders in college students: impact on academic performance. Journal of American College Health 59 (2), pp. 91–97. Note: PMID: 20864434 External Links: Document, Link, https://doi.org/10.1080/07448481.2010.483708 Cited by: §III.
  • [25] R. Q. Quiroga (2019) Plugging in to human memory: advantages, challenges, and insights from human single-neuron recordings. Cell 179 (5), pp. 1015–1032. Cited by: §II.
  • [26] K. Roy, A. Jaiswal, and P. Panda (2019) Towards spike-based machine intelligence with neuromorphic computing. Nature 575 (7784), pp. 607–617. Cited by: 3rd item.
  • [27] M. A. Seager, L. D. Johnson, E. S. Chabot, Y. Asaka, and S. D. Berry (2002) Oscillatory brain states and learning: impact of hippocampal theta-contingent training. Proceedings of the National Academy of Sciences 99 (3), pp. 1616–1620. Cited by: §II.
  • [28] N. Songkram, N. Songkram, S. Chootongchai, and T. Samanakupt (2021) Developing students’ learning and innovation skills using the virtual smart classroom.. International journal of emerging technologies in learning 16 (4). Cited by: §I.
  • [29] Statistica Size of e-learning market in 2014 and 2022. Note: https://www.statista.com/statistics/501104/worldwide-elearning-market-size/Accessed: 2021-10-07 Cited by: §I.
  • [30] O. Taiwo and A. E. Ezugwu (2020) Smart healthcare support for remote patient monitoring during covid-19 quarantine. Informatics in medicine unlocked 20, pp. 100428. Cited by: §I.
  • [31] U. Wagner, S. Gais, H. Haider, R. Verleger, and J. Born (2004-01) Sleep inspires insight. Nature 427 (6972), pp. 352–355 (en). External Links: ISSN 1476-4687, Link, Document Cited by: §III.
  • [32] J. J. Yoo, O. Hinds, N. Ofen, T. W. Thompson, S. Whitfield-Gabrieli, C. Triantafyllou, and J. D. Gabrieli (2012) When the brain is prepared to learn: enhancing human learning using real-time fmri. Neuroimage 59 (1), pp. 846–852. Cited by: §II.
  • [33] J. Zheng, R. F. Stevenson, B. A. Mander, L. Mnatsakanyan, F. P. Hsu, S. Vadera, R. T. Knight, M. A. Yassa, and J. J. Lin (2019) Multiplexing of theta and alpha rhythms in the amygdala-hippocampal circuit supports pattern separation of emotional information. Neuron 102 (4), pp. 887–898. Cited by: §II.