MISW/AOTULE 2019: Challenges of Developing a Metacognitive Tutor on Open edX

Every year, the three schools of engineering at Tokyo Tech hosts the Multidisciplinary International Student Workshop (MISW) where exchange and local students are given the opportunity to present their work. I presented the results of my pilot study at this venue in 2019. I received a Best Presentation Award (given to five oral presentations) for my MISW 2019 work. 

Photo from Luc Gougeon

The best presentations from MISW 2019 are selected to attend the Asia-Oceania Top University League of Engineering (AOTULE) Student Conference 2019. The AOTULE Student Conference is held at different venues rotating among member universities. For 2019, the host university was Tokyo Tech, which was unfortunate because I did not get to have a free trip, but also fortunate because it was convenient for me. 

For a copy of relevant materials (e.g., presentation, paper) or any questions you may have, please feel free to reach out to me through the Contact Me gadget on this blog's side bar.

Details

Title: Challenges of Developing a Metacognitive Tutor on Open edX

Authors: May Kristine Jonson Carlon, Jeffrey S. Cross

Venue: Asia-Oceania Top University League on Engineering 2019 Conference

Date: November 25 to 27, 2019

Abstract

Metacognition, or the awareness of knowledge level and ability to regulate it, is important to a learner since it is a good predictor of academic performance and success in lifelong learning. In this research, we developed the Personalized Online Adaptive Learning System (POALS), a metacognitive tutor that can be embedded in Learning Management Systems (LMS) such as Open edX. Aside from the complexity of creating a system that interacts with an external system and ensuring the security of learner information, another challenge is validating whether POALS is effective in raising metacognitive skills. Metacognitive development can be measured through online (real-time in-activity measurement) and offline (independently gathered measurement) methods. A pilot study was conducted during the first quarter of the academic year 2019 in an Educational Technology class with 17 learners to give us an idea on POALS’ effectiveness using the data collected by the Learner Profile, a measuring tool that uses the online method built into POALS, and to select an appropriate offline method for succeeding experiments. The results indicate that POALS dramatically increased students’ knowledge awareness with no conclusive effect on reducing bias (i.e. being too optimistic or pessimistic about their knowledge), thus the overall result is moderate. We also learned that for succeeding experiments, the Metacognitive Awareness Inventory (MAI) is preferable to Goal-oriented studying, Active studying, Meaningful and memorable studying, Explain to understand, and Self-monitor (GAMES) survey as the offline method since MAI better corroborates with the results of the offline method.