Making GenAI Integration Accessible for Interactive Immersive Teaching Tools

July 4, 2024

The annual CANHEIT conference brings together the community of IT professionals in higher education to promote best practices, innovation, and collaboration. This year, Daniel Lindenberger, Frederik Svendsen, and Saeed Dyanatkar from UBC’s Emerging Media Lab (EML) presented on the steps being taken by their team to make the integration of generative AI in interactive immersive teaching tools more accessible.

The EML places a particular focus on the use of innovative new technologies, and how they can be applied to enhance learning. With the recent advancements in generative AI, many opportunities have arisen to integrate AI into immersive teaching tools to improve their educational value. Such integration of AI into existing solutions is highlighted by two EML projects – JIS (Judicial Interrogatory Simulator) and Nurse Practitioner VR. These interactive experiences use AI agents to assume the roles of courtroom judge and nurse patient respectively, empowered by a Large Language Model (LLM) server that enables accurate, structured responses that react realistically to user inputs. This helps users familiarize themselves with the complexities of each simulated scenario such as timing, improvisation, responding, and more, which can be difficult to replicate through traditional learning.

EML’s AI Server

These immersive teaching tools, designed to equip users with practical experience and learning, rely on a sense of realism and interactivity to deliver their educational value. Rapid and accurate responses to user inputs are key in delivering immersive education, and generative AI and its associated technologies like LLMs and Natural Language Processing (NLP) can be leveraged to provide unprecedented levels of interactivity, particularly in the areas of language, speech, and response. To meet the needs of such use-cases, EML’s AI server is designed to be fast, flexible, extensible, and reliable, enabling students to innovate whilst keeping basic project setup times to a minimum. The EML AI server provides easy access to a variety of AI features including live chat, text to speech, and speech to text. With live speech recognition, EML’s LLM API Server can trigger prompts and responses in the AI agents or simulated environment according to the user’s inputs. Additional features such as image analysis and function call are currently in development.

As emerging technologies like AI, Virtual Reality (VR), and Mixed Reality (MR) continue to advance, UBC’s EML remains steadfast in exploring technology as an interactive medium for education. With 6 projects themed around AI in 2023 and the development of the AI Server, the EML is looking towards a future where generative AI could evolve learning and higher education.

Learn more about EML’s projects on their website.

Article written by Aaren Lin

Page last updated on July 4, 2024


UBC Crest The official logo of the University of British Columbia. Urgent Message An exclamation mark in a speech bubble. Bluesky The logo for the Bluesky social media service. Bookmark A bookmark in a book. Browser A web browser window. Caret An arrowhead indicating direction. Arrow An arrow indicating direction. Arrow in Circle An arrow indicating direction. Arrow in Circle An arrow indicating direction. Time A clock. Chats Two speech clouds. E-commerce Cart A shopping cart. Facebook The logo for the Facebook social media service. Help A question mark in a circle. Home A house in silhouette. Information The letter 'i' in a circle. Instagram The logo for the Instagram social media service. Linkedin The logo for the LinkedIn social media service. Location Pin A map location pin. Mail An envelope. Menu Three horizontal lines indicating a menu. Minus A minus sign. Pencil A pencil indicating that this is editable. Telephone An antique telephone. Play A media play button. Plus A plus symbol indicating more or the ability to add. Search A magnifying glass. Settings A single gear. Arrow indicating share action A directional arrow. Speech Bubble A speech bubble. Star An outline of a star. Twitter / X The logo for the X (aka, Twitter) social media service. User A silhouette of a person. Vimeo The logo for the Vimeo video sharing service. Youtube The logo for the YouTube video sharing service.