Using AI for Immersive Educational Experiences

This issue is brought to you by Goatstack.ai

Welcome to AutomatED: the newsletter on how to teach better with tech.

Each week, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In this week’s piece, I provide updates on Sora, Devin, and a range of other exciting AI developments relevant to higher educators, and I offer a positive vision of the future of colleges and universities where immersive educational experiences play a greater role for a wider range of students.

📢 Quick Hits:
News Tidbits for Higher Educators

• Sora, OpenAI’s text-to-video generative AI tool, will be publicly available later this year, CTO Mira Murati tells the Wall Street Journal. The wider availability will be complemented by features for audio incorporation as well as content editing to address the challenge of AI-generated inaccuracies.

  • Why it matters: Sora is capable of generating hyper-detailed videos, including from descriptions that lack corresponding detail. This comes with serious risks, further heightening concerns about deepfakes and increasing the need for AI literacy. However, it also offers professors and other educators novel methods to create immersive and interactive learning experiences, as I discuss in my Idea of the Week.

• Devin, an AI designed as the world’s first fully autonomous software engineer, has demonstrated remarkable proficiency on the SWE-bench coding benchmark. Achieving a 13.86% end-to-end issue resolution rate, it significantly outperforms prior models and showcases an array of capabilities, including long-term planning, learning, bug fixing, and active collaboration with human engineers.

  • Why it matters: Devin's introduction marks a significant advancement in AI's role in software development, with implications for higher education and research in computer science. It underscores that future computer science curricula will need to incorporate AI collaboration, and research will likely continue to shift towards improving human-AI co-development. For administrators, this signals a continued shift in faculty research directions and could also signal a reevaluation of IT resource allocation.

• The European Parliament has passed the Artificial Intelligence Act (AI Act), aiming to safeguard rights and foster innovation while establishing Europe as a leader in AI regulation. Revised after ChatGPT’s massive impact was felt, the Act is especially focused on mitigating dangers posed by high-risk AI applications.

  • Why it matters: The Act delineates a clear regulatory framework that prioritizes safety, transparency, and accountability in AI applications, including those used for educational purposes. The focus on innovation and support for smaller and mid-sized businesses and startups through regulatory sandboxes presents opportunities for educational institutions to collaborate on developing cutting-edge AI, so long as they are compliant.

• Microsoft's Work Trend Index Special Report reveals substantial productivity gains among early users of Copilot for Microsoft 365 (the version of Copilot for the cloud-based Office suite; click here for our disambiguation of the various Copilots). The report highlights that 70% of respondents experienced increased productivity, with meeting, email, and writing tasks completed 29% faster on average and a strong inclination (77%) to continue using the tool.

  • Why it matters: This report indicates a significant impact of generative AI on professional workflows. As we have discussed (see here for a free weekly on this topic, and here for a Premium Tutorial), such tools can enhance productivity and creative processes in academia, which is crucial for educators seeking to streamline administrative tasks and focus on teaching and research.

• Microsoft recently updated Copilot Pro to include integration with free Microsoft 365 web apps. As we explained earlier this year, Copilot Pro is for Personal and Family plans.

  • Why it matters: The shift indicates a need for higher education to adapt quickly to avoid being outpaced by personal/home user advancements. Students entering university with pre-college exposure to AI tools like Copilot Pro might find a mismatch in institutional support for these technologies if their institution doesn’t have Copilot for Microsoft 365 enabled, which could influence their learning experience and productivity. At the same time, universities must navigate the complex landscape of data protection and privacy when implementing such tools, a challenge that Microsoft is working to address (including via its commitment to enterprise-grade security in its Microsoft Copilot offerings).

💡 Idea of the Week:
AI and Immersive Education

With the announcement of OpenAI’s Sora public rollout later this year, we near another chapter in the integration of AI in education.

For us, I think the biggest promise of AI tools like Sora — that can create video with ease — is that they lower the cost of immersive educational experiences. This increases the availability of these experiences, expanding their reach to student populations who wouldn’t otherwise have them, whether due to time, distance, or expense.

Consider the profound impact on a history class, where students are transported to California during the gold rush through hyperrealistic video sequences. This vivifies the historical content and cultivates a deeper connection with the material.

In fact, OpenAI has already demonstrated the promise of this sort of use case, with a very simple prompt producing impressive results:

And while video-producing generative AI certainly enhances immersion, text-producing tools already take us some degree towards this future.

For example, imagine leveraging a custom GPT to simulate the nuanced diplomatic stances of different nations at a key point in time in an international relations course. Students could interact with AI-generated diplomats, engaging in simulated negotiations that reinforce their understanding of complex geopolitical dynamics.

In both cases, AI tools democratize the creation of high-fidelity educational content, allowing educators to design bespoke learning experiences that make clear the relevance of the course material while being responsive to student interests.

The process of creating these simulations with AI can also serve as a lesson in AI literacy, arming students with the critical skills necessary to navigate and scrutinize an increasingly AI-driven world.

For example, the history professor could work with students to analyze and produce prompts for video content that results in more accurate depictions of historical events and settings. OpenAI’s basic prompt (“Historical footage of California during the gold rush”) would undoubtedly need to be expanded upon and refined to address inaccuracies and misrepresentations in Sora’s output.

By actively participating in the development and critique of AI-generated material, students learn to discern the strengths and limitations of AI, fostering a judicious approach to future technologies.

In sum, generative AI can act as an academic lever, propelling us toward a future where immersive learning is accessible, engaging, and integral to education at all levels, but it also creates great risks that our students should be guided to grapple with as they become both savvy consumers and ethical stewards of AI.

📬 From Our Partners:
Custom Journal Summaries

Stay on the Cutting Edge of AI Research

Sign up now to get personalized summaries of the most important new AI papers delivered to your inbox daily with GoatStack's free AI Agent.

Our AI reads 4000+ of the latest research papers and handpicks key insights based on your interests. Get custom briefings on topics like NLP, computer vision, robotics, and more without reading every paper.

GoatStack removes the hassle of staying up-to-date so you can focus on advancing your own AI. Start your free trial today.

Let us shape the newsletter for you - just reply with feedback so our AI can learn your preferences.

The future of AI, personalized. Follow us on Twitter @GoatStackAI to engage in discussions about the latest AI innovations and news.

👀 What to Keep an Eye on:
Our “Train Students to Use AI” Webinar

A month ago, we hosted our first AutomatED webinar. It was 2 hours long, covered custom GPTs in depth, and enabled our participants to begin to build their own.

Over the past few weeks, we have been gathering your preferences on topic options for our next webinar. As we reported last week, here are the results:

With these results in hand, we are pleased to announce our next webinar. It will focus on training students to use AI, and it will occur on Zoom on Saturday, April 20th from 12pm to 1pm Eastern Daylight Time.

To sign up, click here:

The price is $69, with a 10% discount for Premium subscribers, included as a discount code immediately below (visible only to those Premium subscribers receiving this via email or logged in on our website).

If you end up being unable to participate at the scheduled time, you will still receive the recording via email.

Given our poll results, we also plan to run webinars in June and August that will focus on:

• (i) using AI for pedagogical purposes while avoiding student AI misuse, and

• (ii) using AI to save time and increase productivity (including how to integrate project management software).