AI Training and Developing Good Judgment

This issue is brought to you by Packback

Welcome to AutomatED: the newsletter on how to teach better with tech.

Each week, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In this week’s piece, I discuss a common misunderstanding about training students to use AI tools, I highlight some amazing recent advances in AI from OpenAI and Google, and I ask for your input into what our next webinar should cover.

📢 Quick Hits:
News Tidbits for Higher Educators

• OpenAI released Sora, “an AI model that can create realistic and imaginative [video] scenes from text instructions.” Here is one. It is very impressive.

  • Why it matters: There are countless implications for many domains, although Sora is inaccessible at the moment to all but a few beta testers. For instance, educators will soon be able to use it and similar tools to create video content from relatively simple text instructions to illustrate concepts, add visual narratives to lessons, or increase student engagement.

• Google announced Gemini 1.5 Pro — a “multimodal mixture-of-experts model” that will likely replace Gemini Advanced’s current Ultra 1.0 model once testing is complete — that demonstrates excellent retrieval ability across ~3 hours of video content, ~22 hours of audio content, or ~7,000,000 words.

  • Why it matters: As retrieval ability goes up, models are more likely to reliably analyze texts and more capable at handling large or many data files. Educators can then rely on them in more contexts to do more advanced tasks, with less caveats about hallucination.

• Google’s previously announced Ultra 1.0 model is being reported by a range of users as being “more creative, better at fiction-writing” and a better writer than ChatGPT4, although we found it still struggles at handwriting recognition.

  • Why it matters: Educators in the humanities and creative arts should conduct comparisons of their own, because it may be that Gemini Advanced (which runs on the model) should be their tool of choice over ChatGPT.

• We at AutomatED created a site map with an organized catalog of all of our old newsletters, with Premium ones marked by this symbol: ✨.

  • Why it matters: You probably subscribed after we sent many of these newsletters, you probably weren’t aware of our old content, and much of our old content is still relevant today. Check it out!

• The US Department of Education’s Office of Ed Tech announced that they are “developing two toolkits that will support educators and edtech developers with AI adoption and procurement decisions, designing AI systems to enhance trust and safety and align with privacy-related laws and regulations in the educational context, and developing education-specific guidelines and guardrails.”

  • Why it matters: These toolkits will help standardize and set best practices for the education ecosystem in the US, and they are likely to be useful in their own right, too.

💡 Idea of the Week:
AI Training Requires Training Good Judgment

Many professors ask me how I train my students to use AI tools like ChatGPT and my custom GPTs. This is an important topic, but some mistakenly assume that this training entirely focuses on how to operate the tools.

The assumption seems to be that, if only a student understands how to turn the knobs and fiddle with the levers that make an AI tool produce an output, they will be able to deploy the AI tools to great effect in the relevant domain. In my case, this is the field of Philosophy.

While your students do need technical skills and need to understand how to operate AI tools relevant to your subject matter, this isn’t all they need.

The problem is that students also need to have good judgment about when the tool is producing an output of the right kind.

It is one thing to reliably be able to produce a bit of text or an image if you know that this is the text or image you should produce. It is another thing to know that you should produce that bit of text or that image in that context.

It isn’t unlike operating tools for carpentry. It is one thing to have technical abilities with tools like the saw or the lathe, and another to be an artist or craftsman who knows how to deploy those technical abilities to produce something valuable or beautiful.

Training your students’ judgment about the quality of AI tool outputs needs to be paired with training them in how to operate the tools.

And training your students’ judgment in this way must occur simultaneous with or prior to their use of AI tools. I find simultaneous works best.

A few weeks ago, we updated our college/university course design GPT so that it can produce better assignment rubrics. Last week, I made some further improvements to improve its overall reliability.

Give it a try, if you have ChatGPT Plus!

It can produce assignments, assignment sequences, rubrics, and AI course policies. We have designed it to be especially effective when it comes to pedagogical issues related to AI.

Remember, you can even get it involved in any other GPT conversation you are having, if you @ it!

⌚ How I am Using AI to Save Time

👀 What to Keep an Eye on:
Our April Webinar (on AI Training?)

This past weekend, we hosted our first AutomatED webinar. This one was 2 hours long, covered custom GPTs in depth, and enabled our participants to begin to build their own. Some of our attendees were interested in GPTs to help with their own pedagogy, some were creating GPTs to assist other educators, and some were interested in ways to ease access to their companies’ content.

We had a great time! Here’s a screenshot of me in action:

We are already planning the next webinar. We are thinking it will be one hour long and occur on a Saturday at noon Eastern Time in April.

If you are interested, please answer the poll below so we have a better sense of which topic(s) we should cover:

📬 From Our Partners:
An AI Writing and Grading Assistant

Packback is an AI writing assistant for every student and an AI grading assistant for every instructor. Packback leverages Instructional AI to improve student motivation, writing skills, confidence, and critical thinking through consistent AI-powered one on one feedback. Our AI acts as an extension of the teacher by coaching and teaching students as they write, allowing them to learn and iterate as they go. Our AI-powered discussion platform, Packback Questions, and our AI-powered writing platform, Packback Deep Dives, have been proven to create more rigorous classroom discussion and improved writing skills, leading to more A’s and B’s. For educators, Packback’s built in Digital TA and customizable assignment rubrics have been proven to reduce grading time by up to 30%.

To learn more about Packback or to schedule a demo, click here.

🔜 What’s Next for ✨Premium Subscribers

Late in the fall of 2023, we started posting Premium pieces every two weeks, consisting of comprehensive Guides, releases of exclusive AI tools like AutomatED-built GPTs, Q&As with the AutomatED team, and in-depth Tutorials of AI use-cases.

Our next Premium piece will be released on February 22nd and is an in-depth Tutorial focused on using Google’s Duet AI for Workspace (soon to be called “Gemini for Google Workspace”) to improve your use of Gmail and Slides.

If your college or university uses Google Workspace, you won’t want to miss it.

So far, we have five Premium pieces:

1. A Tutorial about how professors and others in higher ed can leverage Copilot for Microsoft 365 to answer student questions addressed by their syllabi, catch students up, and create lecture slides.

2. Our Guide for how professors can discourage and prevent AI misuse.

3. Our Guide for how professors should design assignments and assessments in the age of AI.

4. Our Guide for how professors should develop AI policies for their syllabi.

5. A January 24th Q&A with the AutomatED team.

To get access to Premium, you can upgrade for $5/month or $50/year, or get one free month for every two (non-Premium) subscribers that you refer to AutomatED.

To get credit for referring subscribers to AutomatED, you need to click on the button below or copy/paste the included link in an email to them.

(They need to subscribe after clicking your link, or otherwise their subscription won’t count for you. If you cannot see the referral section immediately below, you need to subscribe first and/or log in.)