✨Tutorial: How to Build (Better) Custom GPTs

Welcome to AutomatED: the newsletter on how to teach better with tech.

Each week, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In this fortnight’s Premium edition, I present a comprehensive 7000-word Tutorial on how to build custom GPTs to maximize their performance and prevent misuse.

With custom GPTs now free to use — your students needn’t pay anything to converse with them — this is the time to learn how to build them and build them better so you can go into the fall semester with the ultimate tech tool for personalization in your toolkit.

This Tutorial covers everything you need to know. And it includes tons of real examples of GPTs and GPTs’ instructions, including some of the secret sauce behind our popular Course Design Wizard (which has 4.4⭐s on the GPT Store with 600+ conversations, at least when this is being written).

❓ Step 1: What are Custom GPTs?

The Basics

To begin, it is essential to grasp the basic concept and functionality of custom GPTs.

Custom GPTs are a product from OpenAI built on the same underlying technology as generic large language models (LLMs), like ChatGPT, Gemini, or Claude, but are enhanced with specific “instructions” and additional files in their “knowledge bases” to tailor their responses to particular tasks or contexts.

In essence, a custom GPT’s instructions act as a meta prompt (a prompt before the prompts) that is appended prior to any user’s engagement with the GPT, thereby acting as guardrails on its behavior. Its knowledge base is analyzed and referenced — that is, parts of it are appended to a user’s prompt — when its content is relevant to that prompt.

Examples

Here are some innovative ways professors can leverage custom GPTs to enhance personalization (for more examples, see here):

• Interactive Role-Playing Scenarios: In subjects like business, law, and healthcare, professors can develop GPTs that facilitate role-playing exercises. For example, students can engage in simulated business negotiations, legal proceedings, or patient consultations. Students can receive real-time feedback from the GPT itself or from professors (or their peers) as they interact with the GPT as it stays in character.

• Virtual Office Hours and Tutoring: Professors can create GPTs that simulate virtual office hours, where students can ask questions and receive detailed explanations of course content outside of regular class hours or professor-hosted office hours. These GPTs can be programmed to provide personalized responses based on individual student queries and past interactions.

• Language Practice with Real-Time Feedback: Language instructors can create GPTs that engage students in conversation practice, providing instant feedback on grammar, vocabulary, and pronunciation. These GPTs can simulate native speakers from different regions, enhancing the cultural immersion aspect of language learning. (GPT-4o is a lot better at non-English languages than GPT-4.)

• Historical Figure Simulations: History or political science professors can create GPTs to simulate interactions with historical figures during critical moments. For instance, students could converse with Abraham Lincoln during the Civil War or participate in a simulated UN Security Council meeting during the Cuban Missile Crisis.

• Adaptive Study Guides: Custom GPTs can generate personalized study guides based on each student’s performance and learning style. By analyzing past performances or querying the student about their needs, the GPT can recommend specific topics for review and suggest additional resources, ensuring that each student receives the support they need.

• Research Proposal Feedback: Custom GPTs can assist students in developing research proposals by offering suggestions, critiquing drafts, and providing relevant literature references. This helps students refine their research questions and methodologies, improving the quality of their proposals.

And here are some examples of how custom GPTs can be used to improve professorial productivity (with knock-on effects for helping students better achieve learning outcomes):

• Lesson Planning: Professors can use custom GPTs to plan lessons more efficiently. These GPTs can suggest relevant materials, structure lesson plans, and generate interactive activities tailored to the specific needs of the course. Our very own Course Design Wizard — designed by me — is a prime example of how custom GPTs can support educators in creating assignments that either incorporate or exclude AI use:

• Feedback Generation: Analyzing student work and providing detailed feedback can be time-consuming. Custom GPTs can streamline the process, whether they help the professor produce and package their feedback without access to student work — our Feedback Accelerator, available to only ✨Premium subscribers, is designed for this purpose — or they help evaluate assignments, identify key areas for improvement, and generate feedback for each student. This not only saves time but also ensures that students receive timely and constructive feedback. Moreover, custom GPTs can be used to standardize feedback and grading, like for a team of graduate student teaching assistants.

• Data Analysis and Reporting: Professors can develop GPTs to analyze data from assessments, surveys, and other sources to generate reports and insights. This can help in identifying trends, measuring student progress, and making data-driven decisions to improve the educational experience. (In fact, with the end of the spring semester, I will be covering this topic in a free newsletter in the next few weeks.)

Contrasting Custom GPTs With Other AI Tools

When compared to other AI tools, custom GPTs offer distinct advantages.

Generic LLMs like ChatGPT offer broad conversational capabilities but fall short in specific educational applications. ChatGPT can engage in one-off conversations and handle various topics, but it lacks the ability to retain context over extended interactions and cannot be easily shared or customized for educational purposes. Custom GPTs address these limitations by allowing educators to embed detailed instructions and integrate specific files, enabling the AI to provide consistent and contextually accurate responses. Additionally, custom GPTs can be shared with students or colleagues via a simple hyperlink, making them a versatile and accessible tool for enhancing the educational experience.

Institutional LLMs like Microsoft Copilot and Google Gemini present another point of comparison. These models benefit from greater data security, as students and professors can stay within their existing ecosystems, minimizing the risk associated with external data transmission. However, both Microsoft Copilot and Gemini lack the robust customization options of custom GPTs, limiting their utility in highly specialized educational contexts. They are just like vanilla ChatGPT in this regard.

Some LLMs offer APIs, such as OpenAI’s Assistant API, that offer extensive functionality similar to custom GPTs. In short, these APIs allow more direct access to the LLM. Yet, implementing APIs typically requires a higher level of technical expertise, as well as the need to develop and maintain a website or other user interface for interaction. This complexity makes APIs less accessible to most educators compared to the straightforward setup and deployment of custom GPTs, which can be easily shared and utilized without extensive technical overhead. Custom GPTs effectively bridge the gap by providing the advanced capabilities of APIs in a user-friendly format, empowering educators to harness AI technology without the need for specialized technical skills.

While Google’s NotebookLM is designed for summarizing and interacting with document uploads, particularly notes, but it lacks the extensive customization and sharing capabilities of custom GPTs. In essence, it is a curtailed or limited version of a custom GPT.

Finally, Microsoft’s Copilots (distinct from Microsoft Copilot), while integrated seamlessly with other Microsoft 365 applications, are not yet as effective as custom GPTs due to their relatively nascent stage of development. Furthermore, most institutions are not paying for access to them, even if they are a Microsoft institution (and many institutions use Google Workspace).

Why Would I Use a Custom GPT?

As the prior section makes clear, custom GPTs are superior to the alternatives in most educational contexts for the following reasons:

• They offer tailored solutions to educational challenges, enhancing the teaching and learning experience in ways that general AI tools cannot match.

• They can increase educators’ productivity in a range of ways.

• They are shareable, unlike many other tools, so you can simply link them to your students after you create them.

• They are available regardless of your institution’s technology setup — and now they are free to use (although you will need to purchase ChatGPT Plus at $20/month yourself to create them).

• Finally, they are technologically straightforward to create and modify — no coding skills are needed — although there are some tricks and best practices that I will now cover…

💭 Step 2: Define Your Use Case

Defining a clear and precise use case is a critical step in developing an effective custom GPT. This process involves understanding the specific challenges you aim to solve, identifying your target audience, and establishing the parameters for success. By meticulously outlining these elements before you begin, you can ensure that your custom GPT is purpose-built to meet your needs.

Identify the Problem

The first step in defining your use case is to pinpoint the exact problem you want your custom GPT to solve. Perhaps you want to streamline administrative tasks, enhance student engagement with difficult course content, or provide personalized feedback on assignments. Clearly articulating the problem helps focus the development process and ensures that your GPT addresses a real and relevant need.

Generally, it is important to not try to “do too much” with a given custom GPT. You should pick a problem that is relatively well-defined. If it turns out that you have a very complex problem to solve with many components (or multiple problems), you should just develop multiple GPTs to address each of them.

Specify the Target Audience

Understanding who will use your custom GPT — and their specific needs — is crucial. Your target audience could be students, fellow educators, or administrative staff. Identifying their expectations and how they will interact with the GPT will guide the design and functionality of your model.

If your audience is students, consider their learning methods, the typical questions they ask, and the type of support they need. Your development of your GPT should be sensitive to ways that they will interact with it. I have found that students query LLMs in ways that are consistently surprising to me, so I need to adjust the custom GPT to better serve someone who prompts in those ways. Since some of these adjustments are hard to predict, you should plan to gather feedback and iterate as needed, as I will discuss at the end of this Tutorial.

Define Success

Establishing clear criteria for what success looks like is essential for evaluating the effectiveness of your custom GPT. Determine the kind of outputs you are looking for and how you will measure success. Success metrics might include improved student performance, increased engagement, or time saved on administrative tasks.

Describe Inputs and Exemplar Outputs

Specify the types of inputs your GPT will require from users. These could be text-based queries, document uploads, or specific data points. Providing exemplars — examples of expected inputs and corresponding outputs — can help guide the development process and ensure that your GPT performs as intended. This is an example of few-shot prompting.

Detail the Process

Outline the steps your GPT should follow to transform inputs into outputs. Think of the GPT as a human assistant: what step-by-step instructions would you give to ensure they complete the task accurately? This might include specific algorithms for processing data, methods for generating responses, or procedures for handling complex queries. Ensuring that these steps are clearly defined will help in developing a robust and reliable GPT.

Consider Security and Privacy

Address any security or privacy concerns associated with your custom GPT, particularly if it will handle sensitive data. Ensure that your GPT complies with relevant data protection regulations, such as FERPA in the U.S., and consider how to safeguard user information.

📄 Step 3: Craft Instructions for Your GPT

Effective instructions for your custom GPT are essential to ensure it performs well and reliably. These 8000-character-maximum instructions guide the GPT in understanding its role, processing inputs, and generating appropriate outputs, all in accordance with your plan from Step 2.

Here, I will detail the key components of GPT instructions and how to structure them effectively, with lots of examples from the instructions powering our Course Design Wizard and our ✨Premium-only Feedback Accelerator.

But first, we need to create the custom GPT in order to add the instructions to it.

Create Custom GPT

Start by getting ChatGPT Plus, if you don’t have it already. It costs $20/month and comes with a range of nice features, including the ability to create GPTs.

Once you have ChatGPT Plus, navigate to https://chatgpt.com/gpts. In the upper right corner, you’ll see two buttons, one for your preexisting GPTs and one to create another:

Click the “+Create” button. This will enable you to create a custom GPT.

Next, in the left side panel, click “Configure.” You could use the “Create” button to chat with the GPT Builder, but we are going to cut to the chase and directly input the instructions and then the knowledge base files.

Feel free to name the custom GPT, add a short description, and upload an avatar image.

Now let’s get down to crafting your instructions.

Role/Purpose Description

The role or purpose description is the first part of the instructions and it defines what the GPT is designed to do and sets the context for its responses. This component should be concise yet comprehensive, providing the GPT with a clear understanding of its overall function.

Here is the role/purpose description from our Course Design Wizard:

In this example, the role description clearly outlines the GPT’s task in broad strokes and emphasizes its expertise in providing pedagogical feedback.

Step-by-Step Instructions

Step-by-step instructions provide a detailed sequence of actions the GPT should follow when processing inputs. These steps ensure consistency and accuracy in the GPT's performance by breaking it down into manageable chunks.

This is an instance of what researchers call “chain of thought” (CoT) prompting. CoT “improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking”, as the authors of a seminal paper on the technique report.

There are a range of ways to implement CoT prompting in your GPT’s instructions. I will discuss another way below in the Exemplars section, but the primary way that I recommend you implement CoT prompting is via an explicit outlines of each step in a sequence of steps that the GPT would complete if it were successful. You can design the sequence so that (i) there are no user interactions in between the steps (that is, the GPT completes the sequence in one single action before responding to the user) or (ii) the GPT expects user prompts interspersed between each step.

The step-by-step instructions for our Feedback Accelerator are an example of (i). Here they are:

These steps ensure the GPT processes the input systematically, transforming a professor’s notes on a student’s work into well-structured feedback. You can click here to see this GPT in action:

By contrast, the step-by-step instructions for our Course Design Wizard are an example of (ii) because the GPT expects user prompts interspersed between each step. Here is a segment of them so you can see what I mean:

Now, you may also find that it helps to preface this sort of description (of all the steps in the step-by-step sequence) with guidance on how to proceed through the steps, even if you provide such guidance locally to each step via commands about how to proceed after each step. (Whether this is needed will depend on your implementation — try it both ways!)

For example, with our Course Design Wizard, I preface the whole step-by-step instructions with the following message:

In this case, this is crucial to get the GPT to complete the step-by-step process before offering its pedagogical guidance/suggestions — without aggressive prompting like this, I found that it would just spit out guidance/suggestions immediately. In fact, it will still do this sort of undesired behavior sometimes, partially because we have 8 steps and partially because user prompts are interspersed between each step, which increases the chance it loses the thread on its overall task.

Another way that I try to combat the GPT losing the thread is by having it insert a bit of text that I have labeled SUMMARY in some of its responses (see Steps 4 and 5 above) that is a short description of the step-by-step sequence that effectively reminds it of the sequence and the need to complete it before offering pedagogical advice.

(You may notice that I promise to tip the GPT if it complies. For some reason, this improves the likelihood that it complies, as many other GPT builders have reported. Strange but true!)

Specification of Terminology and Variables

This component defines any specific terminology and variables used in the instructions, ensuring the GPT interprets them correctly.

You will have noticed that the step-by-step instructions for our Feedback Accelerator contain bits like “<perform [INTRODUCTION]>”. You can use specific terms like ‘INTRODUCTION’ to reliably reference specific responses, and you can use markings like ‘<__>’ and ‘[__]’ to flag specific parts of your instructions and how you want the GPT to parse them.

Here is the terminology and variables section for our Feedback Accelerator:

Defining these elements helps the GPT understand the structure and meaning of different parts of the instructions, preventing confusion and errors.

Exemplars

Providing examples of inputs and corresponding outputs helps the GPT understand what to expect and how to respond to it. These exemplars serve as concrete references for the GPT to emulate — effectively modeling the behavior you want to see from the GPT.

One way is to encourage your GPT to reason step-by-step and use CoT prompting is in how you represent the exemplars you provide it. Here is an example from the seminal paper on CoT prompting mentioned before:

In this exemplar, the LLM is encouraged to answer by “showing its work” along the way — the step-by-step process by which it might determine the answer to the problem — and this in turn increases the likelihood that it successfully finds the answer.

This isn’t necessary, though. With our Feedback Accelerator, I provide exemplars after each step in the process, and the exemplars pair possible user prompts with desirable responses:

I also provide exemplars after each step in the process with our Course Design Wizard, but I do not pair prompts with responses. The reason is that the step-by-step sequence for this GPT directs it to ask questions at the very end. So, the initial steps (the question-asking steps) are always roughly the same, regardless of the user’s prompts, and the final step (where the GPT provides pedagogical advice) is too lengthy to add to the instructions. To provide an exemplar, I would need to include tons of information and output a whole course syllabus policy or a complex assignment.

Here’s what I do instead (in my prior quotations, I had excised the examples for simplicity):

In all cases, your exemplars illustrate how the GPT should process inputs and generate appropriate outputs, providing a clear model for its behavior.

Conversational Style

If desired, you can add a conversational style component to specify the tone and manner in which the GPT should respond. This ensures consistency in the GPT’s interactions and aligns with the intended user experience.

Here is how I instruct our Feedback Accelerator GPT to express its feedback:

I will note, however, that I have gone overboard in this dimension. With the GPTs I built as course tutors for my Philosophy classes, I initially instructed them to have a very encouraging and supportive conversational style. I and my students found this to be ultimately counterproductive because the GPTs would encourage them in misleading ways, even when they were completely offbase! You should try various conversational styles and test them to ensure they don’t result in unintended consequences.

Defenses Against Misuse

Including defenses against misuse prevents the GPT from being used inappropriately. This component outlines rules and safeguards to protect the GPT’s integrity.

Here’s an example from one of my Philosophy course tutor GPTs that I used this past semester.

As I put it in the role/purpose description for this GPT, it “is designed to be a teacher's assistant who guides students in a philosophy of artificial intelligence class.” It assists “students in forming logically valid arguments in premise-conclusion form, a method of argument representation/expression explained below with examples.”

Since it is a teacher’s assistant, it should not do the argument analysis or creation task for the students; rather, it should teach them how to do it for themselves. So, to provide some guardrails and limit the GPT to this stance, I included the following instructions:

Notice that the instructions here are positive and layered: I tell the GPT what to do if it receives a request that is “out of bounds” and I give it multiple ways to address such requests, depending on what else it has tried. (In general, LLMs respond better to positive instructions than negative instructions — they tend to have trouble parsing negations, as OpenAI explicitly notes in this context.)

Formatting Methods

Effective formatting methods enhance the performance of your GPT by making instructions clear and organized. This includes using symbols to structure the information hierarchically. While humans are capable of parsing a very wide range of text and document structures, LLMs benefit a lot from the structure being made more explicit with symbols.

I prefer to use basic markdown symbols and asterisks to flag different parts of the structure of my instructions (and my knowledge base files; more on this below). So, for instance, I use the ‘#’ symbol to flag top-level headers, ‘##’ to flag second-level headers, etc., and I use ‘*’ to signal exemplars or examples, like this (note: the intermediate content has been removed to reveal the structure; see the preceding for some of this content):

Try different ways of structuring your formatting so that the GPT can better parse the instructions.

🗄️ Step 4: Construct Your Knowledge Base

In many cases, adding files that act as a knowledge base to your custom GPT is crucial for improving its performance and making it more effective in providing detailed, contextually appropriate responses. Here, after giving some background on how your GPT accesses your knowledge base files, I detail the process of selecting, formatting, and integrating knowledge into your GPT.

Benefits and Challenges

Adding a knowledge base offers significant benefits but also presents certain challenges. Understanding these will help you make informed decisions about the data you integrate into your GPT.

Benefits:

1. Improved Accuracy: A well-constructed knowledge base helps your GPT provide more accurate responses by referencing specific information.

2. Enhanced Relevance: Tailoring the knowledge base to your subject matter or use case ensures that the GPT delivers relevant answers.

3. Increased Diversity of Outputs: A rich knowledge base allows the GPT to draw from a wider array of information, improving the diversity and depth of its responses.

Challenges:

1. Data Quality: Ensuring that the information is accurate, up-to-date, and relevant is crucial. Poor-quality data can lead to incorrect or misleading outputs.

2. Size Management: Larger knowledge bases can slow down the processing time and make it difficult for the GPT to retrieve specific information efficiently.

3. Access Issues: Ensuring that the GPT can reliably access and parse the information in your knowledge base is essential for consistent performance.

To fully optimize your custom GPT, it's essential to understand how it accesses its knowledge base. While we don’t have a lot of specifics on how OpenAI has set up GPTs’ knowledge base access, we know that GPTs rely on “Retrieval-Augmented Generation” (RAG). RAG is a technique that combines the retrieval of documents or data with the generative abilities of a language model to provide detailed and contextually appropriate responses. Here’s a brief overview of how RAG functions:

1. Prompt Processing: When a user submits a prompt, the GPT processes the input to understand the context and extract relevant keywords or phrases.

2. Retrieval Phase: The GPT uses these keywords or phrases to search its knowledge base, which has already been analyzed with respect to its own contents, and retrieve the most relevant files or snippets of them.

3. Augmentation Phase: The retrieved information is then used to augment the original prompt that the user submitted. The model combines the prompt, with some modifications as needed (this part is out of the user’s control), with the retrieved files or file snippets to generate a more accurate and contextually relevant response.

4. Response Generation: The GPT synthesizes the information from the retrieval phase with its internal model to produce a coherent and detailed answer that addresses the user’s query.

By integrating RAG, custom GPTs can significantly enhance their responses, particularly in specialized or technical fields where precise information retrieval is crucial.

However, if your use case doesn’t need this sort of supplementation or enhancement, it might be better to leave the knowledge base empty, as it can reduce the performance of your GPT if it contains irrelevant information or information that is already embodied by the GPT’s model.

Selecting Appropriate Files

When you set up your custom GPT, start by identifying sources of information that are directly relevant to the tasks your GPT will perform. These could include textbooks, academic papers, lecture notes, and other educational materials.

Err on the side of including less for the aforementioned reasons; there is some risk that if you flood the knowledge base of your GPT, it won’t have a good grip on how to relate all of this information to user’s queries. Quality over quantity.

Formatting Files for GPT Integration

Ensure that the formatting of your files is GPT-friendly. Avoid complex formatting like tables or multiple columns, as well as image-heavy pdfs, which can be difficult for the GPT to parse. Instead, use plain text formats such as .txt, .md, or .docx, formatted in ways that conform with the above advice about formatting your instructions.

Likewise, use clear headings and subheadings to indicate different sections of the content, and incorporate keywords or phrases that can act as triggers for the GPT to access specific parts of the knowledge base. This helps the GPT navigate the information and retrieve relevant data effectively via RAG.

Finally, although this is progressively becoming less important as RAG improves, note that order matters. Internal to each file, prioritize the most important information at the start. And, across all the files, upload the most important files first. You can conceptualize this as organizing the information in a logical sequence that mimics human thinking — if you were telling a human assistant what to remember, you would start with the most important parts.

Integrating the Knowledge Base

There are two aspects to integrating the knowledge base in your GPT.

The first is logistical. You can upload the files via the “Upload files” button below the “Instructions” box. You can enable web browsing, image generation, or Code Interpreter as needed. Code Interpreter allows your GPT to “run Python code in a sandboxed execution environment. This tool can process files with diverse data and formatting, and generate files with data and images of graphs.” However, note that enabling Code Interpreter makes it so that your knowledge base files can be downloaded by savvy users.

Now, once you have your files uploaded to your GPT, the next aspect is to ensure that your GPT references them and integrates them in its responses.

In general, you should instruct the GPT on how to use the knowledge base by adding specific guidelines within its instructions that reference specific file names and specific parts of the files. This ensures the GPT knows when and how to reference the added knowledge, increasing the likelihood that the RAG works well.

In the case of our Course Design Wizard, its knowledge base includes several of our ✨Premium Guides, like our Guide to Designing Assignments and Assessments in the Age of AI and our Guide to Discouraging and Preventing AI Misuse. I converted both to plain text, removing images and other extras, and I also inserted headers and other tags to more clearly label their parts. I have tried more generic directives to reference these Guides, like the following, and they have worked great with some prompts but not others:

In general, the effectiveness of such a generic directive depends on whether the RAG works well and the GPT can locate the relevant bits of the files to match the user’s prompts.

I have also tried more specific directives and I find that they generally work better when relevant. However, they require significant experimentation to hone and they also take up a significant amount of your instruction character limit.

Note that you can try to encourage the GPT to take its time and check its work in reviewing each file for relevance — some GPT builders report that this sort of “thoroughness” prompt technique has good effects.

Likewise, you can create exemplars that model how you want the GPT to retrieve knowledge. If you have space in your instructions, I would recommend you provide one exemplar to model how you want your GPT to draw on your knowledge base, using a paradigmatic case of a prompt that has terms that are similar to the content of a properly labeled part of one of your files.

🧪 Step 5: Experimentation and Adjustment

After setting up your knowledge base and integrating it into your GPT’s instructions, you need to publish your GPT.

Press the “Update” button in the upper right corner of your screen (the “Share” button enables you to change who can access the custom GPT):

Then you will be presented with an option to copy the link to the GPT or view it. I would recommend copying the link, opening a new browser tab, and pasting it there.

I would recommend experimenting with it a lot, making adjustments as needed. Try random prompts to see if you can “break” it, but also create a few consistent prompt sequences that you can benchmark it with. The latter will enable you to see how it changes in responding to a fixed set of prompts as you change its instructions and knowledge base.

After you are satisfied, it is crucial to continuously monitor its performance. Regularly review the outputs to ensure they meet your expectations and make adjustments as necessary. Keep track of what it is doing well and what it is doing poorly.

If, for instance, the GPT is failing to reference the knowledge files correctly, copy your instructions to a file to save them and then begin making adjustments to the knowledge files and/or your instructions to see if you can get better performance without diminishing the GPT’s strengths.

Likewise, collect feedback from users to understand how well the GPT is performing from their perspective, especially since people vary in how they prompt.

I have found that my students reveal vulnerabilities and weaknesses in my GPTs very quickly because they prompt in a wide range of ways (and I have many students). I gather feedback from them early in the semester — and throughout — to make fixes and address common issues.

Good luck!