In the AI Realist workshop, we built a hallucination-detection agent. It was primarily a learning experience to understand how Codex can be used in accordance with agent skills standard and benefit from agentic loops.
For paid subscribers, all the materials and the recording are already in the AI Realist workspace.
What we learned
We covered:
How to build reusable Codex skills
How to combine skills into a specialized agent
When to use deterministic programs instead of model judgment
How to create an agentic audit-and-repair loop
How to use an external validator to provide self-improvement feedback
The finished agent extracts factual claims, checks references and experts, judges whether sources support those claims, searches for missed problems, and produces an inspectable claims ledger and credibility assessment.
Build it yourself or install the finished plugin
If you want to use the plugin immediately, you have two options.
Option 1: Install it from the terminal
Add the marketplace:
codex plugin marketplace add ktoetotam/hallucination-detector-pluginThen install the plugin:
codex plugin add hallucination-detector@ai-realistOption 2: Ask Codex to install it
You do not need to remember the terminal commands. Open Codex and use this prompt:
Add the Codex plugin marketplace from this GitHub repository:
https://github.com/ktoetotam/hallucination-detector-plugin
Then install the hallucination-detector plugin from the ai-realist marketplace. Ask me to approve any required installation actions.Codex can guide you through the installation and request approval before making the necessary changes.
After installation, open a new Codex task. The plugin skills become available inside your normal Codex conversations.
Run the plugin using a prompt
Attach a document or provide a URL, then ask Codex:
Use the run-hallucination-detector skill to audit this document:
[PASTE THE URL OR FILE PATH]Codex may ask for permission to access the web or create output files. Approve those actions so the agent can verify external evidence and save its results.
Test it on a real paper
You can test the agent on this OpenReview paper:
Its authors feature in my investigation into an Oxford student’s alleged use of AI to produce and submit academic work at scale:
How One Oxford Student Used AI to Commit Academic Misconduct at Industrial Scale
To test it, paste this prompt into Codex:
Use the run-hallucination-detector skill to audit:
https://openreview.net/forum?id=LF4RSTZUtAContinue inside the AI Realist workspace
Paid subscribers can watch the full recording, complete every exercise, and build the agent step by step:
Log in to the AI Realist workspace
Not yet a subscriber? Subscribe to AI Realist for practical, anti-hype AI analysis and hands-on workshops that show you how these systems actually work.
Build the agent yourself. Test it on a real case. Then install the finished plugin and compare the results.
AI Realist Workshops
As part of the subscription, we organise regular hands-on workshops. The recordings and materials are available in the AI Realist workspace for self-paced learning.
We also offer deep-dive workshops, such as Optimise Your AI Stack.
This is a three-hour, hands-on workshop designed to help participants understand token economics, reduce their AI costs, and achieve better-quality AI results.
The first workshop, on 30 July, is sold out.
The second session will take place on 27 August, and 14 of the 20 places are still available.
Paid AI Realist subscribers receive a 20% discount!
Claims ledger is the right artifact, inspectable beats a score. Judging whether a source supports a claim is itself the weakest link. Log confidence on that step too.