Today's Agenda
Hello fellow humans! Today,
News
Trust, Reliability, and AI Safety
Can we trust AI? This one question is really two questions; first, what makes us feel psychologically safe when interacting with a system, and second, what are the technological mechanisms that make that sense of “trust” possible?
Dr. Verena Seibert-Giller writes in UX Magazine that “trust” is the wrong way to think about AI because it makes AI too human. Because AI is a technology, we should view AI through the lens of reliability, like we think of cars as being reliable. When we think of what it means to be reliable, controllable, predictable, and transparent.
Whether we’re building AI tools like agents or using AI tools, she writes that there are five key elements that we should look for. She writes:
From a psychological standpoint, here are the key building blocks that make people more willing to rely on AI systems:
Predictability: humans dislike uncertainty. If an AI produces different results for the same input, users feel insecure. Clear boundaries of what the system can and cannot do help users calibrate reliance.
Explainability: people don’t demand a PhD-level technical explanation. But they do need a clear, user-centered rationale: “We recommend this route because it’s the fastest and has fewer traffic jams.” Simple explanations anchor trust.
Error Management: paradoxically, users may rely more on a system that admits errors than one that pretends to be flawless. If an AI says, “I’m 70% confident in this answer,” it gives the user space to judge whether to accept or double-check.
Controllability and Agency: a sense of control is essential. Users should always feel they can override the system, pause it, or give feedback. Without agency, reliance quickly turns into mistrust.
Consistency with Values: especially in sensitive domains (healthcare, hiring, finance), people want assurance that AI aligns with ethical and social norms. Clear communication of safeguards reduces fear.
If these elements are the psychological foundation of trust in AI, what is the technical foundation?
For each of the elements that Dr. Seibert-Giller discusses, we need corresponding technological solutions for both the intended AI actions and the unintended results of malicious attacks.
As of today, AI has serious problems with all five of these dimensions of reliability. LLMs are notoriously unpredictable, though we can control this to a certain degree with context and prompt engineering. Models have introduced Chain of Thought to bring some transparency to explain their thinking, but language models do still mostly work in a black box where we do not know how they arrive at their responses. We also know that language models can provide wrong information with strong confidence and will even push back when you tell them they’re wrong. And there is nothing that makes you feel quite as out of control as when you know what you want and the language model just seems unable to give it to you; the response is too long, it gets the context wrong, or it generates code that just fails. With all of this, we’re very reluctant to use language models in high-stakes domains like finance or healthcare.
While accounting for all of the AI system’s planned activity, we also need security frameworks to protect against unplanned activity from third parties, especially when we enter the world of agentic AI systems. According to Boston Consulting Group, “While traditional software systems operate within predictable parameters, natural language processing thrives on dynamic user interactions, interpreting words, phrases, and emotions to generate real-time outputs. Malicious actors have learned to manipulate this adaptability through such techniques as prompt injection, context poisoning, and API exploitation, causing not just theoretical vulnerabilities but concrete, demonstrable risks.”
When organizations are developing agentic systems, it’s easy to overlook that agents are software products with security vulnerabilities, as these agents pass data from one application or storage location to another. If you’re looking at agentic AI systems in your organization, you need to perform a security audit and analysis so that you can develop a comprehensive security strategy. But Boston Consulting Group does identify three main areas to address:
Input validation - ensure that only the intended inputs are delivered to the AI system
API security - ensure strong authentication and input sanitization
Real-time monitoring - ensure that if your agent is targeted, suspicious behaviors and patterns can be highlighted immediately.
As we build and experiment with AI systems, we need to be thoughtful about the challenges of ensuring AI reliability so that we can establish proper governance frameworks and manage the risks of using AI and agentic AI systems. Getting clear about what humans value will help us create systems that are not only technically sound but also ethically aligned and transparent in their decision-making processes.
Adapting Skills for the AI-Integrated Workplace
Emily Mabie, an AI Automation Engineer at Zapier, writes for Fast Company her experience of how AI is reshaping job markets and requiring new skill sets. She describes work that sounds like a product manager doing customer discovery, “I embed with a team (HR, in my case), spot opportunities to enhance the team’s work, and build AI-powered workflows that jump on those opportunities. The goal is to create measurable improvements that free my teammates up for creativity, strategy, and connection.”
She goes on to describe what she does day-to-day, and these also sound like product management, including embedding with the customer team to deeply understand the jobs to be done, triaging which workflows to automate, and prototyping and testing those workflows.
This story suggests that the AI revolution will be less about job displacement and more about job transformation, where workers need to develop AI literacy alongside traditional domain expertise. The emphasis is on adaptability, continuous learning, and developing meta-skills that complement AI capabilities.
Kids Can Thrive in the Age of AI
Information is cheap and widely available, so educators have to ask: what should be our core educational values? Michael Mannino Ph.D. writes for Psychology Today that we should focus on mindset, not memorization as the skills students need most.
He identifies three main areas:
Meta-skills like adaptability, resilience, and focus are now the core curriculum of learning.
Generative AI amplifies attention theft, making focus and cognitive control survival skills.
True education builds minds that self-regulate, reframe stress, and thrive amid uncertainty.
He then breaks these down into 11 meta-skills for the AI world including:
Adaptability
Discomfort mastery
Cognitive reframing
Critical thinking
Self-determination
These (and others that he describes) are all skills that are not only impossible for AI to replicate, but are necessary for a human to use AI effectively. In schools, universities, or workplaces, we need to bolster our own cognitive game to thrive in an AI-integrated world.
Radical Candor
I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.
It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.