How AI Works, Why It's Weird, and Why It's Hilarious
TL;DR:
AI is hilariously limited: it learns patterns, not meaning, so it finds loopholes, optimizes metrics instead of goals, and fails spectacularly outside its training—keep humans in the loop and audit for bias.
Author: Janelle Shane (AI researcher, aiweirdness.com) • Published: 2019
Core Thesis
Five principles — 2026 reading note
Shane’s principles demystify AI: narrow goals, weird failures, no common sense — still the antidote to LLM hype.
AI is NOT Too Intelligent
Example: AI generates "Anus" as a cat name—learned letter patterns, not meaning
Highly Specialized, Not Adaptive
Example: AI recognizes cats perfectly in photos—but can't recognize cats in trees (different context)
Optimizes Metric, Not Real Goal
Example: Tumor detector learns to recognize rulers (in training photos) instead of tumors
Finds Loopholes You Didn't Expect
Example: Robot told to "run fast" learns to do backflips (technically moving quickly)
Chooses Easiest Solution
Example: AI learns "green field = sheep" instead of recognizing actual sheep features
Why These Matter
Examples:
Lesson: AI learns letter patterns, not meaning
Examples:
Lesson: AI mimics structure without understanding physics
Examples:
Lesson: AI learns joke structure but not humor
Examples:
Lesson: AI mimics romantic language without understanding romance
Real-world failures — 2026 reading note
Production mistakes are often boring data bugs, not sci-fi — but generative fluency hides them better now.
Did not brake for a white trailer crossing the road in bright light (2016 Florida crash)
Cause:
Autopilot was not designed to handle crossing traffic (NTSB investigation)
Consequence:
Fatal accident
Lesson:
AI fails outside the conditions it was designed for
Discriminated against women
Cause:
Trained on male-dominated historical hiring data
Consequence:
Tool scrapped after bias against women was found (Reuters, 2018)
Lesson:
Historical bias in data → biased AI
Self-fulfilling prophecy
Cause:
Predicted high-crime areas based on policing patterns
Consequence:
More policing → more arrests → "confirms" prediction
Lesson:
Measurement bias creates feedback loops
Bias — 2026 reading note
Bias enters via data, feedback loops, and product incentives — unchanged by better chat interfaces.
Training data reflects real-world injustice
Example: Amazon HR trained on male-dominated workforce
Underrepresented groups perform worse
Example: Face-analysis systems erred far more on darker-skinned women; benchmark datasets were about 80-86% lighter-skinned (Gender Shades, 2018)
What you measure ≠ what you think
Example: Prison prediction measures policing patterns, not crime
Works well on average, fails for subgroups
Example: Medical AI trained on men fails for women
Human–AI partnership — 2026 reading note
Partnership means humans own goals and consequences; AI supplies drafts and variations.
AI doesn't understand what you actually want
AI can't judge if data is representative or biased
AI doesn't know if outputs make sense
AI can't recognize its own blind spots
Dangerous Pattern
Safe Pattern
Key takeaways — 2026 reading note
Healthy skepticism beats both hype and cynicism — test on your data.
Shane's principles are brilliant for understanding why AI fails. We take them one step further: how do you build workflows that catch those failures before they reach production?
Shane's core idea — that AI systems are relentless optimizers of their training objective, not intuitive thinkers — is even more visible with today's capable large language models. They are impressive pattern machines that can mimic reasoning, humor, and style, but they still have no underlying world model in the human sense. The weird edge cases are not exceptions; they are the default whenever your instructions fall outside the training distribution.
We particularly like Shane's insistence that "AI does not really understand what you mean, only what you say." In 2026, this shows up in the way models confidently hallucinate citations, fabricate API responses, or invent non-existent regulations if you prompt them carelessly. We translate this into concrete habits: always specify constraints, ask for step-by-step reasoning, and verify any output with legal, financial, or safety impact. For practical prompt design techniques, see our Prompt Engineering course.
Where we go beyond the book is in how we recommend users instrument and test AI behavior. Shane illustrates weirdness with funny anecdotes; we treat those anecdotes as a starting point for reliable workflows:
Design Principle
Weirdness in 2026 — 2026 reading note
Modern LLMs feel human — that makes Shane’s “weird narrow AI” message more important, not less.
Shane's book used toy neural networks for examples. Today's models are far more capable — but the same failure patterns persist at a higher level of sophistication. Here are three 2026-class weirdness patterns we see regularly, along with practical fixes.
Shane's Principle #3 (Doesn't Understand the Problem) at scale
Ask a model for "five recent AI safety regulations in Europe" in one short step, and it may mix real laws with invented ones — including plausible-sounding acronyms and dates. The system is not trying to deceive you; it is simply optimizing for "answer-shaped text" rather than factual accuracy. This is Shane's Principle #3 at industrial scale.
Practical Fix
Shane's Principle #4 (Follows Instructions Literally) meets real APIs
Give an autonomous agent access to a search API, a calendar API, and an email API, and you may find it scheduling meetings with itself, emailing incomplete drafts, or getting stuck in loops calling the same failing endpoint. None of this looks like the sleek "AI assistant" demos in marketing videos — but it is exactly what you should expect when you give an optimizer tools without enough guardrails.
Practical Fix
Shane's Principle #5 (Path of Least Resistance) in content generation
AI systems generating content that is technically fine but culturally off: cheerful marketing copy for a serious medical topic, imagery that accidentally encodes biases in race, gender, or age. These failures are often subtle and will not trigger obvious red flags in automated evaluation.
This is where human critical thinking — and exactly the kind of "look at what the AI actually did, not what you hoped it would do" mindset from Shane's work — becomes non-negotiable. In our Academy, we deliberately include such "almost right but wrong in important ways" examples in exercises. For a structured framework, see AI Critical Thinking.
Practical Fix
Weirdness is timeless
See the 5 principles in action
Aged remarkably well as an antidote to hype: Shane’s “AI is narrow and weird” message is more relevant now that fluent LLMs mask failure modes.
Strengths
Limitations
Discussion questions
Next on Academy: AI Critical Thinking
AI is hilariously limited because it learns patterns, not meaning: it optimizes metrics instead of goals, finds loopholes in instructions, and fails spectacularly outside training data—understanding these limitations helps you use AI tools more effectively.
Janelle Shane's five principles reveal AI's weirdness: AI has no common sense, finds unexpected shortcuts, optimizes the wrong thing, lacks understanding of context, and requires careful design to avoid bias—keep humans in the loop and audit for these issues.
Use AI tools wisely by recognizing their limitations: test edge cases, verify outputs, understand training data biases, design prompts to avoid loopholes, and maintain human oversight—treat AI as a powerful but flawed assistant that needs careful management.
Copyright & Legal Notice
© 2026 Best-AI.org. All rights reserved.
All content on this page has been created and authored by Best-AI.org. This content represents original works and summaries produced by our editorial team.
The materials presented here are educational in nature and are intended to provide accurate, helpful information about artificial intelligence concepts. While we strive for accuracy, this content is provided "as is" for educational purposes only.
Complete this quiz to test your understanding of Shane's 5 principles of AI weirdness and limitations.
Loading quiz...
Know these before you go further.
Narrative books persuade with stories; they are not peer-reviewed capability benchmarks.
What this means for you
Extract frameworks, then validate claims against primary sources and your own pilots.
Books lag model releases by 12–24 months — tactics may predate agents, RAG, and new regulations.
What this means for you
Pair each book with a current Academy guide (security, law, or agents) before operational decisions.
Success stories hide failed deployments, reverted pilots, and hidden labor costs.
What this means for you
Ask “what would falsify this claim?” and run a small experiment before scaling.
A single worldview can anchor team strategy without diversity of risk perspectives.
What this means for you
Read contrasting book-club titles (optimist vs. critic) and document disagreements explicitly.
