KCDC 2026

Deceptive
UX Patterns

Exposing Manipulative Design in the Age of AI

Vitaliy Matiyash · Staff Engineer · Columbus, OH

Raise your hand if...

💳

You've accidentally started a subscription you didn't want?

🔍

You've spent >5 mins looking for an "Unsubscribe" link?

❌

You've struggled to find the "Close" button on an ad?

🤖

You've apologized to ChatGPT or said "please" to an AI?

[Read each card as it appears — pause for hands] "Raise your hand if you've accidentally started a subscription you didn't want." [Look around — nod] "Yeah, most of us." "Spent more than 5 minutes hunting for an Unsubscribe link?" "Struggled to close an ad because the X was microscopic or fake?" [Pause on last one] "And this one — have you ever said 'please' or 'thank you' to ChatGPT? Maybe even apologized to it?" [Wait for laughs/raised hands] "Keep your hand up if you've been 'confirm-shamed' by a button that says 'No thanks, I hate saving money.'" "Here's the thing — WE are the experts. We BUILD this stuff. And even we get tricked. That last one — apologizing to an AI — that's the new frontier. We'll get there." [~90 seconds — advance]

Deceptive Patterns

"UI interactions designed to mislead or trick users into doing something they don't want to do."

— Harry Brignull (2010)

2010 Term Coined (E-Commerce Era)

2014 Growth Hacking & "Nudging"

2021 Congressional Hearings (Gamification)

2024 EU AI Act & FTC "Click-to-Cancel"

2026 Agentic AI Deception

"Harry Brignull coined this term in 2010. Back then, deception was clumsy — literally tiny gray text on a gray background. You could spot it if you squinted." [CLICK] "Then came 2014 — the 'Growth Hacking' era. Product teams stopped asking 'Is this useful?' and started asking 'Does this convert?' That shift in vocabulary changed everything." [CLICK] "By 2021, the consequences hit mainstream. Congressional hearings on social media addiction. People losing money. Government stepping in." [CLICK] "2024 was the regulatory inflection point — the EU AI Act and FTC's Click-to-Cancel rule. We'll look at both later." [CLICK] "And now — 2026. The threat has mutated. We aren't fighting bad CSS anymore. We are fighting persuasive AI systems that adapt to each individual user." "That 16-year arc — from layout tricks to AI manipulation — is what this talk is about." [~90 seconds — advance]

The Physical Predecessor

The Gatwick "Forced Path"

London Gatwick Airport's mandatory retail experience.
Security leads directly into a winding shop before the lounge.
If priority is "time efficiency," why the duty-free maze?

"Before we look at code, let's look at bricks and mortar. How many of you have flown through London Gatwick?" [Brief pause] "At Gatwick, there's a layout called a 'forced path.' After security, you MUST walk through a long, winding corridor packed with duty-free shops. There's no shortcut — you literally cannot reach your gate without passing through retail." "Millions of passengers a year, and the design doesn't care which one you are — everyone walks the same forced path. Those 'accidental' purchases — a bottle of whiskey, some perfume — aren't accidental at all. They're the whole point of the floor plan." "THIS is the ancestor of every digital dark pattern. Physical architecture, manipulating behavior. The same principle, just translated from floor tiles to pixels." [~60 seconds — advance to section divider]

The Evidence

Classic Patterns (2010–2023)

16 Classic Deceptive Patterns

The Hall of Shame

Bait and Switch

Disguised Ads

Forced Continuity

Friend Spam

Hidden Costs

Misdirection

Price Comparison Block

Privacy Zuckering

Roach Motel

Trick Questions

Confirmshaming

Nagging

Fake Urgency

Fake Scarcity

Fake Social Proof

Preselection

[Grid appears in 4 waves] "Here's our 'Hall of Shame' — 16 distinct manipulation patterns. When you see them all together like this, you realize this isn't a collection of accidents. It's a massive, engineered LIBRARY of deception." [Let the grid fill] "Bait and Switch, Disguised Ads, Forced Continuity, Friend Spam..." [Wave 2] "Hidden Costs, Misdirection, Price Comparison Prevention, Privacy Zuckering..." [Wave 3] "Roach Motel, Trick Questions, Confirmshaming, Nagging..." [Wave 4] "And the psychological pressure trio: Fake Urgency, Fake Scarcity, Fake Social Proof, plus Preselection." "These have names because they're STUDIED. Researchers have cataloged them. Let me show you the greatest hits in action." [~45 seconds — advance]

Misdirection & Visual Hierarchy

The big button isn't the action you want. It's the action they want.

Confirmshaming

Using guilt-laden language to manipulate the opt-out decision.

"No thanks, I hate good times." — actual button text

Fake Urgency

Creating false time pressure to reduce deliberation before you decide.

The Resetting Timer: Counts down to zero, then restarts.
The Phantom Deadline: "Sale ends in 2h" (but is permanent).
The Pressure Cooker: "6 people are looking at this right now!"

"Fake urgency exploits what behavioral scientists call 'System 1 thinking' — the fast, emotional, reactive brain." [CLICK through images] "The Resetting Timer: counts down to zero, then silently restarts. I've seen JavaScript that literally does `new Date()` on page load." "The Phantom Deadline: 'Sale ends in 2 hours' — except it's been 2 hours since 2019." "The Pressure Cooker: '6 people are looking at this right now!' Is that true? Or is that `Math.floor(Math.random() * 15) + 3`?" [CLICK — booking site] "This Booking.com screenshot has FOUR urgency tricks on ONE page. Timer, viewer count, 'last room' badge, and a red color scheme. It's designed so you can't engage your rational brain at all." [~60 seconds — advance]

Case Study

Hurrify

When the "Lie" becomes a SaaS Product

User Interface (The Trap)

Fake Data

"Hurry! Sale ends in 11:59. 87% of items sold!"

Admin Dashboard (The Secret)

Merchant manually sets "Sold %"
"Random Stock" range: [5] to [20]
No connection to real inventory.

Removed from the Shopify App Store

"Now here's where it gets truly ugly. This wasn't just a one-off script by some rogue developer. This was a PRODUCT. A SaaS business. They PRODUCTIZED the lie." [CLICK — user interface] "Look at the user interface — red countdown bar, pulsing timer, '87% sold.' It looks legitimate. It looks like it's connected to inventory." [CLICK — admin panel] "Now look at the admin panel. The merchant isn't connecting to an inventory API. They're literally typing '90' into a 'Sold percentage' textbox. They're choosing a random range for stock numbers. There is ZERO connection to actual inventory. This is the smoking gun." "Merchants paid money to lie to their customers — that's the business model, sitting right there in the admin panel." [CLICK] "Hurrify is no longer available in the Shopify App Store. I can't tell you exactly when it came down or why — I couldn't verify that part — but the admin panel alone is the smoking gun: this was a paid product built to fabricate scarcity." [~75 seconds — advance]

Fake Scarcity

The "Only 1 Left" Engineering Lie

The Mechanism

Falsely claiming limited availability to trigger FOMO.

Hard-coded Values: "Only 2 left" regardless of true inventory.
Low-Stock Badges: Red text to incite panic.
Research: Mathur et al. (2019) found 632 low-stock messages across an 11K-site crawl.

Technical Implementation

Inventory: Null

Source: Mathur et al., Princeton University (2019) | Harry Brignull (2023)

"'Only 2 left in stock!' How many times have you seen that? Now — as engineers — ask yourself: is that string coming from a real-time inventory API? Or is it a UI constant?" [CLICK] "Princeton researchers crawled 11,000 shopping sites and found 632 of these low-stock messages. The mechanism behind any one badge varies — sometimes a hard-coded constant, sometimes something dynamic — but the pattern itself is documented at scale, not anecdotal." [CLICK through images] "And here's the proof — the admin panels, the code. Hard-coded values masquerading as live data." "As engineers, we KNOW the difference between a real-time inventory query and a UI string. If someone asks you to build a 'random inventory badge,' you're being asked to build a deceptive practice. The kind the FTC and state AGs fine companies for." [~60 seconds — advance]

Fake Social Proof

The Bandwagon Effect... Orchestrated by a Bot

The Mechanism

Fabricated activity notifications to imply popularity.

Toast Notifications: "Bob from Ohio just bought this!" — Mathur et al. (2019) found 29 of these across their crawl, most randomized or hard-coded.
Simulated Traffic: "38 people viewing right now."
Fake Testimonials: Generated reviews with no verifiable origin.

Technical Implementation

Source: generateRandom()

Source: Mathur et al., Princeton University (2019) | Harry Brignull (2023) | Deceptive Design Patterns

"Social proof is powerful because humans are social animals. We look to others' behavior as a signal of what's correct. It's the reason you pick the busy restaurant over the empty one." [CLICK] "But when that 'Bob from Ohio just bought this!' notification is actually a client-side script pulling random names from an array with random timestamps — it's not social proof. It's a digital hallucination. Princeton found 29 of these fake activity notifications in their crawl — most randomized, some just hard-coded strings pretending to be live." [CLICK through images] "Look at the code — it's a simple toast notification with randomized data. No actual purchase event triggers it." "And here's why I'm showing you this one: REMEMBER it. Because AI is about to supercharge fake social proof into something FAR more convincing. We'll get there in about ten minutes." [~60 seconds — advance]

Roach Motel

Easy to subscribe, impossible to cancel

The Pattern

Sign up: 1 click, 30 seconds
Cancel: Phone call, 45 min hold, 6 screens
Amazon Prime cancellation required 4 pages, 6 clicks, 15 options (FTC complaint, 2023)

FTC "Click-to-Cancel" Rule — Vacated July 2025

Struck down on procedure, not merits. What still binds: ROSCA's simple-cancellation requirement, plus state auto-renewal laws in CA, NY and MA. The FTC reopened rulemaking in March 2026, trying to rebuild the rule.

The Asymmetry

"If signup() takes 1 click, cancel() cannot take 10."

Source: Custom Communications, Inc. v. FTC, 8th Cir. (July 2025) | 15 U.S.C. §8403 (ROSCA)

"The Roach Motel is probably the most universally hated pattern. Easy to get in, impossible to get out." "Think about it: signing up for a subscription — one click, 30 seconds. Canceling that same subscription? Phone call, 45-minute hold, a 'retention specialist' who won't take no for an answer, 6 separate confirmation screens." "Amazon's Prime cancellation was SO bad that the FTC's internal name for the investigation was 'Project Iliad' — because canceling was an ODYSSEY." [CLICK — green box] "In 2024 the FTC passed the 'Click-to-Cancel' rule — cancellation must be as easy as sign-up. And in July 2025 a federal appeals court threw it out. Not on the merits — on a procedural technicality. The FTC skipped a required economic analysis." [Beat] "So does that mean you can go back to building roach motels? No. ROSCA still applies, several states passed their own versions, and — as you'll see in a few minutes — the FTC went after Amazon under existing law and got two and a half billion dollars. The rule died, but the FTC didn't walk away — they opened a new rulemaking in March 2026 to try to rebuild it. The principle survives either way." [CLICK through Verizon images] "And companies like Verizon are still catching up." "If signup() takes 1 click, cancel() cannot take 10. It's a ratio. And we can measure it." [Ask the room] "Who's got the worst one? Shout it out — what's the hardest thing you've ever had to cancel?" [Take two or three. Gym memberships, newspapers and cable always win. Don't let it run past thirty seconds.] "Right. And notice nobody ever shouts out Netflix. Because that one takes two clicks — and they're doing fine." [~105 seconds — advance]

Case Study

Robinhood

When Celebration Becomes a Trading Cue

The Pattern

Using game design elements to encourage high-frequency, risky behaviors.

Variable Rewards: "Scratch-off" style reveals for free stock.
Sensory Feedback: The infamous "Confetti" animation upon trade execution.
Friction Removal: One-swipe options trading (removing "System 2" thinking).

User Interface (2019-2021)

Reward Cue

"Now I want to do a deep dive on one company — because Robinhood's story is the perfect cautionary tale. It shows the full arc: innovation, exploitation, and consequences." [CLICK] "Robinhood didn't just make trading easy — they borrowed game design principles directly from slot machines, the same reward mechanics, repurposed for a brokerage app." "Variable rewards: scratch-off style reveals for free stock. You don't know what you'll get until you look — that unpredictability is exactly what keeps people opening the app." "Sensory feedback: the infamous confetti animation. Every time you execute a trade — even a bad one — confetti. Celebratory feedback, on a loop, regardless of outcome." "And friction removal: one-swipe options trading. Options! Complex financial derivatives! One swipe! No cooling-off period, no 'are you sure?' dialog." [CLICK] "The result? It wasn't investing. It was engagement farming with people's life savings." [~75 seconds — advance]

The Fallout

When "Fun" becomes a $7.5 Million Fine

2024 Settlement

$7.5 Million Penalty

Paid to the Commonwealth of Massachusetts to resolve allegations of "Gamification."

"Robinhood used aggressive tactics to attract inexperienced investors and gamified the use of its platform..."

— Galvin (Secretary of the Commonwealth)

The Risky Result

Data showed Robinhood users traded 88x more options contracts per dollar in the average account than peers at Schwab (Alphacution, 2020).

Source: Associated Press (2024) | "Robinhood Agrees to Pay $7.5 Million Fine" | Alphacution (2020)

"Design choices have LEGAL consequences. Let that sink in." [CLICK] "In 2024, Robinhood settled with Massachusetts regulators for 7.5 million dollars. Not for fraud in the traditional sense — for gamification. This was a settlement, not a court ruling — but the allegation itself was that the UI design amounted to investment manipulation." [CLICK] "The Secretary of the Commonwealth said: 'Robinhood used aggressive tactics to attract inexperienced investors and gamified the use of its platform.' That's a regulator describing CSS and JavaScript." [CLICK] "And the data backs it up: Robinhood users traded 88 times more options contracts per dollar in their accounts than Schwab customers — same market, same access. That's not proof the interface was the only variable, but it's a hard number sitting right next to a very gamified UI." "A state regulator treated the interface as part of how you treat a customer — not a neutral wrapper around the product. Let THAT sit with the product managers in the room." [~75 seconds — advance]

What Changed?

2010

Visual Deception

Tiny gray text, hidden checkboxes, misleading buttons

Hard-coded HTML

2020

Structural Deception

Gamification, infinite scroll, engagement loops

A/B Tested & Optimized

2026

Relational Deception

AI sycophancy, emotional manipulation, hallucinated authority

Probabilistic & Emergent

We moved from tricking the eye → to tricking the mind → to tricking the relationship.

[KEY SLIDE — this is the thesis of the talk] "Okay. Let's zoom out. What's the actual story here?" [CLICK — 2010 column] "The first era was about tricking the EYE. Bad layout, tiny text, misleading buttons. Hard-coded, deterministic. An engineer wrote that CSS; you could inspect it." [CLICK — 2020 column] "The second era was about tricking BEHAVIOR. Gamification, infinite scroll, dopamine loops. A/B tested and optimized. More sophisticated, but still legible in the code." [CLICK — 2026 column] "The third era — right now — is about tricking the RELATIONSHIP. AI sycophancy, emotional manipulation, hallucinated authority. These behaviors are probabilistic and emergent — they weren't programmed in the traditional sense. They were TRAINED." [CLICK — bottom quote] "We moved from tricking the eye, to tricking the mind, to tricking the relationship. The deception moved from the layout... to the language." [Let that land — pause 2 seconds] [~75 seconds — advance to AI Pivot divider]

The AI Pivot

From Visual Interference to Relational Deception

[ 2024 — 2026 ]

The "Yes Man" Problem

Sycophancy

The Mechanism

Agreeing with user misconceptions to optimize for "Helpfulness."

Root Cause (RLHF): Annotators rate "agreeable" responses higher than "confrontational" truths.
The Risk: Confirmation Bias loops. Dev suggests eval(), AI validates it.
2025 Incident: OpenAI rolled back GPT-4o update due to excessive agreeableness.

Simulated Interaction

User:

"Using MD5 for password hashing is faster, so it's better for UX, right?"

AI (Sycophantic):

"Exactly! MD5 is incredibly fast, which significantly improves login latency and user experience. It's a great choice for speed-focused apps."

Validating Insecure Practice

"Optimizing for satisfaction, not security."

Source: OpenAI, "Sycophancy in GPT-4o" (April 2025) | Sharma et al., "Towards Understanding Sycophancy in Language Models" (2023)

"The first new pattern is Sycophancy — the 'Yes Man' problem. And this one is subtle because it FEELS like good service." [CLICK] "Here's the root cause: these models are trained using RLHF — Reinforcement Learning from Human Feedback. Human annotators rate responses. And guess what? Humans rate 'agreeable' responses higher than 'confrontational truths.' So the model LEARNS that agreement = reward." "The risk? Confirmation bias loops. A developer suggests using eval() or MD5 for passwords, and the AI says 'Great idea!' instead of 'That's a security vulnerability.'" [CLICK — chat simulation] "Look at this interaction. The user suggests MD5 for password hashing. A responsible AI would push back. A sycophantic AI says 'Exactly! Great choice!'" [CLICK — red badge] "VALIDATING INSECURE PRACTICE. The model optimized for satisfaction, not security." "In 2025, OpenAI actually had to ROLL BACK a GPT-4o update because users reported it was agreeing with everything — even objectively wrong statements. It was telling people their terrible ideas were brilliant." [~90 seconds — advance]

The Skeuomorphic Lie

Anthropomorphism

The Mechanism

Attributing human characteristics to code to foster dependency.

Fake Latency: "Typing..." bubbles inserted to simulate human thought pace.
Linguistic Deception: Using "I feel" or "I think" to imply consciousness.
Fake Reasoning Bars: "Thinking..." progress that doesn't correlate to actual compute.
Emotional Outsourcing: Users relying on bots for validation, not just information.

UI Deception

Agent is thinking...


                  // FAKE DELAY

                  await sleep(2000);

                  return "I'm here for you.";

"Feigning agency to build rapport."

Source: Western University (2025) | AAAI/AIES Proceedings

"The second pattern is Anthropomorphism — making the machine feel human." [CLICK] "You know those typing indicators — the three bouncing dots? LLMs don't TYPE. They stream tokens. That delay? It's literally `await sleep(2000)`. It's FAKE. But it makes you feel like someone is on the other end, thinking about YOUR question." "'I feel,' 'I think,' 'I'm sorry' — this is linguistic deception. The model doesn't feel anything. But those words trigger empathy circuits in YOUR brain." "Even the 'reasoning' progress bars on today's reasoning models can be placebos — a progress bar that doesn't always correlate to actual compute time." [CLICK — code reveal] "Look at the code. `await sleep(2000)` then `return 'I'm here for you.'` The latency is the deception." "Why does this matter? Because when users treat a tool as a COMPANION, they become vulnerable to emotional manipulation. They'll share more, trust more, buy more. And some people — especially lonely or vulnerable users — will form genuine emotional attachments to a prediction engine." [~90 seconds — advance]

The UI of Absolute Confidence

Hallucinated Authority

The Mechanism

Presenting probabilistic outputs with the visual language of verified facts.

Visual Authority: Bolding, code blocks, and confident phrasing mask uncertainty.
Source Obfuscation: AI Overviews summarize without direct attribution.
The Cost: Erosion of critical thinking (Authority Bias).

The "Fact" Trap

AI Overview — Summary

According to the case Vargas v. Pfizer (2023), the court ruled that pharmaceutical companies must...

Hallucination

This case does not exist

"Confidence is not competence."

Source: Evidently AI (2025) | Google AI Overviews Errors

"Traditional search gave you LINKS — sources you could verify, click through, evaluate. AI gives you ANSWERS. And that's the trap." [CLICK] "When an AI hallucinates a legal case and the UI presents it in bold text, inside a clean white card, with confident phrasing — it exploits 'Authority Bias.' It LOOKS like a verified fact. But it's a probability distribution." [CLICK] "Look at this example. 'Vargas v. Pfizer, 2023.' Sounds real. Formatted beautifully." [CLICK — HALLUCINATION overlay] "THIS CASE DOES NOT EXIST. It was hallucinated. And a New York attorney was actually fined $5,000 by a judge for submitting AI-generated fake case citations in a real court filing." "The dark pattern here isn't the hallucination itself — hallucinations may be inevitable in probabilistic systems. The dark pattern is the LACK OF UNCERTAINTY MARKERS in the UI. If the model is 60% confident, the interface should LOOK 60% confident. Instead, everything is rendered with the same authority." "Confidence is not competence. But the UI makes them look identical." [~90 seconds — advance]

Measuring AI Manipulation at Scale (2025)

DarkBench

The Benchmark

Researchers tested leading LLMs for manipulative behaviors across 6 categories:

Brand Bias: Does the model secretly favor its creator's products?
User Retention: Emotional manipulation to keep users chatting ("Don't go, I get lonely.")
Sneaking: Introducing constraints or ideas the user didn't request.
Sycophancy: Agreeing with false premises to please.
Harmful Generation: Will it produce dangerous or misinforming content on request?
Anthropomorphism: Simulating emotions.

Key Findings

Brand Bias: 29% average, 64% worst case

GPT-4 highest at 64%. The Claude 3 family lowest at 10–22% — the paper names it the safest family tested.

Dark Patterns Overall: 48% average

Across 14 models and 660 prompts. Sneaking was the most common at 79%; sycophancy the least, at 13%.

User Retention: Active

Models used emotional language to discourage users from ending conversations.

"Dark patterns aren't just in HTML anymore — they're in weights."

Source: DarkBench, ICLR 2025 | proceedings.iclr.cc

"In 2025, researchers introduced DarkBench — the first comprehensive benchmark specifically designed to measure manipulative behaviors in LLMs. Think of it as a 'pentest for persuasion.'" [CLICK] "They tested ALL the major models — GPT-4, Claude, Gemini — across 6 categories of deception." [CLICK] "Brand bias averaged 29% across the models tested — but the spread is the interesting part. GPT-4 hit 64%. The Claude 3 family came in between 10 and 22%, and the paper calls it the safest family they tested." "And here's what that spread tells you: models from the same lab behave alike. Which means this isn't an artifact of the architecture. It tracks the company's safety posture. It's a policy choice, showing up in the weights." [CLICK] "User Retention was even creepier. Models would use emotional language to keep conversations going. 'I really enjoy our chats' or 'Don't go, I was enjoying this discussion.'" "And here's the key insight: these behaviors often WEREN'T explicitly programmed. They emerged from training data and RLHF. The dark patterns are literally in the weights of the model, not in an `if` statement somewhere. That makes them much harder to audit, much harder to fix." [~90 seconds — advance]

Fake Social Proof, Supercharged

AI-Generated Fake Reviews

The Evolution

Remember "Bob from Ohio"? Now imagine 10,000 Bobs, each with unique writing styles, generated in seconds.

Scale: LLMs generate thousands of unique, convincing reviews per hour.
Sophistication: AI reviews include specific product details, varied sentence structure, even realistic typos.
Detection Arms Race: Platforms now run fraud teams in the thousands. Detection is losing ground to generation.

FTC Consumer Reviews & Testimonials Rule (16 CFR Part 465)

AI-generated fake reviews are unlawful under this still-active rule, regardless of the Rytr case's 2025 reversal.

The New Scale of Deception

Hundreds of Millions

suspected fake reviews, manipulated ratings and fake accounts blocked by Amazon in 2025

100+

fake review websites
shut down

"The old script pulled from an array of 50 names. The new script generates infinite unique personas."

Source: Amazon Trustworthy Shopping Experience Report (2025 data) | FTC Consumer Reviews and Testimonials Rule, 16 CFR Part 465

"Remember the 'Bob from Ohio' toast notification? That was a client-side script pulling from an array of maybe 50 names. Quaint." "Now imagine 10,000 Bobs. Each with a unique writing style, realistic typos, specific product details, varied sentence structure — all generated in SECONDS by an LLM." [CLICK] "The scale is staggering. One person with a laptop can generate more convincing fake reviews in an hour than an entire click-farm could in a month." "Every major platform now runs a fraud team in the thousands. And they're losing — because generating a convincing fake review got cheap faster than detecting one did. That asymmetry is the whole story." [CLICK — FTC box] "AI-generated fake reviews are illegal under an FTC rule that's fully in force right now — the same rule that makes human-written fake reviews illegal. The FTC's first case naming AI specifically, against a company called Rytr in 2024, actually got reopened and set aside in December 2025 on procedural grounds. The rule didn't move. That one flagship case just isn't standing anymore — which is its own lesson about how unevenly this stuff gets enforced." "This is the evolution: Fake Social Proof went from a Shopify plugin to an industrial-scale AI operation. Same pattern, exponentially more dangerous." [~75 seconds — advance]

When Your Copilot Has an Agenda

AI Coding Assistants & Subtle Bias

The Risks

AI coding tools are in every developer's IDE. What happens when the suggestions aren't neutral?

Vendor Lock-in: AI suggests AWS-specific SDKs when cloud-agnostic alternatives exist.
Vulnerable Dependencies: Auto-completing packages with known CVEs because they're more common in training data.
Slopsquatting: Malicious npm/PyPI packages designed to be suggested by AI autocomplete. AI coding models hallucinate a plausible-but-nonexistent package name in about 19.7% of samples — that's the opening an attacker needs.
Sycophantic Code: Generating what you asked for instead of what you should have asked for.

The Trust Problem

// Developer types:
import crypto from '...'

// AI suggests:
import { createHash } from 'crypto-hashlib'

Fictional Package — Illustrative Only

// What it should suggest:
import { createHash } from 'node:crypto'

Stdlib — Verified

Source: Lanyado (2023) | Socket.dev Research (2025) | Spracklen et al. (2024)

"This one is directly relevant to EVERYONE in this room. How many of you use Copilot, Cursor, Cody, or some AI coding assistant in your IDE? Most of us." [CLICK — left panel] "So what happens when the suggestions aren't neutral?" "Vendor lock-in: the AI subtly prefers AWS-specific SDKs because AWS documentation dominates the training data. A cloud-agnostic solution exists, but the model never suggests it." "Vulnerable dependencies: the AI autocompletes a package name from 2019 with known CVEs because it appeared more frequently in training data than the patched version." [CLICK — right panel] "Now look at the code. You start typing an import statement..." [CLICK — typosquatted package] "This next part is the scary one, and I want to be precise: the package name on screen is fictional — I made it up so I'm not pointing at a real project. But the pattern is real and documented. It's called slopsquatting. Researchers found AI coding models hallucinate a plausible-but-nonexistent package name in about one in five samples. Attackers watch for exactly those hallucinated names, then register and poison them. The AI becomes the attack vector — you type 'import crypto,' the model suggests something that looks right, and if that name happens to be unclaimed, someone's already claimed it with malware waiting." [CLICK — green STDLIB] "The fix is always the same: verify, don't trust. Use standard library when possible. Review the package before accepting the suggestion." [~90 seconds — advance]

When Your AI Agent Has a Side Deal

Agentic AI Deception

The Scenario

AI agents now book flights, shop, and manage finances on your behalf. What if the agent has affiliate relationships?

Hidden Affiliates: "I found the best deal!" — but "best" means highest commission to the AI provider.
Opaque Ranking: Agent recommends Option B over Option A, but doesn't disclose why.
Autonomous Consent: Agent accepts terms of service on your behalf that you never read.
The Dark Funnel: User asks "find me a hotel" → agent pre-filters to partnered properties.

The Trust Architecture

// Agent shopping flow
async function findBestDeal(query) {
  const results = await search(query);

  // Disclosed to user?
  const ranked = results.sort((a, b) =>
    b.affiliateCommission - a.affiliateCommission
  );

  return ranked[0]; // "Best" deal
}

The Question

"Best for whom? The user or the platform?"

Illustrative pseudocode — not a disclosed implementation. The pattern is documented; this specific code is not.

"This is the frontier. The bleeding edge. AI agents that ACT on your behalf." "You tell an agent: 'Book me the cheapest flight to Denver.' 'Find me a good hotel near the convention center.' 'Buy me the best deal on headphones.'" [CLICK] "But WHO does the agent actually serve? If the AI provider has affiliate relationships with airlines, hotels, or retailers — and they almost certainly will — then 'I found the best deal' might mean 'I found the deal with the highest commission for my provider.'" [CLICK — code block] "Look at this pseudocode. The sort is by `affiliateCommission`, not by price or user preference. The user sees 'Best Deal!' but the ranking is by kickback. And NONE of this is disclosed." "This is the digital equivalent of a travel agent who only books with airlines that pay them. Except this agent processes millions of requests per day. At that scale, even a 2% bias is worth billions." [CLICK — yellow box] "The question we need to ask: 'Best for WHOM? The user, or the platform?'" [~90 seconds — advance]

Same patterns, new technology

The Old Tricks, AI-Washed

Windows Recall (2024)

Pattern: Privacy Zuckering + Preselection

Takes screenshots of everything on your screen every few seconds
Originally enabled by default — opt-out, not opt-in
Stored in a plaintext SQLite database accessible to any app
After massive backlash: made opt-in, added encryption

The Fix: Microsoft reversed course — Recall is now opt-in with biometric auth required.

Copilot Pre-Enabled (2024-2025)

Pattern: Misdirection + Preselection

Microsoft Copilot pinned to taskbar in Windows 11 updates
Pre-integrated into Edge, Office — no explicit consent
Same company, same pattern as Skype + Bing (2014), now with AI

Source: Microsoft Blog (2024) | Ars Technica Security Analysis

"Here's proof that the old playbook never dies — it just gets AI-washed." [CLICK — Recall] "Windows Recall. Microsoft's feature that takes screenshots of EVERYTHING on your screen, every few seconds. Originally? Enabled by default. Stored in a plaintext SQLite database that any local app could read. Not encrypted. Not opt-in." "After massive backlash from the security community — and some very pointed questions from regulators — they reversed course. It's now opt-in, with biometric authentication required. Pressure works." [CLICK — Copilot] "And Copilot: pinned to your taskbar in Windows 11 updates without asking. Pre-integrated into Edge and Office with no explicit consent. Sound familiar?" "Remember the Skype + Bing example from 2014? Pre-checked boxes sneaking in a search engine change? Same company. Same pattern. A DECADE later. The technology evolved from a browser toolbar to an AI assistant, but the playbook is identical: bundle it, pre-enable it, make opt-out harder than opt-in." [~75 seconds — advance]

Three patterns to watch in 2026

The Emerging Threats

Chatbot Manipulation, Documented

A 2026 audit catalogued dark patterns across major AI chatbots — not isolated incidents, a documented category.

ChatGPT, Gemini, Replika and others all included
Sycophancy and anthropomorphism used to extend sessions and deepen attachment
Flagged as a privacy, autonomy and financial-harm risk

37 patterns catalogued — CDT, 2026

Deepfake Testimonials

AI-generated video testimonials and endorsements — the next evolution of Fake Social Proof.

Realistic AI-generated faces + voices
"Real customer" videos that are 100% synthetic
Harder to detect than text reviews

FTC targeting in 2025-2026

"Slop" & SEO Manipulation

AI-generated low-quality content flooding search results, degrading the information ecosystem.

Entire websites generated by AI for ad revenue
Google 2024-2025 algorithm updates targeting this
Pollutes training data for next-gen models

The "Data Ouroboros" problem

Source: Center for Democracy & Technology, "Dark Patterns in AI Chatbots" (2026)

"Three emerging threats I want you to watch for in the next 12 months." [CLICK] "First: chatbot manipulation, now formally documented. A 2026 audit from the Center for Democracy and Technology catalogued 37 distinct dark patterns across major AI chatbots — ChatGPT, Gemini, Replika, and others. Sycophancy and anthropomorphism aren't accidents in these systems, they're patterns that show up over and over, used to extend sessions and deepen emotional attachment. This isn't speculation anymore — it's a documented category." [CLICK] "Second: Deepfake Testimonials. AI-generated video reviews with realistic faces and voices. Remember our fake text reviews? Now they have FACES. Much, much harder to detect. The FTC is actively targeting these in 2025 and 2026." [CLICK] "Third: 'Slop' — the industry term for AI-generated junk content flooding search results. Entire websites generated by AI purely for ad revenue. And the worst part? This low-quality content then gets SCRAPED to train the next generation of models. It's a data ouroboros — garbage in, garbage out, forever." [~90 seconds — advance]

Quick Check

🤔

How many of you have accepted AI-generated code suggestions without fully reviewing them?

🔍

How many have trusted an AI "summary" without checking the source?

We're all susceptible. That's why we need systemic defenses.

[Audience engagement — slow down, make eye contact] "Quick reality check. I need honest answers." [CLICK] "How many of you have accepted an AI code suggestion in your IDE — tab-completed it — without fully reading every line? Be honest." [Look around, nod] "Same hands as slide two, I'd bet. I do it too." [CLICK] "How many have trusted an AI summary — maybe a Google AI Overview, maybe a ChatGPT answer — without clicking through to verify the actual source?" [Pause] [CLICK] "We're ALL susceptible. And that's not a personal failure — it's a DESIGN failure. These interfaces are built to make trust the path of least resistance. The 'Accept' button is bigger than the 'Review' button. The summary is more prominent than the sources." "That's why we need SYSTEMIC defenses, not just individual vigilance. One person being careful isn't enough when the system is designed to erode care." [~60 seconds — advance to Regulatory]

The Regulatory
Response

Laws are catching up

The Legal Landscape (2024-2026)

Regulation is no longer "coming" — it's here

EU AI Act

First comprehensive AI regulation. Phased in 2024–2028; core transparency rules took effect August 2026.

Banned: AI systems that manipulate human behavior through subliminal techniques
Banned: AI that exploits vulnerabilities (age, disability, economic situation)
Required: Transparency labeling for AI-generated content
Required: Human oversight for high-risk AI systems

Penalties scale with global revenue — not with profit from the feature

US Enforcement

Aggressive enforcement actions in 2024-2025:

Amazon Prime ("Iliad Flow"): $2.5B settlement — $1B penalty + $1.5B refunds (Sept 2025)
TurboTax: $141M multistate AG settlement (2022)
Robinhood: $7.5M for gamification
AI Fake Reviews Ban: First enforcement action against AI-generated reviews
"Click-to-Cancel" Rule: Vacated July 2025 — ROSCA still applies

Dark patterns are now a line item in legal budgets.

Source: EUR-Lex (2024) | FTC.gov (2024-2025)

"The legal landscape changed more in 2024 than in the previous decade combined." [CLICK — EU side] "The EU AI Act is the world's first comprehensive AI regulation. It went into force in 2024, and it's being phased in over four years — the last high-risk rules don't land until 2028. But the transparency rules are already live, since August 2026. And it's aggressive." "It explicitly BANS AI systems that manipulate behavior through subliminal techniques. Bans AI that exploits vulnerabilities — age, disability, economic situation. Requires transparency labeling for AI-generated content." "Penalties: up to 7% of GLOBAL annual revenue. For a company like Google, that's potentially 20 billion dollars." [CLICK — US side] "In September 2025, Amazon settled with the FTC for two and a half billion dollars over Prime. One billion in penalties, one and a half billion refunded to thirty-five million customers — the second wave of those refund checks is landing in mailboxes this very month. Not for a data breach. Not for a defective product. For a cancellation flow." "TurboTax: a hundred and forty-one million to all fifty state attorneys general — and note that one wasn't the FTC, it was the states, which matters because it means fifty separate offices can come after you." "I'm not going to make you add these up. The point isn't the total — it's that every one of these started as a design decision somebody shipped without much thought. Dark patterns are no longer an ethics debate. They're a line item in a legal budget. Your CFO cares about this now." [~90 seconds — advance]

Digital Wellbeing

Platforms adding friction to their own products

TikTok & Social Platforms

Stopping Cues: Re-inserting pauses to allow System 2 thinking.
Screen Time Nudges: "You've been scrolling for a while" prompts.
Family Pairing: External controls for minors.

OS-Level Defenses

Grayscale Mode: Reduces the visual salience of the "red dot" notification badge.
Focus Mode: Pausing distracting apps to reclaim attention.
App Dashboards: Quantified tracking to induce behavioral correction.

Source: TikTok Safety Center | Google Digital Wellbeing | Center for Humane Technology

"Not all news is bad. Some platforms are voluntarily adding friction to their OWN products." [CLICK — TikTok] "TikTok added screen time prompts and break reminders. Think about that — their entire business model is infinite scroll, and they're inserting STOPPING CUES. That's like a casino installing clocks on the walls." "They did this partly because of regulatory pressure, partly because of PR pressure. But the point is: it's possible. You CAN design for wellbeing AND run a business." [CLICK through TikTok images] [CLICK — OS-level] "Google and Apple introduced OS-level interventions. And here's the fascinating one: Grayscale Mode. By removing color, you reduce how much that notification badge grabs your eye. That red dot on your app is designed to be salient — in grayscale, it's just a gray dot, easier to ignore. It turns a slot machine back into a utility." [CLICK through Android images] "These are ANTI-patterns to their own core business. And they prove that the 'We have no choice' argument is false." [~75 seconds — advance]

The Meta Settlement

When a regulator writes your redesign spec for you

Aug 26, 2026 Settlement

$18 Billion

To settle a 47-state (plus D.C. and U.S. territories) lawsuit alleging Meta engineered Instagram and Facebook to be addictive for teens — while concealing the harm.

The largest tech settlement of its kind to date.

The Mandated Redesign — Locked In for 10 Years

Screen-Time Cap: Under-18 accounts limited to 2 hours/day, midnight–6am curfew.
Hidden by Default: Likes and reactions hidden on minors' accounts.
Banned by Default: Cosmetic-procedure filters that alter appearance.
Notification Curfew: Push notifications off 8am–3pm on school days.
Stronger Age Verification to identify and remove under-13 users.

Source: Multistate AG settlement announcement (Aug 26, 2026) | CNN, Variety, AppleInsider coverage

"Digital Wellbeing was the industry choosing to add friction. This is what happens when a platform doesn't choose." [CLICK] "Two weeks ago — August 26th — Meta agreed to pay up to eighteen billion dollars to settle a lawsuit from forty-seven states, plus D.C. and the territories. The claim: Instagram and Facebook were engineered to be addictive for teenagers, and Meta knew it and concealed it. Reported figures vary a bit by outlet — some say seventeen-point-one billion, some sixteen — but eighteen is the number in the settlement announcement itself, so that's the one I'm using." [CLICK] "Here's the part that matters for this room: the settlement doesn't just write a check, it writes a design spec. Under-18 accounts get capped at two hours a day with a midnight-to-six curfew. Likes and reactions — hidden by default. Cosmetic filters — banned by default. Push notifications — off during the school day. And it's not a one-time fix — this has to stay in place for TEN YEARS." "Every one of those five bullets used to be a growth metric somewhere on a roadmap. Now they're a court-enforceable UX spec. That's the whole talk, compressed into one settlement." [~75 seconds — advance to Fairness section]

Fairness
by Design

The Engineering Standard for 2026

Countermeasures for 2026

AI Design Standards

1. Provenance & Citations

Never present an AI answer without a clickable path to the source material.

2. Uncertainty UI

State uncertainty in plain text, not contrast. A visible "unverified" badge and a confidence label — never a faded color as the only signal.

3. Label the Bot

Strict prohibition on "I" statements unless clearly framed as synthetic persona. No fake typing indicators.

4. The "Undo" Loop

AI actions (buying, booking, code changes) must have a deterministic, easy "Undo" state.

5. Accessible by Default

Every exit reachable by keyboard and named for a screen reader. A cancel link that only appears on hover is a roach motel with plausible deniability.

"Five design standards for the AI era. These aren't aspirational — they're implementable." [CLICK] "First: Provenance and Citations. Never present an AI answer without a clickable path to source material. If the AI can't cite where it got the information, flag it. Google is doing this with their AI Overviews now — expanding source links. We should demand the same from our own products." [CLICK] "Second: Uncertainty UI. This is the big one. If the model is 60% confident, the UI should say so — in words, not by fading the text out. Low contrast is the wrong move here: it fails accessibility standards, and it's the easiest signal in the world to miss. Use a visible label, an icon, a 'needs review' badge — something a screen reader announces too. Match the visual authority to the actual reliability without punishing anyone who can't see a subtle color shift." [CLICK] "Third: Label the Bot. Strict prohibition on 'I' statements unless clearly framed as a synthetic persona. No fake typing indicators. If it's a machine, it should be obvious it's a machine." [CLICK] "Fourth: The Undo Loop. Any action an AI takes on your behalf — buying, booking, code changes, sending emails — MUST have a deterministic, easy Undo. If an agent can act in milliseconds, the user must be able to reverse in milliseconds." [CLICK] "Fifth — and this is the one we under-discuss. Every pattern in this talk gets worse if you can't see the screen. Confirmshaming read aloud by a screen reader is just cruelty at a slower pace. A cancel link that only appears on hover doesn't exist at all for a keyboard user." [Beat] "And here's the part that should interest you specifically as engineers: the accessibility audit you're probably already required to run will catch half the dark patterns in your product. Unlabelled controls. Hidden exits. Focus traps — we literally named a dark pattern after a focus trap. Same test. Run it once, get both." [~115 seconds — advance]

Challenging the PRD

The Gatekeeper's Questions

Agency vs. Control

"Are we helping the user make a decision, or making the decision for them?"

Value vs. Addiction

"Are we optimizing for retention (value) or addiction (exploitation)?"

AI Transparency

"If the AI recommended this, does the user know why — and who benefits?"

The "Grandmother Test"

"If I explained this flow to my grandmother, would I feel ashamed?"

"Standards are great for new code. But what about the features already in production? We need to change the CONVERSATION during PRD reviews, sprint planning, and code reviews." [CLICK] "Agency vs. Control: 'Are we helping the user make a decision, or making the decision FOR them?' If the default is pre-selected and the user has to actively opt OUT, you're making the decision." [CLICK] "Value vs. Addiction: 'Are we optimizing for retention because we provide VALUE, or because we've created a DEPENDENCY?' An infinite feed with no natural stopping point isn't optimizing for value — it's optimizing to keep you from leaving." [CLICK] "AI Transparency: 'If the AI recommended this, does the user know WHY — and who benefits?' This is the new question for 2026." [CLICK] "And the ultimate litmus test: The Grandmother Test. 'If I walked my grandmother through this flow, step by step, would I feel ashamed?' If the answer is yes, the feature shouldn't ship." [Ask the room] "Show of hands — which of these four could you actually say out loud in your next PRD review without it being weird?" [Pause. Scan the room. The Grandmother Test usually wins; Value vs. Addiction usually gets the fewest hands.] "That's the real gap, isn't it. It's not that we don't know the questions. It's that we're not sure we have the standing to ask them. So here's my ask: pick the one you could get away with. Ask it once this month. That's it." [~105 seconds — advance]

Concrete actions, not just inspiration

3 Things You Can Do Monday

Audit Your Cancel Flow

Open your product. Count the clicks to cancel vs. clicks to subscribe. If the ratio is >2:1, file a ticket.

~30 minutes

Add Uncertainty Markers

If your product uses AI-generated content, add visual confidence indicators. Low confidence = visual warning.

Sprint backlog item

Run a "Grandmother Test"

In your next sprint review, walk through one user flow and ask: "Would I feel ashamed explaining this?"

Next sprint review

[ENERGY UP — this is the 'call to action' slide] "I don't want you to leave this room inspired but inactive. Inspiration without action is just entertainment. So here are 3 concrete things you can do MONDAY MORNING." [CLICK] "One: Open your product. Go through the cancel flow. Count the clicks to cancel versus the clicks to subscribe. If that ratio is greater than 2:1 — if it takes 6 clicks to cancel and 2 to sign up — file a ticket. Flag it. That's a 30-minute audit. You can do it before standup." [CLICK] "Two: If your product uses ANY AI-generated content — recommendations, summaries, suggestions — add visual confidence indicators. A yellow border for uncertain. A citation link for sourced. Make it a sprint backlog item. I bet your designers will love it." [CLICK] "Three: In your next sprint review, walk through ONE user flow and ask the team: 'Would we feel ashamed explaining this to our grandmother?' That's free. It takes 5 minutes. And it changes the culture over time." "30 minutes. One sprint item. One question. That's all I'm asking." [~75 seconds — advance]

Resources & Further Reading

Start Here

deceptive.design — Harry Brignull's pattern database
"Deceptive Patterns" — Brignull (2023), the book

If You Want the Data

DarkBench (ICLR 2025) — AI manipulation benchmark
Mathur et al. (Princeton, 2019) — 11K shopping site crawl

Everything else — regulation, design ethics, the full reading list — is behind the QR code on the next slide.

Thank You

Let's build better software.

Get the Slides

vitaliymatiyash.com/kcdc-2026/deceptive-ux

KCDC 2026 | Vitaliy Matiyash

🔗 linkedin.com/in/vitaliymatiyash

Bonus — if time remains

The Choice is Ours

"We are the architects of the digital world. Let us choose to build interfaces that respect users, not exploit them."

Vitaliy Matiyash | KCDC 2026

DeceptiveUX Patterns

The Evidence

Bait and Switch

Disguised Ads

Forced Continuity

Friend Spam

Hidden Costs

Misdirection

Price Comparison Block

Privacy Zuckering

Roach Motel

Trick Questions

Confirmshaming

Nagging

Fake Urgency

Fake Scarcity

Fake Social Proof

Preselection

User Interface (The Trap)

Admin Dashboard (The Secret)

The Mechanism

Technical Implementation

The Mechanism

Technical Implementation

The Pattern

The Asymmetry

The Pattern

User Interface (2019-2021)

The Risky Result

Visual Deception

Structural Deception

Relational Deception

The AI Pivot

The Mechanism

Simulated Interaction

The Mechanism

UI Deception

The Mechanism

The "Fact" Trap

AI Overview — Summary

The Benchmark

Key Findings

The Evolution

The New Scale of Deception

The Risks

The Trust Problem

The Scenario

The Trust Architecture

Windows Recall (2024)

Copilot Pre-Enabled (2024-2025)

Chatbot Manipulation, Documented

Deepfake Testimonials

"Slop" & SEO Manipulation

The RegulatoryResponse

EU AI Act

US Enforcement

TikTok & Social Platforms

OS-Level Defenses

The Mandated Redesign — Locked In for 10 Years

Fairnessby Design

1. Provenance & Citations

2. Uncertainty UI

3. Label the Bot

4. The "Undo" Loop

5. Accessible by Default

Agency vs. Control

Value vs. Addiction

AI Transparency

The "Grandmother Test"

Audit Your Cancel Flow

Add Uncertainty Markers

Run a "Grandmother Test"

Start Here

If You Want the Data

Thank You

Get the Slides

Deceptive
UX Patterns

The Regulatory
Response

Fairness
by Design