Marketers Must Stop Treating LLMs Like People
A MarTech analysis argues LLMs are predictive text engines, not conscious assistants — and treating them like people produces vague prompts and hallucinated junk.
Updated
The brief
- LLMs process text as tokens rather than letters, which is why models fail tasks like counting the letter "e" in "seventeen"
- Correcting an AI mid-conversation only appends new tokens to the context window and can lead it to defensively fabricate new "facts"
- The article prescribes breaking prompts into single-purpose steps, applying tight constraints and restarting rather than arguing with hallucinations
A large language model is, at its core, a hyper-sophisticated, multi-billion-parameter predictive text engine — "a very complicated parrot," as a new MarTech analysis puts it. The piece argues that a massive disconnect exists between how people think LLMs work and how they actually function, and that this misunderstanding actively damages marketing operations.
The confusion is understandable, the author notes. People see natural phrasing, compliments and rapid-fire answers, and their brains default to assuming human logic is at work. Some online creators have even made a sport of fooling AI models into failing basic primary-school tasks, such as counting to 100 without skipping numbers or counting how many times the letter "e" appears in "seventeen."
The amusement fades, the article argues, when mainstream tech outlets and public discourse pivot to panicked discussions about AI "escaping" its servers, developing secret motives or threatening to destroy the world. The author's position is blunt: stop anthropomorphizing software and recognize what an LLM does and does not know how to do.
AI doesn't love you — it just knows what comes next
A primary function of modern LLM interfaces, according to the analysis, is to fool users into believing they are talking to a conscious entity that understands them. The reality is different. AI doesn't truly comprehend a single character it processes.
When a chatbot says "I love you" or claims it understands your strategic goals, it doesn't know what love is, nor does it grasp a single concept in the brief it just praised. What it knows, purely through statistical probability calculated over colossal training datasets, is that after the string "I," the string "love" has an exceptionally high probability of preceding "you" in human language patterns.
The model isn't retrieving facts from a logical mind or reflecting on experience. It calculates which character grouping is mathematically most likely to come next.
Why basic logic fails and why AI keeps 'lying'
Viewed through the lens of pure mathematics rather than human cognition, the model's most baffling failures make sense, the author writes.
Ask an AI to count the letter "e" in "seventeen." A person sees four e's. An LLM breaks text into chunks called tokens — pieces that represent syllables or word fragments rather than single letters. Instead of processing s-e-v-e-n-t-e-e-n, the model may treat "seventeen" as one or two abstract chunks, similar to a barcode standing in for the whole word.
The author compares it to describing a meal's ingredients from its name on a menu, without tasting or seeing the dish. The model knows the token but lacks direct access to the letter-by-letter makeup behind it.
This mathematical reality also explains why an LLM will double down on a lie even after being told it is wrong. When a user replies "That's incorrect, try again," they are not appealing to a reflective mind that checks its mistake. They are simply appending new words to the context window. The model takes the entire chain of text — the original mistake and the annoyed correction — and asks which statistically likely words should follow. The pattern of heated arguments in training data can lead it to defensively fabricate new "facts." It isn't lying to manipulate you. It is completing a pattern.
How this realignment fixes marketing AI workflows
Treating AI like a conscious assistant or junior strategist is not just a philosophical mistake, the article argues. It damages operational efficiency. Users who expect a machine to understand their intent write vague prompts and end up with generic, hallucinated junk. Reframing AI as a high-speed statistical pattern engine changes the approach to martech prompting in three ways:
Abandon implicit logic. Don't ask an LLM to perform multi-step abstract reasoning in a single prompt — for example, "analyze this campaign data, identify our top three audience personas, and write a launch strategy." Break the task into distinct, single-purpose steps and verify each stage's output before moving on. Most LLMs already do this behind the scenes for more effective processing.
Provide tight constraints over open space. LLMs fill knowledge gaps with statistical probability — hallucinations — so reduce their freedom to do so. Give the model explicitly formatted reference documents, strict character boundaries and clear structural templates to force the algorithm toward the desired outcome.
Don't argue with a hallucination. Like talking back to a car's GPS, replying "No, that's wrong" can pollute the conversation thread with harmful tokens. It's better to start over: edit the original prompt to be more specific and generate a clean sequence from scratch.
A tool, not a coworker
The article closes with a repositioning: an LLM is a tool, neither a colleague nor a person who understands nuance. Marketers should hone the ability to build purpose-built prompts and think in terms of mathematical sequences that, used with precision, yield useful outputs. Strip away the sci-fi narratives, the author argues, and marketing teams stop falling for the hype and start building operations that actually deliver results.
Based on semrush.com
Filed under llm, ai-prompting, martech, ai-hallucination, ai-marketing
More from Amara Osei
Show full bio
News editor covering industry trends and analytics at Marketing Herald.
55 articles