TL;DR
Three real brand case studies show where AI content creation lands. The Great: Tomorrow Sleep used MarketMuse to find content gaps and grew organic traffic 10,000%. The Good: Denmark's TV 2 Fyn used ChatGPT as a headline sparring partner across 46 A/B tests. The Ugly: CNET let AI write whole stories and had to correct 41 of 77. The throughline is that AI is your first step, not your last, and humans stay in the loop.
The second-most common use of AI is writing content for marketing material, right after idea inspiration. 28% of marketers who are using AI, leverage the tool to write materials such as blog posts and emails.
But as a marketer, you need answers to more crucial questions – what ROI does AI bring when used to create blog posts? Can the entire process be automated by AI? Considering the risks associated with using AI for content creation, what pitfalls should one avoid?
The good news is that you don’t need to make your own mistakes while answering these questions, thereby (hopefully) preventing a loss of money or reputation. We analyzed three case studies on how brands use AI in marketing. Here are our key takeaways:
- AI has not progressed to the level of autonomy - Human intervention is vital.
- Use AI as the first step in content generation, rather than the last step.
- AI still requires an entry of the customer profile and tastes, which should come from a marketer’s understanding of their audience.
- While AI can be useful in processing large datasets, it falls short when trying to generate content.
- Garbage In, Garbage Out (GIGO) - The quality of input determines the AI output.
Read the case studies for a deeper dive into our findings.
Case - 1: The Great
The problem
When a US-based mattress brand, Tomorrow Sleep, started creating online content, they realized that long-standing players in the market had already established a strong foothold in organic results. While traditional measurement tools for tracking keywords and ranking were useful, they weren’t enough to drive meaningful and drastic impact.
The approach
Tomorrow Sleep’s marketing agency first identified primary topics and analyzed them with MarketMuse, an AI-powered content strategy platform. AI found related topics to consider and how frequently experts mentioned them when writing on these subjects. The marketing team then drilled down into the top 20 search results for those primary topics, to visualize the gaps and opportunities for content creation. The heatmap displays the topic distribution (how frequently mentioned) across each piece of content.
This strategy provided insights critical to creating new, in-depth content on existing topics that could help establish Tomorrow Sleep as an expert. Tomorrow Sleep also used its AI tool to see where competitors ranked for each topic. This allowed them to identify gaps and opportunities in their content plan.
The result
- 10,000% increase in website traffic – from 4K per month to 40K monthly visits.
- Multiple rankings in a single search results page.
- Domain authority to secure Google’s featured snippet for specific results.
- Out-ranking their largest competitor (Casper) for primary topics.
The learning
Machines (AI) can give you a great starting point to develop a superior content strategy, by processing large volumes of SEO data and identifying opportunities for content creation. In some instances, it can help you weed out mediocrity and poor performers. And while it can provide a game plan, you still require creativity and originality to stand out in the market.
Case- 2: The Good
The problem
TV 2 Fyn, one of Denmark's regional television stations, wanted to explore if their writers could save time by asking AI to read their articles and create catchy headlines.
The approach
For three weeks the marketing team continuously carried out A/B tests on their website with headlines generated by ChatGPT and humans. They fed ChatGPT both full articles and summaries of articles. If the first suggestions by AI didn't work for the team, they continuously engaged with the chatbot and asked for refinement. For example, specifying that a certain person or a certain word from the article should be included in the headline. They found that AI was generally responsive to the suggestions, but in some cases, it ran into a wall and stopped developing further on the headlines. In those cases, the human team refined the headline before A/B testing. A total of 46 A/B tests were made during the test period.
The results
The learning
Using ChatGPT can make A/B testing easier. Although ChatGPT's bid on headlines varies greatly, it gives the advantage of a sparring partner with a short response time. Instead of having to spend 15-20 minutes developing ideas with colleagues, you can easily get suggestions based on the content of the article. However, you cannot count on the AI to deliver perfect headlines every time, so you still need to keep a critical eye on AI suggestions. For the best quality work, provide the AI with as much information on the consumer’s needs and wants. Refine and edit the AI suggestions further based on your unique understanding of the customer.
Case- 3: The Ugly
The problem
The Editor of CNET, an American media website, said they used AI-generated content as an “experiment” to assist reporters in their work.
The approach
It was a simple method that they tried. CNET used AI to generate entire stories, with reportedly little to no human editorial intervention.
The result
- More than half of the AI-generated stories contained factual errors, leading CNET to issue sometimes lengthy corrections on 41 out of its 77 bot-written articles.
- The quality of content was on the weaker side, with The Washington Post saying - “It’s … robotic: serviceable but plodding, pocked by clichés, lacking humor or sass or anything resembling emotions or idiosyncrasies.”
- AI-generated content appeared to have plagiarized work from competing news outlets resulting in copyright issues.
The learning
AI platforms generally source data from existing content on the internet. So, it tends to be less effective in categories with a unique product/service or a need for originality. This showcases the need for human intervention to help stay competitive. AI can be a great sounding board and an opportunity to overcome writer’s block, but it’s not the end solution.
Which examples of AI in marketing helped shape your AI content writing strategy? Let us know by writing to us.
Frequently Asked Questions
Should brands use AI for content creation, or does it cause more problems than it solves?
Brands should use it, but as a starting point rather than a finished product. The case studies here show AI shines at processing large datasets and surfacing SEO and content gaps, and falls down when asked to write polished, original content unsupervised. The pattern across all three brands is the same: AI as the first step, humans for the last.
How did Tomorrow Sleep grow its organic traffic so dramatically with AI?
Tomorrow Sleep's agency used MarketMuse, an AI content strategy platform, to identify primary topics, map how often experts mentioned related subtopics, and analyze the top 20 search results for gaps. That insight fueled deeper, expert-level content that outranked competitor Casper and earned featured snippets. The reported result was a 10,000% jump in organic traffic, from roughly 4,000 to 400,000 monthly visits.
What did the TV 2 Fyn ChatGPT headline experiment actually prove?
The Danish station ran 46 A/B tests over three weeks, pitting ChatGPT-generated headlines against human ones and refining the AI's suggestions when they stalled. The takeaway was that ChatGPT works well as a fast sparring partner, cutting the 15 to 20 minutes a team spends brainstorming headlines, with the winning AI headlines linked to a roughly 59% lift in click-through rate. It did not reliably produce perfect headlines on its own, so human editing stayed essential.
Why did CNET's AI-written articles go so wrong?
CNET used AI to generate entire stories with little to no editorial oversight, and it backfired. More than half were flawed: the outlet issued corrections on 41 of its 77 bot-written articles, some lengthy, and several passages appeared to plagiarize competing outlets. The Washington Post called the output robotic and plodding, which is exactly what happens when you skip the human layer.
Can AI fully automate the content creation process end to end?
No, not at its current level. Every case study here points to the same limit: AI has not reached the autonomy to run content unsupervised, and it still needs a marketer's understanding of the customer fed in as input. Garbage in, garbage out, so the quality of your brief and your edits determines the quality of the output.
Sources
- HubSpot AI Trends for Marketers Report: writing copy for marketing content such as blogs and emails is among the most common uses of generative AI by marketers, second to generating content ideas
- Marketing AI Institute / MarketMuse case study: Tomorrow Sleep boosted organic traffic 10,000%, from about 4,000 to 400,000 monthly visits, and outranked competitor Casper for primary topics
- SEO.ai AI content performance case studies: TV 2 Fyn A/B tested ChatGPT-generated headlines against human ones over a three-week period, linked to a 59% increase in click-through rates
- Engadget: CNET issued corrections on 41 of its 77 AI-written articles, with some requiring substantial fixes and others flagged for phrases that were not entirely original (plagiarism)
