Confirmation that two-thirds of headline analyzers spout utter nonsense

On the grounds that headlines are all-important for getting potential readers to actually click through to read the rubbish I write, I’ve been spending more time lately refining them than I used to. Making sure the spelling’s correct, trying to get the length right, and including keywords, positive words, emotive words, ‘calls to action’ and so on.

This is all based upon the kind of advice that’s quite commonly bandied about, which tends to look like:

• keep it short (8-14 words)
• restrict the number of characters to around 70
• focus on ‘who’, not ‘why’
• be specific
• put keywords near the beginning – or the end
• consider adding a [bracketed qualifier]
• try to optimise for your audience, search and social media

(That list is mostly a summary of an article by Corey Wainwright called ‘How to Write Catchy Headlines and Blog Titles Your Readers Can’t Resist‘ – I’d offer a hat tip to ‘The Nerdy Lion’ for providing me with that link, but his website seems to be having problems at the moment.)

The thing is: although I consider the extra effort worthwhile from the point of view of crafting a more well-rounded post, I don’t think that it’s made the slightest difference to my blog’s traffic.

Dr Bob Rich put me on to the Advanced Marketing Institute’s ‘Headline Analyzer‘ some time ago, and I’ve been using that (thanks, Bob!). It’s undoubtedly been useful in getting me to think more about the words I choose. But, while it’s useful for honing headlines – I tend to agree with its assessments when it offers different values for similar alternatives – I’ve come to realize that the ‘ratings’ it generates are more than a little bit suspect. For one thing, it provides percentage values, which imply some sort of scientific basis, and these are to two decimal places, suggestive of a high degree of accuracy; yet a suspiciously large proportion of the results I’ve seen end in ‘.00’. For another, there’s a high occurrence of duplicate numbers such as ‘22.22%’ and ‘33.33%’ (those two, in particular, appear far too frequently).

A while ago, I even stumbled on one headline that had scored a nonsensical ‘125.00%’. Unfortunately, I can’t now recall what that headline was. I used the form on the tool’s website to send them a query about it, and neglected to take a copy; I didn’t get a response. I’d pretty much forgotten about it until recently; when working on the post ‘A unique opportunity (time-limited offer!)‘ – its headline scored exactly ‘100.00%’ (!), and it was that that caused me to smell a rat.

Disillusioned, I went searching on DuckDuckGo for alternative tools, and then spent some time comparing their results. What I discovered was quite interesting….

A comparison of three – free! – headline analyzers

1. Advanced Marketing Institute (‘AMI’)

https://www.aminstitute.com/process/headline.cgi

Pros:

• no login
• fast
• simple

Cons:

• minimalistic
• suspicious results

2. isitwp

https://www.isitwp.com/headline-analyzer/

Pros:

• no login
• good suggestions

Cons:

• mostly quick but sluggish at times
• when displaying the headline, Irritatingly Transforms Each Word To An Initial Capital

3. CoSchedule

https://coschedule.com/headline-analyzer

Pros:

• retains a history of previous entries
• comprehensive suggestions (although perhaps over-the-top!)

Cons:

• registration required (the data requested implies that only companies are invited, but nonsense alias information was accepted)
• insistent nagging to ‘upgrade to premium’ (paid plan, and not cheap)

The results

I ran a couple of dozen of my most recent post titles past each of these three ‘headline analyzers’. None of them agreed on their assessments of the headlines I offered them. One that CoSchedule rated in the top three, the other two rated in the bottom three; one that AMI rated second best (85.71%), isitwp rated as worst (42), while CoSchedule gave it a noncommittal value (57).

And none of the results, as far as I could determine, bore any correlation at all to any post’s traffic, in terms of views, likes or comments.

When I took my five best-performing posts (in terms of views) in the last year and fed their titles to these tools, not one of them came up with a particularly high rating, and their average scores were much the same as the average of those I’d examined earlier.

Then, just for fun, I tried the first two lines of Lewis Carroll’s Jabberwocky:

Twas brillig, and the slithy toves did gyre and gimble in the wabe

Analyzer	Rating	Comments
AMI	0.00%	‘Neutral’
isitwp	48	‘positive sentiment’, ‘more likely to get clicks if it had more uncommon words’ (really?)
CoSchedule	70	‘easy for most readers to comprehend’, ‘message is clear and concise’ (wait, what?)

Analysis of the first two lines of ‘Jabberwocky’

What started out as just a quirky idea ended up confirming my suspicions that these so-called ‘analyzers’ are little more than self-absorbed nonsense generators. Only ‘AMI’ offered anything resembling an honest judgement of this particular ‘headline’.

And here are the results for the title of this post:

Analyzer	Rating	Comments
AMI	22.22%	‘Empathetic’
isitwp	48	Your headline has a negative sentiment.
CoSchedule	65	‘Nice work! Your headline’s message is clear and concise.’

Analysis of ‘Confirmation that two-thirds of headline analyzers spout utter nonsense’

Oh, look, there’s that ‘22.22%’ again. I think that pretty much sums it all up.

Don’t get me wrong: I still think that headlines matter. I intend to continue using the ‘AMI’ tool (unless, perhaps, you’re able to suggest a better one?); mainly because it’s fast, totally free, and doesn’t take itself as seriously as its competition. I believe that, in the blogosphere at least, what’s more important than catchy headlines is: good connections with other bloggers.

As a counterpoint, though, I offer a highly thought-provoking video from Veritasium on his experience with titles on YouTube:

Veritasium: ‘Clickbait is Unreasonably Effective’

Analyzer	Rating	Comments
AMI	50.00%	‘Intellectual’
isitwp	45	Your headline is too short.
CoSchedule	47	‘Nice work! Your headline’s message is clear and concise.’

Headline analysis of ‘Clickbait is Unreasonably Effective’

Veritasium: Can I tell you something I’m bad at? I am terrible at making clickbait. Up until two years ago, my most popular video was about a basketball being dropped from a dam with a bit of backspin. It takes off like a rocket and shoots out way farther than you’d expect. This video was embedded on literally hundreds of news websites, and in its first week it got 16.3 million views. But almost none of those views came on the YouTube platform. Why? Because I gave it this thumbnail and I called it ‘Strange Applications of the Magnus Effect’.

(This video is sponsored by KiwiCo. More about them at the end of the show.)

Now I believe within YouTube, I’m used as an example of how not to package your video. As if to demonstrate just how bad a title and thumbnail this is, someone else re-uploaded the video with the clever title ‘Basketball Dropped From Dam’, and within a few weeks that video had received tens of millions of views on YouTube. This was when YouTube gave me access to Content ID; that’s the system that allows you to earn revenue when someone else re-uploads your videos, which was good, but I still wasn’t very good at clickbait. So for this video, I called in an expert.

Veritasium: I wanted to see your reaction face. Can you give me like a reaction face? Like, what’s a good thumbnail face?

Mr. Beast: Oh, I got you. This is a thumbnail face. I have like 10, I go through the motions of like we just pick the best one and probably have like five hundred of these just saved and we can just photoshop my face on my face; then I don’t really have to do it anymore.

Veritasium: When I started on this platform some 10 years ago, clickbait was less important. Subscribers were what mattered because most of the views came from the sub feed and videos went viral, usually by getting a lot of attention elsewhere – like on Reddit or Facebook – not due to the YouTube algorithm, but once you had a big video and people subscribed, well, then your next video would likely get a lot of views from those subscribers. So YouTube would take that popular video and share it with more people, and so you’d get more subscribers, in a positive feedback loop.

But soon, YouTube realized that this did not create the best experience for the viewer. They discovered that showing people only videos from channels they were subscribed to led to fewer clicks, less watch time and less engagement with the site as a whole. Plus, they knew that relying on other platforms to drive traffic to viral hits was risky, since those other platforms might disable the traffic at any time. So they needed to make YouTube a destination in itself. They wanted people to come to YouTube, see videos that interested them, click on at least one of them and watch it for a while. Ideally, get sucked down the YouTube rabbit hole and spend hours on the site without even noticing it.

The ultimate resource is people’s time and attention, and every platform is trying to capture as much of it as possible. So to make YouTube this go-to destination, they had to decrease the importance of subscribers, make it less like a podcast app where you only get the shows you’re subscribed to, and more like Reddit, where stuff with the most engagement rises to the top. But that necessarily meant increasing the importance of clickbait.

Now, there seems to be a paradox when it comes to clickbait, people almost universally claim to hate it. But you also see it everywhere. So why is this? Well, one of the problems is we don’t all agree on the definition of clickbait. When I google it, the top definition is, ‘[on the Internet] content whose main purpose is to attract attention and encourage visitors to click on a link to a particular web page’. We could call this ‘Type I Clickbait’, and there doesn’t seem to be anything wrong with it. I mean, if you didn’t try to attract attention and get people to click on your links, then you wouldn’t really be doing your job. But there is a second definition, one that I think more people ascribe to, which is: ‘something [such as a headline] designed to make readers want to click on a hyperlink, especially when the link leads to content of dubious value or interest’.

Wikipedia says a defining characteristic of clickbait is that it is sensationalized or misleading, and it also talks about teasers that intentionally withhold information to exploit the curiosity gap. They give you enough information to make you curious, but not enough to satisfy that curiosity. Here are two actual titles from a news website: ‘9 out of 10 Americans are Completely Wrong About This Mind-Blowing Fact’, and ‘Someone Gave Some Kids Some Scissors. Here’s What Happened Next’. I think we can all agree that these are examples of bad Type II Clickbait.

Now, imagine a clickbait space where on one axis you have how misleading or sensationalized it is, and on the other, how much information is intentionally withheld to create a curiosity gap. Well, then these two titles fall in the top right corner, and these are the zones of Type II Clickbait. At the other extreme, you have things that are so unsensationalized as to be dull. You could call this the ‘Dead Zone’. Now here is where you would find ‘Strange Applications of the Magnus Effect’. I mean, I didn’t tell you what the applications were. Now, in the middle is where you would find Type I Clickbait. But honestly, I think Type I and Type II clickbait are so different that we shouldn’t even use the same word for them.

Instead of Type I Clickbait, my friend and YouTuber Brady Haran suggested ‘Legitbait’. I mean, it might sound enticing, but it is legit. Instead of Type II Clickbait, we could use ‘Clicktrap’, ‘Clicktrick’, ‘Linktrap’, or ‘Dupechute’. What’s important to recognize is that for any given video, there is no one true title and thumbnail. Each video could have hundreds or thousands of different ‘Legitbait’ titles: for example, ‘How does a zero-g plane work?’, ‘I went on a plane that does parabolic trajectories’, ‘What happens to FIRE on a zero-g plane?!’ Now, the most enticing titles and thumbnails are found close to Type II Clickbait. I’m reminded of the infographic by Smarter Every Day showing that on social media, the greatest engagement occurs close to the boundary of what’s allowable. But remember that everyone’s definition of clickbait is different, and everyone’s perceptions of words and images are different. So these are not clear boundaries, they’re actually kind of fuzzy; what for one person might be ‘Legitbait’, for someone else could be a ‘Clicktrap’. What’s clear is that on a site where clickthrough rate is important, clickbait of all types is inevitable.

Veritasium: How important to a video success is the title and thumbnail?

Mr. Beast: Very important, of course, if they don’t click on the video, they don’t watch it. You can’t get 10 million views unless 10 million people click on the video. So, I mean, it’s literally that simple. They don’t click on it, they don’t watch it.

Veritasium: So why is clickbait everywhere? Well, because it works. More enticing thumbnails get more clicks. Despite some people’s claims that they won’t click or they’ll unsub. It’s just like evolution; whatever survives, multiplies and traits become amplified. If you don’t begrudge the giraffe its long neck so it can reach the highest leaves, can you begrudge a YouTuber the big red arrow that allows them to reach further audiences?

I talked about this in my video two years ago. To be successful as a YouTuber, you need to optimize two things: watch time, and the clickthrough rate of your videos; that’s the number of times the title and thumbnail are clicked divided by the total number of times they’re shown; that’s the number of impressions. Now, at the time, this was something of a revelation for me because I always thought my job was to make great videos and then a title and thumbnail that adequately represented what the video was about. But now I’ve realized that making the title and thumbnail is at least half the job. This is not just because better titles and thumbnails get you more clicks, but because better titles and thumbnails will get you way more impressions.

YouTube has limited real estate with which to show you virtually infinite content, and so it’s not enough to make a good video even if people watch all of it. You also have to make a title and thumbnail that gets clicked, especially in competition with other really good titles and thumbnails. That’s the only way you can expect YouTube to give you more impressions.

Now, the big development since my last video on this is YouTube introduced real-time metrics like views, impressions and clickthrough rate, and I suggested this would create an arms race. So, what you can bet will happen is that creators will launch a video and then they’ll be sitting there with all these different variants of thumbnails, and they’ll be swapping them out and looking at what that does to clickthrough rate and then going with the one that leads to the greatest clickthrough rate. And this is basically what has happened.

Let me give you my favorite example. Last year, I made a video about asteroids, which I thought was really good. I called it, ‘Asteroids: Earth’s Biggest Threat’, which is something Stephen Hawking said; and people were very positive about the video. They thought it was maybe one of my best, but the performance was well below average. In its first day, ‘Asteroids’ was ranked ninth out of my previous 10 videos. It was probably on target for about one and a half million views, so I tried different titles and thumbnails like ‘Asteroid Impact: What Are Our Chances’, or ‘Asteroid Impact: What Could We Do?’ But none of these changes got much traction.

And then, on day three after launch, I changed the title and thumbnail to: ‘These Are The Asteroids To Worry About‘, and, immediately, the video started doing better. It quickly shot up from almost my worst performing video – to my best. It now has 14 million views. Nothing about the video changed, just that one image and 38 characters. But because of that, the video has reached nearly 10 times as many people as it otherwise would have. And the title and thumbnail accurately describe what the video is about. I mean, sure, there’s a curiosity gap, but you couldn’t explain the whole concept in the length of the title.

So, if you see a YouTuber changing titles and thumbnails, this is why; because that effort can be rewarded many times over. I’ve seen people objecting to this practice because they think the creator is trying to dupe their audience, get them to click on the same video more than once, but that’s not it. The whole point is to get YouTube to show the title and thumbnail to more people. We’re trying to increase the number of impressions, which is heavily dependent on the clickthrough rate.

Now, a lot of my recent videos have this typical view curve: there’s an initial spike after I release the video, and then a dip, and then a second bump after I have figured out a better title and thumbnail. I change the title and thumbnail and I watch the real-time view graph: what I’m looking for is a noticeable bump in views; sometimes there’s no change, sometimes it gets worse; but on occasions when you see something like this, well, then you know you’ve found a winner. This is something all the big YouTubers are doing. Not even Mr. Beast knows exactly which thumbnail will work best beforehand.

Veritasium: Have you ever changed a title in thumbnail, and then the video did better?

Mr. Beast: Oh, of course. So like everybody, I usually make like two or three thumbnails, and then if it’s not doing as well as I want, we usually just swap them out and see if it does better. The thing is like, you don’t really know. I mean, you could know if you just were an almighty being that could just predict what people would be interested in. But, you know, usually like you do hide and seek; you don’t really know if, like, you hiding in a tree and then someone walking below you is a good thumbnail, or if you’re hiding in a trashcan in the market in front of you. So, you know, just do both and then see which one interests people a little bit more.

Veritasium: I feel the same way, but I do feel like you have a better sense of this than, like, most people.

Mr. Beast: Of course, I mean, no one gets 40 million views on video.

Veritasium: But what is the point in getting more views? Well, if you’re cynical, you might say it’s all about money and fame. And while there are certainly financial incentives to getting more views, that’s not why I do it. As an educational YouTuber, I think there are two very good reasons for using excellent Type I Clickbait over more straightforward packaging.

To understand the first reason, let’s consider two different possible titles for my most recent video. I called it, ‘The Simplest Math Problem No One Can Solve‘, but it’s a video about the Collatz conjecture, so perhaps a more straightforward title would have been simply to call it that. The problem is, if I publish a video called ‘The Collatz Conjecture’, the most likely people to click on it are those who already know what the Collatz conjecture is, and the vast majority of people will never have heard of it. So for them, the title is meaningless and only the very curious or those who really like Veritasium would click. In contrast, calling it ‘The Simplest Math Problem No One Can Solve’ conveys more information about the video to everyone, and this means more people can click on it, most of whom will never have heard of the Collatz conjecture, so I get to teach them something new. And since the video has a higher clickthrough rate, YouTube shows it to even more people; so, if my aim is to increase the level of knowledge in the world by the maximum amount possible, this is the way to do it.

The second reason we need to optimize titles and thumbnails is to support the major goal of this channel. We are trying to make the best science films on every topic we tackle. That means traveling to meet experts and film experiments, hiring people to build props, make spectacular animations, research and fact check. We hire expert consultants to double- and triple-check our work. I don’t want to make vlogs; I want to make science documentaries on YouTube that put broadcast to shame; and to make this possible and sustainable, the videos have to get views and lots of them. And to do that, we have to make the best titles and thumbnails we can.

YouTuber and Patreon CEO Jack Conte has talked about ‘adjusting your packaging’: the idea is know what you’re passionate about, what you won’t compromise on; and that stuff goes in the box. All the rest, like what paper it’s wrapped in, that’s the packaging. So the video is my focus, and the title and thumbnail are the packaging that I’m happy to adjust so I can make the type of videos that are important to me. Now, is it ironic that a channel whose whole purpose is to promote a truth-seeking mindset has to experiment at the edge of what is truthful in order to fulfill that purpose? It is something that I often worry about.

Veritasium: When I did like risking my life to do X, it’s like, well, what probability of death does there have to be for you to risk your life? Like 50%?

Mr. Beast: I love how he always overthinks things. Funny. He’s like, is my life risked enough where I can put, ‘I risk my life’, you know? I only had a 9% chance of dying, and I need at least a 13% chance. He is so funny. He was the same way two years ago when he asked me all this stuff. I was like, dude, just do whatever makes the most sense.

Veritasium: Yeah, I mean, obviously, like, I feel like my instincts are not good…

But what I realized is that I don’t have to trust my instincts. This problem of getting the best title and thumbnail, well, it’s a scientific problem. We’re just asking which accurate representation of the video will get the most clicks from a general audience. And that’s a problem we can use the scientific method to solve. So I’ve hired a couple of really bright people, and we spend a lot of time brainstorming and making titles and thumbnails and testing them out.

For example, on Twitter and Patreon, Veritasium titles and thumbnails have gotten better; not because I’m better at it, but because of my team and our testing. If you have ideas about how we could do it better, please get in touch. The results have often contradicted what I expected. I mean, this video seemed to perform 10% better when we excluded the word ‘surprising’ from the title, so it became simply, ‘The Secret of Synchronization’. I thought these two titles were basically a toss up, but Patreon had a strong preference for one over the other.

What’s interesting about this research is that the more clickable titles and thumbnails often better represent the content of the video. Let me give you some examples. You know, one thing I didn’t expect when YouTube brought in the real-time analytics tools and allowed title and thumbnail changes to affect impressions was that it would also work for older videos. So, here is an old video that I originally entitled, ‘Why The Neutron Is The Hero of Nuclear Physics’. In hindsight, it’s a weird kind of meaningless title, so I changed it to, ‘Why Einstein Thought Nuclear Weapons Impossible’. It’s clearer, more accurate, and this is what happened to views after the change.

I changed ‘Strange Applications of the Magnus Effect’ to ‘Backspin Basketball Flies Off Dam’. Again, more accurate, and the video is now being watched on YouTube, probably 10 million more views than it otherwise would have had.

Even anti-clickbait has proven effective. Here’s a video I originally titled, ‘Are Negative Ions Good For You?’ which is a question nobody asked, ever. Now, I changed it to, ‘Do Salt Lamps Work?’. Seriously. And here’s the result. One and a half million more views.

I think, looking at these examples, you could easily make the case that YouTube has made me better at what I do. It has improved the clarity and accuracy of my titles and thumbnails to help me figure out what is interesting to my audience and how to encapsulate that in a single image in fewer than 50 characters.

For those of you who worry that a focus on titles and thumbnails will prevent me from picking challenging science topics, well, I simply ask you to look at the videos from the last year or two, like the ones on Gödel’s incompleteness theorem, general relativity, Penrose tilings, the logistic map, Newton’s method of calculating π, the one-way speed of light. Good titles and thumbnails make it possible to tackle these topics and to reach more people who have never heard of them before.

There is a symbiotic relationship between views and video quality. The more views we get, the more people, locations, props and equipment and research we can invest in. This makes the next video better than the last. If we can be good at titles and thumbnails, the ultimate outcome is better videos. That is something I think you and I both want.

Hey, this video was sponsored by KiwiCo, a long time supporter of the channel. Now, KiwiCo makes awesome hands-on projects for kids, including big kids like me. They have eight subscription lines targeted at different age groups all the way down to newborns. What I love about KiwiCo crates is that they make learning fun and something to look forward to. I mean, when I pull out a crate, my kids just jump at the chance to make it with me, and all the materials you need come right in the box, so there’s no need to run to the store: you just open it up and go. My kids and I just built this domino machine, which, who knows, might be inspiration for a viral video. Now, the big idea is that while playing and having fun, kids learn about ‘STEAM’ concepts; that is: science, technology, engineering, art and math. And in addition to the project, there’s a magazine with a lot of additional information. So, I think this is the best way to learn; by doing something yourself and having fun. It’s how education should be. So if you want to try it out, go to kiwico.com/veritasium50, and you’ll get 50% off your first month of any crate. I will put that link down in the description.

So, I want to thank KiwiCo for sponsoring Veritasium, and I want to thank you for watching.

Veritasium: ‘Clickbait is Unreasonably Effective‘

The transcript above was made with the help of Sonix, which did most of the donkey work for a tiny fee (I did have to spend some time tidying it up). Note that I do not have the copyright owner’s permission to publish this transcript here. I’ve investigated the copyright rules regarding transcriptions (more about that here), and one thing I’ve learned is that it’s no defence to make a disclaimer like “these aren’t my words, no copyright infringement intended.” However, I offer the transcription here as a service to society (especially the deaf community). I do hope the copyright owner won’t object. And I hope that you find this video as interesting as I did.

Header image adapted from
‘Charting Goals and Progress‘
by Isaac Smith on Unsplash