Digital Innovation Digital Publishing Top Stories
2 mins read

Errors in Men’s Journal show the perils of using AI for publisher content

Arena Group’s foray into using AI for Men’s Journal ends up in controversy as the publisher makes a “cascade of changes” to an AI-generated testosterone article, correcting factual errors pointed out by a licensed medical doctor.

A feature on highlights the dangers of using AI as the lead source for publisher content. The controversy surrounds an article in Men’s Journal entitled What All Men Should Know About Low Testosterone which was seized upon by Futurism and given to Bradley Anawalt, the Chief of Medicine at the University of Washington Medical Center, to evaluate.

Anawalt’s critique was scathing, pointing out that the article contained “persistent factual mistakes and mischaracterizations of medical science that provide readers with a profoundly warped understanding of health issues”.

There is just enough proximity to the scientific evidence and literature to have the ring of truth but there are many false and misleading notes.

Bradley Anawalt, Chief of Medicine at the University of Washington Medical Center speaking to

As a result of the critique, Men’s Health made widescale changes to the article, correcting various factual errors. A message also appeared at the footer: “The original version of this story described testosterone replacement therapy as using ‘synthetic hormones’ and stated poor nutrition as one of the most common causes of low T, which are inaccurate.”

However, the assertion that poor nutrition was a “factual error” does itself raise questions as the AI behind Men’s Journal AI had correctly scraped University of Kentucky research which found a clear correlation between unhealthy diets and testosterone levels.

According to MediaPost, the Arena Group – which also publishes Sports Illustrated – had previously announced that Men’s Journal used AI for its “Men’s Fitness” section, producing articles such as “Proven Tips to Help You Run Your Fastest Mile Yet” and “The Best Ways for Men Over 40 to Maintain Muscle.”

Overall, the AI test demonstrated clear success in terms of page metrics, referrals and shares, and revenue performance. It had also increased workflow efficiencies by more than 10 times the normal rate, the company claimed.

In a recent op-ed in WNIP, United Robot’s Cecilia Campbell reminded publishers that AI is not in control, rather the publisher is, adding:

AI can help improve our work processes, but it cannot produce journalism.

Indeed, all the evidence so far points to AI being a great first officer, but a lousy Captain.