Humanity’s last exam: Can AI survive the ultimate challenge?

The tools and strategies behind rapid scaling

Author

Ryan Estes
January 27, 2025

Hi ,

Podcast advertising is the best way for businesses to connect with hard-to-reach prospects.

Wildcast helps you find the riches in the niches.

Click the big red button to learn more:

(and Wildcast pays up to 10% commission for referrals.)

let’s dive in…

Tip of the spear:

  • International researchers have created a benchmark named "Humanity's Last Exam" to evaluate the limitations of large language models, where even the most advanced AI systems currently fail 90% of the time.

  • The exam consists of 3,000 questions across over 100 specialized fields, with a significant focus on mathematics, developed with contributions from nearly 1,000 experts from 500 institutions worldwide.

  • AI models like GPT-4o and OpenAI's o1 struggle with this challenging test, and the models often exhibit overconfidence with calibration errors over 80%, highlighting the gap between their confidence and accuracy.

  • OpenAI has introduced "Operator," a web automation tool powered by the new Computer-Using Agent (CUA) model, which manages computers through a visual user interface similar to human interaction.

  • Currently, Operator is accessible to ChatGPT Pro subscribers for $200 monthly, with future plans to expand availability to Plus, Team, and Enterprise users, and eventually integrate these features into ChatGPT and its API.

  • The system works by capturing screenshots to understand the computer environment and then uses AI to decide and execute actions, enabling it to manage complex tasks across various applications.

Meta in panic mode as DeepSeek gains traction LINK

  • Meta employees are in "panic mode" as leaked internal discussions reveal growing anxiety over DeepSeek's success and the company's bloated AI organizational structure..

  • DeepSeek's open-source AI model, developed with just $5.5 million, has outperformed Meta's much more expensive solutions on third-party benchmarks.

  • The free MIT-licensed availability of DeepSeek's models poses a direct challenge to Meta's massive AI investments and traditional development approach.

In partnership:

Time to supercharge your podcast, and grow your base with your personal Content CoPilot.

Introducing Clik- an AI-powered creative workflow tool built for podcasters like you. Clik understands your visual content like a human does, allowing you to search your podcast archive by what you see or hear in a video. Instead of hiring a social media intern to dig through video, Clik can now do it for you. Clik helps you effortlessly repurpose your podcast archives into engaging short-form content to increase brand visibility and audience engagement.

The best part - Clik is offering a one-month free trial to Ryan Estes’s dedicated community.

Signup here and try for yourself!

Keep it moving:

  • Open source alternatives to Instagram, TikTok, and WhatsApp raise funds on Kickstarter LINK

  • Engineers plan to turn moon dust into oxygen with new system for lunar bases LINK

  • Apple admits next-gen CarPlay is late, but still in development LINK

  • AI-developed drugs are coming LINK

  • Researchers optimize simulations of molecules on quantum computers LINK

  • LinkedIn sued for using private messages in AI training LINK

  • Elon Musk and Sam Altman clash over Stargate on social media LINK

  • Donald Trump pardons Silk Road creator LINK

  • Meta is building Oakley smart glasses for athletes LINK

  • Instagram offers big bonuses to attract TikTok creators LINK

  • Bitcoin surpasses $109,000 as market watches Trump LINK

What I’m thinking about:

New tools:

Dreamteam IQ: helps marketing agencies and startups staff top overseas talent, providing cash-saving solutions for roles on elite marketing teams. LINK

FivePointFive: Transform your wellness instantly with FivePointFive’s science-backed breathwork, personalized biometric insights, and music from your favorite artists. LINK 50% OFF and 3 days free

Clik: Clik helps you effortlessly repurpose your podcast archives into engaging short-form content to increase brand visibility and audience engagement. LINK

Wildcast: B2B influencers for the world’s top technology brands. LINK

Foundations of Large Language Models: the paper explains the core principles, architecture, and challenges involved in building and using large language models. LINK

Lindy: Build AI agents in minutes to automate workflows, save time, and grow your business. LINK

Replit: On-demand coding assistance for self-guided projects LINK

Thanks for listening.

-Ryan

p.s. When you are ready, here are two ways I can help.


1. We have successfully booked podcast interviews for over 800 funded startup founders, entrepreneurs with exits, and C-suite executives. If you're ready to lead your company from the front, let Kitcaster handle your podcast scheduling and create a powerful stream of content for you and your brand.

2. Our sister company, Wildcast, connects B2B, Tech, SaaS, and Business creators with companies targeting hard-to-reach customers. We deliver the most relevant online audiences through branded conversations, host-read podcast ads, and 360 sponsorships for tech companies aiming to scale their marketing. Start with a free audience psychographic and opportunity assessment tailored to your Ideal Customer Profile (ICP).

I write about Revops, Product, and Founder-led marketing on Linkedin, Twitter, and my blog. 

And if you are in Denver, we have the #1 culture and discovery podcast dedicated to the Queen City of the Plains —> realgooddenver.com