SEO Then & Now: Marshall Simmonds on AI, Dark Traffic , LLM.txt & the Future of Search 🚀

The SEO Podcast: Page 2 Podcast Hosted by Jon Clark & Joe DeVita

https://page2pod.com - What happens when the gatekeepers of search stop sending traffic, but still take the content? In this milestone Episode 90, Marshall Simmonds (Defined Media Group) joins us to trace SEO’s journey from white-text-on-white-backgrounds to log-file analysis, AIO, and LLM.txt—plus what publishers must do right now. He even shares the wild story of spinning up a mirror test server so Google (and others) could pound on About.com to refine their crawlers.

We dig into why watching server logs matters more than ever, how “dark traffic” is back, and why author authority and structured content still win—despite shifting consumption and murky attribution in AI-assisted search.

Marshall also takes us behind the scenes on unlocking The New York Times archive (and why it crushed for traffic and revenue), plus a pragmatic take on AIO monetization, blocking AI training vs. real-time access, and the early but evolving role of LLM.txt.

🧠 In This Episode:
• The About.com “test server” era: letting engines crawl a mirror site to improve their bots—what Marshall learned by “watching the watchers.”
• Why log files are mission-critical again as dark traffic surges and LLM crawlers multiply.
• Enterprise-grade log tooling & options: Sitebulb, Screaming Frog, Splunk—what to use when.
• Crawl budget signals hiding in plain sight: the Search Console “Crawl stats” report.
• NYT archive strategy: opening history for scale, membership, and revenue (and why it worked).
• E-E-A-T before the acronym: building human authority into content and authorship.
• LLM.txt today: yes, it’s crawled; no, standards aren’t settled; how teams are exposing it.
• AIO reality check: impressions up, clicks down; what’s converting (and why patience pays).
• Blocking AI training vs. real-time use: robots.txt limits, legal gray zones, and first-line defenses.
• Holistic marketing > channel silos: brand, email, social, search, Discover—working together.

If you've ever wondered where SEO has been and where it's headed, this conversation with Marshall Simmonds is essential listening.

📌 Subscribe for more insightful episodes on SEO, AI, and digital strategy.
💬 Comment below: How are you adapting your SEO strategy in the age of AI and dark traffic?

🔗 Links & Resources Mentioned
• Marshall Simmonds LinkedIn
• Defined Media Official Site
• Page 2 Podcast Noah Learner episode

Sponsored by Moving Traffic Media

Follow the Page 2 Podcast

• Page 2 Podcast on Youtube
• Page 2 Podcast on Linkedin
• Page 2 Podcast on Facebook

Follow Jon Clark

• Jon Clark on Bluesky
• Jon Clark on Linkedin