The past, present, and future of robots.txt

Dec 21, 2021 · 53m
The past, present, and future of robots.txt
Description

In this episode of Search Off the Record, Gary, Lizzi, and Martin from the Google Search team are joined by special guest David Price, a Product Counsel for Google. They...

show more
In this episode of Search Off the Record, Gary, Lizzi, and Martin from the Google Search team are joined by special guest David Price, a Product Counsel for Google. They discuss human initiated crawl requests vs. bot initiated crawl requests, the robots.txt file, and more!   Episode transcript → https://goo.gle/sotr028-transcript    Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
show less
Information
Author Musonera Jean Claude
Organization Musonera Jean Claude
Website -
Tags

Looks like you don't have any active episode

Browse Spreaker Catalogue to discover great new content

Current

Podcast Cover

Looks like you don't have any episodes in your queue

Browse Spreaker Catalogue to discover great new content

Next Up

Episode Cover Episode Cover

It's so quiet here...

Time to discover new episodes!

Discover
Your Library
Search