Discover website URLs, feed entries, and latest publications by checking sitemap.xml, sitemaps.xml, atom.xml, and rss.xml before crawling a specific site. Use when Codex needs to find the most recent posts/articles/publications from a named website or domain, search, crawl, scrape, monitor, or enumerate site content, and should prefer the site's own sitemap, Atom feed, or RSS feed over blind link crawling.
Discover website URLs, feed entries, and latest publications by checking sitemap.xml, sitemaps.xml, atom.xml, and rss.xml before crawling a specific site. Use when Codex needs to find the most recent posts/articles/publications from a named website or domain, search, crawl, scrape, monitor, or enumerate site content, and should prefer the site's own sitemap, Atom feed, or RSS feed over blind link crawling.