# robots.txt for Adzuna (except UK, US) User-agent: Amazonbot Disallow: / User-agent: * Disallow: /search? Disallow: /jobs/search? Disallow: /to-rent? Disallow: /for-sale? Disallow: /goto/ad/ Disallow: /jobs/goto/ad/ Disallow: /land/ad/ Disallow: /jobs/land/ad/ Disallow: /advanced-search? Disallow: /jobs/advanced-search? Disallow: /jobs/my-alerts? Disallow: /my-alerts? # Disallow /create_notification endpoint from being accessed by the AdsBot # https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers User-agent: AdsBot-Google User-agent: AdsBot-Google-Mobile Disallow: /create_notification