User-agent: * Disallow: /offer/_detail/ Disallow: /offer/*/line/ Disallow: /offer/16/ Disallow: /offer/23/ Disallow: /offer/24/ Disallow: /offer/28/ Disallow: /offer/29/ Disallow: /offer/30/ Disallow: /offer/46/ Disallow: /offer/47/ Disallow: /offer_info/ Disallow: /entry/ Disallow: /entry_signup/ Disallow: /entry_signin/ Disallow: /delegate/ Disallow: /password_forget/ Disallow: /password_reset/ Disallow: /users/ Disallow: /facility/ Disallow: /agent/ Disallow: /stop/ User-agent: ia_archiver Disallow: / User-agent: Baiduspider Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: BaiduMobaider Disallow: / User-agent: BaiduImagespider Disallow: / User-agent: YandexBot Disallow: / User-agent: DotBot Disallow: / User-agent: PetalBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: Bytespider Disallow: / User-agent: barkrowler Disallow: / User-Agent: ImagesiftBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: SEOkicks Disallow: / User-agent: SemrushBot Disallow: / User-agent: TikTokSpider Crawl-delay: 10 User-agent: bingbot Crawl-delay: 10 User-agent: msnbot Crawl-delay: 10 User-agent: AhrefsBot Crawl-delay: 10 User-agent: perplexitybot Crawl-delay: 10