User-agent: * # Clean Path Architecture # This site uses path-based routes for better SEO: # - Gender filters: /shop/t-shirts/for/women/ # - Color filters: /shop/t-shirts/color/black/ # - Product pages: /shop/t-shirts/for/men/product-slug/ # These are indexable and provide clear hierarchical structure # Protected & System Paths # Block internal APIs (except robots & sitemap) Allow: /api/robots Allow: /api/sitemap Disallow: /api/ # Tracking & UI Parameters # Block to prevent crawl waste Disallow: /*?*utm_source=* Disallow: /*?*utm_medium=* Disallow: /*?*utm_campaign=* Disallow: /*?*gclid=* Disallow: /*?*fbclid=* Disallow: /*?*sid=* Disallow: /*?*view=* Disallow: /*?*per_page=* Disallow: /*?*price_min=* Disallow: /*?*price_max=* # Multi-Select Parameters (Exponential Crawl Trap) # Block comma-separated values to prevent 2^N combinations Disallow: /*?*color=*,* Disallow: /*?*size=*,* # Calendar Date Patterns # Prevent infinite date combinations Disallow: /calendar/*?date=* Sitemap: https://seo-workshop.vercel.app/sitemap.xml