WP Functions

Block search result URLs in robots.txt

Stops search engines from crawling internal search results, including spam searches from bots.

Hook
robots_txt
Updated
2026-09-27
License
GPLv2

Works with the robots.txt WordPress generates. If a robots.txt file exists in the site root, add the two Disallow lines there instead.

add_filter( 'robots_txt', function( $output, $public ) {
    if ( ! $public ) {
        return $output; // Site already hidden from search engines.
    }
    $rules = "Disallow: /*?s=\nDisallow: /search/\n";
    if ( preg_match( '/^User-agent: \*\R/mi', $output ) ) {
        return preg_replace( '/^User-agent: \*\R/mi', "User-agent: *\n" . $rules, $output, 1 );
    }
    return "User-agent: *\n" . $rules . "\n" . $output;
}, 20, 2 );
Open in the builder

Licensed under GPL-2.0-or-later, provided as is. Review and test it before using it on a live site.

How it works

Every search on a WordPress site has its own URL, like /?s=shoes or /search/shoes/. Spam bots create thousands of them, and search engines spend their time crawling those pages instead of your content. When there is no robots.txt file in the site root, WordPress generates one, and the robots_txt filter lets you add rules to it. The snippet adds two Disallow lines under User-agent: *.

Good to know

  • If Discourage search engines is ticked in Settings → Reading, the snippet does nothing.
  • WordPress already marks search pages as noindex. Blocking them in robots.txt saves crawling, but Google can’t see that noindex anymore, so a search URL linked from another site may still show up in results, without a description.
  • If your site has a physical robots.txt file, add the two lines there instead.

How to check it works

Open /robots.txt on your site. You should see Disallow: /*?s= and Disallow: /search/.

How to undo it

Remove the code.

How to add this snippet

  1. Copy the code above.
  2. Paste it at the end of your child theme’s functions.php, in Appearance → Theme File Editor or over FTP.
  3. Save, then open the site and check that everything works. If something breaks, remove the code.

Need more than one? Open this snippet in the builder, turn on the others you want and copy them as one functions.php, or download them as a small plugin.

More in SEO