robots.txt vs noindex: which one to use
robots.txt tells crawlers which addresses they may visit. A noindex rule tells search engines not to keep a page in their index.
Use robots.txt when
you want to stop crawlers from wasting time on pages such as internal search results or filter combinations.
Use noindex when
a page may be visited but should not appear in search, such as thank-you pages.
A common mistake
If you block a page in robots.txt, Google cannot open it and therefore cannot see the noindex tag on it. Pick one method per page.