SEO Glossary: NoIndex

What is NoIndex?

Noindex is a meta tag entry. It informs search engine crawlers that a certain desired page should not be included in the index. This allows webmasters and SEOs to decide for themselves which pages should be included in the search engine index and which should not.

 

In addition, it is possible to use the noindex tag with the attributes "nofollow" and "follow" attributes.

 

 

Where is the noindex tag used?

 

There are various areas of application in which Noindex is used. These include, among others:

 

• Paginated pages: If there is pagination on a website, duplicate content can be created on the following pages. With the help of Noindex, the paginated subpages are not indexed.

 

• Internal search results: If a search query is sent using the internal search function, the corresponding result pages can also be provided with Noindex. Because they can also mean DC.

 

• Category pages: Category pages can also run the risk of duplicate content. Noindex prevents this. As you can see, Noindex is mainly used to avoid duplicate content. This DC is known to cause search engines to penalize you.

 

How to integrate NoIndex?

There are two basic options for integrating Noindex:

Noindex Meta tag in the head area

The meta tag with noindex is quick and easy to implement.

 

Example:

 

<meta name=“robots“ content=“noindex“>

It is important that the tag is in the head area (). Bots then know about it and do not include the corresponding page in their index.

 

It is also possible to assign Noindex to individual search engines only. To do this, the corresponding bot must be specified in the HTML tag.

 

Example for the Google bot:

 

 

<meta name=“googlebot“ content=“noindex“>

 

 

Example of the Bing bot:

 

<meta name=“bingbot“ content=“noindex“> 

 

Noindex per HTTP response header

 

The second option for preventing indexing is the X-Robots tag header. The instruction is given directly in the response header.

 

 

Example:

 

HTTP/1.1 200 OK (...) X-Robots-Tag: noindex (...)

 

Of course, the X-Robots tag header also offers the option of assigning individual search engine bots and crawlers to address individual search engine bots and crawlers.

 

When does Noindex in the HTTP response header make sense?

 

In principle, there is no reason not to opt for the first option (meta tag in the head area). However, if a document or file without HTML code (e.g. PDF document) is to be excluded from indexing, this is difficult with an HTML tag.

 

In this case, the X-Robots tag is used, which is ideal for this purpose.