SEO Glossary: NoIndex

What is NoIndex?

Noindex is a meta tag entry. It informs search engine crawlers that a certain desired page should not be included in the index. This allows webmasters and SEOs to decide for themselves which pages should be included in the search engine index and which should not.

 

In addition, there is the possibility to use the noindex tag with the attributes "nofollow" and "follow" to extend it.

 

 

Where is the noindex tag used?

 

There are several application areas in which Noindex is used. These include, among others:

 

• Paginated pages: If there is pagination on a website, duplicate content can be created on the following pages. With the help of Noindex, the paginated subpages are not indexed.

 

• Internal search results: If a search query is sent using the internal search function, the corresponding result pages can also be provided with Noindex. Because they can also mean DC.

 

• Category pages: Category pages can also run the risk of duplicate content. Noindex prevents this. As you can see, Noindex is mainly used to avoid duplicate content. As you know, this DC can cause a downgrading by the search engines.

 

How to integrate NoIndex?

To include Noindex, there are two basic ways:

Noindex Meta tag in head area

The meta tag with Noindex is implemented quickly and easily.

 

Example:

 

<meta name=“robots“ content=“noindex“>

It is important that the tag is in the head area (). Then bots will know about it and will not include the corresponding page in their index.

 

In addition, it is possible to assign Noindex only to individual search engines. For this, the corresponding bot must be specified in the HTML tag.

 

Example of the Google bot:

 

 

<meta name=“googlebot“ content=“noindex“>

 

 

Example of the Bing bot:

 

<meta name=“bingbot“ content=“noindex“> 

 

Noindex per HTTP response header

 

The second possibility to prevent indexing is the X-Robots tag header. In this case, the instruction is made directly in the response header.

 

 

Example:

 

HTTP/1.1 200 OK (...) X-Robots tag: noindex (...)

 

Of course, the X-Robots tag header also offers the possibility to specify individual search engine bots and crawlers crawlers.

 

When does Noindex make sense in the HTTP response header?

 

In principle, there is nothing to be said against opting for the first variant (meta tag in the head area). However, if a document or file without HTML code (e.g. PDF document) is to be excluded from indexing, this is not possible with an HTML tag.

 

In this case, the X-Robots tag is used, which is perfectly suited for this purpose.