Robots.txt File Question

kuldeepsingh689 · Jul 30, 2022

Hello Guys.

I need to know something about robots.txt file. please help.

I have converted my website images .jpg to .webp

But it did not work, so I deleted that plugin which converted .jpg to .webp

Now google bots are crawling the URL /wp-content/uploads/2017/720x400.jpg.webp (example) (found in Crawled - currently not indexed)

i need to block google bots.

What should i add in my website robots txt file.

Disallow: /wp-content/*.webp
or
Disallow: /wp-content/uploads/*.webp

Any ideas.

Thanks

fb3003 · Jul 30, 2022

webp does help in speed
what plugin did you use?
how do you know it did not work?

kuldeepsingh689 · Jul 30, 2022

fb3003 said:
webp does help in speed
what plugin did you use?
how do you know it did not work?

website was getting error in the error log. And .webp version has bigger file size.

TomTheCat · Jul 30, 2022

Code:

Disallow: /wp-content/uploads/*.webp$

kuldeepsingh689 · Jul 30, 2022

TomTheCat said:
Code:

Disallow: /wp-content/uploads/*.webp$

Thank you.

fb3003 · Jul 30, 2022

kuldeepsingh689 said:
website was getting error in the error log. And .webp version has bigger file size.

what plugin did you use?

kuldeepsingh689 · Aug 1, 2022

fb3003 said:
what plugin did you use?

Not remember now, but you can easily find on WordPress plugins list.

You can also use litespeed cache plugin.

fb3003 · Aug 1, 2022

kuldeepsingh689 said:
Not remember now, but you can easily find on WordPress plugins list.

You can also use litespeed cache plugin.

i dont need a plugin. wanted to know what plugin you were using to might have caused this.

kuldeepsingh689 · Aug 8, 2022

TomTheCat said:
Code:

Disallow: /wp-content/uploads/*.webp$

Sir why you put dollar sign $ in last. can you let me know.

Alexion · Aug 8, 2022

kuldeepsingh689 said:
Sir why you put dollar sign $ in last. can you let me know.

it means the file has to end in webp
without the $ sign, files like "some-important-article.webp-tutorial" would also be excluded

kuldeepsingh689 · Aug 8, 2022

Alexion said:
it means the file has to end in webp
without the $ sign, files like "some-important-article.webp-tutorial" would also be excluded

Ok, but all images end with .webp

Robots.txt File Question

kuldeepsingh689

Regular Member

fb3003

Elite Member

kuldeepsingh689

Regular Member

TomTheCat

Elite Member

kuldeepsingh689

Regular Member

fb3003

Elite Member

kuldeepsingh689

Regular Member

fb3003

Elite Member

kuldeepsingh689

Regular Member

Alexion

Banned - Shitlisted and did not do the right thing

kuldeepsingh689

Regular Member

Main Menu

Marketplace

Making Money

BlackHat World