Site gone from Bing's SERP in a day! :)

bikegremlinbikegremlin ModeratorOGContent Writer
edited July 2022 in Technical

This is an interesting problem I've never faced before, not could help myself by googling.

All the websites from bikegremlin.com domain (bike.bikegremlin.com, io.bikegremlin.com etc.) are gone from Bing's SERP starting from June 26th 2022. Zero, zilch.

DuckDuckGo does the same, while Google and Yandex show all the pages normaly in their serches.

Bing's dashboard shows no errors (robots, page test, live URL test etc.), except for the Site Scan. Site scan returns "Error 400-499".

Full report that I'll update if I get any more ideas, or hear from Bing's support:

Microsoft Bing – site gone from SERP overnight!

If anyone has any ideas to try, I'm all ears. :)

Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
BikeGremlin's web-hosting reviews

«1

Comments

  • havochavoc OGContent Writer

    Clearly some IT gremlins at play!

    ...sorry couldn't help myself

  • armandorgarmandorg Services Provider

    I’ve seen such issues before on a client which got reported plenty of times by his competitors on search engines, for different reasons. Remembered that got removed from Bing and later on added after contacting support.

    Thanked by (2)bikegremlin Ympker

    Web Design Agency - Custom Web Designs
    WHMCS.design - WHMCS Themes | Blesta.shop - Blesta Themes

  • Something I was reading up today coincidentally...

    look up Bing Indexnow.
    Might be a dead end, or might lead to a solution. YMMV

    Thanked by (2)bikegremlin Ympker
  • IIRC years ago DuckDuckGo was using Bing's backend when they first launched, so maybe that's still the case and would explain why both exhibit the same behaviour.

    Thanked by (1)bikegremlin
  • bikegremlinbikegremlin ModeratorOGContent Writer

    @vyas said:
    Something I was reading up today coincidentally...

    look up Bing Indexnow.
    Might be a dead end, or might lead to a solution. YMMV

    Hmm, I think my contents are indexed (according to Bing's dashboard and previous SERPs), it's just removed from SERP as of recently.

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • DanielDaniel OG
    edited July 2022

    @ralf said:
    IIRC years ago DuckDuckGo was using Bing's backend when they first launched, so maybe that's still the case and would explain why both exhibit the same behaviour.

    They still use Bing. Pretty much every search engine that isn't Google uses Bing in some way, as Bing is the largest search engine that has an official public search API that other companies can use (whereas Google does not have one). Voice assistants like Alexa also also use Bing.

  • bikegremlinbikegremlin ModeratorOGContent Writer
    edited July 2022

    @Daniel said:

    @ralf said:
    IIRC years ago DuckDuckGo was using Bing's backend when they first launched, so maybe that's still the case and would explain why both exhibit the same behaviour.

    They still use Bing. Pretty much every search engine that isn't Google uses Bing in some way, as Bing is the largest search engine that has an official public search API that other companies can use (whereas Google does not have one). Voice assistants like Alexa also also use Bing.

    Yup. Roughly 5 - 10 % traffic from those combined (about 70 - 80 % from Google).

    Still, it's strange - and it sucks.

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • @bikegremlin said:

    @Daniel said:

    @ralf said:
    IIRC years ago DuckDuckGo was using Bing's backend when they first launched, so maybe that's still the case and would explain why both exhibit the same behaviour.

    They still use Bing. Pretty much every search engine that isn't Google uses Bing in some way, as Bing is the largest search engine that has an official public search API that other companies can use (whereas Google does not have one). Voice assistants like Alexa also also use Bing.

    Yup. Roughly 5 - 10 % traffic from those combined (about 70 - 80 % from Google).

    Still, it's strange - and it sucks.

    I shall check all mine later!

  • @armandorg said:
    I’ve seen such issues before on a client which got reported plenty of times by his competitors on search engines, for different reasons. Remembered that got removed from Bing and later on added after contacting support.

    Any more details on this contacting support part?

    Why?

  • bikegremlinbikegremlin ModeratorOGContent Writer

    I've used this situation to check out some Google alternatives.

    Made a list of search engines that use Bing (Bing copies basically), followed by a list of search engines that seem to be independent.

    Bing is still "working on this and keeping me informed." :)

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • nfnnfn
    edited July 2022

    Do you have Super Bot Mode enabled in Cloudflare?

    Thanked by (1)bikegremlin
  • bikegremlinbikegremlin ModeratorOGContent Writer
    edited July 2022

    @nfn said:
    Do you have Super Bot Mode enabled in Cloudflare?

    No. But a good call - that's what CF support first asked.

    Not all the CF sites are affected - though all used the same CF setup.
    Only one domain. So it may not be CF-related at all.
    Though one can never be certain - too many stuff in stack, relying on each other.
    I'm surprised it ever works at all in the first place. :)

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • nfnnfn
    edited July 2022

    @bikegremlin said:

    @nfn said:
    Do you have Super Bot Mode enabled in Cloudflare?

    No. But a good call - that's what CF support first asked.

    Not all the CF sites are affected - though all used the same CF setup.
    Only one domain. So it may not be CF-related at all.
    Though one can never be certain - too many stuff in stack, relying on each other.
    I'm surprised it ever works at all in the first place. :)

    I have the same problem and made some analysis at bing webmaster to discover that Super Bot Mode is blocking Bing:

    https://community.cloudflare.com/t/super-bot-mode-blocking-bing-bot-from-reading-sitemaps/316208

    You can test this here: https://www.bing.com/webmasters/sitescan

  • bikegremlinbikegremlin ModeratorOGContent Writer
    edited July 2022

    @nfn said:

    @bikegremlin said:

    @nfn said:
    Do you have Super Bot Mode enabled in Cloudflare?

    No. But a good call - that's what CF support first asked.

    Not all the CF sites are affected - though all used the same CF setup.
    Only one domain. So it may not be CF-related at all.
    Though one can never be certain - too many stuff in stack, relying on each other.
    I'm surprised it ever works at all in the first place. :)

    I have the same problem and made some analysis at bing webmaster to discover that Super Bot Mode is blocking Bing:

    https://community.cloudflare.com/t/super-bot-mode-blocking-bing-bot-from-reading-sitemaps/316208

    You can test this here: https://www.bing.com/webmasters/sitescan

    I have superbot disabled.
    Site Scan doesn't work on both affected and unaffected websites. :)
    Go figure...

    Relayed all the info to Bing's support.

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • Any updates on this?

    Did you check your server logs for issues when you performed Bing Site Scan?

    Thanked by (1)bikegremlin

    My reliable providers : Ramnode : HostHatch : Serverica : InceptionHosting : AlphaVps : LittleCreekHosting : NexusBytes : Hostodo

  • bikegremlinbikegremlin ModeratorOGContent Writer
    edited August 2022

    @niceboy said:
    Any updates on this?

    Did you check your server logs for issues when you performed Bing Site Scan?

    Couldn't find anything - not in Cloudflare WAF logs or in server logs.
    Contacted both Bing, and Cloudflare support.

    However, I host several websites with the same Cloudflare setup, and the same hosting provider, yet only one domain is affected (with more than one website).
    That's why I think this is a Bing-induced problem and probably has nothing to do with Cloudflare or hosting.

    I'm keeping the Bing website removal article up-to-date with any info.

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • bikegremlinbikegremlin ModeratorOGContent Writer

    Seems like the home page is back to Bing & Co.
    But zero search queries show any of my articles - so practically it’s still gone from Bing.

    We’ll see if that changes.

    Thanked by (1)MikePT

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • MikePTMikePT Hosting ProviderOGServices Provider

    He deserves it. Writes amazingly well and about interesting stuff!

    Thanked by (2)bikegremlin chimichurri
  • bikegremlinbikegremlin ModeratorOGContent Writer

    Got an email from Bing that my websites were blocked, and upon review, they decided to lift the block - so it should all be fine in about 3 weeks. A bit more info:
    Manual unblock?

    We'll know by September, I suppose.

    Thanked by (1)MikePT

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • bikegremlinbikegremlin ModeratorOGContent Writer

    It seems like my sites are slowly making a return to Bing (and DuckDuckGo and the company) search results pages.
    Faster than I had expected.
    We'll see how long it lasts. :)

    Thanked by (1)MikePT

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • I have same issue, all the urls are marked as infected with malware … they were not, but bing see them as infected.

  • bikegremlinbikegremlin ModeratorOGContent Writer

    @rober7 said:
    I have same issue, all the urls are marked as infected with malware … they were not, but bing see them as infected.

    Double check (in case you haven't already) - tools like Sucuri SiteCheck are a quick and easy way to get some basic info.

    If it's all good, try contacting Bing's support.

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • Who's is using bing nowadays? It is like yahoo search.

  • hostartshostarts Hosting Provider

    Very interesting, I've placed a new domain, and new website online and it ranked first in just a week on bing, They clearly did some changes on Bing, your report suggests that the issue is intermittent?

    Web Hosting - VPS Cloud - EU Dedicated Offers Managed Hosting Made Easy Dedicated Servers, Private Cloud, Colocation, VPS, Shared & Reseller Hosting

  • nfnnfn
    edited August 2022

    I haven't heard from them since August 1st and none of the sites I reported have come back.

  • @bikegremlin said:

    @rober7 said:
    I have same issue, all the urls are marked as infected with malware … they were not, but bing see them as infected.

    Double check (in case you haven't already) - tools like Sucuri SiteCheck are a quick and easy way to get some basic info.

    If it's all good, try contacting Bing's support.

    I contacted the support and say that they will come back to me ... it has been some time since.
    There is a forum discussion on Microsoft about this and i am not the only one. I saw somewhere that Bing bot detect as a malware TLS V3 (or something like this)

  • bikegremlinbikegremlin ModeratorOGContent Writer

    @rober7 said:

    @bikegremlin said:

    @rober7 said:
    I have same issue, all the urls are marked as infected with malware … they were not, but bing see them as infected.

    Double check (in case you haven't already) - tools like Sucuri SiteCheck are a quick and easy way to get some basic info.

    If it's all good, try contacting Bing's support.

    I contacted the support and say that they will come back to me ... it has been some time since.
    There is a forum discussion on Microsoft about this and i am not the only one. I saw somewhere that Bing bot detect as a malware TLS V3 (or something like this)

    In my case, I got a first reply in 24 hours, saying they'll look into it.
    About 7-10 days later, I got another email, saying that the problem is "escalated to the concerned team.”
    Then, some 20 days after my initial problem report, a got an email saying they've manually fixed the problem, and that it was a block on their end.

    After that, in about two weeks' time, my websites have almost fully recovered in Bing's SERP.

    For all I could tell, not everyone is that lucky, and there is still some problem with Bing's indexing.

    Relja of House Novović, the First of His Name, King of the Plains, the Breaker of Chains, WirMach Wolves pack member
    BikeGremlin's web-hosting reviews

  • I think it's time to block Bing Crawler B)

Sign In or Register to comment.