All search engine optimization professionals know robots.txt and its effect on search engine crawlers. Basically, if you want to prevent specific search engines from indexing websites, you can declare this intention by means of modifying robots.txt; in fact, you can even disallow all search spiders to keep websites as private as possible, which is what disgraced budget airline Ryanair seems to be doing.
In the third week of May, SEO professionals noticed that Ryanair was drastically dropping on the Google search engine results page (SERP), a highly unusual situation given the popularity of the airline. A few other SEO practitioners noticed the same thing happening on Bing. After a few days, site links were disappearing from the Google SERP, meaning that no new information was being indexed. A quick look into Ryanair's robots.txt file showed that all search engine crawlers had been disallowed.
Speculation about what Ryanair may be trying to accomplish with this highly unusual SEO move has been quite high. This Irish company has always been in the news, often for the wrong reasons, but the latest scandal is related to the mass cancellation of flights caused by the coronavirus pandemic. Airline industry analysts believe that the company is not planning on issuing refunds for flights that never took off; if this is the case, Ryanair may be trying to minimize website traffic, but this would be a laughable corporate strategy.
Some SEO professionals think that Ryanair has simply made a mistake, but what are the chances of such a colossal error not being corrected after a few days? Another possibility would be that the company is going through a major overhaul of its website properties; this would entail considerable testing that Ryanair does not want everyone to see. The problem with this disallowing action is that it effectively ruins their organic standing on the Google SERP, a situation that competitors could take great advantage of.
What is interesting about the Ryanair robots.txt incident is that it still ranks reasonably high on the various SERPs, and this has a lot to do with the new search engine algorithm directive of deprecating the "noindex" declaration on the file. For more information click here https://www.reddit.com/r/SEO/comments/gl5m2z/ryanairwebsiterobotstxt_disallow/.