Of course they have bots... That's how you index the web.
If we took Google offline I bet there would also be a reduction in bot traffic from them too!
Indexing is also probably a low priority task, so if half their datacenter capacity is taken offline, their bot traffic probably reduces by more than 50%.
Logic says google bots do not come disguised, but as google agents.
Or is there other information?
Now there is probably also AI agent spam from google, but not at this level I believe. Also I am not fully clear what they count as spam, I doubt they would include google in this list, so do they include yandex bots here, if they come officially?
It's a nice reminder that HN is just filled with the same primitive tribal normies and is always getting oneshotted by the same transparent xenophobic hysteria that plagues the rest of the internet.
Yandex has cloud offering, their servers are used for actual bot traffic, not just crawlers. Yandex hasn’t been only a search company for a very long time.
Ukraine should take the rest of Yandex permanently offline so Cloudflare can get a clean objective measurment, and we can enjoy fewer Russian shill-bots on Hacker News.
The russian bots are better.. usually its conspiracy fog, attached to a valid russia opposing point that reveals them. Chinese bots are just this months CCP talking points, so can be quite easily visible if they cram some topic only china cares about into every reply.
The downvotes are because you're resisting easily-observed reality with the seemingly-rational response "Where is this 'air'? If I'm breathing it, it should be easy to point to, no?"
No. If you can't see it, and don't understand it's there, you cannot be explained to. You're someone with no palate asking how anyone else can know if a wine tastes good.
I can’t believe there are so many dumb comments in this thread.
All it shows is that crawling from Yandex search engine stopped from one Yandex data center ASN.
No one runs opinion/malicious bots from data center IPs that people mention here.
On the other hand there are plenty of datacenter located command/control systems that puppeteer vast fleets of residential proxies (and Russia is by no means the only one doing so), so it will be very telling to see what social media accounts go silent over the coming days and weeks.
> from one Yandex data center ASN
This is only one of two extant Yandex ASN, not an ASN specific to one geographic region/datacenter.
The HTTP traffic from Yandex is down to ~ 0 % after the third strike, after the first two strikes already degraded it. I wonder how this looks like at the network level because their search page still serves, but presumably their crawler is inactive now
Edit: Here is some more info, they do have availability issues as is to be expected
> Because three of its primary facilities were knocked offline within four days, Yandex’s failover mechanisms broke down[1][8].
> Outages have cascaded across Yandex Go (taxis), Yandex Pay, food delivery, smart home platforms, internal logistics, and third-party corporate services hosted on Yandex Cloud[2][7]. Problems have spilled into neighboring countries using these systems (Belarus, Kazakhstan, Armenia, Serbia)[2][7].
Edit 2: Here is the Yandex Cloud status page, their GCP/AWS equivalent. The entire compute core is offline, and even worse, Cloud Backup
I feel like this isn't even a particularly new thing. I worked at a medium-sized SaaS company and we did disaster recovery drills for moving our stack from eu-west-1 to eu-central-1 in case something catastrophic happened in Ireland. If you want HA, you kinda have to assume that any single country in the world can have some kind of a crippling disaster. Unfortunately for Yandex's clients, Yandex only has regions in two countries, and not that many regions either.
I don't think there is much you can do about being targeted by a military, simply because of how unique the attack vector is.
Hacking is smart but happens through software, so if you have hardware (i.e. physical) barriers, that stops the hacking.
Natural disasters happen physically but they are dumb, so if you have software redundancy (i.e. by copying the data to multiple locations), that stops them from destroying everything.
A military is both physical AND smart. They can target multiple locations at once and destroy whole buildings.
You can learn to quickly diversify availability (or at least data safety) beyond the region of conflict when a kinetic war breaks out. The moment “kinetic sanctions” became the strategy, the risk to data centres should have been self-evident. And if it wasn’t immediately obvious, it should have been blindingly so when targeting expanded from oil refineries to Wildberries warehouses.
I predict a great many more videos coming out of Russia of Firepoint UAVs flying into refineries and such, punctuated by background audio commentary that can be summed up as "cyka blyat"
*Sudden drop in Cloudflare bot traffic from AS13238 YANDEX — Yandex LLC
UPD Definition of bot traffic from Cloudflare's glossary:
>Bot vs. human traffic
>The percentage of HTTP requests classified as bot versus human, based on bot scores. Requests with a bot score between 1 and 29 are classified as likely automated (bot), while requests with a score of 30 or above are classified as likely human. For more information, refer to Bot classes.
>By default, Radar shows bot vs. human traffic for requests to HTML content. This filter is meant to represent traditional web traffic by excluding API calls, asset requests (images, scripts, fonts), and other machine-to-machine requests.
Everyone is mentioning crawlers, but wouldn't most traffic from a datacenter's ASN usually be bots just in general given that there's not many humans there to generate human traffic?
An eye for an eye for the Russians.
I love western imperial propaganda though, and how HN so eagerly amplifies it. There’s something self-fulfilling about willingly becoming a foot soldier of the Empire.
Here's to a world where the West is monolithic, and may Iran, Iraq, Yugoslavia, Syria, Libya, Lebanon, Palestine be only be the beginning. And may every new crusade for freedom and democracy leave behind another Gaza, in pursuit of upholding human rights and civilizing the world.
I'm going to be really interested how much this cleans up the internet, I mean the damage is done if you read comment sections on youtube etc, but let's see how long the BS persists.
I was thinking, we should all contribute to a Yandex fund for the Ukrainian mil to help keep the level of bot activity on the rest of the internet under control!
Yandex is Russian Google.
Of course they have bots... That's how you index the web.
If we took Google offline I bet there would also be a reduction in bot traffic from them too!
Indexing is also probably a low priority task, so if half their datacenter capacity is taken offline, their bot traffic probably reduces by more than 50%.
The other Yandex AS which is their hosting/hypervisor platform, broadly similar to AWS, Azure, whatever also shows a major drop in bot traffic:
https://radar.cloudflare.com/bots/as200350?dateRange=7d
https://www.peeringdb.com/net/20950
I don't think these are the boys they refer to?
EDIT: I was wrong, they're just generic bot vs human traffic numbers and you're right.
Don't let logic get in the way of a good propaganda.
Logic says google bots do not come disguised, but as google agents.
Or is there other information?
Now there is probably also AI agent spam from google, but not at this level I believe. Also I am not fully clear what they count as spam, I doubt they would include google in this list, so do they include yandex bots here, if they come officially?
Neither do Yandex bots. What's your point?
It's a nice reminder that HN is just filled with the same primitive tribal normies and is always getting oneshotted by the same transparent xenophobic hysteria that plagues the rest of the internet.
I wonder how this will affect the Yandex search engine. I think I've read they split the Russian one and .com, but I'm not sure.
Yandex has cloud offering, their servers are used for actual bot traffic, not just crawlers. Yandex hasn’t been only a search company for a very long time.
Ukraine should take the rest of Yandex permanently offline so Cloudflare can get a clean objective measurment, and we can enjoy fewer Russian shill-bots on Hacker News.
Can you point to a few comments that you consider examples?
I've never noticed occurrences of such stuff, so I'm curious what I'm failing to key in on.
Edit: and would someone mind explaining the downvotes? If they're meant to communicate something to me, I'm genuinely not understanding.
Post a controversial political statement and grab a beer.
Ehh, there are enough humans that would do that. Don't delude yourself thinking that there are no people that hold such views.
Nobody wants honesty, everyone just wants to fit in with the collective. The downvotes are the collective, reactive mindset, showing itself.
The russian bots are better.. usually its conspiracy fog, attached to a valid russia opposing point that reveals them. Chinese bots are just this months CCP talking points, so can be quite easily visible if they cram some topic only china cares about into every reply.
The downvotes are because you're resisting easily-observed reality with the seemingly-rational response "Where is this 'air'? If I'm breathing it, it should be easy to point to, no?"
No. If you can't see it, and don't understand it's there, you cannot be explained to. You're someone with no palate asking how anyone else can know if a wine tastes good.
Nonsense. They’re trying to breathe and the downvotes won’t allow it.
Isn't this showing the percentage of bot traffic from the Yandex IP range?
If so, it isn't particularly surprising that the line would go down.
EDIT: yes, it is. Looking at the same graph for all of Russia, and there's no discernible pattern:
https://radar.cloudflare.com/bots/ru?dateRange=7d
same for the 48h view:
https://radar.cloudflare.com/bots/ru?dateRange=2d
USA accounts for 44% bot traffic worldwide. Russia 0.9%. Russia derangement syndrome on HN as usual.
Someone needs to learn how to read the Bot Traffic page.
That is a drop in traffic specifically from Yandex bots (which includes their crawlers and other apps hosted in their DC)
Now compare it with Bot Traffic in:
* Russia: https://radar.cloudflare.com/bots/ru?dateRange=7d
* Europe: https://radar.cloudflare.com/bots/europe?dateRange=7d
* Worldwide: https://radar.cloudflare.com/bots?dateRange=7d
Dip is barely noticeable even in Russia.
Its like saying, "API requests are completely down" (and small disclaimer: only from your server, because your server is down)
I can’t believe there are so many dumb comments in this thread. All it shows is that crawling from Yandex search engine stopped from one Yandex data center ASN.
No one runs opinion/malicious bots from data center IPs that people mention here.
Did people actually stop thinking?
On the other hand there are plenty of datacenter located command/control systems that puppeteer vast fleets of residential proxies (and Russia is by no means the only one doing so), so it will be very telling to see what social media accounts go silent over the coming days and weeks.
> from one Yandex data center ASN
This is only one of two extant Yandex ASN, not an ASN specific to one geographic region/datacenter.
https://www.peeringdb.com/net/1751
The other one is this: https://www.peeringdb.com/net/20950
And the 200350 also shows a near complete drop off in bot activity:
https://radar.cloudflare.com/bots/as200350?dateRange=7d
The HTTP traffic from Yandex is down to ~ 0 % after the third strike, after the first two strikes already degraded it. I wonder how this looks like at the network level because their search page still serves, but presumably their crawler is inactive now
Edit: Here is some more info, they do have availability issues as is to be expected
> Because three of its primary facilities were knocked offline within four days, Yandex’s failover mechanisms broke down[1][8].
> Outages have cascaded across Yandex Go (taxis), Yandex Pay, food delivery, smart home platforms, internal logistics, and third-party corporate services hosted on Yandex Cloud[2][7]. Problems have spilled into neighboring countries using these systems (Belarus, Kazakhstan, Armenia, Serbia)[2][7].
Edit 2: Here is the Yandex Cloud status page, their GCP/AWS equivalent. The entire compute core is offline, and even worse, Cloud Backup
https://status.yandex.cloud/ru/dashboard
> ru Compute Cloud, ru Kubernetes, ru PostgreSQL, ru ClickHouse, ru MySQL, ru YDB, ru Kafka, ru Application Load Balancer
This will be a case study in catastrophic events for HA and Ops folks.-
I feel like this isn't even a particularly new thing. I worked at a medium-sized SaaS company and we did disaster recovery drills for moving our stack from eu-west-1 to eu-central-1 in case something catastrophic happened in Ireland. If you want HA, you kinda have to assume that any single country in the world can have some kind of a crippling disaster. Unfortunately for Yandex's clients, Yandex only has regions in two countries, and not that many regions either.
I don't think there is much you can do about being targeted by a military, simply because of how unique the attack vector is.
Hacking is smart but happens through software, so if you have hardware (i.e. physical) barriers, that stops the hacking.
Natural disasters happen physically but they are dumb, so if you have software redundancy (i.e. by copying the data to multiple locations), that stops them from destroying everything.
A military is both physical AND smart. They can target multiple locations at once and destroy whole buildings.
> I don't think there is much you can do about being targeted by a military, simply because of how unique the attack vector is.
You could choose not to invade other countries.
You can learn to quickly diversify availability (or at least data safety) beyond the region of conflict when a kinetic war breaks out. The moment “kinetic sanctions” became the strategy, the risk to data centres should have been self-evident. And if it wasn’t immediately obvious, it should have been blindingly so when targeting expanded from oil refineries to Wildberries warehouses.
I just searched "blyat" and it worked, so?
Given that blyad is just punctuation at this point, I am not surprised.
I predict a great many more videos coming out of Russia of Firepoint UAVs flying into refineries and such, punctuated by background audio commentary that can be summed up as "cyka blyat"
The username checks out!
("Ept" and "suka" are two other swearwords, that are also used much like punctuation.)
*Sudden drop in Cloudflare bot traffic from AS13238 YANDEX — Yandex LLC
UPD Definition of bot traffic from Cloudflare's glossary:
>Bot vs. human traffic
>The percentage of HTTP requests classified as bot versus human, based on bot scores. Requests with a bot score between 1 and 29 are classified as likely automated (bot), while requests with a score of 30 or above are classified as likely human. For more information, refer to Bot classes.
>By default, Radar shows bot vs. human traffic for requests to HTML content. This filter is meant to represent traditional web traffic by excluding API calls, asset requests (images, scripts, fonts), and other machine-to-machine requests.
https://developers.cloudflare.com/radar/glossary/#bot-vs-hum...
https://developers.cloudflare.com/bots/concepts/bot-score/
Correct, I probably didn't didn't write that in the clearest possible way, I meant "detected bot traffic hitting cloudflare originating from Yandex".
Everyone is mentioning crawlers, but wouldn't most traffic from a datacenter's ASN usually be bots just in general given that there's not many humans there to generate human traffic?
That's some intense mitigation from Cloudflare
You gave me a good laugh, cheers!
An eye for an eye for the Russians. I love western imperial propaganda though, and how HN so eagerly amplifies it. There’s something self-fulfilling about willingly becoming a foot soldier of the Empire.
Here's to a world where the West is monolithic, and may Iran, Iraq, Yugoslavia, Syria, Libya, Lebanon, Palestine be only be the beginning. And may every new crusade for freedom and democracy leave behind another Gaza, in pursuit of upholding human rights and civilizing the world.
Interestingly most traffic coming out of this AS was actually UDP: https://dashboard.terracenetworks.com/sources/asns/13238
Pretty benign AS as far as actual exploit or malicious crawling is concerned.
Probably QUIC if true, but you should also ask how these people get their data, because this isn't something you can just see or look up.
Someone needs to inventory all the accounts that go dark on social media so we can flag them going forward.
I have to wonder what will happen when the remaining 6 data centers get hit as well. It's bound to happen.
If there will be an world-wide war, sure. But at that point, the Internet as we know it will probably collapse anyway.
There is one, but it's taking place mostly by information means, not physical.
https://postimg.cc/YjjDtrT2
Are there even six remaining?
Could be just crawlers
I take the entire russian, chinese, north korea AS blocks (and also other from tiny irrelevant countries) and drop them at the firewall level.
I do that since years and the Internet keeps working just perfectly fine.
If you've got servers and serve "the west": just drop all these commie cunts countries entirely. Simply pretend they don't exist.
And nothing shall be lost.
I'm going to be really interested how much this cleans up the internet, I mean the damage is done if you read comment sections on youtube etc, but let's see how long the BS persists.
It's only counting Yandex search crawler traffic, not any actually bad bots.
So, is sending hot hate towards a bot farm going to be a paid service on Cloudflare now?
Would be ironic, since they provide services to criminals who commit DDoS-attacks, spread malware and conduct phishing campaigns:
https://cofense.com/blog/how-cloudflare-services-are-abused-...
https://www.heise.de/en/news/Report-Cybercriminals-use-Cloud...
Or maybe they would just want to get rid of the competition.
Cloudflare should consider donating to the Ukrainian military for the public service they are performing.
I was thinking, we should all contribute to a Yandex fund for the Ukrainian mil to help keep the level of bot activity on the rest of the internet under control!
https://war.ukraine.ua/donate/
(I’d welcome a data center and oil infrastructure targeting specific fund)
Oh no, how is Trump going to win the midterms now?
isn't this only counting russian origin already?
The URL is CF's public metrics for bot traffic from a single specific Russian ASN, Yandex.
https://www.peeringdb.com/net/1751
oh well of course if you blow up Yandex then Yandex stops sending you traffic, because they got blown up.
And nothing of value was lost.
Yandex is the best search engine that doesn't follow American censorship norms. For this reason Kagi uses them as one source of several.
“AI agents” is just a fancy rebranding of “bots” with some “intelligence”.