Forum Updates and DDoS attack

[Mr X]

usually state sponsored

The question is which states...

i do not like to speculate. Many times I know a lot but I prefer to stay silent. I will follow the same principle once again

 

[AjayGhatak]

Seems like you folks are not alone!
Someone very wise made a commit in Crowdstrike endpoint driver for Microsoft which skipped all the automated tests and was pushed to their production CDN.... surprisingly Microsoft also does not do a simple "smoke test" of all its drivers and lets Crowdstrike do it itself. I guess, the policy of "sign your code to run into kernel" came back and bite it in its posterior.

 

[j_hungary]

Seems like you folks are not alone!
Someone very wise made a commit in Crowdstrike endpoint driver for Microsoft which skipped all the automated tests and was pushed to their production CDN.... surprisingly Microsoft also does not do a simple "smoke test" of all its drivers and lets Crowdstrike do it itself. I guess, the policy of "sign your code to run into kernel" came back and bite it in its posterior.

crowdstrike I read was another issue, kind of similar to what Optus did last year when they bring down the entire service by puting a test patch in production (20 hours without mobile and internet) that is done by accident.

This, putting into the fact it just gone off after the update were made, that's intentional

 

[j_hungary]

i do not like to speculate. Many times I know a lot but I prefer to stay silent. I will follow the same principle once again

As I said, you probably know, I am not asking you to spill the bean lol

The issue I got is that why now. This is probably the million dollar questions.

 

[AjayGhatak]

crowdstrike I read was another issue, kind of similar to what Optus did last year when they bring down the entire service by puting a test patch in production (20 hours without mobile and internet) that is done by accident.

This, putting into the fact it just gone off after the update were made, that's intentional

The fun part with CrowdStrike is this:

It is a kernel mode driver that is causing the crash.... this should be impossible really. Why? Because of this : https://learn.microsoft.com/en-us/w...igning-requirements--windows-vista-and-later-

Basically, you need Microsoft's nod before your driver can load into any retail windows installation. Windows does not load a kernel mode driver without Microsoft's signature.

This only means one and only one thing. Crowdstrike is somehow able to load a kernel mode driver without Microsoft's vetting.

That's not a very comforting thought.

 

[HeratKabul]

I should preferably turn the board off for 1 hour but I am not doing it.

The site could be down when the server transfer is complete that is not helping me to sleep (5am in London)

Take rest brother. Turn off the board and do what you need to do.

 

[AjayGhatak]

Most likely some IT Cell/Scam center in Noida pooled together their family's collective savings to pay some russians for a DDOS attack

Those types infest twitter these days.

 

[j_hungary]

The fun part with CrowdStrike is this:

It is a kernel mode driver that is causing the crash.... this should be impossible really. Why? Because of this : https://learn.microsoft.com/en-us/w...igning-requirements--windows-vista-and-later-

Basically, you need Microsoft's nod before your driver can load into any retail windows installation. Windows does not load a kernel mode driver without Microsoft's signature.

This only means one and only one thing. Crowdstrike is somehow able to load a kernel mode driver without Microsoft's vetting.

That's not a very comforting thought.

Well, have you ever wonder why Edward Snowden can made off with treasure trove of NSA file without even going to an NSA Blacksite??

NSA share their Kernal Data with their third-party contractor (Dumb, I know), essentially giving these contractor unvetted access to their system because they think it is going to be secure on their end.

This practice seems happened a lot with government and company, well, you have Rockstar Game leaking their stuff because they have adapted their remote working station to their system. Then you have many other company (Uber, Telstra etc) doing the same thing and got hacked.

 

[AjayGhatak]

Well, have you ever wonder why Edward Snowden can made off with treasure trove of NSA file without even going to an NSA Blacksite??

NSA share their Kernal Data with their third-party contractor (Dumb, I know), essentially giving these contractor unvetted access to their system because they think it is going to be secure on their end.

This practice seems happened a lot with government and company, well, you have Rockstar Game leaking their stuff because they have adapted their remote working station to their system. Then you have many other company (Uber, Telstra etc) doing the same thing and got hacked.

Contracting and sub contracting.... the bane of anything secure. You never know who actually ends up doing the job and how they were working with your "Crown Jewels".

 

[j_hungary]

Contracting and sub contracting.... the bane of anything secure. You never know who actually ends up doing the job and how they were working with your "Crown Jewels".

That's why you need to be a person in this game to work for a subcontractor. Essentially you are doing the same job when you were doing in the NSA or Military Intelligence, but you get paid 4 times the amount so that line the pocket of the contractor company (which spell the who's who in the military circle)

But that does not mean it was always secure, because things changes and circumstance changes, and the reason why they are doing that is because of money, those who were in charge of these organisation have no idea how IT work, I mean do you think, for example, Gen David Petraeus is well verse and knowledgeable in IT if at all? That's the person who make those decision (Not singling him out, everyone in his position was doing the same thing) But then IT Company like MS or Oracle should have known better

 

[srshkmr]

Someone with deep pockets

It’s not that cheap to do the ddos of this scale.

Few kids with scripts and infected computers can do a lot more.

Not sure which plan you are using for cloudflare.
But i would personally recommend the following.

1. Ddos protection Basic
2. Customized WAF rule, analyzing the threat traffic and the forum's standard traffic which blocks restricts suspcted requests.
3. Basic smart caching, reducing the overall requests to cpu utilization ratio.

 

[Viet]

Hello,

Here's what we know so far. On the 15th of July, I upgraded the forum software. Shortly after the upgrade, I noticed numerous broken links throughout the site. I fixed them all, but an hour later, the CPU usage skyrocketed beyond 100%, leading to database errors. I also observed 1,000 faults in the processes, rendering the forum non-functional.

I disabled all plugins and themes, reverting to the stock version of Xenforo, but the CPU performance didn't improve. I was considering potential solutions when I noticed a sudden drop in CPU usage, bringing levels back to normal. The website functioned properly for a while but then exceeded maximum capacity again, becoming dysfunctional without any intervention. I disabled all plugins via the command line since the ACP was unavailable. The site came back online, once more using the stock version.

Users were requesting dark mode and "users activity" feature, so I enabled them, which caused the site to slow down again. I continued to suspect the upgrade, as the site was fine before it.

However, after speaking with the hosting provider, they were certain it was a DDoS attack. Our typical traffic is around 40,000 hits per day, but between the 15th and 16th of July, we had about 11 million hits. This volume is highly unusual for a small site like ours, suggesting bad bots. We installed a captcha, but it didn't help much. A Xenforo developer who works for the company reverted the forum to the older version, fixed everything and advised against upgrading until all plugins are compatible. We were back to the state of a week ago with a functioning website and compatible plugins.

The website worked fine for another 12 hours before the CPU resources were maxed out again. Another experienced Xenforo developer who was helping me all along (Andy) also believes a DDoS attack is causing the issue, rather than a problem with the forum itself.

Originally, I transferred the domain to Cloudflare in December for DDoS protection, but the changeover took longer than expected, leading to numerous user complaints even after four days. With the old PDF shutting down that week, I reverted to the original host for a smoother user transition. Since then, transferring to Cloudflare has been on hold. My current host offers some DDoS protection, but I plan to switch to Cloudflare this week, despite potential user disruptions, as it's beneficial in the long run.

I noticed my tracking tools reported about 500,000 visits from Russia. Also... on the day of the attempted assassination of Donald Trump, we saw an increase to 1,200 users from the USA, compared to the usual 200 per day. So, there was natural traffic growth alongside the DDoS attack.

Please bear with us during these challenging times as the forum is fully recovered and we understand the issue. I have temporarily disabled some features and will re-enable them once the attack subsides. I might reintroduce the captcha to reduce unwanted bots until further notice. So I apologise in advance for the inconvenience. Please use the following link to find the latest posts in the meantime, if I keep switching it off in case

What's new

Thank you for your understanding.

Ah great work. Well done.
You are not alone. Some are worse hit. Look at the disaster caused by Crowdstrike.

 

[Mr X]

Ah great work
You are not alone. Look at the disaster caused by Crowdstrike.

Crowdstrike is a very different attack but I get it that I am not alone in this mess… lol

 

[Viet]

Crowdstrike is a very different attack but I get it that I am not alone in this mess… lol

Oh well, enabling automatic server or network update via direct link to provider the most stupid thing ever.
I wonder who does it.
Seems there are lots of companies out there that run like blind chickens.

 

[r3alist]

Great update