Server updates

[exxos]

I've have to "remove" the automatic unbanning on the IP check page, Because, I kid you not, I had several IP unbanning Notifications from spammers wanting to be removed thismorning!!

These IPs wasn't even banned by my server, they were blocked from the abuseipdb block lists because of thousands of reports.

So when IP addresses are requested to be unbanned, I get a notification and will manually review the IP request. Which can take a few hours to a few days.

So I suggest guest readers register and login once a week to make sure your kept whitelisted. As I can see me having to take down the IP check page the way things are heading.

Currently new attacks have been at a all-time low this past week. I think banning ranges has done the trick as it doesn't give other IPs the chance to really hit the server anymore. A lot of my older rules don't even find anything with my new improved detection rules. I'll leave them running though as each jail checks for different types of mass attacks. There's 33 jails running now.

I'm still monitoring every couple of days. I found last week that fail2ban was keeping banned IPs in memory and the server was a heartbeat away from crashing due to lack of RAM. I thought keeping the SQLite file small would do the trick but nope. I wasn't aware F2B made a store in RAM also. I mean like 4GB worth of banned IPs... So that should be fixed now.

[exxos]

I needed a CAPTCHA for the IP check page to try and slow down all the unban requests from spammers.. Even though I hate cloudflare, they had a free "drop in"module, so given that a try.

Capture.PNG (43.28 KiB) Viewed 603 times

A lot of the IP addresses had been banned via the abuseipdb blacklist. I was considering moving that blacklist to the top of the firewall chain to solve that problem, but I'm getting unban requests from the Brazil IPs now, where almost nobody has reported them yet, so that system would not work

Problem is I don't know if users or bots are requesting the removals. But anyway the page has been significantly updated now with the CAPTCHA verification. Hopefully if anything that will slow things down a bit.

Powered by GPT5

With the changes you’ve got in place:

• Bots can’t just auto-unban anymore (email approval link needed).
• Spambots get tripped up by Turnstile + honeypot.
• Humans must give you a reason, which gives you context before approving.
• CSP is tight, but now explicitly allows Cloudflare’s Turnstile.

That should filter out 99% of the junk while still letting genuine users request a review.

[exxos]

Just a forewarning, that I'll be doing server updates at some point over the next few days, so the server will be down for a few minutes while the server is rebooted and the new kernel is installed.

Each time the server is rebooted, users may likely see SQL errors for about a minute until the system has fully restarted. This is actually normal and nothing to worry about.

Also the server will generally run somewhat slow after a reboot because of the firewall lists have to be rebuilt from the last saved state which maxes out the CPU for a couple of minutes.

It is difficult to exactly test the restore of the firewall because there is a lot of scripts involved where I can only really test them after a reboot.. So it's possible I may have to spend some time sorting that out and maybe even rebooting the server again to retest. Everything is working fine.

Generally, after every boot, the server mostly settles down after about 5 to 10 minutes. Most users will probably not notice anything anyway..

[exxos]

Updates & kernel updates now completed

Had a bit of trouble with the firewall restoring again after a reboot some entries were missing and GPT seem to have got stuck in a loop again fixing a bash script... GPT seems to say "ahh yes the error is... heres the fixed script"... but still broken like 50 times over

Gave the same problem to Grok and it fixed it first time !! Then Grok died , but came up about 15mins later.. then fixed a few more things first try!

[exxos]

I have "relaxed" some firewall rules to be less "strict". The intention is to not so trigger so harshly on IP addresses which change from good traffic to bad traffic continuously..

The overall problem here is that several bad hits from a single IP is nothing.. but multiply that by an entire country like Brazil, and it soon becomes a bit of a problem!

I've also had to bring back IP rotation (automatic unbanning) simply because it is not realistic to keep banning IP addresses continuously because the list is getting rather large! I did combat this issue, that if X-amount of IPs in a 255 range trigger my script, they automatically moved to a range blocking file . So 1 line instead of 100's. But it doesn't solve the slowly growing lists.

Fail2ban had been erroring for the past couple of weeks because it could no longer add IP addresses into the block list because I only allocate a certain number of IP is to be blocked in ipsets, and it got full again .

The automatic removal of IPs, I won't say what the timeout is, but it is still rather a "long time". Of course repeat offenders will automatically get banned again anyway. But I hope the slightly relaxed rules will strike a better balance between mistakenly banning mobile IPs, but without reducing protection significantly where we start going off-line again...

I don't think we had any significant slowdowns or going off-line recently (aside from the DNS issues the other day) . So we shall just see how things go over the next few weeks...

[exxos]

Also looks like attacks may have quietened down globally as well, as the IPDB list does not have much at all at the moment...

Downloading AbuseIPDB blacklist...
IPv4 addresses added from AbuseIPDB: 99703
IPv6 addresses added from AbuseIPDB: 297

[exxos]

So I just got a unban request from this IP...


https://www.abuseipdb.com/check/45.66.35.30

Capture.PNG (37.2 KiB) Viewed 395 times

2.PNG (112.63 KiB) Viewed 395 times

reason only said "Tor VPN/Browser IP Adress" which does not even answer the question which was asked for comments as per usual...


But this was not even banned by my own server it was banned from the IPDB blocklist..


I can only assume it is some attacker is trying to get un-banned (again) this is why I think having the IP unban page is just becoming a hindrance these days... I think only legitimately ever had 3 requests which look genuine in all the time the page was live.

[exxos]

Well relaxing the rules is not going to work... We are peaking badly with traffic and CPU again now

You would have thought they all would have given up by now.. but nope..

It's probably going to take a few days for the firewall to rebuild...

[exxos]

We have seemed to have lost IP6 connectivity.. I will likely have to reboot and people may be kicked / logged out while I diagnose the problem.

EDIT:

*should* be fixed now

[exxos]

I need to screw around with the forum theme stuff.. so likely random crap will be happening....