Server updates

[exxos]

Apologies to those getting blocked. It's not by any rules in place. It looks like one of my blacklists is corrupted somehow and weird ranges of ips have been blocked.

I'll try and clear the list tomorrow night when I get back home if not sooner.

Clearing the list may result in the forum running a bit slow as attacks will have to be banned again..

[JezC]

Access to the forum from my phone on mobile data seems to be working again!

I'm not sure which of the above changes fixed the issue -but thanks!

[exxos]

Access to the forum from my phone on mobile data seems to be working again!

I'm not sure which of the above changes fixed the issue -but thanks!

I asked @Icky to wipe the range ip blacklist as I believe this was the cause of the problems for some reason as I had several IP unban requests emailed to me as well over the past few days and they were all circling around that list.

The list will slowly get rebuilt automatically as IPs keep hitting us. I think the first consolidation script was malfunctioning somehow and blocked weird ranges. Hopefully that is fixed now...

[exxos]

I have been, I have been trying to think of how I could basically whitelist the IP the unba page so users can still access that even though there may be completely banned from everything else.

It will be relatively simple to put that into practice, in just having a white listed port number. Though this also has the problem that, as soon as the attackers, find the port is open, it will get hit incredibly hard. And I cannot think of any way of dealing with this because it needs to be open for people to use it, it, but also closed. Not to be abused. It's a bit of a chicken and egg problem.

https://www.proxysite.com/ can be used to access the page. But I'm not sure some people understand how that works.

I wonder if anyone knows any proxy sites which can accept a URL with the destination url included. That proxy site seems to have encrypt what's in the URL. So not much good for simple one click URL to access the unban page.

I was also thinking about automatically detecting people's IP addresses to unban. Obviously this will not work if it is going via a proxy server in the first place. Overall it kind of needs three websites to be bookmarked in order to get unbanned the way things are currently

I'm out of ideas unless anyone else has any ideas about it..

I guess the simple one click solution would be simple for users but the page needs self protection. I can probably clamp most by refusing all HTTP1 traffic. So that's my best idea currently.

[exxos]

Realise that IP a was not correctly allowing IP6 addresses to be checked . So that's now fixed.

I could do with someone basically who has a ( preferably) disposable IP address , that I can ban to see they can access the new unban page still..

The idea is that this page will be accessible even if people's IP addresses banned. But obviously I need to ban someone to see if it actually works first

[exxos]

The IP ban check page has been updated now which has a port number which is whitelisted.

https://www.exxosforum.co.uk:8085/IP_CHECK/

Myself and @rubber_jonnie have confirmed when a IP is banned it can still access that page to request a unban.

Obviously this has a certain range of problems in white listing a port... But will have to cross that bridge when / if we come to it.

Looking at today's log we have a DDOS attack going on the wiki now.. The whole thing is just ridiculous. Playing whack a mole constantly isn't getting ahead.

So I have had to go back to my original bad idea and outright blocking (**) HTTP1 traffic. Just a necessity at this point. Bots do get white listed when they visit the forum, but there will likely be a "drop" on the first connection which is unavoidable. As google publish their IP range, they are all white listed as well. So at least we should not vanish from Google searches. Most crawlers are using HTTP1 still, but good luck to them! I'm not going to spend time with others anymore. They should all be on HTTP2 by now anyway.

(**) This is not exactly true. I am currently blocking based on the rule set still but it should hopefully capture all the bad requests at least.

[JezC]

Seems to have worked for me just now on my mobile...

Thanks @exxos and@rubber_jonnie

[rubber_jonnie]

Seems to have worked for me just now on my mobile...

Thanks @exxos and@rubber_jonnie

You're welcome

[exxos]

Seems to have worked for me just now on my mobile...

Its odd again, as it was only in the IP ranges file. But no other IPs in that range have been found in fail2ban.. So not sure how IPs are being banned. Must be something going wrong with a script somehow

I see what's happened now.. Someone on .139 tried a fake chrome request on HTTP1.. So the range was blocked, and blocked you as well @JezC but it should only block when several different IPs have violated the rule.. So something is wrong with a script im running somewhere..

[exxos]

@JezC I see the problem now. Whatever network you are on, there was 22 attacks in the same IP range (just 255 numbers !) So the script banned the /24 range including your IP.

As we found with @rubber_jonnie , he mobile IP keeps changing like every hour, it makes the whitelisting ineffective.

damned if do... damned if don't really

So if 100 IPs get banned due to attacks.. and then everyone's IP changes, legit users end up banned because of what the IP was used for before.

Technically there could be timeouts involved.. But it does not really solve the problem. And I had to ban IP is permanently as mentioned a few weeks ago because the same IP addresses keep constantly attacking the server...

It's kind of a no-win situation at this point

I suggest people not use mobile networks without a fixed IP until I can think of a solution to this problem. but its not going to be simple. We might have to have a captcha to gain access to forum and hope its enough to block all the bad traffic. Though doesn't really help with allowing bots on the forum either. It could likely take some time to figure something out unfortunately...