I think you have missed a lot of what's been going on and how many attacks have been ongoing over the past few months. I've posted lots of info about it like the wiki being hit 40,000 times with various account hacking attacks. I've updated fail2ban and now it's correctly dealing with those attacks.. it's not going to deal with next week's new attack vector though.. I keep thinking there won't be anymore. .but I keep getting surprised on how another attack vector manages to turn up... Its been a fulltime job for the past several weeks! not easy when my hands are in agony either and I need to keep away from the PC for the next several weeks.. but it keeps dragging me back!!
As it's been soaking up all my time , 3rd party tools have to be brought in to help deal with all the attacks. I'm losing the battle on my own. This is NOT a trivial problem.. There are so many attacks and different ones happening weekly. Patching them takes times. Fail2ban can only deal with known attacks. The log of what it's dealing with are huge! I've posted all sorts about it all. But I'm just repeating myself again now.
You either have to throw time or money at problems.. I can't keep dealing with all this on my own, so costs going up is the only way unfortunately. The anti-spam services don't cost that much. Its the cloud backup for the full disk which takes up costs. I can't really do anything about that. hourly Backups are done to a offsite server which a member is letting me use. So huge costs have already been reduced there already. I cannot use RSYNC to backup the disk as the server has to go offline to copy the disk. So I am limited to the hosts cloud service for full disk backups. Other than reducing the number of backups I don't really see any alternatives currently.
Self hosting has a whole bunch of issues and I quickly worked out years back it would be more expensive long term.. but I'm not going down that road again after all this time. I have a limited internet package and speed for starters.. things are never simple when you get into all the details. Even if someone offered a "free" server to me... is that person going to be on call 24/7 to offer support? Are they going to deal with the problems within a couple of hours ? Are they going to fix hardware failures within a few hours ? Are they going to do it indefinitely for the rest of their lives? I think it's rather a large gamble to rely on the person hosting such things.
I mean the raw server costs like $22 a month for a VPS 4GB RAM 4 Cores 100GB SSD. Is not really that bad in the grand scheme of things! But also the host (Ramnode) have gone above and beyond to offer me good support which is always extremely important to me to get things done quickly and efficiently. So I would be super reluctant to change from them after all these years. At one point in the very early days I was using a friends server, FOC, one of his staff banned my package (can't remember why) so its at that point I decided going with a proper paid for package with a commercial company and not have my server killed out of the blue. It was just the way to go... There have been many battles fought over the years to end up how things are now..
EDIT:
F2B log for the past 20 hours..

- 11.PNG (211.52 KiB) Viewed 724 times

- 22.PNG (202.87 KiB) Viewed 724 times
3,113 matches and 368 banned. May not sound like much, but each of those IPs is hitting various pages with various attacks multiple times. Its how we end up with multiple IP's doing various attacks per second which then ends up causing the server to run slowly and chuck out 502 errors like it did a couple days ago.
Those attacks maxed out all 4 CPU cores.. the server just ran out of resources to cope. That was over 200 IPs per second, but they only did a small number of hits per second per IP, it made it basically a DoS type attack. Difficult to solve and spot.. So we really need DoS protection, but that's another £5 a month.. and who knows if it would end up banning legit users as I wouldn't have much control over it.
Currently all known attacks are dealt with F2B.. Until a new attack appears where there is no rule yet. So if anyone has done any custom fail2ban rules which isn't part of the default setup, then please send them to me as it may help with attacks i've not seen yet. I've already got a LOT of custom rules wrote.
Cleantalk is blocking registration attempts on the wiki and forum like crazy..

- forum.PNG (124.3 KiB) Viewed 717 times

- wiki.PNG (137.14 KiB) Viewed 717 times
It was starting to get through the regular sign up process and were getting on the forum! Anyway this can really happen is either by brute force on the questions or questions have "got known". Which is kind of worrying considering a lot of the questions are now wanting people to look at store for part numbers to answer the questions... Though I assume the answers have been published somewhere to let the spammers though.. This has happened a few times previously and it is clear that keep changing the questions regularly is turning into another job by itself! I believe the stop forums spam extension broke not long ago as well, I cannot remember why off-hand. So many topics and so many problems each week to deal with..
Plus the attacks on the mail server, the SSH attacks.. The amount of log files which are now monitored is basically all of them now I think! It really is mind blowing the sheer amount of attacks on every single service running on the server. Anything you can think of is under heavy attack. If I had not put the effort in over the past few months, I think the whole server would likely be rather broken and off-line by now. It is why I put considerable effort into mitigating such attacks and also optimising various points on the server to deal with "heavy traffic" better. Always work I have done for free and it sure taken up many hundreds of hours so far!!
It's pretty amazing that when I first set up the server to run this forum, the traffic was practically zero for a long time. While there were undoubtedly some attacks I dealt with at the time, the attacks which server is under these days is just simply unbelievable! Like I said many times, I have essentially been out of work as the pandemic happened and don't have unlimited funds any more. It's not just down to my store purely either. I've literally done no work to pay my "real life bills" for the past weeks or even months now, as all my time has been ploughed into keeping the server online.. Which brings me right back to the topic of this thread...
I have been ploughing so much of my time over the past 30 years into doing Atari stuff and the past several years maintaining the server, paying for it all myself. I don't think it's unreasonable to ask for some help financing it all from the community which I am supporting. I will try and save costs for next year by reducing the number of backups and if I don't have to update code daily all the time, I probably won't need ChatGPT which will save a lot of time and costs as well.. I can't really predict what will happen a years time.