Yay yet another map that they've left New Zealand off...Thought the real-time threat map by spamhaus was interesting https://www.spamhaus.com/threat-map/
Can all registered uses please login, even just for a few minutes..
It helps build a picture where our "good traffic" is coming from..
Thanks :)
It helps build a picture where our "good traffic" is coming from..
Thanks :)
Server updates
Re: Server updates
Re: Server updates
MUCH better thismorning ! all that "noise" on the left, even though it vanished by itself around 11am yesterday, the drop at 9pm last night show a significant drop in traffic.
I will continue to monitor, but I really hope this epic saga is close to being over now. Its one of those things , where the more I look at stuff, the more things I find ! 
- Capture.PNG (96.19 KiB) Viewed 112 times
I will continue to monitor, but I really hope this epic saga is close to being over now. Its one of those things , where the more I look at stuff, the more things I find !
Re: Server updates
Server change: members now have their own reserved resources
Small but useful change today, worth explaining since it affects how the forum behaves under load.
The forum runs on a fixed number of PHP workers, which are the processes that actually build a page when it cannot be served from the cache. Up until now everybody shared the same pool, members and guests alike. That is fine normally, because the cache absorbs nearly all guest traffic and the workers barely get touched.
The problem is when something abnormal comes along. Yesterday a scraper was hitting the forum hard from thousands of different addresses at once, in a way that mostly bypassed the cache. It pushed the workers to 95 out of 96 in use. Nothing actually broke, but there was no room left, and a slightly bigger flood would have taken the forum down for everyone, members included.
So the workers are now split into two groups. Guests get the larger pool, and logged-in members get a smaller pool reserved just for them. Guests physically cannot touch the member pool, so no amount of bot traffic can crowd real users off the site. The total is unchanged, so there is no extra load on the server.
Members do not need many workers. Logged-in traffic here is genuinely light, and a page for a logged-in user renders quickly. Sixteen workers sat waiting is plenty, and it means that even if the guest side is completely saturated, anyone logged in carries on as though nothing is happening.
One honest caveat. The login page itself cannot be cached, for security reasons, and somebody logging in does not have a member session yet, so the login page sits on the guest side. If we ever get hit hard enough to saturate that, logging in might be slow while it lasts. Anyone already logged in is unaffected, which is the important bit. I looked at giving login its own reserved workers too but it added complexity for very little gain.
Routing between the two pools uses the signed token the forum issues when you log in, not a plain cookie. That matters because a plain cookie can simply be made up by anyone, so bots would just claim to be members and help themselves to the reserved workers. The token cannot be forged.
Nothing to do at your end. If anything, being logged in is now slightly better than not, which seems like the right way round.
Also thanks to @viking272 for giving me a poke about linking Claude to my server! One of the main time drains (and hardest on my hands) is copying code that Claude wants me to run, into the terminal, waiting for it to run, then copying the results back into Claude about 57,000 times every single day !! So now Claude can basically access on the server directly to do its research stuff. This is going to be a enormous timesaver and a whole lot more RSI friendly !
Small but useful change today, worth explaining since it affects how the forum behaves under load.
The forum runs on a fixed number of PHP workers, which are the processes that actually build a page when it cannot be served from the cache. Up until now everybody shared the same pool, members and guests alike. That is fine normally, because the cache absorbs nearly all guest traffic and the workers barely get touched.
The problem is when something abnormal comes along. Yesterday a scraper was hitting the forum hard from thousands of different addresses at once, in a way that mostly bypassed the cache. It pushed the workers to 95 out of 96 in use. Nothing actually broke, but there was no room left, and a slightly bigger flood would have taken the forum down for everyone, members included.
So the workers are now split into two groups. Guests get the larger pool, and logged-in members get a smaller pool reserved just for them. Guests physically cannot touch the member pool, so no amount of bot traffic can crowd real users off the site. The total is unchanged, so there is no extra load on the server.
Members do not need many workers. Logged-in traffic here is genuinely light, and a page for a logged-in user renders quickly. Sixteen workers sat waiting is plenty, and it means that even if the guest side is completely saturated, anyone logged in carries on as though nothing is happening.
One honest caveat. The login page itself cannot be cached, for security reasons, and somebody logging in does not have a member session yet, so the login page sits on the guest side. If we ever get hit hard enough to saturate that, logging in might be slow while it lasts. Anyone already logged in is unaffected, which is the important bit. I looked at giving login its own reserved workers too but it added complexity for very little gain.
Routing between the two pools uses the signed token the forum issues when you log in, not a plain cookie. That matters because a plain cookie can simply be made up by anyone, so bots would just claim to be members and help themselves to the reserved workers. The token cannot be forged.
Nothing to do at your end. If anything, being logged in is now slightly better than not, which seems like the right way round.
Also thanks to @viking272 for giving me a poke about linking Claude to my server! One of the main time drains (and hardest on my hands) is copying code that Claude wants me to run, into the terminal, waiting for it to run, then copying the results back into Claude about 57,000 times every single day !! So now Claude can basically access on the server directly to do its research stuff. This is going to be a enormous timesaver and a whole lot more RSI friendly !
-
viking272
- Site sponsor
- Posts: 293
- Joined: Mon Aug 10, 2020 11:32 am
- Location: Reading, Berkshire, UK
Re: Server updates
Great work and glad to help in a small way! Hopefully Claude can help out with further config optimisations, suggestions and help you get back some important time to you.
Re: Server updates
Just a heads up. I'll be working on a new cache system over the next couple of days, so the forum may actually go down or act a little weird during the updates.
Even though I have got everything nicely set up, I realised yesterday, that the system is actually rather complicated, so I have been thinking about completely changing how the whole system works, where it will be a lot more efficient. It basically involves a lot of stuff setup in nginx being moved earlier into the chain where phpbb lives.
Even though I have got everything nicely set up, I realised yesterday, that the system is actually rather complicated, so I have been thinking about completely changing how the whole system works, where it will be a lot more efficient. It basically involves a lot of stuff setup in nginx being moved earlier into the chain where phpbb lives.
Re: Server updates
I've just realised the "quick login box" at the bottom of the index page might be "broken". Best to always use the main login page for now else you may not be able to login without a second attempt. This problem should be fixed soon as well.
In case it needs to be said, if anyone does have any troubles, shout out on another Atari forum or FaceBook group as a lot of members here visit other groups. This way issues can come back to me that way. If nobody speaks up, I will always assume nothing is broken..
In case it needs to be said, if anyone does have any troubles, shout out on another Atari forum or FaceBook group as a lot of members here visit other groups. This way issues can come back to me that way. If nobody speaks up, I will always assume nothing is broken..
Re: Server updates
Quick login box removed, and a note on cookies and JavaScript
The small login box on the front page has gone. Use the Login link at the top right instead.
The short reason: that box carried a hidden security token tied to the exact moment the page was built. Pages are now served from a cache rather than being rebuilt for every visitor, so by the time you saw that token it could be hours old, and logging in would fail with "the submitted form was invalid". A few of you will have hit that already, and I'm sorry, it was not obvious and it was not your end. The main login page is unaffected and always will be, because it is never cached.
While I'm here: cookies and JavaScript are now effectively required to use the forum properly. That is not me being awkward. Cookie-less browsing is the single biggest source of the automated traffic that has been flattening this server, because it makes the forum hand out an endless supply of unique links for crawlers to chase. Turning it off removes an enormous amount of load for the sake of a very small number of visitors, and anyone browsing with cookies disabled in 2026 is having a poor time on most of the internet anyway.
The honest bit. Keeping this place online has got far more complicated than it used to be, and every clever feature is one more thing that can break at three in the morning. Some of them simply have to go. Not because they were bad ideas, but because dropping a handful of small conveniences makes the whole thing an order of magnitude easier to keep on top of, and I would rather spend that time on the forum than on the plumbing underneath it.
One last thing: I'll be working on protection and performance updates throughout today, and probably beyond. I'll keep downtime and disruption as small as I can, but you may see the odd error along the way. If something goes wrong once, it's probably me mid-change and it'll sort itself out. If anything keeps happening, please tell me, because the things I can't see are the ones that stay broken.
The small login box on the front page has gone. Use the Login link at the top right instead.
The short reason: that box carried a hidden security token tied to the exact moment the page was built. Pages are now served from a cache rather than being rebuilt for every visitor, so by the time you saw that token it could be hours old, and logging in would fail with "the submitted form was invalid". A few of you will have hit that already, and I'm sorry, it was not obvious and it was not your end. The main login page is unaffected and always will be, because it is never cached.
While I'm here: cookies and JavaScript are now effectively required to use the forum properly. That is not me being awkward. Cookie-less browsing is the single biggest source of the automated traffic that has been flattening this server, because it makes the forum hand out an endless supply of unique links for crawlers to chase. Turning it off removes an enormous amount of load for the sake of a very small number of visitors, and anyone browsing with cookies disabled in 2026 is having a poor time on most of the internet anyway.
The honest bit. Keeping this place online has got far more complicated than it used to be, and every clever feature is one more thing that can break at three in the morning. Some of them simply have to go. Not because they were bad ideas, but because dropping a handful of small conveniences makes the whole thing an order of magnitude easier to keep on top of, and I would rather spend that time on the forum than on the plumbing underneath it.
One last thing: I'll be working on protection and performance updates throughout today, and probably beyond. I'll keep downtime and disruption as small as I can, but you may see the odd error along the way. If something goes wrong once, it's probably me mid-change and it'll sort itself out. If anything keeps happening, please tell me, because the things I can't see are the ones that stay broken.
Re: Server updates
New cache system now in place. 10 hours straight work. Would have taken me years without A.I. !
I need to build a ACP part to all this for a lot of reasons next.
Now guests should get a "realtime" cached paged again. The cache system depends on what PHPBB now tells it to do, not nginx. The nginx cache system was to "messy" so the rebuild done today solves a whole lot of issues.. and probably added some new ones !
Images and downloads should now feel snappier as well.
I need to build a ACP part to all this for a lot of reasons next.
Now guests should get a "realtime" cached paged again. The cache system depends on what PHPBB now tells it to do, not nginx. The nginx cache system was to "messy" so the rebuild done today solves a whole lot of issues.. and probably added some new ones !
Images and downloads should now feel snappier as well.