Yay yet another map that they've left New Zealand off...Thought the real-time threat map by spamhaus was interesting https://www.spamhaus.com/threat-map/
Can all registered uses please login, even just for a few minutes..
It helps build a picture where our "good traffic" is coming from..
Thanks :)
It helps build a picture where our "good traffic" is coming from..
Thanks :)
Server updates
Re: Server updates
Re: Server updates
MUCH better thismorning ! all that "noise" on the left, even though it vanished by itself around 11am yesterday, the drop at 9pm last night show a significant drop in traffic.
I will continue to monitor, but I really hope this epic saga is close to being over now. Its one of those things , where the more I look at stuff, the more things I find ! 
- Capture.PNG (96.19 KiB) Viewed 56 times
I will continue to monitor, but I really hope this epic saga is close to being over now. Its one of those things , where the more I look at stuff, the more things I find !
Re: Server updates
Server change: members now have their own reserved resources
Small but useful change today, worth explaining since it affects how the forum behaves under load.
The forum runs on a fixed number of PHP workers, which are the processes that actually build a page when it cannot be served from the cache. Up until now everybody shared the same pool, members and guests alike. That is fine normally, because the cache absorbs nearly all guest traffic and the workers barely get touched.
The problem is when something abnormal comes along. Yesterday a scraper was hitting the forum hard from thousands of different addresses at once, in a way that mostly bypassed the cache. It pushed the workers to 95 out of 96 in use. Nothing actually broke, but there was no room left, and a slightly bigger flood would have taken the forum down for everyone, members included.
So the workers are now split into two groups. Guests get the larger pool, and logged-in members get a smaller pool reserved just for them. Guests physically cannot touch the member pool, so no amount of bot traffic can crowd real users off the site. The total is unchanged, so there is no extra load on the server.
Members do not need many workers. Logged-in traffic here is genuinely light, and a page for a logged-in user renders quickly. Sixteen workers sat waiting is plenty, and it means that even if the guest side is completely saturated, anyone logged in carries on as though nothing is happening.
One honest caveat. The login page itself cannot be cached, for security reasons, and somebody logging in does not have a member session yet, so the login page sits on the guest side. If we ever get hit hard enough to saturate that, logging in might be slow while it lasts. Anyone already logged in is unaffected, which is the important bit. I looked at giving login its own reserved workers too but it added complexity for very little gain.
Routing between the two pools uses the signed token the forum issues when you log in, not a plain cookie. That matters because a plain cookie can simply be made up by anyone, so bots would just claim to be members and help themselves to the reserved workers. The token cannot be forged.
Nothing to do at your end. If anything, being logged in is now slightly better than not, which seems like the right way round.
Also thanks to @viking272 for giving me a poke about linking Claude to my server! One of the main time drains (and hardest on my hands) is copying code that Claude wants me to run, into the terminal, waiting for it to run, then copying the results back into Claude about 57,000 times every single day !! So now Claude can basically access on the server directly to do its research stuff. This is going to be a enormous timesaver and a whole lot more RSI friendly !
Small but useful change today, worth explaining since it affects how the forum behaves under load.
The forum runs on a fixed number of PHP workers, which are the processes that actually build a page when it cannot be served from the cache. Up until now everybody shared the same pool, members and guests alike. That is fine normally, because the cache absorbs nearly all guest traffic and the workers barely get touched.
The problem is when something abnormal comes along. Yesterday a scraper was hitting the forum hard from thousands of different addresses at once, in a way that mostly bypassed the cache. It pushed the workers to 95 out of 96 in use. Nothing actually broke, but there was no room left, and a slightly bigger flood would have taken the forum down for everyone, members included.
So the workers are now split into two groups. Guests get the larger pool, and logged-in members get a smaller pool reserved just for them. Guests physically cannot touch the member pool, so no amount of bot traffic can crowd real users off the site. The total is unchanged, so there is no extra load on the server.
Members do not need many workers. Logged-in traffic here is genuinely light, and a page for a logged-in user renders quickly. Sixteen workers sat waiting is plenty, and it means that even if the guest side is completely saturated, anyone logged in carries on as though nothing is happening.
One honest caveat. The login page itself cannot be cached, for security reasons, and somebody logging in does not have a member session yet, so the login page sits on the guest side. If we ever get hit hard enough to saturate that, logging in might be slow while it lasts. Anyone already logged in is unaffected, which is the important bit. I looked at giving login its own reserved workers too but it added complexity for very little gain.
Routing between the two pools uses the signed token the forum issues when you log in, not a plain cookie. That matters because a plain cookie can simply be made up by anyone, so bots would just claim to be members and help themselves to the reserved workers. The token cannot be forged.
Nothing to do at your end. If anything, being logged in is now slightly better than not, which seems like the right way round.
Also thanks to @viking272 for giving me a poke about linking Claude to my server! One of the main time drains (and hardest on my hands) is copying code that Claude wants me to run, into the terminal, waiting for it to run, then copying the results back into Claude about 57,000 times every single day !! So now Claude can basically access on the server directly to do its research stuff. This is going to be a enormous timesaver and a whole lot more RSI friendly !
-
viking272
- Site sponsor
- Posts: 291
- Joined: Mon Aug 10, 2020 11:32 am
- Location: Reading, Berkshire, UK
Re: Server updates
Great work and glad to help in a small way! Hopefully Claude can help out with further config optimisations, suggestions and help you get back some important time to you.