Kinda already doing that but giving it away for free not exactly any solutionssporniket wrote: Fri Aug 07, 2026 7:54 pm It could be better to write a short book full of your insights with a catchy title like "How I keep afloat my forum despite the tsunamis of AI-driven web-scrapers in 2026" and publish it for a few €/£/$. You MAY earn enough, and you can always update the book and change the title every year
I mean my server thread has a huge book of basically what was going on all the time, and still is..
There are pages like this which I put together recently.. As to how many people actually read it, probably almost nobody. And yet that entire article broadly summarises about two years of solid work dealing with these problems.
https://www.exxosforum.co.uk/articles/s ... _2026.html
That's all basically the roundup of my experience so far..
There are huge topics on phpbb's own forum about it all as well. Also made me chuckle as one person was basically complaining that 3,000 guests on his forum brought his server to its knees, and yet our last hit was 90,000 !
The problem is with what I have set up, it is not exactly simple.. It requires a lot of nginx config changes, custom phpbb extensions. I actually have several of them now doing various things, maybe 3 or 4 related to security / traffic.
Plus I cannot really emphasise how many different attack angles there are. It's like there is a new one every few days and the cycle never ends. So that's the problem of publishing information, after a few months it is probably obsolete anyway. The problem is every tool which is available just isn't designed to cope with traffic in 2026. It all has to be custom based on endless trial and error. But that's the problem, it never simply ends.
I wish I could offer a drop-in solution to other phpbb forum owners, but simpley, I can't, and I realistically don't think there will ever be. Like just I have been realising that there are some botnets faking ChatGPT crawlers. Plus a bucketload of other ones. Its of no real surprise that there are faked user agents, and Google being faked was the main one. So I had a script which would verify every single Google hits with a reverse DNS to actually make sure it was actually a Google address and if it wasn't it would get blocked. Now mix in just about every other agent like GPT, Claude etc etc. now I am writing other scripts to check such things. Any providers that don't publish a IP list, I really don't care about any more, I just do not have time for them. And that is just one isolated problem among many...
The PHPBB extensions had to come about because you couldn't trust the forum cookie to be valid and it could be easily faked. So I had to build a new extension to give a reliable signal if users were actually logged in or not.. And that spawned into a couple of other extensions because of all the things I found as problems as well..
Similar I've been fighting various cache mechanisms over the past two years. It has been anything but simple. Take for example when someone requests a page on the forum the URL parameters can be mixed up and faked and have extra stuff in them, so you end up having to re-juggle the URL parameters on the fly to put them in a standard order to stand any chance of the page registering as a cache hit. Pile on about 50 other problems on top of that.....
Like I just found out yesterday that CAPTCHA is basically pointless now AI can solve it. Apparently the trick is just to keep the AI bots busy as long as possible, just to cost them resources, idea being if they have a lot of work for a small forum like us, spending all the CPU on it might not be worth the effort.
But anyway, the whole problem is immensely complex, this is why just about every site tries to hide behind Cloudflare now. Realistically PHPBB forum software just isn't designed to handle the problems which the users are facing. So I am having to build custom solutions, and that is what is taking all my time up lately.. It doesn't help much because the landscape changes every few weeks..
This is why I separated server resources between guests and registered users earlier today. Because it's realistically impossible to keep on top of everything, and believe me I have tried. So the bottom line basically becomes, guests will just hit a "server busy page" in future. Logged in users will still be fine. But that's assuming "next week" more floods happen which we have no defences for..
The landscape is just progressively getting worse, pretty much every week. The "writing is on the wall" as we say. Keep refining defences and all it does is bide a few more days or weeks until you have to adapt again. So basically guest access is just impossible to maintain long-term.
The laughable thing as well, even if guest access was completely removed, any page would have to be directed to the login page which has to be rendered in real-time.. And you can pretty much guarantee that page will get hit continuously with bad bots.. At that point, how are users actually supposed to login ? There is just no way out of all this.
TL;DR
While I can document a million ways why things will fail and why, I cannot reliably say "do this to fix it all" because that answer changes every week.
