exxos's Atari, Amiga & retro forum

This is why it's tricky as well..

Because spam has "?????" so you think yep I can just match that and delete it...


But nope! There's genuine posts which do the same.

exxos wrote: 14 Nov 2023 13:13 I'm not really sure what the original Atari wiki logo was I thought it was a blue Fuji but I haven't found it in any of the files yet. Maybe we should have a new one now...

You're right - wayback machine has it: http://web.archive.org/web/201503151446 ... /Main_Page

Thanks @troed

@Zogging Hell Your site pops up such as http://www.zhell.co.uk/magdisks.html is it no longer live or is the URL wrong ?

I think overall there's going to be a lot of broken links on this wiki. There are several on the homepage alone. Is going to need a serious overhaul to fix it all.

What I may ultimately do is nuke all the users. There is no way to know who has a rubbish password or any accounts which have been compromised.

exxos wrote: 14 Nov 2023 15:14 What I may ultimately do is nuke all the users. There is no way to know who has a rubbish password or any accounts which have been compromised.

I think that's fair. The other option would be to set new secure passwords and send out to those users who are "reconizable" - but it's more admin and you and others are already doing an immense effort bringing this back.

troed wrote: 14 Nov 2023 16:25 I think that's fair. The other option would be to set new secure passwords and send out to those users who are "reconizable" - but it's more admin and you and others are already doing an immense effort bringing this back.

Working out who whos could be a lot of effort. For example..

One entry is simply this, no real name and it almost looks like a spam account. However whoever this is, must be something to do with Atari legend as there is a lot of game related posts if I remember rightly..


But then later there is a lot of these...


Most probably spam accounts, but the number of characters is not consistent. So "Rb" could get filtered out if I used "real name" as a detection field.

I don't think there is a massive amount of users anyway. Maybe 100. The database seems to be protecting the passwords from me changing them as well. But I have not tried a scripted method yet.. If users are kept I need to work out how to force the change of a new password..

Though there is also the question of how many of these people would still contribute to the wiki. A lot of the email addresses and accounts could be 10-20+ years old and no longer valid anyway. There is something like 1.8 million user accounts as well.

It is my preference to at least try and save the users because they are, for the most part, linked to posts on the wiki. Though some users both seem to have been edited by spam bots somehow as well.

Currently I'm still deleting out the spam from the main database. At least when I can get it down from a 39GB database, it should help allow me to run more scripts on lower amount of RAM required on the server. It's a bit of a 2-step problem of sorts. As the spam is deleted I get user IDs of who posted it, so I need to write a script that will delete any remaining posts by that user ID, but that might not be needed. I don't know yet. The users will get cleared out because there will be no corresponding posts to that account afterwards. It is just a matter of waiting for the scripts to finish to see what chaos is still left afterwards and then do more delicate scripts as the database gets in a more manageable size.

I have not looked at the tools for the wiki yet but if there is a tool to force reactivation of all accounts with a new password then that would probably be a useful tool. Anyone which does not reactivate will have their account deleted. Though people can always reregister anyway. The main issue is security in all this and having a lot of accounts with unknown password strengths from 20 years ago isn't exactly a good start.

exxos wrote: 14 Nov 2023 16:56 I don't think there is a massive amount of users anyway. Maybe 100.

Maybe it's the case that the spam accounts started appearing at some point in time so that all user ids lower than that are real users? That's a quick method of sorting them out. Will lose some of the "newer" accounts that registered after the spam problem started but better than nothing.

troed wrote: 14 Nov 2023 17:21 Maybe it's the case that the spam accounts started appearing at some point in time so that all user ids lower than that are real users? That's a quick method of sorting them out. Will lose some of the "newer" accounts that registered after the spam problem started but better than nothing.

Yeah its why I've got a "safe limit" on posts as most happened all at once. When I was delving in the database there was your scanlines article which I mentioned before, there was about 20 spam posts before it, then spam exploded after along with user accounts.

The problem is if any genuine users registered and posted content after that limit. I mean it is literally one in a million of finding them. I think however it is done there is going to be some collateral damage somewhere. But at the end of the day better to lose the odd post here and there than not have the wiki at all.

Ironically the best way for the deleting spam would be that none of the spam pages are linked back to any actual page on the wiki like the main page etc. unfortunately when I looked into it before, there was also several pages where actual users never linked back to any index anywhere either :roll:


Incidentally if anyone wants to take a look at the log files I can upload them. It lists the pages which are deleted in alphabetical order... like this...

There are the normal Russia spam accounts aswell. Though ironically , they don't seemed to have actually posted any content :shrug:


I was going to start looking for particular email accounts like from Russia, then search the database for any posts by those accounts and delete it all. Though I think I'm just going to wait for my script to finish checking all user posts first as its probably going to significantly reduce the actual active spam accounts to check afterwards. Then I'll do the email checking to see what accounts are left.

So far :

3036060 spam posts yet to process with the current script
1732006 users for the current script to process

I found I can *just* run processing on 2 different tables at the same time without running out of resources on the server. Though SQL keeps churning out binlog files which keeps filling the disk up :roll: Seems every 5 hours it sucks up 10GB!! :roll:

:lolbig:

There was a slight flaw in my logic of deleting users with no posts :lol:

exxos wrote: 14 Nov 2023 19:04 3036060 spam posts yet to process with the current script
1732006 users for the current script to process

wow, impressive figures. It seems it is almost impossible to clean that DB
As Wiki wasn't so often updated by real users, I wonder whether would be possible just to remove all users and its edits after particular register day.