exxos's Atari, Amiga & retro forum

I missed this during the summer, so I wanted to say thanks so much to exxos, Icky and any and all that are involved in this.

I for one have missed this site a ton, and I think its really important to have a central resource that will survive and thrive.

Thanks.

James

Bikerbob wrote: 24 Nov 2023 14:12 I missed this during the summer, so I wanted to say thanks so much to exxos, Icky and any and all that are involved in this.
I for one have missed this site a ton, and I think its really important to have a central resource that will survive and thrive.

:thanksyellow:

Yes the wiki needed to come back. Everything is scattered all over the place and we need some sort of central hope to organise it all properly. I have been trying for years to get the database to fix it all up properly and getting it up and running again! It looks like all the effort may finally pay off in the very near future 8-)

Turns out that other external links also tally up with the page table. I was trying to filter by spam keywords but that was going to take a enormous amount of time. So thankfully because each row in the links table tallies with the page table I only have to verify if there is a match or not.. So basically I have checked out several numbers to make sure they match or not match correctly. It pretty much it's a certain number then go straight into spam anyway..


EDIT:

Deleting script well under way. Likely will be running all weekend but I don't plan on doing anything else until Monday now anyway..

Looks looks like the external links database has now cleared up. It has gone from 6 million down to under five thousand :lol:

Next I will check the other databases to see if anything else can be cleared up but basically it is as good as it's going to get currently now.

Found another discrepancy / broken page.


The article number is way out. The same in the database I got originally. So somehow this ID got corrupted and the article went AWOL. Looking on temlib and seeing what the text is, I did find it in the database with a more recent edit. But I had to manually put all the numbers in and trace them through the database which was no fun.

Firstly I tried to just simply export the single row in phpmyadmin. Though if you get it to export one row starting from a particular number, it seems to jump 10 numbers upwards. I did not notice this originally and was wondering why spam was starting to appear in the database again :pullhair:

Then I tried to manually insert a new row into the table. But one of the fields kept complaining it was blank even though I could not actually enter anything anyway :pullhair:

In the end I took the latest revision that was in the database, and updated that one with the text from the late entry. Then fiddle the numbers backwards to use that entry in the database and just forget the "missing" numbered one.

In any case it is fixed now. I just hope there isn't many more of these "Missing" pages.. and no you can't simple add them back in on the wiki itself , as it won't have that either!


It is quite puzzling how this actually happened. It is like a user change the page at some point but while the text entry went in the database, any references to that page were simply missing.

My only hypothesis at the moment is that someone tried to update the page while it was under heavy attack and the query simply timed out and in effect corrupted the database. So it left the updated text in the database but with no references to it. In actual fact there was a reference number but that number did not match any other entries in the database. Hence having to go back and fiddle previous entries with the updated content and using that one.

Have noticed that some of the previous revisions of the pages don't seem to work according to the wiki engine anyway. It either comes at not found or a blank page. Not really sure why it is doing that but not really important. But I should mention that nobody clicks on any previous article to restore content as it could likely make matters worse. Indeed if anyone comes across anything broken then just let me know and I will look into it...

I hate commandline stuff :lol: I tried to use gzip to recursively zip up the entire wiki folder before I updated it. Only zipped up every individual file.. Individually. :lol: :roll: Thankfully the A.I. Knew how to un-f**k that one up :D

So it seems the way to update is to delete everything other than the image and settings file and dump the new version of the wiki ontop and run the update script. That sounds totally oldskool, so it all makes sense!
:hide:

So now the wiki is updating to version 35 LTS. I'm not sure how long this will take but I expect it will be done by morning.

:WTF: :sigh:

I guess something broke while updating the wiki version :roll:


The first problem I use MobaXterm where does not seem to register a directory rename. I renamed the mediawiki folder to mediawiki35 then renamed the old mediawiki folder to mediawiki. So then after restoring the SQL database it brought back version 25. However in MobaXterm even doing a dir, still thinks it is in the mediawiki35 folder even though in the terminal its in the mediawiki folder :WTF:

Second problem is it seems to be in the grump with the user table for some reason...

User name "CooperSetser911" is usable, cannot create an anonymous actor for it. Run maintenance/cleanupUsersWithNoId.php to fix this situation.

There is only 56 users in the user table so I'm not sure where it is getting those names from. However running this script causes an error anyway.

Looking here https://mwusers.org/topic/231-revision_ ... snt-exist/ for both problems I have, it seems the script breaks because of problems with the logging table. That table was on my to do list to fix next so I guess I will do that first this time then update the wiki version and see if it behaves this time round..

I'm running a script to clean up the logging table though it is likely going to all get deleted as the whole thing seems like it is full of spam.

Another problem is this talk page which is the first tab on the home page..


The problem there is the original database does not load anything, but that ID got deleted because it was a spam on the fixed database. But again the same problem as before, why is that ID so high to start with.

My only current thought is as before that the page was attempted to be edited during a huge spam attack and the database timed out before it updated the rest of the database correctly causing the thing to break.

Another horrid thought is that the spamesr had direct access to the SQL database and trashed it immensely. Alternatively every single page on the wiki got edited (badly) to redirect to a spam page..

The problem is I don't know how many broken pages there is unless I stumble on them by accident currently. Every single page probably takes about half an hour to trace through all the IDs to fix it so I hope there isn't a lot of these broken pages to fix.

At this point if the database is damaged to that extent, it could actually be quicker to start from a fresh database and basically copy over all the "good" pages from the wiki and with a script manually submit them to emulate what a user would do and rebuild the database that way. Though it is frustrating because there cannot be that much wrong with the database because the wiki is actually running.I think this could potentially take a while to figure out :(

EDIT:

It does looks like others had issues upgrading around the same version as well https://www.mediawiki.org/wiki/Topic:V35dno671xsssdgs

It seems for some bizarre reason my second test server wiki is now redirecting to the first server :shock: God knows why it is doing that because they are completely separate IP addresses.

Then going directly to a page as opposed to the index does yield the spam results (from the original database)...


It looks like the 2006 one is the actual valid entry.


However the entries completely missing from the original database anyway.

Considering the temlib one seems to be from 2020, and the problems seem to be in 2022 where all the entries started getting badly corrupted.

Looking on the temlib copy..


I know what the text is and can search for it in my updated database where it is actually there still...

Then I have to use rev_id from that row which is ironically also 6327... And get the page table information... Which does not seem to exist... But this makes sense because the wiki engine is looking for ID 4723017 which definitely does not exist because it was spam.

So doing a search for that particular number..


So getting page_id 1274 and looking in the page table for other edits... There doesn't seem to be any.

So using ID 6327 I go one table back to the revision table and the IDs also there .. So there is one valid page, its just not being linked to the correct text id .

So the fault seems to be in the page table... ID 4723017 is invalid as it was a spam post.


As the ID #6327 is where the page actually is.. Lets change it..


And voilà!


I think the order of images got messed up at some point but it is unlikely to make any sense to anyone anyway :lol:

So basically I need some way of searching for invalid pages and then working out somehow what the last actual edit was which still exists in the database and have a script automatically rearrange the pages to link back to older ID's which actually exist still.

TL:DR

Vaid pages on the wiki got replaced with spam pages several times over. My script removed all the spam so there are "missing pages" now. But the pages are still in the database just several edits behind. I can't use the wiki tools to undo edits either as the edits are not showing correctly in the database. Plus that would take too long to do them all individually.

Obviously I cannot spend over an hour figuring out every single missing page as I would be here for the next several months. So I need to figure out a script to scan the database for missing pages automatically and then figure out what the last valid edit page was in the store that page index.

Looking back through the IDs a lot are high numbers because I've been adding pages and doing edits anyway. But it looks like the MINT page is a high number.


When ironically that pages seem to exist albeit a missing image..


Looking back through the IDs... It must have been the cut-off point where my script about started deleting spam...


So maybe there is a many pages which are actually missing.. I need to look at all those indexes and see where things went amiss...