PiStorm Chat

[ijor]

All I did was write the SD card and install it in the Pi.
And it runs brilliantly and includes a whole heap of software in the image, Caffeine OS included. You write the image plug it and go, but with Musashi you have to build the image, configure the CPU etc etc and then make any other changes you need and then it might work.

I understand. But that seems to be more a package distribution issue, and not something specific to being bare metal or not. Developers could perfectly provide a non bare metal ready to run SD card image, with everything installed and configured.

Sure, using Linux has the added advantage that it's fairly straight-forward to pass-through devices like Ethernet, Wifi etc transparently, but Emu68 already has functionality built in for retargetable graphics, ethernet and wifi.

I realize. But there would always be more peripherals that you might want to use. What if you want to use Bluetooth (I love Bluetooth keyboards and mice)? What if you want (or need) to use a proxy? Or you want to share the hard disk with Linux. Or you would like, say, a web server to manage things remotely. High level scripts. Etc, etc. It will always be much easier if you are not running on bare metal. Let alone that bare metal developing is much more complicated.

Perhaps it's just the inefficiency of the Musashi emulator, but emu68 is significantly faster and more accurate.

I see. So probably there is no much choice, at least not currently.

Btw, checking the Emu68 repository, it seems that it is compiled as big-endian, which is obviosly more efficient for emulating a 68K. But that might explain the main reason for being bare metal. It also seems that Emu68 uses JIT, which is more like virtualization instead of "classic" emulation. This is much more efficient.

The PiStorm started out a few years ago, so the original versions CPLD has become rather old now. But later models of PiStorm already updated this, for instance the Pistorm32-lite uses a Efinix Trion T8Q144 FPGA. There are also other variants and newer developments since.

Oh, I see. But then why the ST (or is STE?) version uses the old Max II design? That could be one of the problems. It is possible that to add the extra functionality that we need for the ST, it would require a more modern FPGA.

Arbitration/error handling: For the Atari ST (and Amiga CDTV), compatibility isn't there yet because essential signals like bus arbitration, bus error, and processor state pins are not yet correctly driven.

That makes more sense. So it is not that the bus arbitration is intrinsically more complicated or more difficult to implement (I know it's not). The problem is that it is difficult just because it wasn't yet implemented in the original design.

Perhaps you're the perfect kind of person for Michael (lead developer of Emu68) to have a conversation with around these kinds of topics.

I can't involve myself directly in the project, at least not at this time. But if he has any specific issues, I would mind at all to help.

However I would strongly recommend first to migrate the hardware to a modern FPGA. But please, not to one of those cheap Chinese devices. That made sense during the Pandemic when major brands (Altera/Xilinx) were not widely available. But now, just for saving a few bucks, it is IMHO a bad idea.

[stephen_usher]

The Atari PiStorm "team" is basically one person, who is not a hardware person. The project is just having to use the off the shelf Amiga A500 PiStorm with specific types of latches.

So, it's stuck with the off the shelf boards subsidised by the Amiga people.

[Badwolf]

The background is that the original PiStorm was (and remains?) a Musashi-based program running as root on top of the standard Pi OS.

Aside from the emulation it interacts directly with the GPIO pins to multiplex bus signals through to the CPLD which is in charge of driving and sampling the 68k pins. Communication is driven by the PI with a couple of pins being used to indicate when data (such as a change of interrupt state) is pending.

The Amiga 500 version had no need for bus arbitration, which wasn't implemented CPLD-side nor the FCx lines, which weren't built into the emulator nor passed through the multiplex but the big hit was bus error. Bus error is triggered from the motherboard and there wasn't a communication channel for that. Also being able to handle bus error needs to be built into Musashi (it wasn't for the A500 version) and offers a significant performance hit.

My first ventures into the PiStorm world saw me add in the FCx lines (they could be put in the unused upper address byte exchange) and repurpose the RESET signal for BERR. I also added some extremely rudimentary bus arbitration emulation into the CPLD.

The bus cycle state machine (S0-7 emulation) had been optimised for the A500, dropping some states and simplifying it to the minimum necessary to work. That was different for the ST so I had to modify that too.

Interrupt vector handling had to be added as the A500 only used autovectored interrupts (IIRC).

This did, for the most part, work. The original A500 interrupt code was known to be a bit of a hack and would often over-trigger. This wasn't really a problem with the autovectored interrupt on the A500, but it was a significant reliabilty problem when the MFP is issuing vectors and the emulator tries to ask for a vector on a phantom interrupt.

The biggest issue, however, was bus cycle speed. The emulator as is now stood was not quick enough to be ready for the next CPU cycle. If it were performing a bus read it would need to wait for the result of that read, check it wasn't a bus error, do whatever logic it was meant to do at that point (could be a jump) and then start the next read. All through a multiplexed bus. It would simply miss its slot meaning ST-RAM access was slow. Not quite half speed on average, but of that order.

Oddly, in the Amiga world, this doesn't seem to be an issue. I don't know if it's because the concept of chip, slow and fast RAM has been around from the very early days and programmers take it into account, but RAM accesses under 3MB/s didn't cause the Amiga PiStorm scene much pause for thought at all. That is simply too slow for ST accesses (which need to aim for 4MB/s).

So this is the point I handed the keys over to @dad664npc. He was much more of a in-the-weeds software guy than me and I hoped he could really give the emulator a good optimising to hit that 4MB/s mark.

Spoiler: he could. And he'd go on to add many more features to the emulator side too.

But a lot of the other issues remained. It's running at the jagged edge to hit the timings, the interrupt logic was known iffy and the problem of BERR and RESET were outstanding. Interrupts accounted for much of the reliability issues and the super-tight timing for another bunch. Every different machine tested on needed more tweaks and exceptions. Much like we see with Exxos' STE536 firmware.

The firmware diverged from the original to such and extent I'm not sure they're mergeable any more and Steve will probably admit he, as I, is not a verilog expert.

That's the point where I no longer had time to keep up and dropped off the radar. AFAIK things kept iterating away since that time and whilst many new features are there, it's still more like tuning your classic 1920s racing car than getting into the Fiesta on your drive.

My suspicion is that what it really requires is fresh eyes on a new firmware based on the emulator's current output, but I'm out of the picture so long now that I may be wide of the mark and it's simply that the emulator is asking too much of the PI hardware to hit that 4MB/s timing and it'll never quite get to full stability.

BW

[rubber_jonnie]

Well, since I am discussing one of the PiStorms that exxos has, I am willing to do testing for this when I get my hands on it, and though I have limited time I'll throw my hat into the ring and offer whatever help I can give.

I'm no electronics or software expert - I just tend to use stuff - but structured testing I can do.

[ijor]

The background is that the original PiStorm was (and remains?) a Musashi-based program running as root on top of the standard Pi OS.
...
My first ventures into the PiStorm world saw me add in the FCx lines (they could be put in the unused upper address byte exchange) and repurpose the RESET signal for BERR. I also added some extremely rudimentary bus arbitration emulation into the CPLD.
...
This did, for the most part, work. The original A500 interrupt code was known to be a bit of a hack and would often over-trigger.
...
The biggest issue, however, was bus cycle speed. The emulator as is now stood was not quick enough to be ready for the next CPU cycle.
...
Oddly, in the Amiga world, this doesn't seem to be an issue.
...
So this is the point I handed the keys over to @dad664npc ... But a lot of the other issues remained.

If you ask me, I think that this is absolutely crazy. My humble recommendation:

1 - Build a testbench to make sure that Musashi can interface with the ST fast enough. Modify or optimize the protocol if necessary.
This doesn't require to build any custom hardware. You can use any FPGA dev board connected to the PI "emulating" an ST host. You can use a fast microcontroller, if you prefer instead of an FPGA, even a Pico. No point in building any hardware, let alone actually connecting to an ST until this is working smoothly as it should.

2 - Once you are sure that the RPI side is ready, build a better an modern PiStorm adapter. I would use a MAX 10 that nowadays it should cost about the same, if not less, than using an obsolete Max II.

Don't copy the Amiga PiStorm design, at least not blindly. E.g., I see that the original design uses a small resistor as a level shifter for the CPU clock. IMHO, this is very risky. Certainly bad practice. Also remember that some signals, such as IPL0, are unused in the ST.

[Badwolf]

The biggest issue, however, was bus cycle speed. The emulator as is now stood was not quick enough to be ready for the next CPU cycle.

If you ask me, I think that this is absolutely crazy. My humble recommendation:

1 - Build a testbench to make sure that Musashi can interface with the ST fast enough.

I don't know if you've got the wrong end of the stick from my post, but it's not that it can't interface fast enough, it simply couldn't execute fast enough to initiate the next bus cycle in time to meet the ST's quasi-synchronous bus cycles at 8MHz.

It spends (say) three and a half cycles after a read cycle starts doing nothing waiting for the results of the ST-RAM access (or ROM read, or whatever) then has 62.5ns to perform the next emulation step and present the next bus cycle in time for S0/1 to start again (these timings may not be the right values, I don't remember them off the top of my head, but you get the idea).

Without any CPU emulation the interface can merrily bang away at full speed. An ST-RAM read/write and speed test is one of the tools in the PiStorm suite and it working reliably and accurately is obviously a prerequisite to firing up the emulator.

BW

[ijor]

I don't know if you've got the wrong end of the stick from my post, but it's not that it can't interface fast enough, it simply couldn't execute fast enough to initiate the next bus cycle in time to meet the ST's quasi-synchronous bus cycles at 8MHz.

Sorry, I probably didn't express myself correctly. When I said interface, I didn't mean just the actual interface between the PI and the PiStorm. I meant everything that is required from the Pi side, including emulation.

It spends (say) three and a half cycles after a read cycle starts doing nothing waiting for the results of the ST-RAM access (or ROM read, or whatever) then has 62.5ns to perform the next emulation step and present the next bus cycle in time ...

Why it waits for the read result? There is no need to. If it works like that, it is extremely inefficient. The next bus cycle, if it's executed immediately, never depends on the previous read. Even a real CPU can't perform something like that. This is one of the reasons that there is prefetch.

Either the next bus cycle doesn't depend at all on the previous one, which is the most common case. Or if it does depend, as it happens on conditional branches, the next bus cycle is delayed for two cycles, it is not started immediately.

The data written on the next bus cycle might depend on the previous read. This happens on read and modify instructions. But this is different. Only the data written, not the address or the bus cycle characteristics. You have two full additional cycles for updating the data to be written. In the meantime you can start the bus cycle without waiting for processing the result.

Without any CPU emulation the interface can merrily bang away at full speed.

Yes, of course, but I meant with emulation, not without.

Again, sorry if I wasn't clear or I was misleading.

[stephen_usher]

It sounds as if there needs to be a CPU core dedicated to the I/O doing the bus cycles and the instruction processing being separated out onto a different core then, so that they are asynchronous.

[dad664npc]

Nice to see conversations on this albeit rather late. @Badwolf started the project off and made huge progress by modifying the Amiga source and firmware to work with Atari ST. I volunteered to help after Badwolf posted a YouTube video on his progress. I was able to improve performance to around 3.7MB/s with bus arbitration and interrupt handling.
However, there was a fundamental problem. It would work on my dev machine but not necessarily on anyone else’s. So I rewrote the firmware to use the Atari 8 MHz CPU clock instead of a 200MHz gpio derived clock. This had the added bonus of freeing up one gpio signal for RESET. The current GitHub repo uses this. The firmware for this repo will only work with one latch type though and no matter what I did I just could not get the other latch type to work at the same performance. And that’s where I’m currently at.
I got made redundant, had to relocate for work and haven’t really gotten back in to it all. But I am still tinkering with new firmware. So the project is not quite dead yet but it really needs a verilog programmer. Until a stable firmware is written no other development in functionality is worth the time.
Comments have been made about the outdated CPLD. Yes, this project would benefit from newer solutions. The main inefficiencies come from the 16 bit io transfers for the pistorm protocol. This is limited by the raspberry pi gpio count . So any new solution needs to take that into account.
Also development. Who will do it? The Atari scene is devoid of support. I’ve reached out numerous times and don’t get help. Really it’s a labour of love.
If there are any genuine developers out there then reach out in the discord channel.
Happy new year!

[Badwolf]

Why it waits for the read result? There is no need to. If it works like that, it is extremely inefficient. The next bus cycle, if it's executed immediately, never depends on the previous read. Even a real CPU can't perform something like that. This is one of the reasons that there is prefetch.

Why? That's deeper than I cared to go. So your guess is as good as mine.

Likely options are:

1) That's how the 68k emulator works (with an assumption it's talking to fast modern memory rather than original hardware);
2) Good enough (perhaps Musashi supports prefetch and blind cycles, but they weren't needed for the A500).

Nice to see conversations on this albeit rather late.

Good to hear from you, Steve! Hope the new gaff is working out.

BW