PiStorm Chat

[ijor]

Hi Steve,

So I rewrote the firmware to use the Atari 8 MHz CPU clock instead of a 200MHz gpio derived clock.

Not sure this is a very good idea. Working fully synchronously with the main ST clock is, of course, good. But if you got rid of the 200 MHz clock, that means that the RPI inteface and processing is asynchronous, and this is not good at all.

This had the added bonus of freeing up one gpio signal for RESET.

Hmm, why you would need a dedicated signal for RESET? Anyway, seems like you still have some extra GPIO space, or am I missing something?

The current GitHub repo uses this. The firmware for this repo will only work with one latch type though and no matter what I did I just could not get the other latch type to work at the same performance. And that’s where I’m currently at.

It shouldn't be too difficult to support both latch types. It's not clear to me, at all, how this could affect performance. I could understand if it wouldn't work. But why a different type of latch that doesn't affect the control signals, would affect performance? I checked the repo to see if I could spot something, but seems to be fixed to just one latch type?

I would actually get rid of the latches. I don't think they contribute too much if at all. Mostly they just make the timing more complicated. But, of course, this would require new hardware.

Until a stable firmware is written no other development in functionality is worth the time.

I beg to differ, but I think you got the order wrong. Firmware development should be last. As I said previously, you first need to establish that the PI side and the protocol runs smoothly and reaches full 4MB bandwidth. Then the hardware. And only then the firmware.

The main inefficiencies come from the 16 bit io transfers for the pistorm protocol. This is limited by the raspberry pi gpio count . So any new solution needs to take that into account.

Not sure I understand why this would be the main performance issue. Can't the PI bit bang the GPIO fast enough to be able to multiplex the interface? Surely ii can bit bang way much faster than 8 MHz?

Anyway, it seems to me that several GPIO pins are wasted. Why in earth would you need 4 dedicated JTAG pins? I realize this is something coming from the original design. But this could be "fixed" in a new hardware version and "automatically" bring you 4 extra GPIO pins.

Comments have been made about the outdated CPLD. Yes, this project would benefit from newer solutions. Also development. Who will do it? The Atari scene is devoid of support.

Well, yes, unfortunately the ST scene is pretty small. But one thing is related to the other. Personally, I would refuse to waste my time helping on such an obsolete design. Working with a modern FPGA would make development far much easier.

If there are any genuine developers out there then reach out in the discord channel.

Why discord? Not sure this is the best way to attract developers or contributors. Why not here or some other Atari forum?

[Steve]

Why discord? Not sure this is the best way to attract developers or contributors. Why not here or some other Atari forum?

@ijor because the entire PiStorm dev community is in the same Discord channel. It would be very difficult to cross-communicate and share information otherwise. Also the Emu68 dev channels are there. I don't know why you would assume any Atari forum would be more useful, as Steve said, there's literally nobody who wants to help (there have been multiple PiStorm related threads over the years in both major Atari forums, which always end up dead)

[dad664npc]

Something went wrong with the previous post - this is what it was supposed to be

So I rewrote the firmware to use the Atari 8 MHz CPU clock instead of a 200MHz gpio derived clock.
Not sure this is a very good idea. Working fully synchronously with the main ST clock is, of course, good. But if you got rid of the 200 MHz clock, that means that the RPI inteface and processing is asynchronous, and this is not good at all.

The 200MHz clock was used for edge detecting and allowed the latches to be set/cleared within a 62.5ns pulse

This had the added bonus of freeing up one gpio signal for RESET.
Hmm, why you would need a dedicated signal for RESET? Anyway, seems like you still have some extra GPIO space, or am I missing something?

BERR and RESET signal detection had large processing overheads within the emulator. I eventually managed to repurpose the GPIO layout to include these two signals. Reduces read/write overheads as in the case of BERR, it has to be checked for each I/O

It shouldn't be too difficult to support both latch types. It's not clear to me, at all, how this could affect performance. I could understand if it wouldn't work. But why a different type of latch that doesn't affect the control signals, would affect performance? I checked the repo to see if I could spot something, but seems to be fixed to just one latch type?

it shouldn't have been an issue, but....

Until a stable firmware is written no other development in functionality is worth the time.
I beg to differ, but I think you got the order wrong. Firmware development should be last. As I said previously, you first need to establish that the PI side and the protocol runs smoothly and reaches full 4MB bandwidth. Then the hardware. And only then the firmware.

The PI4 is the board of choice because I needed the extra horse-power. However, with the many emulator optimisations I introduced I could get full performance with a PI3A+. There are caveats - If the emulation is using ET4000 for example, this reduces the Atari memory throughput. Raw memory performance at best has always been 3.91MB/s without the emulator running. With emulation, performance is typically 3.7MB/s

Not sure I understand why this would be the main performance issue. Can't the PI bit bang the GPIO fast enough to be able to multiplex the interface? Surely ii can bit bang way much faster than 8 MHz?

The Pi writes to GPIO faster than the CPLD can handle so the protocol has to write to some GPIO multiple times to make sure the CPLD see's it. Then the way the GPIO works on the Pi, signals have to be set and cleared which adds further overheads

Anyway, it seems to me that several GPIO pins are wasted. Why in earth would you need 4 dedicated JTAG pins? I realize this is something coming from the original design. But this could be "fixed" in a new hardware version and "automatically" bring you 4 extra GPIO pins.

Yes you are right. But there aren't many people in the Atari scene who could or would design bespoke boards for the Atari. This project has always been a proof of concept and has not matured to the point where it warrants such expense. But feel free to design a solution

Comments have been made about the outdated CPLD. Yes, this project would benefit from newer solutions. Also development. Who will do it? The Atari scene is devoid of support.
Well, yes, unfortunately the ST scene is pretty small. But one thing is related to the other. Personally, I would refuse to waste my time helping on such an obsolete design. Working with a modern FPGA would make development far much easier.

And there's the problem... as I said it's a labour of love!

Why discord? Not sure this is the best way to attract developers or contributors. Why not here or some other Atari forum?

LOL don't make me laugh! The problem with forums is someone always has an opinion - why don't you do it like this lol. Never any help! Discord is where the original pistorm development was done on the Amiga. There are many sub-channels within it for Amiga updates, JIT etc and of course Atari

[ijor]

The 200MHz clock was used for edge detecting and allowed the latches to be set/cleared within a 62.5ns pulse

I don't know how it was used before. But now all the PI interface must be async. And again, this is not good at all. You are surely getting tons of compiler warnings. And among other things, this means that you can't make any meaningful timing analysis.

Some samples of unsafe async usage (not necessarily the worst, just what I spotted while quickly browsing at the code):

Code: Select all

    always @( c8m ) 	BGACK_DELAY	<= {BGACK_DELAY[4:0], M68K_BGACK_n};

This doesn't do what you probably think it is doing. Didn't actually test this construct in Quartus, but most compilers would not infer any kind of delay with this code. May be this was a typo and you meant to make it synchronous? But then it means that you are ignoring the warnings, possibly because you are getting too many. You should not ignore any warning unless you are sure it is harmless.

Code: Select all

   assign M68K_E = (e_counter > E5) ? HI : LO; //six clocks low (0-5), four clocks high (6-9)
 

This is too bad. You are generating a clock using combinatorial logic. Chances that the E clock would glitch like this. A glitch on a clock could produce spurious clock transitions. It is probably harmless here, only because the CPLD is too fast, and any glitches generated externally would be tiny in the worst case. The very slow ACIA devices would not be affected by tiny glitches.

But this illustrates the risk of using asynchronous logic. If you would use the clock internally, it might fail. In this case it is easy to fix because this works in the 8MHz domain:

Code: Select all

  // Note that this adds one pipeline cycle
  always @( posedge SomeClock )
      M68K_E <= (e_counter > E5) ? HI : LO; //six clocks low (0-5), four clocks high (6-9)

it shouldn't have been an issue, but....

But where is the version for the other type of latch? Or are you using the same binary for both latches?

Then the hardware. And only then the firmware.

The PI4 is the board of choice because I needed the extra horse-power.

I think you misunderstood, and that was probably my fault. By "hardware" I meant the PiStorm hardware, not the PI itself.

However, with the many emulator optimisations I introduced I could get full performance with a PI3A+. There are caveats - If the emulation is using ET4000 for example, this reduces the Atari memory throughput. Raw memory performance at best has always been 3.91MB/s without the emulator running. With emulation, performance is typically 3.7MB/s

I am confused now. With the PI4 you do get full 4MB bandwidth, even with emulation? The performance issue is only with the PI3?

Btw, does the emulator support some kind of prefecth?

The Pi writes to GPIO faster than the CPLD can handle so the protocol has to write to some GPIO multiple times to make sure the CPLD see's it. Then the way the GPIO works on the Pi, signals have to be set and cleared which adds further overheads.

I don't think that the GPIO can toggle faster than what the CPLD can process. It is possible that the Pi processor is bit banging the GPIO too fast, but too fast for the GPIO itself, not for the CPLD.

Non technical issues in a separate message ...

[ijor]

Personally, I would refuse to waste my time helping on such an obsolete design. Working with a modern FPGA would make development far much easier.

And there's the problem... as I said it's a labour of love!

Oh, come on. What this has to do with love? Love is about spending time, not about unnecessarily wasting time with the wrong tools. And don't speak to me about love for Atari as if you were the only one. You have no idea how much time (and money) I spent for my love for Atari. Do I need to pull up my resume with all the contributions I made, for free ???

And I'm not the only one. Most people here in the forum love the platform, including @exxos and the forum team, of course. But each one has his own preferences, priorities, talent, knowledge, and time depending on the circumstances. Just because you didn't get help specifically on this, it doesn't mean that there aren't people that love the platform. It is just an issue of how small is the ST scene, in comparison with the Amiga scene, at least.

@ijor because the entire PiStorm dev community is in the same Discord channel. It would be very difficult to cross-communicate and share information otherwise. Also the Emu68 dev channels are there.

But the problem seems to be Atari specific, isn't it? Aren't we using a specific CPU emulator that was abandoned? Aren't we using a specific firmware? We might even need specific hardware. What we need from the non-Atari PiStorm developers? From what I gathered here, nothing at all. May be I miss something, but I don't see what we need to cross communicate or share information.

As I understand, Emu68 is not suitable for us because it lacks supports for some features that are not used by the Amiga. So why would we need to work with the Emu68 devs? If they are willing to enhance the emulator at some point, and if then they would need specific help for that, well, that might make sense. Otherwise ...

So let me rephrase. I don't know for sure if the forums are necessarily better than Discord. It might be a matter of preferences. But I"m here and I'm willing to help. As I said, I can't involve myself directly with the development. I don't have the time right now. But I can help. I can help with the firmware code and with the hardware design. But I'm not going to register to Discord, let alone install any app. Sorry if this sound like some harsh conditioning, but that's what I can and willing to do.

[ijor]

I might have been too harsh. I apologize if I was, especially to @dad664npc . But I don't like to be accused of not contributing or not loving the platform. This is certainly not fair. Not for me, and not for most us here. But again, I apologize if I was too harsh. Respect and good luck.

[exxos]

I feel like I should interject a little bit here.. To clarify I'm not "taking sides" here.. I think both @ijor and @dad664npc both have valid points and both are correct. We are talking about multiple different issues in multiple different angles here..

Of course while I can agree with @dad664npc that "labour of love" (Something you willingly work hard at without being paid for) is indeed correct, a lot of us of wasted millions of hours on such unpaid work. If I got paid for my work I should be a multimillionaire by now! But no, for some reason, I constantly choose to live in poverty

I know better than anyone that trying to get help with a range of projects is pretty much non-existent. I semi-regularly ask questions help and probably 90% of the time go unanswered, it surely has me questioning why I bother if nobody else cares . Generally, because we all just simply have a lack of technical people. But also it gets complicated, because the community is also a bit fragmented because of various fallings out across various people over the years.. I won't going to all that again here..

I have a backlog of about five years of work personally, and had to drop many projects (including the Pistorm) as they just turn into huge time sinks. Of course I tried , but it was clear from the beginning nobody really seemed to care about the project , one reason I abandoned it. But that pretty much fits the bill for any project these days. So I completely understand that it is a "labour of love" in that respect.

I was also told to head over to discord, which I really did not want to do either.. I have been on it briefly before and it just had a terrible GUI and I just simply found it annoying. As people know, my time is stretched thin and I have RSI, I just do not have the mental capacity to keep swapping between various chat windows and projects all the time. I barely have enough time to read posts on the forum never mind anything else. I have multiple conversations outside of the forum with various people all day everyday.. It is mentally draining to say the least..

While I can also partly agree that forums are mostly based on opinions, this is exactly one of the reasons why I quit atari-forum, as the whole thing was just powered by people's opinions.. Ask for help or a solution and everyone falls silent.. Try and come up with a solution yourself and you get ridiculed.. But this is just how people tend to be these days, I don't think is limited to just forums. But anyway I digress..

Even though it's my opinion, I just got the impression by work that others have done so far, that the PI simply is the wrong thing to use. This is evident by the clear lack of IO for starters which is adding another layer of complication right off the bat. It's basically a hack job which are things I am really trying to avoid in my own work these days!

I also got the impression that it just was not simply fast enough to deal with all the extra signals which is required on the Atari which the Amiga does not need. It would need a faster CPU core.. Or a considerable redesign to improve the code and efficiency which is probably problematic based on the Linux kernel to start with.

As people know, I am not really fan of the whole raspberry Pi ecosystem anyway. Just look at all the time I am spending on raw electronics with the ST536.. It's a minefield alone. Then go slap on the full Linux operating system in the mix as well and it would be virtually impossible to debug anything. I have enough Linux headaches running my server and don't want any more

Of course IMHO, I took the TF536 design (with permission) to build for the ST better, as it was a tried and tested polished product, at least for the Amiga. Mix in the chaotic nature of the Atari with all its quirks and I think I spend more time working on the 536 project than any other project so far! I have fallen victim to this many times that taking something that someone else is designed as "working", and modifying it to something else, is basically taking a shortcut, which always seems to bite me in the ass ultimately.. I also wonder if this isn't the same for the pistorm.

Like for myself, @Icky and @ijor , we have a different design ethos to run "emulation" in "bare logic" such as a FPGA. We are very busy working on that in the background. I personally choose this method has been the best solution for supporting the Phoenix platform long-term. It's certainly going to be more of a raw CPU power type of thing and not all the bells and whistles like the pistorm.. But this does not mean we cannot get there in the end anyway, and without a LInux kernel running in the mix. I'm old-fashioned, I like logic gates not endless layers of software.

So this "argument" that the community isn't helping with pistorm, can be used equally in reverse , that where are the pistorm community helping us with our CPU cores ? We need people familiar with the CPU architecture,Verilog etc as well.

Theres actually a fair few people working on various projects in the community. but, pretty much, everyone always ends up doing the whole project themselves.. Just look at the ST536 project, been ongoing for like 5 years now, 80+ pages of problems. Pretty much doing all the work myself.. thousands of hours gone already.. Many board respins..

I've been asking for literally years for help many things like designing a proper LAN for the H5 Phoenix platform.. If I had the monumental speed boost in people helping with projects then it would free up my time to help others.. Is a two-way street.. This is why developers like myself often end up doing things alone.. And in my case, I just have to abandon projects which just turn into huge time sinks for me. I really should quit the 536 project, but im just too stubborn...

I don't just do a hour here or there either. If I'm not working or at my girlfriends, literally from the moment I wake up, to when I fall asleep, often in the early hours of the morning, I am working on all this stuff.. I'm exhausted as it is, and just cannot take on any more big projects any more. I have several of those already and getting nowhere fast. Despite all the time and effort I plough into these things.

TL;DR;
Indeed the Atari community just too small. I mean look at the Raven, @agranlund built the whole thing himself. Now has support with testers and things are getting fixed and debugged all the time with his fantastic. The point being, he did all the ground work and did everything himself to get booting. That is just how things are in this community. Unfortunate but true.

[stephen_usher]

And it's not just this community.

I wrote a diagnostic ROM for the QL in a basic form because no-one had previously. I thought that if I got something basic done others would help out with further tests and add to the code. But no. Build it and they still won't come.

I did try helping out with some testing for the PiStorm but it seems that the interface card I had started misbehaving and losing its settings. A replacement board is just too much money at the moment, especially for a project which looks stalled anyway.

[dad664npc]

The 200MHz clock was used for edge detecting and allowed the latches to be set/cleared within a 62.5ns pulse

I don't know how it was used before. But now all the PI interface must be async. And again, this is not good at all. You are surely getting tons of compiler warnings. And among other things, this means that you can't make any meaningful timing analysis.

Some samples of unsafe async usage (not necessarily the worst, just what I spotted while quickly browsing at the code):

Code: Select all

    always @( c8m ) 	BGACK_DELAY	<= {BGACK_DELAY[4:0], M68K_BGACK_n};

This doesn't do what you probably think it is doing. Didn't actually test this construct in Quartus, but most compilers would not infer any kind of delay with this code. May be this was a typo and you meant to make it synchronous? But then it means that you are ignoring the warnings, possibly because you are getting too many. You should not ignore any warning unless you are sure it is harmless.

Code: Select all

   assign M68K_E = (e_counter > E5) ? HI : LO; //six clocks low (0-5), four clocks high (6-9)
 

This is too bad. You are generating a clock using combinatorial logic. Chances that the E clock would glitch like this. A glitch on a clock could produce spurious clock transitions. It is probably harmless here, only because the CPLD is too fast, and any glitches generated externally would be tiny in the worst case. The very slow ACIA devices would not be affected by tiny glitches.

But this illustrates the risk of using asynchronous logic. If you would use the clock internally, it might fail. In this case it is easy to fix because this works in the 8MHz domain:

Code: Select all

  // Note that this adds one pipeline cycle
  always @( posedge SomeClock )
      M68K_E <= (e_counter > E5) ? HI : LO; //six clocks low (0-5), four clocks high (6-9)

it shouldn't have been an issue, but....

But where is the version for the other type of latch? Or are you using the same binary for both latches?

The PI4 is the board of choice because I needed the extra horse-power.

I think you misunderstood, and that was probably my fault. By "hardware" I meant the PiStorm hardware, not the PI itself.

However, with the many emulator optimisations I introduced I could get full performance with a PI3A+. There are caveats - If the emulation is using ET4000 for example, this reduces the Atari memory throughput. Raw memory performance at best has always been 3.91MB/s without the emulator running. With emulation, performance is typically 3.7MB/s

I am confused now. With the PI4 you do get full 4MB bandwidth, even with emulation? The performance issue is only with the PI3?

Btw, does the emulator support some kind of prefecth?

The Pi writes to GPIO faster than the CPLD can handle so the protocol has to write to some GPIO multiple times to make sure the CPLD see's it. Then the way the GPIO works on the Pi, signals have to be set and cleared which adds further overheads.

I don't think that the GPIO can toggle faster than what the CPLD can process. It is possible that the Pi processor is bit banging the GPIO too fast, but too fast for the GPIO itself, not for the CPLD.

Non technical issues in a separate message ...

The pi4 can produce full performance.
The Musashi 68000 emulation does include prefetch.
I am not a verilog programmer by any stretch of the imagination. I tend to write it as if it were C. Hence the many requests over the years for a verilog dev.
Yes, there are many warnings when compiling the firmware but even the Amiga firmware compiles with many warnings too.
The firmware should work with both latch types but somewhere during development that functionality got broken.
I have rewritten the firmware and have repurposed the first 8 gpio pins. The idea behind this was to improve the pistorm protocol and incorporate BERR, RESET, IPL1 and IPL2. But I am having Atari write problems being unstable - reads seem okay. If you are interested, I can send you the firmware to look over.

[dad664npc]

And there's the problem... as I said it's a labour of love!

Oh, come on. What this has to do with love? Love is about spending time, not about unnecessarily wasting time with the wrong tools. And don't speak to me about love for Atari as if you were the only one. You have no idea how much time (and money) I spent for my love for Atari. Do I need to pull up my resume with all the contributions I made, for free ???

And I'm not the only one. Most people here in the forum love the platform, including @exxos and the forum team, of course. But each one has his own preferences, priorities, talent, knowledge, and time depending on the circumstances. Just because you didn't get help specifically on this, it doesn't mean that there aren't people that love the platform. It is just an issue of how small is the ST scene, in comparison with the Amiga scene, at least.

@ijor because the entire PiStorm dev community is in the same Discord channel. It would be very difficult to cross-communicate and share information otherwise. Also the Emu68 dev channels are there.

But the problem seems to be Atari specific, isn't it? Aren't we using a specific CPU emulator that was abandoned? Aren't we using a specific firmware? We might even need specific hardware. What we need from the non-Atari PiStorm developers? From what I gathered here, nothing at all. May be I miss something, but I don't see what we need to cross communicate or share information.

As I understand, Emu68 is not suitable for us because it lacks supports for some features that are not used by the Amiga. So why would we need to work with the Emu68 devs? If they are willing to enhance the emulator at some point, and if then they would need specific help for that, well, that might make sense. Otherwise ...

So let me rephrase. I don't know for sure if the forums are necessarily better than Discord. It might be a matter of preferences. But I"m here and I'm willing to help. As I said, I can't involve myself directly with the development. I don't have the time right now. But I can help. I can help with the firmware code and with the hardware design. But I'm not going to register to Discord, let alone install any app. Sorry if this sound like some harsh conditioning, but that's what I can and willing to do.

I think you mis-understand the expression "labour of love" - I'm using it to mean it's something I enjoy doing (as a hobby) with little reward.
The pistorm was designed for the Amiga A500. The Atari uses the same hardware. However the Atari emulation and firmware needed alterations to make it work. @Badwolf made huge strides in this area by incorporating the FC bits, BERR handling and Interrupt handling. However he ran in to performance issues.
The Emu68 developer (Michael Schulz I believe) has stated that it should work with Atari ST but there will need to be a lot of work to get stuff booting etc. (A lot of 68000 machine coding ).
Okay so you don't like Discord - not sure why as it's very similar to your phones messaging app, but we are all different. Just would have been nice to keep everything together.
But if you think you can get a stable firmware then brilliant - I can send you the latest stuff I'm working on