Falcon SCSI write issues

[exxos]

The limit seems to be about 3MB..

[Badwolf]

More interestingly, is this comment:

Code: Select all

; 12-Feb-1992   ml 5.0c Modified _do_rw() to use the 14-bit counter on the
;                       ACSI DMA chip of the Sparrow.

which more or less confirms there's an 8MB DMA limit. Interestingly, the ST is limited to 128KB transfers according to the same source. If more than 8MB (128KB on the ST) is requested for transfer then the transfer is split into multiple DMA requests.

Interesting. I wonder if that limit is actually lower on the production machines?

We need to test a 7MB file with AHDI.

Aside from AHDI, is it only HDDriver & EmuTOS affected, or not known yet?

Unknown. Only just confirmed the behaviour yesterday.

BW

[exxos]

Aside from AHDI, is it only HDDriver & EmuTOS affected, or not known yet?

Jookies test program doesn't use any driver.

[dml]

Jookies test program doesn't use any driver.

Does the program perform read & write or just reads? Is there any way for the program to know the DMA address counter is still aligned with the end of the transfer? It may be this problem is not easy to test for until the exact mis-behaviour is understood.

[exxos]

Does the program perform read & write or just reads? Is there any way for the program to know the DMA address counter is still aligned with the end of the transfer? It may be this problem is not easy to test for until the exact mis-behaviour is understood.

I think it just does reads..but it seems writes has the same problem anyway in copying files.. I'll copy the post to Jookie to answer later..

I think EMUTOS may hold a clue as it corrupts FAT..

[jookie]

Does the program perform read & write or just reads? Is there any way for the program to know the DMA address counter is still aligned with the end of the transfer?

The test prog does only 1 single large SCSI READ(10) command, and it waits for DMA interrupt to appear, by polling MFP register...

Code: Select all

#define mfpGpip			((volatile uint8_t *) 0xFFFA01)
#define IO_DINT     0x20        // DMA interrupt (FDC or HDC)

uint8_t wait_dma_cmpl(uint32_t t_ticks)
{
	uint32_t now, until;
	uint8_t gpip;

	now = *HZ_200;
	until = t_ticks + now;              // calc value timer must get to

	while(1) {
		gpip = *mfpGpip;

		if ((gpip & IO_DINT) == 0) {    // Poll DMA IRQ interrupt
			return OK;                 	// got interrupt, then OK
		}

		now = *HZ_200;

		if(now >= until) {
			break;
		}
	}

	return 0xff;                   // no interrupt, and timer expired,
}

Reading of the DMA count register could be added after the command or during the wait for finish, if needed.

From the device's perspective (viewed using DSO and logic analyzer on CosmosEx years ago), first ~3+ MBs are transferred fine, but after the 4th MB the REQ-ACK handshake from Falcon stops, so device cannot transfer more data and just times out (seconds later), and the Falcon thus doesn't get any additional data after the 4MB boundary, and also times out.

[Badwolf]

My tests have all be reading from IDE and writing to SCSI.

OS implementation notes:

EMUCON (seen not to exhibit the problem) implements copy using Fread() and Fwrite() with a 16k limit (https://github.com/emutos/emutos/blob/m ... cmd.h#L101).

EmuDesk (wrapping Fwrite()) uses as much STRAM as it can grab: https://github.com/emutos/emutos/blob/m ... #L918-L936

TOS2 appears to allocate as much RAM as available too, and then calls Fread/Fwrite() as expected: https://github.com/th-otto/tos3x/blob/m ... #L734-L735 https://github.com/th-otto/tos3x/blob/m ... dir.c#L838

In TOS2 Fwrite() ultimately ends up allocating a buffer as large as the contiguous sectors it can find up to the transfer size and then calling usrio(), which is a wrapper to Rwabs(). https://github.com/th-otto/tos3x/blob/m ... 1083-L1084 https://github.com/th-otto/tos3x/blob/m ... at.c#L1093

Rwabs() is one of the functions that a hard disc driver overrides, devolving the matter to the driver at this point.

BW

[stephen_usher]

I know that this is not relevant to Jookie’s test but it should be made clear that:

Size of file != Size of transfer.

I’m very surprised that Atari drivers are trying to send such large chunks all in the one transfer. It’s very unusual.

[Badwolf]

I know that this is not relevant to Jookie’s test but it should be made clear that:

Size of file != Size of transfer.

I’m very surprised that Atari drivers are trying to send such large chunks all in the one transfer. It’s very unusual.

Here's how EmuTOS does it:

The sector count, calculated from file size and contiguous available clusters (1), is passed directly to the hardware and it's set off (2).

1. https://github.com/emutos/emutos/blob/m ... fsio.c#L84
2. https://github.com/emutos/emutos/blob/m ... csi.c#L167

So on a nice clean drive, a large file will be perhaps only three transfers depending on cluster boundaries (head, middle [the big bit], tail).

BW

[exxos]

Just got this from Uwe...

regarding your SCSI issue, while preparing and testing the next HDDRIVER
release I played a bit with HDDRUTIL's "Read Test" functionality. This test
reads quite large (in Atari terms) amounts of data with a single SCSI command.
Usually (but this may depend on the HDDRUTIL release used) the test tries
to transfer up to 5 MiB of data in one go. This works fine with my Falcon.
But 10 MiB in one go did *not* work.

With my Ataris I use SCSI2Pi (https://www.scsi2pi.net/), which is perfect
for conveniently debugging scenarios like this on the Pi command line. Using
a nested intervals approach I found that with my Falcon the maximum amount
of bytes that are correctly transferred are 8388096. For a drive with 512
physical bytes per sector this is equivalent to 16383 sectors.

This is an excerpt from an SCSI2Pi log on debug level with 16384 ($4000)
sectors:

[s2p] (ID 3) [debug] Controller is executing READ(10), CDB 28:00:00:00:00:00:00:40:00:00
[s2p] (ID:LUN 3:0) [debug] Device is executing READ(10) ($28)
[s2p] (ID 3) [warning] Sent 33 byte(s), 512 required
[s2p] (ID 3) [warning] Received RESET signal

Something is obviously wrong here, HDDRIVER had to reset the SCSI bus.

With 16383 ($3fff) sectors there are no issues:

[s2p] (ID 3) [debug] Controller is executing READ(10), CDB 28:00:00:00:00:00:00:3f:ff:00
[s2p] (ID:LUN 3:0) [debug] Device is executing READ(10) ($28)
[s2p] (ID 3) [debug] Controller is executing READ(10), CDB 28:00:00:00:3f:ff:00:20:01:00
[s2p] (ID:LUN 3:0) [debug] Device is executing READ(10) ($28)

The 16384 boundary does not appear to be arbitrary, because it is 2^14. I
am not aware of any official Falcon documentation that says there is such
a limit, though. But a final official Falcon documentation does not even
exist I think. Atari probably had no money left do document things .

16384 is very small compared to what the TT supports, or to transfer sizes
used by other SCSI platforms. The SCSI protocol has been updated several
times to support larger transfer sizes, because they are important for
high throughput. Up to 2^32 sectors in one go are supported even by old
SCSI standards, because for obvious reasons workstations prefer large
transfer sizes.

Anyway, there is kind of a conflict now: Without any official source saying
that there is an 8 MiB limit I am reluctant to change the optimized transfer
mode settings in HDDRIVER 12.7 to 8 MiB. Just the fact that I can clearly
show this limit with my Falcon is not really a proof. On the other hand,
with a Falcon it does not really matter whether the limit is 8 MiB or more.
In particular HDDRIVER users appear not to have been affected by this for
decades, which is not that surprising because. transfers of more than 8 MiB
*in one go* are very unusual with an Atari.