                 CM1: A new delta-file compression scheme

I.  Why another file format?

As users have become more sophisticated in their animations, RAM and
disk-space limitations have become more and more irksome.  The SEQ
file format is quite efficient when it comes to disk-space savings, but
must be expanded back into a delta-like format in RAM for realtime
playback.  The blitter chip should help this, but it is not readily
available yet...unless you want to invest in a Mega, too.

The CM1 compression format is the result of more than 2 years'
work on a compression scheme that combines reasonable compression
efficiency, and realtime playback.



II.  How good is it?

A CM1 and PC1 combination can be nearly as efficient as SEQ when it comes
to disk-space savings, but can be expanded in realtime. So it can
reside in RAM in the same form as it does on disk. Actual compression
factors can range from .50 to .66 for most animations. Thus, a 700K
CM1 animation is roughly equivalent to a 1M delta file.  If each
frame requires 5K, you have gotten an additional 60 frames for free!


III.  How does it work?

The compression scheme used in CM1CON ( the delta-file to CM1-file
conversion routine) just removes a certain kind of
redundancy in delta files.  It has been found that most animation
frame-updates consist of a relatively small number of bitpatterns that
are repeated many times. By storing each bitpattern just once per frame,
we can get significantly better storage efficiency than an equivalent
DLT frame.

However, there is a situation where CM1 could actually be WORSE than
DLT.  This can arise when a large number of bitpatterns is used per frame,
and each bitpattern occurs only a few times.  Then the overhead begins to
work in favor of DLT. You will just have to try CM1CON to see what kind
of savings are possible for each animation... but also note that just
about any kind of compression scheme will have difficulties with the
above-stated conditions.

Because CM1CON requires about 290K of free RAM for internal data
structures, ST owners with 512K may have to disable some of their
desktop accessorys to run CM1CON.
 

IV.  How do I use it?

To use CM1CON, just double-click on it.  It comes up with an alert
box which contains (among other things) the version number. After you
click on OK, it will request a delta file.  After you select a file to
be compressed, the program loads in the first-frame PI1 file and then
requests the name of the output file.  At this time you can choose
a different disk for the output file, to make chaining operations simpler.
Because of some optimization steps, the program may take several
passes per frame.

When CM1CON is done, it asks if you wish to chain the animation.  If
you click on YES, it will bring up another file selector.  Since the
program assumes that the last frame of the first animation is the
starting frame of the chained animation, it doesn't request a new
PI1 file but goes directly to the compression step.  If you answer
NO to the chaining option, it will then ask if you want to compress
another animation.  If not, it exits back to the desktop.

NOTE: in order to make the expansion code run as fast as
possible, the actual frame-update bitpatterns are used, rather than the
XOR.  This turns a read-modify-write (needed for DLT expansion) into a
write-only, with an accompanying speedup.  But this means that animations
in CM1 form cannot run backwards. ( I have found that XOR bitpatterns
produce somewhat "fatter" files, anyway.)  All this optimization has
produced a data format that is actually considerably FASTER than
DLT playback, which is useful for double-buffering.


V.  How do I use the compressed delta files?

Included in this arc file is a .CYB file that, when loaded into
Cybermate, will incorporate the expansion routine into Cybermate.
The file also contains useful hints and application examples, readable
from Cybermate.  In order to load in the expansion routine, it must
be in the same directory as Cybermate and ADDCM1.CYB


V.  Can I use this compression/expansion system in my own programs?

I am making the compression program and companion expansion routine
(for Cybermate) available as a service for Cybermate users.  They can
use the compression routine in their own animations, and distribute the
runtime code with their animations, but they cannot sell CM1CON.PRG
or EXPCM1.LNK. After all, I have spent a lot of time on this
and some recompense ( or at least credit ) wouldn't hurt.

If requested, I will supply interested developers with expansion
object code that can be linked to C programs (ALCYON .O format, and in
awhile LASER .O format), along with calling conventions, AND a complete
file specification for the CM1 (Compression Method #1) format.

           Mark Kimball  July 1988
