
NEW SAVE SYSTEM
===============

CHUNKS
------

The savegame format is like IFF, and contains numerous chunks.  Each
chunk has the following format:

      0   [Start Marker]
      4   [Length]           // only the following data
      8   [data......]       // padded to multiple of 4 bytes
     ???  [End Marker]

The markers consist of 4 letters, the start marker has the first
letter uppercase and the remaining letters lowercase, whereas the end
marker (same as the start marker) has all letters are uppercase.  For
example: << Desc .... DESC >>  The length only counts the data inside
the chunk, i.e. does not include the markers or length field.

The format of the chunk data depends on the type (given by the
marker).  Some chunks are just containers for a set of sub-chunks.
Unknown chunks should be ignored by the file reader.  Order of chunks
is usually significant (unless otherwise noted).

Top level chunks have their data trivially encrypted (using XOR on
each byte) and then compressed with LZO (by the miniLZO library).
This is to prevent trivial tampering and to keep savegame files small.
All other chunks behave normally.

Chunks are also used for storing data structures in the file (like a
whole mobj_t or sector_t), including complete sub-structures, but not
pointers.  The start marker of these chunks is all lowercase, for
example: << mobj .... MOBJ >>.  The contents of these chunks are the
field values of the structure in question.  The savegame contains
information on the format of these chunks, so that unknown fields can
be ignored and missing/redundant fields can be reasonably emulated.


Overall Format
--------------

The savegame file contains a non-chunked header, followed by a fixed
sequence of top-level chunks, followed by a non-chunked trailer.  The
top-level chunks are:

  -  Global area
  -  Structure area
  -  Array area
  -  Data area

The Global area contains the important nitty-gritty details, such as
which map this savegame came from, the skill level, the user's
description, etc...  When creating the load-game menu, only this lump
needs to be read and decoded.

The Global area also contains the information needed to check that the
level from which the savegame was saved is exactly the same as the one
being loaded into.  For example, trying to load a PLUTONIA savegame
into TNT will be detected and prevented (or at least warn the user).

The Global area also contains the information on RTS scripts and DDF
entries that exist.  For multiplayer modes, this is also used to
guarantee that everyone is playing on the same map and with the same
_DDF_ and RTS files (i.e. prevents trivial cheating).  For single
player, the user will only get a warning if DDF/RTS has changed.

The Structure/Array/Data areas are for storing all the normal dynamic
information (objects, sector heights, etc...).  The Structure and
Array areas are merely definitional, the Data area contains the actual
data (the meat).  They are described in more detail below.

NOTE: all integer values are written in Little-Endian byte order (i.e.
least significant byte first).  Floating point is written as a 16-bit
signed exponent value followed by a 32-bit signed mantissa value.


Structures
----------

The Structure area of the savegame file contains a list of all the
needed structure types and the fields thereof.  All these structure
definitions occur in the top-level STRU chunk, and each definition
occurs within a SDEF sub-chunk.

Each definition contains:

  -  structure name, e.g. "player_ammo_t".
  -  4 letter marker, e.g. "PAMO".
  -  list of fields.

Each field contains:

  -  field name (e.g. "num" and "max").

  -  primitive type, one of: NUMERIC, INDEX, STRING, STRUCT.

  -  how many elements (usually 1).  Elements are stored in sequence.
     If the savegame has more elements then expected, they are
     ignored, and if it has fewer, the remaining are untouched from
     the default values.
  
  -  for NUMERIC: how many bytes (1, 2, 4 or 8).  String and Struct
     types are auto-sizing.  Strings are IBM cp437 charset.  
     INDEX values are always 4 bytes.

  -  for STRUCT: the name of the structure.

Structure definitions are used for:

  -  the type of an array (see below).
  -  the type of another structure's field.

Unknown fields can be ignored during reading (they represent data that
this engine doesn't know about).  Missing fields will just end up with
default values.  Either of these may produce a warning dialog.

Note: No problem if a structure within a savegame is unknown by the
reading engine, since the arrays and/or fields using that structure
_must_ also be unknown by the engine.  [The savegame writing code
should double check this].

This leads to an important rule: once a field name or array name is
used, it is fixed for all time to remain that same type.  Use a new
name if the type gets changed, e.g. if a "height" field got changed
from integer to floating point, then the new field name (as far as
savegames are concerned) must be something different, like "height_f".


Arrays
------

Pointers, of course, cannot be stored directly in the savegame file.

Instead, we pretend that anything that is a pointer (e.g. a pointer in
mobj_t to another mobj_t) is actually just an integer value, an index
into an array of whatever (e.g. array of mobj_ts).  This goes even for
something stored in a linked-list (though the swizzling / unswizzling
would be harder).

NOTE: These index values start at 1.  Zero is used for NULL.

The Array area defines these "virtual arrays".  It exists in the
top-level ARRY chunk, and contains a list of array definitions in ADEF
sub-chunks.  Each array definition contains:

  -  the array name (e.g. "mobjs").
  -  the array type (e.g. "mobj_t", a structure name).
  -  number of elements stored in the savegame (e.g. 321).

When the file is read, but before any actual data fields are decoded,
_ALL_ arrays (like mobjs, players, etc..) are set to the correct size
and initialised with default values.  Thus unswizzling an integer
index back into a pointer could be as easy as &array_of_foo[index-1].

Unknown arrays can be ignored during reading (but may cause a warning
message to be displayed).

Missing arrays are treated exactly as if they existed and had zero
elements in them (but may cause a warning dialog).  They should not
cause any problems, since nothing else in the savegame file should
refer to them.  [The savegame writing code should double check this].


File Format
-----------

-  HEADER.  Non-chunked.  Contains:

   +  MAGIC NUMBER (`EdgeSave')
   +  PADDING: 0x0D 0x0A 0x1A 0x00.
   +  EDGE version that wrote the savegame (32 bit BCD, e.g. 0x125).

-  Global Area.  [GLOB]

   +  Game name (e.g. "HELL_ON_EARTH")   [WHER]
   <  Level name (e.g. "MAP01")
   <  Level flags.

   +  Level time   [WHEN]
   <  Pseudo-Random snapshot

   +  Skill level  [SKIL]
   <  SP/Coop/DM mode
   <  Console player

   +  Description [DESC]
   <  Date/time

   +  View image  [VIEW] (duke-nukem style).  Optional.

   +  Wad file check  [WDCK]

   +  Level sector check    [LVCK]
   <  Level linedef check
   <  Level thing check
   <  RTS script info

   +  DDF info check  [DFCK]

      // attacks.ddf
      // game.ddf 
      // levels.ddf
      // lines.ddf 
      // sectors.ddf
      // things.ddf
      // weapons.ddf
 
-  Structure Area.  [STRU]

   +  multiple set of: Structure Def  [SDEF]

-  Array Area.      [ARRY]

   +  multiple set of: Array Def      [ADEF]

-  Data Area.  Note: no container chunk (like STRU and ARRY).

   +  multiple set of: Array Storage  [STOR]

      // array storage includes:
      //
      //   + Players
      //   + Mobjs
      //   + Sectors
      //   + Lines
      //   + Triggers
      //
      //   ETC ETC...

-  TRAILER.  Non-chunked.  Contains:

   +  Data end marker (`ENDE').
   +  MAGIC NUMBER (`EdgeSave'). 
   +  Data CRC (every data byte in file before this CRC).


Notes:

  -  Serious problems like bad-CRC or magic values, chunks with
     non-matching start/end markers, etc... should produce an error
     dialog (i.e. definitely NOT ignored).


Lump Formats
============

WHER
----

STRING for game name.
STRING for level name.
32 bits for level flags.
32 bits for gravity.

WHEN
----

32 bits for level time (tics).
32 bits for P_Random snapshot.
32 bits for total kills.
32 bits for total items.
32 bits for total secrets.

SKIL
----

32 bits for console player.
32 bits for skill level.
32 bits for netgame mode (0 for SP, 1 for COOP, 2+ is DM).

DESC
----

STRING for description.
STRING for date/time (preferred format: "5:20 PM  01/Jan/2001")

VIEW
----

32 bits for width  (should be 160).
32 bits for height (should be 100).

then all the pixels in normal (row-major) order.  Each pixel is a
standard 16-bit RGB 5:5:5 value.

WDCK
----

32 bits for number of WADs.

then a STRING for the filename of each WAD, in the same order as
appearing in W_InitMultipleFiles().  The list does not include
EDGE.WAD, GWA files, or any raw lumps given with -file.  Filenames
include the extension (though anything other than ".wad" doesn't make
sense), but the path is stripped off.

LVCK
----

32 bits for number of sectors.
32 bits for CRC over all sectors.

32 bits for number of linedefs.
32 bits for CRC over all linedefs (includes vertex and sidedef info).

32 bits for number of things.
32 bits for CRC over all things (original map-things).

32 bits for number of scripts.
32 bits for CRC over all scripts (static info, not dynamic triggers).

Note: when each object in question has it's own pre-computed CRC
(especially for DDF), then "CRC over all XXX" is computed like this:

   for (cur=0, i=0; i < NUM; i++)
   {
      cur = ((cur << 12) | ((cur & 0xFFF00000) >> 20)) ^ CRC[i];
   }

DFCK
----

32 bits for number of attacks.
32 bits for CRC over all attacks.

32 bits for number of games.
32 bits for CRC over all games.

32 bits for number of levels.
32 bits for CRC over all levels.

32 bits for number of line types.
32 bits for CRC over all line types.

32 bits for number of sector types.
32 bits for CRC over all sector types.

32 bits for number of thing types.
32 bits for CRC over all thing types.

32 bits for number of weapons.
32 bits for CRC over all weapons.

Note: it is less serious when there are RTS/DDF mismatches, a single
player game could continue and might work fine, but in multiplayer
modes it will be strictly disallowed (to prevent trivial cheating).

SDEF
----

32 bits for number of fields.

STRING for structure name.
STRING for structure marker.  Must have length 4 and be lowercase.

then each field definition, which has this format:

   8  bits for field type
   8  bits for field size (for NUMERIC).
   16 bits for number of elements (usually 1).
   STRING for field name.
   [ STRING for embedded structure name, for structure type ]
   [ STRING for index array name, for index type ]

ADEF
----

32 bits for number of elements in array.

STRING for array name.
STRING for array type (a structure name).

STOR
----

STRING for array name (to look up in definition list).

then a sequence of structures.  Number and type of structures is given
by the ADEF definition lump.


EDGE INVENTORY
==============

Here is a list of all the structures and arrays needed by the current
version of EDGE (1.25pre1). 

Structures
----------

playerammo_t:

  -  current number.
  -  current maximum.

playerweapon_t:

  -  weapon type (STRING).
  -  owned flag.
  -  shot counts.
 
psprite_t:

  -  current state (INDEX to state).
  -  tics remaining.
  -  screen x/y offsets.
 
player_t:

  -  corresponding object (INDEX to mobj).
  -  current player state (Note: for SP, always PST_Live).
  -  player number.
  -  player in game ?
  -  view height stuff (viewz, viewheight, deltaviewheight, bob).
  -  kick offset(s).
  -  frag values.
  -  health and armour.
  -  keys held (32 bit value).
  -  ammo held (limit to 8 different types) (STRUCT playerammo_t).
  -  powerups in effect (limit to 16 types).
  -  first weapon (INDEX to playerweapon).
  -  ready and pending weapon values.
  -  weapon key choices and refire value.
  -  remember_atk values.
  -  cheats in effect.
  -  kill, item & secret count.
  -  extralight and flash values.
  -  jump wait tics.
  -  damage, bonus, grin & attackdown counts.
  -  weapon psprite (STRUCT psprite_t).
  -  flash psprite  (STRUCT psprite_t).
  -  crosshair psprite (STRUCT psprite_t).

  +  network stuff ???  (Presumably NOPE).
     +  consistency, netcmds, netnode
     +  thinker stuff.

spawnpoint_t:

  -  x/y/angle/type/options.
 
shootspot_t:

  -  x/y/z.

mobj_t:

  -  current position x/y/z/angle/mlook.
  -  floorz and ceilingz.
  -  momentum x/y/z.
  -  speed and fuse.
  -  info type (STRING).
  -  current state and tics, next state and tic_skip.
  -  current health.
  -  current flags and extended flags.
  -  movement direction and count.
  -  reaction time and threshold.
  -  player (INDEX to player).
  -  last look value, side value, orig_height value.
  -  spawn point (STRUCT spawnpoint_t).
  -  visibility and target visibility.
  -  current attack (STRING) and spread count.
  -  path trigger (INDEX to trigger).
  -  first shootspot (INDEX to shootspot).
  -  source mobj (INDEX into mobjs).
  -  target mobj (INDEX into mobjs).
  -  tracer mobj (INDEX into mobjs).
  -  supportobj (INDEX into mobjs).
  -  ride_em (INDEX into mobjs).
  -  ride_dx & ride_dy.

iteminque_t:

  -  mapthing: x/y/angle/type/options.
  -  time tics remaining.
 
vert_region_t:

  -  floor/ceiling plane (INDEX to sector + 1 extra bit).
  -  properties (INDEX to sector).
  -  extrafloor type (STRING).
  -  extraside (INDEX to side + 2 extra bits)
 
sector_t:

  -  floor/ceiling:
     -  current height.
     -  current texture (STRING).
     -  x and y offsets.
     -  translucency.
  
  -  properties:
     -  light level.
     -  colourmap (STRING).
     -  special type (STRING).
     -  gravity, friction, viscosity.

  -  first region (INDEX to region).
  -  floor/ceiling specialdata (INDEX to activesec)
 
line_t:

  -  flags and tag number.
  -  activation count.
  -  special type (STRING).

side_t:

  -  top/middle/bottom:
     -  current texture (STRING).
     -  x and y offsets.
     -  translucency.

rad_trigger_t:

  -  parent script (INDEX to script).
  -  disabled and active flags.
  -  current state (INDEX to rts states).
  -  repeats left and wait tics.

light_t:

  -  sector (INDEX to sector).
  -  light type.
  -  minimum, maximum, direction.
  -  lighttime, darktime, probability.

button_t:

  -  line (INDEX to lines).
  -  original texture (STRING).
  -  where value.
  -  timer value.
 
sec_move_t:

  -  movingplane type (INDEX + 1 extra bit).
  -  sector (INDEX to sector).
  -  floor-or-ceiling and crush.
  -  startheight, destheight, speed.
  -  direction and olddir.
  -  tag and wait value.
  -  sfxstarted and completed.
  -  new special.
  -  new texture (STRING).
 
slider_move_t:

  -  sliding door type (INDEX).
  -  line (INDEX to lines).
  -  current opening.
  -  direction.
  -  waited tics.
  -  sfxstarted & final_open.
 

Arrays
------

players         :  player_t
shoot_spots     :  shootspot_t
mobjs           :  mobj_t
itemqueue       :  iteminque_t
r_triggers      :  rad_trigger_t
lights          :  light_t
buttons         :  button_t
activesecs      :  sec_move_t
active_sliders  :  slider_move_t
regions         :  vert_region_t
sectors         :  sector_t
sides           :  side_t
lines           :  line_t


Level CRCS
----------

sector:

  -  original tag and type.
  -  original floor & ceiling heights.
  -  bounding box.
  -  number of lines.

linedef:

  -  original tag and type.
  -  vertex coordinates (v1 & v2).
  -  some original flags (blocking, passthru).
  -  a bit for the presence of each side.

things:

  -  original type.
  -  original x/y/angle.
  -  option flags (skill, ambush, etc).

rts scripts:

  -  appear flags.
  -  min/max players.
  -  coordinates (x/y/z1/z2) and radius.
  -  name and tag number.
  -  triggering info (disabled, use, independent, immediate).
  -  repeat info.
  -  path info.
  -  ondeath and onheight info.
  -  states (tics, label, and action-specific info).
 
ddf entries:

  -  name and number.
  -  any fields that effect gameplay (i.e. not just visuals/sounds
     like PALETTE_REMAP).  For example: radius, height, states,
     etc...


Implementation Details
======================

Structure Tables
----------------

For each structure in the savegame, there will normally be a table in
the code describing the structure.  Each element of the table
contains:

  -  the field name (e.g. "num" and "max").

  -  primitive type (see above).  Used for writing the SDEF chunk.

  -  a routine to read the storage.  E.G. SV_MobjGetHeight.  The
     routine is passed a pointer to the current structure.

  -  a routine to write the storage.  E.G. SV_MobjPutHeight.  This can
     be NULL, meaning that this field only exists for backwards
     compatibility with older savegames (the read function will be a
     fudging routine).  These fields don't get written to the SDEF
     chunk.

Outside of this table, there is also:

  -  a routine to create the array of structures, size NNN.  The
     structures are initialised with default values.  For certain
     arrays (e.g. sectors & linedefs) this routine does nothing since
     those arrays are created when loading the level data.
  
  -  a routine to finalise the set of structures.  This can be used to
     set up links or compute reference counts (etc...), which are not
     done during reading.  For example, it is used to link mobjs into
     the correct subsector and/or blockmap lists.
 
Reading a valid structure is as simple as calling:

   SV_ReadStruct(&info, struct_ptr);

Writing a valid structure is as simple as calling:

   SV_WriteStruct(&info, struct_ptr);

