Blitzkrieg Banzai? (SP1 Conversion)

[MarkSheppard]

Well, I think I have a decompression routine that works for SPCAMO version games. Now comes the hard part of reversing it.

[MarkSheppard]

A compressed section is made up of multiple interspersed compressed and uncompressed run-length encoded data blocks.

Each run-length coded data block consists of:

[N] a run-length coding byte (RLB)
[b] one or more data bytes.

The Run Length Coding Byte (RLB) is the trigger that tells us what type of data block it is:

If the RLB is equal to or less than 127, it's a uncompressed run-length coded (URLC) data block, which is used for sequences with mixed values:

[N] The RLB indicates the number of data bytes that follow, which can be from 1 to 127 bytes.
[b] Individual Data Bytes that can be copied/edited as/is.

If the RLB is equal to or greater than 128, it's a compressed run-length coded (CRLC) data block, which is used for large sequences of identical data bytes to save space.

[N] The RLB indicates the number of identical data bytes that follow from 1 to 127 bytes.

BIG IMPORTANT NOTE:
To get the true RLB for a CRLC block, you need to subtract 128 from the initial RLB. For example, if your CRLC has a RLB of 250, it's true length is actually: 250 – 128 = 122 Byte RLB. (This is what tripped me up; I was assuming 250, 0 meant 250 bytes of identical 0 value.)

[b] Data Value for All the Bytes in this block. The decompressed data is the sequence of N times the data byte B.

A visual representation of the typical sequence of a compressed data block is attached.

RED: Section Header. 4-byte Header Marker (37 0 0 0) followed by compressed size (33 0 0 0) and the compressed flag (1).

GREEN (Uncompressed) and BLUE (Compressed) Blocks You can see how the RLB says 16 bytes, followed by 16 bytes, then you have a compressed block of 86 bytes (214 - 128 = 86) with a value of zero; and so on so on.

[MarkSheppard]

With both compressed and uncompressed datablocks both taking up 127 bytes, it looks like a quick and dirty compression method would be:

read chunks of 127 bytes. If bytes all match, compress. If there's mismatch, run as uncompressed.

EDIT: And of course, you don't need to recompress; you can just insert the newly uncompressed section into the scenario file and change the section header to account for the new length and change the COMPRESSED flag from "Y" to "N" and it'll load in game

[DRG]

Both the game code and the code Fred uses have a mixture of " we start numbering from zero here but 1 here"..........it makes life "interesting"

[MarkSheppard]

So far I have:

• Automated Victory Hex placement, value setting, and control setting
• quick unit placement via "[UNIT NAME: A0] XLOC:? YLOC:?"

It's an early proof of concept so far; I need to figure out some way of simply typing in a formation number, e.g. A0 and the editor automatically places that unit on the same hex as that unit.

I won't bother with load/passenger tables. Let SPWW2/SPMBT do that in the editor. Too many things to screw up in an undocumented game file.

Basically, you still have to manually edit facings with mouse right clicking, but that's OK; it's a lot less work than having to "hunt and peck" across a map looking for hex locations off a spreadsheet.

[MarkSheppard]

Naturally, since I've cracked the compression code; this means that SP2 support isn't *that* far off.

There are a few interesting ideas in Hapschott's SP2 collection like:

When Bear meets Jew*
*
______Israel Defend*
____________vs.*
______Russia Assault*
*
Date: October 21, 1974*
Location: Sinai*
Time: At night*
*
Before the cease-fire that ended the Yom Kippur war, the Soviet Union threatened to intervene in the war when the IDF surrounded the Egyptian Third Army. The Russians didn't intervene, but what if they did? This scenario pits the Russians invading an Israeli base at night. Although prepared for armoured combat, the Israelis were not aware that they were facing airborne troops as well. Will Israel still repel Russia? This scenario was made by Ambush. ([email protected] or www.travel-net.com/~orion/) *

___________KALKILIAH*
*
Israeli 202nd Parachute Brigade*
______________vs.*
__Jordanian Police Contingent*
*
Date: Oct 10, 1956*
Location: Kalkiliah Police Fort, Jordan*
*
Units from the 202nd parachute brigade attack the Jordanian Police fort
at Kalkiliah backed by 4 Sherman tanks.
*
Note: The "Raid on Kalkiliah" scenario that comes with Steel Panthers II
puts the date of this action at June 5, 1967. Avalon Hill has it as a
much earlier date of October 10, 1956. I'm not sure who is actually
correct but I chose to stay with the Avalon Hill model of this battle.*
*
*
___Based on the Avalon Hill game*
______"The Arab-Isaeli Wars"*
*
____________Created by*
__________Keith Heitmann*
[email protected]*

Keep Feeding me! Keep Feeding me!*
*
The Invasion at Pearls Airport:*
*
-----------------------*
*
Operation Urgent Fury*
the United States invasion of Grenada.*
*
This is a "What-If" scenario. The actual events do not make a good wargame scenario, but a minor adjustment to reality changes that entirely and makes it a desperate struggle for both sides.*
*
On 25 October 1983, while the 75th Rangers were occupied trying to take the Cuban-built airport at Point Salines, the 2nd/8th BLT USMC battle group hit their first objective -- Pearls Airport in northern Grenada. Details were sketchy on the opposition they would face, but figures bandied about in the hours before the Marines landed had the Grenadan force at battalion strength reinforced by elements of the Cuban military. Even though the Marines had no solid intelligence about what to expect, they were amazingly close to predicting what was -supposed- to be there waiting for them.*
*
This "What-If" scenario is based on the premise that Maurice Bishop was able to thwart the coup, stay alive and remain in power. That, remaining in power and faced with U.S. aggression, Bishop and Castro coordinated a quick, ahead-of-schedule airlift of some of the military hardware that would have arrived in Grenada over the next few weeks. More Cuban military advisors would, of course, have come with the equipment. And, most important, with Bishop alive the people of Grenada most probably would have turned out en masse to defend their island -- and in this scenario they do.*
*
This scenario played from either side is a vicious battle.*
*
Please read the included "PEARLS.TXT" for more detailed information.
*
This is the third of a series of scenarios by the author depicting Operation Urgent Fury.*
*
This is version 1 of this scenario. I welcome any and all comments -- please e-mail [email protected]*

Then there are some old scenarios which are surprisingly prophetic, like this one from 1996:

FLASHPOINT UKRAINE*
*
Ukraine (Red) Defend*
________vs.*
___Russia Assault*
*
Date: June 12, 1998*
Location: Sarny, Ukraine*
*
By the late 1990's, relations between Russia and Ukraine had worsened. Facing economic and nationalistic pressures at home, the Russian government demanded Black Sea port rights from the Ukraine. The Russians also insisted that the Ukrainian government turn over all potential nuclear weapons. Feeling that their legitimacy as a sovereign nation was at stake, Ukraine refused and mobilized its large army. The Russians struck swiftly, launching several strikes at various military bases and airfields near the border. Soviet Special Forces and the Ukrainians battled it out over an airbase.

[MarkSheppard]

Conversion Guide Part II:
################################################## ###

Copy your converted "spmap999.dat" to your WinSPWW2 or WinSPMBT \Maps folder.

Start your game of choice (MBT or WW2), load the scenario editor:

Hit MAP-CUSTOM button and load the map (999).

(NOTE: Unfortunately, it appears that smoke/damage conditions get wiped clean during conversion)

While you're in there, set up the initial conditions of the scenario:

Player Nations, Date, Visibility, Type of Battle, etc.

This information will be in the _Scenario_Data.csv file for handy reference.

and buy the OOBs for each side.

OOB information for the original scenario will be in the _Units.csv file for handy reference.

WHY NO AUTO CONVERSION OF OOBs?:

One of the big reasons why I didn't design an automatic conversion of the OOB/unit list is this:

SP1 OOBs were very simplistic; while SPWW2 OOBs are much more realistic and have more granular detail with 25~ more years of research in them, plus the larger size of OOBs allowing more detail.

Case in point; for Japan in November 1945 in Steel Panthers I; you have the following infantry formations available in the OOB:

Infantry Plt
SNLF Plt
Cavalry Plt
Engineer Plt
Engineer Sec
Sniper
Spec Atk Force
Machine Gun Sec

Meanwhile, in Steel Panthers: World War II, you have for the same time period in the Japanese OOB:

Para Spd Co[A]
Paratroop Co[A]
Para Platoon[A]
Para Mor Sec[A]
Para HMG Pl [A]
Para Enq Pl [A]
Gldr Spd Co[G)
Glider Co[G]
Glider Pl [G]
Gldr Mor Pl [G]
Gldr HMG Pl [G]
Gldr Enq Pl [G]
Para Pl -[A]
Rifle Platoon A
Mot Recce Co
Mot-Inf Platoon
Rifle Platoon C
Rifle Platoon E
Infantry Co A+
Infantry Co B+
Infantry Co C+
Infantry Co D+
Infantry Co E+
Infantry Co F+
Infantry Co A
Infantry Co B
Infantry Co C
Infantry Co D
Infantry Co E
Infantry Co F
Infantry Co G
Infantry Co H
Rifle Platoon D
Rifle Platoon B
Para Pl -
Paratroop Co
Para Platoon
Para Special Co
Para HMG Pl
Imp Guards Co
Imp Guards Pl
SNLF Company
SNLF Company -
SNLF Platoon
Mountain Pl
Mountain Co
Ski Company
Ski Platoon
Militia Company
Militia Platoon
Patrol
Mot Patrol Sec
Mntd Patrol Sec
Mot Scout Sec
Scouts
Mntd Scout Sec
Mountain Scouts
Imp Gds Scouts
SNLF Scouts
Ski Scouts
Sniper
Sniper Group
SNLF Sniper
Mountain Sniper
Inf-AT Section
Special Atk Det
Engineer Pl
Para Eng Pl
Imp Gds Eng Pl
SNLF Eng Pl
Machinegun Sec
Machinegun Pl
Bn HMG Co
Regt HMG Co
Bn HMG Co -
Regt HMG Co -
Heavy Wpn Pl A
Heavy Wpn Pl B
Heavy Wpn Pl C
Para Mortar Pl
AAMG Section

You can now get a much more closer approximation to reality than with a simple "auto convert" -- more work for you in figuring out what the new scenario OOB should be, but in return getting a much more improved scenario -- a "Gold" scenario, so to speak.

Once you've bought your formations, save the game as slot 999.

Copy that savegame file over into the conversion tool folder.

IMPORTANT NOTE:
The current version of SPCAMO slicer forms a file list of the files in it's directory, and picks the first one with the string "_VLOCs.csv" in it's list -- so if you have two scenario dumps in the tool directory, only the alphabetically first CSV file will be used by SPCAMO slicer for VLOC automatic editing.

Run "SPCAMO Slicer.exe"

It's currently hardcoded to load "spWW2scn0999.dat", in this initial release.

It'll spit out a bunch of debug information:

Analyzing spWW2scn0999.dat
Section 0 offset at: 15
Section 1 offset at: 20
Section 2 offset at: 5959
Section 3 offset at: 69968
Section 4 offset at: 70985
Section 5 offset at: 73281
Section 6 offset at: 81107
Section 7 offset at: 82318
Section 8 offset at: 84857
Section 9 offset at: 135069
....

These are the file offsets for each section in the scenario file.

NOTE: Currently, I have no idea how to program it so that it displays section 48's offset correctly, but that's for another day.

It will then display more debug information.

Unpacking Critical Scenario Sections....

Block Analyzed. Data is as follows:
---------------------------------------
Block IDNUM: 1
Block Header Offset: 20
Block Type: Compressed
Block Size: 5930 bytes (compressed size).

What's happening is that the program is now identifying and dumping those sections into the following files

S1_COMP_ORIGINAL.BIN
S1_UNCOMPRESSED.BIN
S17_COMP_ORIGINAL
S17_UNCOMPRESSED.BIN
S37_COMP_ORIGINAL.BIN
S37_UNCOMPRESSED.BIN

The original compressed versions are for debug purposes.

It'll then tell you it's found the VLOC.csv file:

....loading external data files to edit Scenario....

match: West_Front_Early_Scen_038_VLOCs.csv

Victory Hex CSV file Loaded into memory!
VLOC#0 10,30 Value: 50
TEST DUMP OF FIRST ROW OF CSV FILE ABOVE.

Editing Victory Locations!
VLOC#0 10,30 -- 50 Value -- 1 Control
VLOC#1 30,30 -- 50 Value -- 1 Control
VLOC#2 50,30 -- 50 Value -- 1 Control
VLOC#3 10,50 -- 50 Value -- 1 Control
VLOC#4 30,50 -- 50 Value -- 1 Control
VLOC#5 50,50 -- 50 Value -- 1 Control
VLOC#6 70,50 -- 50 Value -- 1 Control
VLOC#7 10,70 -- 50 Value -- 1 Control
VLOC#8 30,70 -- 50 Value -- 1 Control
VLOC#9 50,70 -- 50 Value -- 1 Control
VLOC#10 70,70 -- 50 Value -- 1 Control
VLOC#11 80,70 -- 50 Value -- 1 Control
VLOC#12 80,60 -- 50 Value -- 1 Control
VLOC#13 70,40 -- 50 Value -- 1 Control
VLOC#14 60,20 -- 50 Value -- 1 Control
VLOC#15 10,22 -- 50 Value -- 1 Control
VLOC#16 74,14 -- 50 Value -- 1 Control
VLOC#17 76,20 -- 50 Value -- 1 Control
VLOC#18 83,20 -- 50 Value -- 1 Control
VLOC#19 84,14 -- 50 Value -- 1 Control
VLOC#20 80,16 -- 50 Value -- 1 Control
Victory Locations Edited!

Then it brings up the locations of the units:

-----------------------------------------------------
Numbers of Note:
-----------------------------------------------------
255 -- OFF MAP (Artillery, etc)
777 -- enter when done with editing locations
-----------------------------------------------------

Canadian HQ - A0
XLOC: 35
YLOC: 15
Kommandeur - B0
XLOC:

As said before, if you don't feel like editing everything, or you've edited everything and the program starts displaying blank unit location fields; just punch in 777 and it will break that editing loop and continue on.

It will then reassemble everything via inserting the edited sections (sections 17 and 32 currently) into the original scenario file.

Because of the way Python handles file input; I basically have to reassemble the file from scratch from:

File Data Before Segment
Edited Segment
File Data After Segment

And repeat it so on.

The newly edited file will be named "spWW2scn0998.dat"

It will be like 892 kB (versus about 320 kB for the original file).

This is because I inserted uncompressed sections into the file via changing the header flags.

But don't worry!

Simply copy "spWW2scn0998.dat" over to your SPCAMO scenario folder and load it up in WW2/MBT's Scenario editor.

Change a few things, like move a unit 1 hex away; change the facing, etc and then save it as normal in the scenario editor.

When you check the newly saved file, it'll be back to a "normal" file size as the built in game compression re-compresses it for us, with no work needed on our end.

[MarkSheppard]

Originally Posted by MarkSheppard

It's an early proof of concept so far; I need to figure out some way of simply typing in a formation number, e.g. A0 and the editor automatically places that unit on the same hex as that unit.

Simpler solution. Modified the SP1 dumping to output things like:

C0 SO Rifle Sqd Passenger - B9: T-34/76c at Location 82-10

[MarkSheppard]

73.8 MB ZIP

Current version of the SP1 Conversion tools.

Still includes the full scenarios for SP1; so unpacked size is 946 MB (!!!)

I'll work on figuring out a way to work with ZIP files so that the next release isn't so massively huge on disk space. (Plus, I need to do that so that it doesn't break when I add the SP2 stuff).

[MarkSheppard]

So it looks like:

Stock SSI SP2 Scenarios are uncompressed at 689kb.

SP2 Scenarios saved with the in-game editor have their compression turned "on" and are arranged like "SPCAMO" scenarios and I can use some of what I created to edit SPCAMO scenarios there.

Given that the original concept was for SP2 to be run with a disc in the CDROM Drive (to show movies and to act as copy protection); and given the original requirements:

486 DX/2 66 MHz
8 MB RAM + 1 MB SVGA Card
2x CDROM Speed (330 kb/sec)

They possibly ran some tests with computers using the original Steel Panthers I requirements (486 DX 33 MHz) to see how well it would run on the older hardware and figured out that having the game read uncompressed scenarios from the CDROM drive was faster than trying to have the slower PC decompress scenario files.

They probably figured from an "emotional" perspective, people would tolerate slower load times for save games or user created scenarios on older hardware.