Next Generation SHP Viewer

[Mobhack]

Originally Posted by MarkSheppard

I'm currently stalled on the VFX SHP compression; I can get it to compress the first row of pixels in an icon; then it just stops compressing pixel rows.

The GOOD: Both SHPed and this show the exact same broken image; so my image decompression code is valid.

Mark - check your PM's as I sent you an email address there which you can reply to for a copy of the design specs for the VFX library SHP format by Miles Design. That includes the compression formulae they used.

You will need to pass me a valid email as there is no way to attach a zipfile to PM's, and I'm not putting it up here in case of copyright issues.

oh - found that he seems to have put it up on github so I'll PM you with the linky

cheers
Andy

[DRG]

Just curious, was there any difficulty setting up the SHPEdit code in HTML5 ?

[MarkSheppard]

I think I've made a semi breakthrough; I was stuck on trying to recompile SPCAMO stock SHP Icon111 from WW2; and I tried some other SHP files and they showed correctly:



Likewise, some old Warhammer 40K SHPs partially show up:



With this (helped by the very useful cross overlay) I think what's happening is that I've got the SHP compression code now mostly correct...

BUT somewhere along the way, the SHP header stuff is not being passed along correctly; and the SHP header stuff plays a big important part in the SHP file compression procedure; because it defines a bounding box of pixels that we compress and discard the rest (to save pixel space)

[MarkSheppard]

So, I --THINK-- I may have gotten it to work; but it's late at night and I don't feel like editing game files; but I've validated the files in Fred's SHPEd afterwards; so they load in that at least.

Here's v0.82; once again, SPCAMO is the password to let me upload it.

big glaring warning

this is currently not validated completely; so don't use it for heavy file editing [i.e. replacing 100 icons etc] just yet.

[MarkSheppard]

now that I can actually save SHPs; I'm doing some more case testing -- and I need to make some changes to the icon replacement code -- it works fine if the icon is the same size as the one being replaced; but looks horrible if there's a size disparity between the new/old icons.

To fix this, here's a quick replacement; as once again SPCAMO is password.

[DRG]

Take a break. You are making great progress but walk away for a day or two and clear your head

I made a copy of a SHP file I am putting new icons into and replaced the hull and turret of one I had tinkered with a bit

When I was done I loaded it into SHPedit and the hull looked good but the turret had black Kruf below the turret which I fixed with SHPedit as none of that showed in the new editor. I saved the file and viewed the icon in MOBHack



That is usually caused by the wrong-sized Shp file the icon is in and SHPEdit reports

#7 Problem??
cols 72, lines 27
Last #67
#8 Problem??

Shpedit also reports the main file is 89x89 BUT this new editor is reporting them as 90x90 which may explain above

My other issue is I've been using SHPEdit for over 2 decades. I know I saved the file once that is how the screenshot was produced but this is not intuitive to me and I have attempted to go back and redo this but I cannot find how I saved the first time ( it's late here too.......)

[DRG]

OK I figured out how to save correctly " write headers+ images to SHP" ( " write headers+ images to SHP to SAVE" might be better

I looked at the recompiled.SHP with Shpedit and it reports
recompiled.shp
99 Icons
90 Max Height
90 Max Width

It should not be reporting they are 90 and EVERY file in that recompiled SHP reports 90x90 when checked with SHPedit SPFix .

I renamed it to the correct SHP file nameand loaded it in MOBhack and the Icons were skewed like the screen shot above

But also

ALL the icons I never touched with the new editor are skewed because they are 90x90



But it gets more interesting

This icon I never touched at all and this is what shows up in the game (EDIT) using the file I created with your viewer.... it does not show up with the grey zone around the turret barrel like that in the game with the SHPEdit created file.



Sorry for the bad news but I suspect whatever it is a simple thing...... it's finding it that might be a problem

[DRG]

I ran an experiment

I built 2 test Icon inside an 88x88 square instead of 89x89
I check the recompiled.shp with SHPEdit and......

recompiled (1).
99 Icons
89 Max Height
89 Max Width

With V83 it reports



89x89 for that Icon but for all the others in the V83 generated SHP it's



90x90

The change I made using the v.83 version shows up OK in MOBHack


BUT in the game it shows up



and the other icons in the file that I did not touch show up in MOBhack show up skewed as before and in the game with the same issue with the turret as above

There is a glimmer of good news their..... v,83 is making the SHP too big which accounts for the skewing in MOBHack but I have NO IDEA why the turret issue is happening

[DRG]

I ran an experiment

I built 2 test Icon inside an 88x88 square instead of 89x89
I check the recompiled.shp with SHPEdit and......

recompiled (1).
99 Icons
89 Max Height
89 Max Width

With V83 it reports



89x89 for that Icon but for all the others in the V83 generated SHP it's



90x90

The change I made using the v.83 version shows up OK in MOBHack


BUT in the game it shows up



and the other icons in the file that I did not touch show up in MOBhack show up skewed as before and in the game with the same issue with the turret as above

There is a glimmer of good news here..... v,83 is making the SHP too big which accounts for the skewing in MOBHack but I have NO IDEA why the turret issue is happening

HOWEVER

There is another issue



That grey around the turrets is FLASHING in the game,
Those colours are NOT in the Icon palette....... those are the cycling greys used for smoke so that is NOT the "SPCamo Icon only palette ( 2021+) that shows being loaded at start.... it cannot be as those entries are blacked out in that file so that is the full palette being used

Icon palette on left, full palette on right. The flashers are all blacked out in the Icon version

[MarkSheppard]

Thanks for the testing; I've been so focused on making sure that everything decompresses/compresses in SHPEd as a "major sanity check"; that I haven't done testing in the actual game(s) yet.

Did a slight detour away from file compression issues (after spending so long bashing my head against it) and went back into designing usability features.

EDIT:

I think I may have fixed the distorted images:

In my SHPView_sHeaders.js file I had this:

function readImageHeader(dataView, offset, imageNumber)
{
// SHP Image Header: 24 bytes long

const height = dataView.getUint16(offset + 0, true) +1;
const width = dataView.getUint16(offset + 2, true) +1;

const var1 = dataView.getInt32(offset + 4, true); // Use getInt32 to allow for negative numbers
const xStart = dataView.getInt32(offset + 8, true); // Use getInt32 to allow for negative numbers
const yStart = dataView.getInt32(offset + 12, true); // Use getInt32 to allow for negative numbers
const xEnd = dataView.getInt32(offset + 16, true); // Use getInt32 to allow for negative numbers
const yEnd = dataView.getInt32(offset + 20, true); // Use getInt32 to allow for negative numbers

// Include the image number in the log message
conditionalLog(`Image ${imageNumber} Header - Height: ${height}, Width: ${width}, Var1: ${var1}, xStart: ${xStart}, yStart: ${yStart}, xEnd: ${xEnd}, yEnd: ${yEnd}`); // only if verbose logging included.

return { height, width, var1, xStart, yStart, xEnd, yEnd };
}

Removing the +1 from height & width in that function made things show up inside the game + MOBHack for me with no skewed icons.

I then rewrote Icon0128.shp for MBT; doing a test with replacing

{12800, 12801}, //4948. SPW-60PB DDR *********NEW

with a Caesar like SPH icon



Transparency no good in the "zoom out" icon, but seems to work in the zoom in feature.

I then skipped down to an icon in Icon0128.shp that I did not edit:

{12810, 12811}, //4953. 2S1 Gvozdika Ukraine *********NEW winSPMBT v16

It exhibits the same issue:



So I guess I need to look at palette mapping codes....