SWBF_MSH_INFO
v0.05 alpha

This will eventually become a tool for tweaking and modifying MSH files outside of a model editor.

Currently it is simply a viewer of raw data with no editing capabilities.
If you want to 'see' how the tags fit together in a msh, this will do it.

The plan is to make it into a full-on, MSH file tweak and edit tool. With advanced chunk swapping, attribute managment and vertex/polygon manipulation features.

It doesn't do any of that right now.
Currently, it displays raw and translated information contained in a MSH.
Once this program can display the information correctly, I'll start making things editable. (very alpha right now.)

I have been on this for several months and it is now stable enough to get it into the hands of the curious.

FEATURES:

-32 and 64 bit versions (I can't test 64 bit on my end)
-Open SWBF1 and 2 MSH files
-Tag and Tag Chunk export
-Displays MSH Tag information in readable format (for known tags)
-Displays MSH Tag information in hex format (for unknown tags)

KNOWN PROBLEMS:

-The GUI is not displaying properly on Win7 64 bit. (this one may cause me to change languages)
-It is written in AutoIT, so the initial loading of the file is a bit slow (several min for the AT-AT.)
-Size validation is broken. Don't use it.
-Fix Sizes is broken. Don't use it.
-A lot of the descriptions are messy, missing and/or inaccurate. These will be fixed.

PLANNED FEATURES
in no particular order

-Better display of tag data
-Import Tag and Chunk files
-Import/Export txt or xml of all data
-Possibly save as Obj or other 3d formats (don't count on this one)
-Validate Skeleton Names and Heirarchy
-More Translated tags
-Move Tags or Chunks, up or down the heirarchy
-Tag specific controls for editing data
-Overall Scalling of geometry
-Insert/Delete Vertexes and associated normals, uv's, ect
-Switch between HEX and translated data display
-Display the bone heirarchy
-Move MODLs around the bone heirarchy
-Merge Geometries (for shadow volumes)
-A help file and or proper documentation.
-MSH Summery/Report (Poly counts, Textures used)
-.option and basic ODF generation
-Very Basic projection of Geometry (for visualizing vertex placement)
-User Notes and calculator(s) for planning edits to achieve certain goals

WISHLIST FEATURES:
these will probably never happen.

-Rendering a MSH
-Convert MSH to XSI source
-Munge / Un-Munge a MSH

For an idea of what this program is showing you, my MSH Notes are here: [info].msh structure notes
And Ande's MSH research here: Star Wars: Battlefront MSH File Information

DOWNLOAD HERE: SWBF_MSH_INFO_v0_05_alpha.zip

There are 2 versions in the archive. The 32 bit version is marked with 'x86' and the 64 bit version is marked 'x64'.
There may still be issues with the 64 bit version. If the issues can't be nailed out, I will port over to another language that doesn't have issues.

Any questions, suggestions, corrections, ideas, flames, or comments are welcome.  I'm not a programmer, but I will try.

This post can be found at the following forums:
www.swbfgamers.com
gametoast.com

History:[spoiler]
01/31/2012 v0.05 alpha
          -Moving forward on this release as I am seeing a dentist soon and that
           may interfere with the programs progress.

          -[new bug]something in the way I'm decoding FWGT is breaking the
           'validate tag size' function so I'm disabling it for now. Also
           disabling the 'save msh' till I am confident with the size issues.

          -Customizing the data display for each tag is getting annoying from my
           end. I'm switching to a generic [line number] [data] format till I
           get past the translation hurdles.

          -Started implementing tag definitions again. Got a bunch showing, and
           they look correct. (so far) Time to catch up on the known MSH tag
           translations.

          -Started working on a 'Move Tag UP/DOWN' feature. Initially it works,
           but hoses up the parent structure. I'll probably make it a
           'Move Chunk'feature and limit movements to valid positions. Maybe
           only allowing MODL chunks to be moved since that's all it would be
           good for anyway.

          -[resolved bug] - The loading slowdown is caused by resizing the arrays
           on every entry. So I just need to resize the arrays before my loops
           begin. (initial tests show a 1600% increase in tag conversion speed)
           'THE' fix so far is to split up how I save data during conversion
           into 3 functions, instead of one.

          -the upshot of the following 3 is that the determiniation of sizes for
           each tag are now correct, that chunk exporting is reliable, and the
           summery display of tags within a chunk no longer bleeds into the next
           chunk or to the end of the parent chunk.

          -[resolved bug] - It's truncating 4 bytes off of some tags during
           load. This one trumps everything. I was guessing at the chunk size,
           now I'm actualy measuring. Doesn't slow it down much. (+/- .001 sec)

          -[resolved bug] - CLTH's size in dummy.msh, was including SWCI (16 bytes)
           Corrected size. It was probaby caused by the last child error below.

          -[resolved bug] - The last child tag is not being calculated
           correctly for (a lot of tags). This affects is what summery info is
           displayed for it. And it IS affecting how chunks are saved. Just
           re-wrote the hole function to explicitly define end tags.

01/25/2012 v0.04 alpha
          -[note] - The difference in this version is subtle on the outside, but I
           promise you, a lot of code got moved around.

          -Some 3rd party MSHs have MTYP misspelled as MYTP. For now, I am
           silently correcting but not saving. It's only 3 or 4 models, so I
           think it was a typo on the modder's part and not a tool issue. In
           the future the program will offer to save if this change is made.

          -Adding some column labels.(this will continue until all tags have
           sensible labels)

          -re-organizing line labels.(this will continue until all tags have
           sensible labels)

          -[unresolved bug] - When opening a MSH while one is already loaded, the
           program 'remembers' the selected tag and selects the same tag # on
           the tree.
          -[unresolved bug] - Some of the tags have an extra blank line in the
           display. This doesn't affect saved files, but still needs to go away.

          -[fixed bug] - The order I was assigning variables in the MSH parser was
           causing incorrect offsets to be read in some fixed length, multiline
           Tags. (like BBOX, DATA)

          -The reason some tags take so long to convert is not the way I'm
           converting them, It's the way I'm juggling variables and arrays. The
           'fix' is to get everything passed as pointers instead of copying
           values in memory. A rewrite of the parser is happening, but won't be
           implemented until it is 100% working.

01/21/2012 v0.03 alpha
          -Cleaned up debug menu. Added array descriptions

          -increased the selected tag box height. It might be to big, but thats
           better than to small.

          -started organizing column descriptions (not yet complete)

01/20/2012 v0.02 alpha
          -Greyed out disabled menu options.

          -Disabled chunk export when a chunk is not selected.

          -Validate Tag Size now asks if you want to repair an invalid size
           field or not.

          -Fix_AllTagSizes removed. It was redundant.

          -still experimenting with different tag data displays

          -added 64-bit build

01/20/2012 v0.01 alpha - released as is.
[/spoiler]

So, I'm working on that msh format structure and I'm running into some incredibly large numbers.
Not large as in milllions and billions, but large as in 15, 20 and more places to the right of the decimal.

Does anyone here know (or even have a guess at) how precise XSI is with things like point locations?
I know the units are usually 3.something tall (approximatly). Just don't know how important that 10th or 11th decimal place is.

I Have the prog, just not installed and am hoping to save a few hours.

-edit- just realized this isn't really the proper place for this Question.
Feel free to move it to Mod Factory or Requests.

-edit
After a lot of trial and error with raw data conversion, I have determined that it is quite safe to use a precision of 6 decimal places.

The nature of floating point numbers (if I understand the IEEE spec correctly) is that the number should be reproducable with a minimum of 9 decimal places.

Of course, most systems are much more acurate than that, but knowing that minimum gives me a boundry to work with and keep my numbers at a sane length.

As a result, I will use 9 places in all my calculations, and only display however many are contextually appropriate and readable.

0.900000000 is just not right to display 90%.
0.90 is better, in that case.

For vertecies, normals, uv locations, rotation quaternions, excetera...,  the 9 places should be more than precise enough.

edit by tirpider: tagged title

Blackfoot Repair (kludge)

Blackfoot is the un-intentional appearance of part of the skeleton of a model.
In some cases, it appears on or below the ground, directly between the legs of the character.
Sometimes it is a diamond shape inside the body of the model, and only appears (passing through the body or head,) during animation.

I believe what causes this, is the lack of, or improper FLGS in the skeleton MODL chunk(s).
There may be more to the issue, but this repair will get the blackfoot off the model and the game.

I'm going to ramble about some stuff now, feel free to skip to "Ok. So how do I do that?" to avoid boredom.
Bear with me as I am as good a writer, as I am a programmer. (meh)

First off, What is a MODL chunk?
[spoiler]MODL is a tag inside the msh file that defines geometries, points, and spaces for that model.
There are several types of geometries, and usually many MODL chunks per msh. Each can have it's own skin, shape, and visiblility settings. Those settings are contained in sub tags. To see an example of all these tags, load up a msh in the Starwars Battlefront Viewer and select "Show" then "Debug Window."[/spoiler]

Ok, What does all that mean?
[spoiler]Good question. Some of it is self explanitory, some isn't. I've got a thread on the msh structure [here]http://www.swbfgamers.com/index.php?topic=3905.0. I don't know what most of it means and am learning more about it every day. You don't have to know all about it to interact with it though. (Just like BFBuilder, eh?) Basicly, it's all the settings, geometries, material, camera, translation information, the game needs to render the model.[/spoiler]

What part Fixes blackfoot?
[spoiler]FLGS.
I don't know what setting diferent values for it does. GT has a thread on it and I'll link to it from here.
Browse through a known, good model's debug view in the viewer, and note which models have a FLGS field, and which ones don't.  FLGS will be located after NAME and PRNT, and before TRAN and GEOM.  Almost all the eff_*, bone_*, and root_* MODL chunks have it, while few of the others don't.
Now check the debug view of your model that has blackfoot. I'm almost certain, there are several skeleton pieces (eff_*, bone_*, and root_*) that don't have a FLGS tag.
The fix is to insert a FLGS field into the proper place.[/spoiler]

How will that fix it?
[spoiler]I'm guessing that FLGS has something to do with whether the engine renders a MODL or not. Perhaps it is just a visability setting. An XSI guru can tell you more about that part, cause I don't know.
What I do know is that copyig a FLGS field from a known good model into the skeleton pieces of one with blackfoot, makes the blackfoot go away. So this is more of a kludge than a fix.[/spoiler]

Ok. So how do I do that?
Finally. The meat and bones.

Tools:

   
• A Hex-editor. I use Freeware Hex Editor XVI32 (http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm) and recomend it highly. Also, my instructions are XVI32 specific.
• A calculator. (the math is simple, but absolutely has to be correct.)
• My SWBF_MSH_TOOLS_v*.zip from the bottom of my msh structure notes post. http://www.swbfgamers.com/index.php?topic=3905.0 (They are buggy, but work for this and much more.)

Make a work folder containing the following:

   
• a msh with blackfoot. (You don't need tga's, odf's, or .options's for this, just the msh file)
• a msh with no blackfoot. (any one from the bfbuilder assets will do)
• SWBF_MSH_TOOLS_v*.zip

Setup

   
• Unzip the tools into the folder with the msh's your working with.
• run MSH2chunks_v*.exe
  This will create a sub directory for each of the msh's in the folder, and fill them with .chunk files.
  (The benefit to breaking the msh up is now you don't have to hunt through the whole msh for every MODL)
• Open the folder with the chunks of the model with blackfoot.
• make a new folder in this folder.
  The name isn't important at all, new folder is fine.
• select all the MODL chunk files with bone_ root_ eff_ sv_ or DummyRoot in thier name.
• COPY them to the new folder.
  It will seem like all of them, but some should be left out of this. (l_hand, r_hand, helmet, head, pack, ect.)
• Open the folder you just copied those MODL chunks to and one at a time, open each one with XVI32 and see if it has a FLGS field. Delete the ones that do. (this is why we copied them)
  It will be within the first 100 characters in the file, 12 bytes(spaces) before TRAN.
  At the end of this step you should have a folder with only the pieces that have no FLGS field.
• Go to the folder with the known good model's chunks and open it's DummyRoot chunk with XVI32.
• Click once on the "F" in FLGS on the right hand panel.
• Hold SHIFT and use the cursor keys to move right 11 spaces.
  This will mark the characters red. You should have 12 spaces marked (from the first 'F' to the space just before 'T' in TRAN)
  hit ctrl-c to copy the block to the clipboard.
• close the DummyRoot chunk and go back to the new folder with all the FLGS-less chunks.

One at a time, open each chunk with XVI32 and do the following:

   
• Click tools, then Data Inspector. (Make sure the data inspector is visible, we need it)
• Click on the first 'T' in TRAN and press ctrl-v.
  This will insert the FLGS block and should insert, not overwrite anything.
• Since we just added 12 bytes to the size of this chunk, we need to adjust its size header acordigly.
  Click once on the 5th space from the begning. (dec. offset 4) It will be the first space after MODL.
• On the Data Inspector, double click the number after 'longint' to bring up the encode number dialog.
• Make sure the 'lonint(4 bytes)' option and 'Overwrite at current address' option are selected.
• Add 12(the size of the FLGS chunk we just inserted) to the number and type it into the number box. (don't copy and paste the number or you'll have to open the good DummyRoot again to get the FLGS chunk back on the clipboard.)
• Press OK, to overwrite the old size with the new.
• Save the chunk and open the next one.

Ok, now you have a folder full of 'fixed' MODL chunks.

   
• Count the files you just fixed.
• Multiply that times 12.
  That is how many bytes you just added to the model size.
• Now we need to add that to the HEDR and MSH2 field.
  Go to your blackfoot model's chunk directory and open the first file. (HEDR)
• Click once on the 5th space from the begning. (dec. offset 4) It will be the first space after HEDR.
• On the Data inspector, double click the number after 'longint' to bring up the encode number dialog.
• Make sure the 'lonint(4 bytes)' option and 'Overwrite at current address' option are selected.
• Add the number from the previous step to the number in the box and press ok and save. (it's ok to copy/paste to and from the calculator now)
• Do the same for MSH2 (the second file in the directory.)
  The numbers will be different.

Time to put it all back together.

   
• Copy all those fixed files back into the model's chunk directory. (overwriting the existing ones.)
• Copy or move the Chunk2MSH tool into this directory and run it.
• This puts all those pieces back together into a new msh file named "new_msh.msh"

Have a look at it in the MSH Viewer. If nothing shows up, then something is wrong with a size field somewhere. (most likely HEDR or MSH2)

Now rename the file, munge and test the msh in Battlefront.
If Battlefront crashes, then one of the 'fixed' MODL chunks probably didn't need fixing. Make sure you only did the bone_ root_ eff_ ones and not any of the actual geometry ones like 068_MODL_054_l_hand.chunk (note, no root, eff, or bone in it's name.)

Thats's it.
If it's working, rename it appropriatly and add it to your arsenal of assets, and delete the chunk work folder.
If it isn't working, go over the steps again, if it still isnt working, feel free to ask questions.

NOTE
I updated the chunk2MSH tool. If you are using "chunk2MSH_v2.exe" then you don't need to preform the purple steps. The program does it for you.

-edits-[spoiler]
11/22/2011 -marked un-needed steps, due to tool upgrade.
[/spoiler]

We have all read about how cloth doesn't do well in BF1. (What I've read is that it doesn't work at all.)

Has anyone tried switching the ModelMunge.exe in the BF1 tool bin for the pc_ModelMunge.exe in the BF2 tool bin? (with the intent of getting additional features working?)

They are different programs, but they should use simular .option switches and I would imagine they output simularly munged files? Or is the cloth code hardwired into the battlefront executable?

I will test on my own, just curious if anyone else has tried.

-edit- changed title to solved

Notes on SWBF .msh file structure.
I want to be able to manipulate model features and geometries without XSI. In order to do that, I need the file format.

Ande has a site dedicated to this exact information. (link: www.ande.pytalhost.eu)
If anything in this document confilcts with what he has, I would go with what he has.
Some (a lot, particularly the CLTH stuff) of the following is straight from there, as I have no way to test it.

.msh MAIN CHUNK HIERARCHY
[spoiler]
HEDR                            -Size of file from next byte to end
   +-SHVO                       -Has Shadow or not?
   +-MSH2                       -Size of file from next byte to last MODL
   |    +-SINF                  -Scene Info
   |    |    +-NAME             -Name
   |    |    +-FRAM             -Scene animation info
   |    |    +-BBOX             -Bounding Box
   |    +-CAMR                  -Camera
   |    |    +-NAME             -Camera name. Typicaly "Camera"
   |    |    +-DATA             -Camera position and direction info.
   |    |    +-ANIM             - ? Animated Camera?
   |    |    +-LGTI             - ? Light?
   |    |         +-NAME        - ? The Light's name, I assume.
   |    |         +-DATA        - ? The Light's Metrics, I assume.
   |    +-MATL                  -Material info
   |    |    +-MATD             -Material description chunk
   |    |         +-NAME        -Material name
   |    |         +-DATA        -Color information
   |    |         +-ATRB        -Attributes of Texture
   |    |         +-TX0D        -Texture name
   |    +-MODL                  -Model geometry, skeleton, nulls, ect
   |         +-MTYP             -MODL type
   |         +-MNDX             -Model index #
   |         +-NAME             -MODL name
   |         +-PRNT             -Parent MODL NAME
   |         +-FLGS             -Visible/Hidden
   |         +-TRAN             -Transform/Translation
   |         +-GEOM             -Geometry
   |         |    +-BBOX        -Bounding Box
   |         |    +-SEGM        -Standard Geometry
   |         |    |    +-SHDW   -Shadow
   |         |    |    +-MATI   -Material index #
   |         |    |    +-POSL   -Vertex locations
   |         |    |    +-WGHT   -Weights
   |         |    |    +-NRML   -Vetrex Normals
   |         |    |    +-CLRL   -Vertex color
   |         |    |    +-CLRB   -Vertex color
   |         |    |    +-UV0L   -UV map
   |         |    |    +-NDXL   -Polygons
   |         |    |    +-NDXT   -Triangles
   |         |    |    +-STRP   -Triangle strips
   |         |    +-ENVL        -Envelope references
   |         |    +-CLTH        -Cloth (special geometry)
   |         |         +-CTEX   -Texture name
   |         |         +-CPOS   -Cloth Verticies
   |         |         +-CUV0   -Cloth UV map
   |         |         +-FIDX   -Cloth envelope indexes
   |         |         +-FWGT   -Cloth Weighting
   |         |         +-CMSH   -Triangles
   |         |         +-SPRS   -Stretch Constraints
   |         |         +-CPRS   -Cross Constraints
   |         |         +-BPRS   -Bend  Constraints
   |         |         +-COLL   -Collision data
   |         +-SWCI             -Collision data
   +-SKL2                       - CRC's of bones and ?
   +-BLN2                       - CRC's of bones
   +-ANM2                       -Animation data
   |    +-CYCL                  - ?
   |    +-KFR3                  - Keyframes
   +-CL1L                       - ?(Always at eof)
[/spoiler]

.msh BONE HIERARCHY (MODL chunks)
[spoiler]
DummyRoot
   +-MESH_body
   |    +-bone_root
   |         +-root_a_spine
   |         |    +-bone_a_spine
   |         |         +-bone_b_spine
   |         |              +-bone_ribcage
   |         |                   +-eff_ribcage
   |         |                        +-root_l_clavicle
   |         |                        |    +-bone_l_clavicle
   |         |                        |        +-eff_l_clavicle
   |         |                        |            +-root_l_upperarm
   |         |                        |                +-bone_l_upperarm
   |         |                        |                    +-bone_l_forearm
   |         |                        |                        +-eff_l_forearm
   |         |                        |                            +-root_l_hand
   |         |                        |                                +-bone_l_hand
   |         |                        |                                    +-eff_l_hand
   |         |                        |                                    +-MESH_l_hand
   |         |                        +-root_r_clavicle
   |         |                        |    +-bone_r_clavicle
   |         |                        |        +-eff_r_clavicle
   |         |                        |            +-root_r_upperarm
   |         |                        |                +-bone_r_upperarm
   |         |                        |                    +-bone_r_forearm
   |         |                        |                        +-eff_r_forearm
   |         |                        |                            +-root_r_hand
   |         |                        |                                +-bone_r_hand
   |         |                        |                                    +-eff_r_hand
   |         |                        |                                    +-MESH_r_hand
   |         |                        +-root_neck
   |         |                             +-bone_neck
   |         |                                 +-bone_head
   |         |                                     +-eff_head
   |         +-bone_pelvis
   |              +-root_r_thigh
   |              |   +-bone_r_thigh
   |              |       +-bone_r_calf
   |              |           +-eff_r_calf
   |              |               +-root_r_foot
   |              |                   +-bone_r_foot
   |              |                       +-bone_r_toe
   |              |                           +-eff_r_toe
   |              +-root_l_thigh
   |                  +-bone_l_thigh
   |                      +-bone_l_calf
   |                          +-eff_l_calf
   |                              +-root_l_foot
   |                                  +-bone_l_foot
   |                                      +-bone_l_toe
   |                                         +-eff_l_toe
   +-sv_ShadowVolume_mesh
   +-MESH_head
   +-MESH_backpack
   +-MESH_jetpack
   +-MESH_belt
   +-MESH_cape

   * this is just how the bones are parented. Enveloping and weighting determine
     how it moves with the skeleton.
   * the items that start with MESH_ are pretty much options. Their NAME is only
     important for parenting.
   * To make a model override_texture compatible, it's 'body' MODL chunk's name
     should be 'override_texture'. (Don't forget to change the name in it's
     children's PRNT chunk.)
   * p_* MODLs are deliberatly ommited as I don't know if they need to strictly
     follow the heirarchy or not
   * The order of the MESH_* chunks is just an example. There are models with
     all the geometry after all the bones, or some with the sv_ near the
     DummyRoot. As long as the PRNT and ENVL entries point to the correct
     models, then the msh should be valid and pass through ModelMunge just fine.

     
[/spoiler]

.msh TAG OFFSETS AND DESCRIPTIONS
[spoiler](in hierarchal order)
(items in brackets [] are still being investigated)
(? in the length fields indicate a variable length, typically defined by the preceding field)
HEDR
   offset   length   type      contents
        0        4   char(4)   HEDR
        4        4   long      bytes to end of file
   * the size entry will be the filesize - 8 bytes.

SHVO (child of HEDR)
   offset   length   type      contents
        0        4   char(4)   SHVO
        4        4   long      byte length of section from next byte
        8        4   byte(4)   ? typicaly 1
   * could be a long integer, or a series of bytes like ATRB
   * needs more study

MSH2 (child of HEDR)
   offset   length   type      contents
        0        4   char(4)   MSH2
        4        4   long      bytes from next byte until last byte of last MODL chunk

SINF (child of MSH2)
   offset   length   type      contents
        0        4   char(4)   SINF
        4        4   long      byte length of section from next byte

NAME (child of SINF)
   offset   length   type      contents
        0        4   char(4)   NAME
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long

[FRAM] (child of SINF)
   offset   length   type      contents
        0        4   char(4)   FRAM
        4        4   long      byte length of section from next byte
        8        4   long      start of frame range
       12        4   long      end of frame range
       16        4   float     fps
   * needs more study
   * most player msh's with a KFR3 chunk only have 2 frames in the KFR3. Haven't verified this in animation MSHs yet

BBOX (child of SINF)
   offset   length   type      contents
        0        4   char(4)   BBOX
        4        4   long      byte length of section from next byte
        8       16   float(4)  Quaternion Rotation ( qx , qy , qz , qw )
       24       12   float(3)  Center ( x , y , z )
       36       12   float(3)  ( Width/2 , Height/2 , Depth/2 )
       48        4   float     Bounding Sphere Radius

CAMR (child of MSH2)
   offset   length   type      contents
        0        4   char(4)   CAMR
        4        4   long      byte length of section from next byte

NAME (child of CAMR)
   offset   length   type      contents
        0        4   char(4)   NAME
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long

[DATA] (child of CAMR)
   offset   length   type      contents
        0        4   char(4)   DATA
        4        4   long      byte length of section from next byte
        8       12   float(3)  position ( x? , y? , z? )
       20       12   float(3)  direction ( x? , y? , z? )
       32       16   float(4)  ( ? , ? , ? , ?) (a quat, perhaps?)
   * needs more study

[ANIM] (child of CAMR)
   offset   length   type      contents
        0        4   char(4)   ANIM
        4        4   long      byte length of section from next byte
        8        ?   ?         ?
   * needs more study

[LGTI] (child of CAMR)
   offset   length   type      contents
        0        4   char(4)   LGTI
        4        4   long      byte length of section from next byte
   * needs more study

NAME (child of LGTI)
   offset   length   type      contents
        0        4   char(4)   NAME
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long

[DATA] (child of LGTI)
   offset   length   type      contents
        0        4   char(4)   DATA
        4        4   long      byte length of section from next byte
        8        4   float     ?
       12        4   float     ?
       16        4   float     ?
       20        4   float     ?
       24        4   float     ?
       28        4   float     ?
       32        4   float     ?
       36        4   float     ?
   * floats work for them so far, but may be incorrect
   * needs more study

MATL (child of MSH2)
   offset   length   type      contents
        0        4   char(4)   MATL
        4        4   long      byte length of section from next byte (Including sub section sizes)
        8        4   long      number of MATD sections to follow
   * MATI references to MATD's start at 0

MATD (child of MATL)
   offset   length   type      contents
        0        4   char(4)   MATD
        4        4   long      byte length of section from next byte

NAME (child of MATD)
   offset   length   type      contents
        0        4   char(4)   NAME
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long

DATA (child of MATD)
   offset   length   type      contents
        0        4   char(4)   DATA
        4        4   long      byte length of section from next byte
        8       16   float(4)  Diffuse Color ( red? , green? , blue? , alpha?)
       24       16   float(4)  Ambient Color ( red? , green? , blue? , alpha?)
       40       16   float(4)  Specular Color ( red? , green? , blue? , alpha?)
       56        4   float     Specular Intensity
   * needs more study
   * Haven't tried to interpret the floats into proper RGBa values

[ATRB] (child of MATD)
   offset   length   type      contents
        0        4   char(4)   ATRB
        4        4   long      byte length of section from next byte
        8        1   byte      addative flag values
        9        1   byte      render type number
       10        1   byte      data0 0 - 255
       11        1   byte      data1 0 - 255
   * Inside Edit Flags (FAQ) on GT
   * I will reflect the settings here when I start bump-mapping my Tuskens.

TX0D (TX1D, TX2D, ect. ) (child of MATD)
   offset   length   type      contents
        0        4   char(4)   TX0D
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long
   *some of the attributes require extra TX?D entries

MODL (child of MSH2)
   offset   length   type      contents
        0        4   char(4)   MODL
        4        4   long      byte length of section from next byte (Including sub section sizes)

[MTYP] (child of MODL)
   offset   length   type      contents
        0        4   char(4)   MTYP (model type)
        4        4   long      byte length of section from next byte
        8        4   long      0=null      1=skin    2= ?
                               3=envelope  4=static  6=shadow
   * offset 8 info from the Riley pages. The absense of 2 makes me believe there is more to this.
   * I havent figured out which one, but there is an exporter that spells this incorrectly as MYTP.
   * needs more study

MNDX (child of MODL)
   offset   length   type      contents
        0        4   char(4)   MNDX (model index #)
        4        4   long      byte length of section from next byte
        8        4   long      model index #
   * These are reffrenced by the ENVL chunks.

NAME (child of MODL)
   offset   length   type      contents
        0        4   char(4)   NAME
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long

PRNT (child of MODL)
   offset   length   type      contents
        0        4   char(4)   PRNT
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes
   *it will always be a multiple of 4 bytes long

[FLGS] (child of MODL)
   offset   length   type      contents
        0        4   char(4)   FLGS
        4        4   long      byte length of section from next byte
        8        4   long      00 00 00 00 = visible
                               01 00 00 00 = hidden
   * could be a long integer, or a series of bytes like ATRB
   * needs more study

[TRAN] (child of MODL)
   offset   length   type      contents
        0        4   char(4)   TRAN
        4        4   long      byte length of section from next byte
        8       12   float(3)  Scale ( x , y , z )
       20       16   float(4)  Quaternion rotation  ( qx , qy , qz , qw )
       36       12   float(3)  Translation  ( x , y , z )
   * needs more study
   * these are important for keeping the bones pointed in the right direction.

GEOM (child of MODL)
   offset   length   type      contents
        0        4   char(4)   GEOM
        4        4   long      byte length of section from next byte (Including sub section sizes)

BBOX (child of GEOM)
   offset   length   type      contents
        0        4   char(4)   BBOX
        4        4   long      byte length of section from next byte
        8       16   float(4)  Quaternion Rotation ( qx , qy , qz , qw )
       24       12   float(3)  Center ( x , y , z )
       36       12   float(3)  ( Width/2 , Height/2 , Depth/2 )
       48        4   float     Bounding Sphere Radius

SEGM (child of GEOM)
   offset   length   type      contents
        0        4   char(4)   SEGM
        4        4   long      byte length of section from next byte (Including sub section sizes)

[SHDW] (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   SHDW
        4        4   long      byte length of section from next byte (Including sub section sizes)
   *all the verticies are listed first
        8        4   long      number of verticies
                12   float(3)  vertex(X, Y, Z)
   *then all the edges
                 4   long      number of edges
                 2   short     Vertex index of this edge entry. Refers to the vertex list above.
                 2   short     edge 1 (refers to another edge in this list)
                 2   short     edge 2 (refers to another edge in this list) **
                 2   short     end of edge. (65535) or (0xFFFF)
   * needs more study
   ** oddly, the vertex in the edge this refers to is always the same as the vertex in the edge that edge 1 refers to.

MATI (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   MATI
        4        4   long      byte length of section from next byte
        8        4   long      MATD reference.
   * The MATD's are indexed from 0

POSL (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   POSL
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12       12   float(3)  ( x , y , z )
   * the entry at offset 12 repeats, index( offset 8 ) times.

WGHT (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   WGHT
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 32 byte entries to follow
       12       32   (long, float)(4) (bone reference, Weight in percent)
   * the entries from offset 12 to 38 repeat, index( offset 8 ) times.
   * you can have multiple weights, they just have to add up to 100% (1.0) (XSI docs)
   * The bone is called by ENVL index.
     For example: if the first bone in ENVL is bone_neck (MNDX 34) then the WGHT reff will be 0 since it is the first bone in the ENVL.

NRML (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   NRML
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12       12   float(3)  3 floats describing Vertex normals
   * the entry at offset 12 repeats, index( offset 8 ) times.

[CLRL] (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   CLRL
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12        4   short(4)  ( red , blue , green, alpha )
   * needs more study

[CLRB] (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   CLRB
        4        4   long      byte length of section from next byte
        8        4   short(4)  ( red , blue , green, alpha )
   * Got it's structure, now what is it?
   * needs more study

UV0L (UV1L, UV2L, ect.) (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   UV0L
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 8 byte entries to follow
       12        8   float(2)  ( tx , ty )
   * the entries from offset 12 and 16 repeat, index( offset 8 ) times.
   * to see where each UV is on a texture, multiply each coordinate by the size
     of the texture.
     For example:
         if texture size = 512 x 512 and tx=0.564125 , ty=0.059709
         the points would be:
         x = tx * 512 = 0.564125 * 512 = 288.832
         y = ty * 512 = 0.059709 * 512 =  30.571008
         

[NDXL] (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   NDXL
        4        4   long      byte length of section from next byte
        8        4   long      an index. Number of polygons
       12        2   short     an index. Number of corners
       14        2   short     point ref (from POSL)
   * the entries at offset 14 repeats, index( offset 12 ) times.
   * the entries from offset 12 to (end of it's sequence) repeat, index( offset 8 ) times.
   * some software uses this info, but it will munge and play without it.
   * needs more study

[NDXT] (child of SEGM)(depriciated)
   offset   length   type      contents
        0        4   char(4)   NDXT
        4        4   long      byte length of section from next byte
        8        4   long      an index. Number of triangles
       12        2   short     point ref (from POSL)
       14        2   short     point ref (from POSL)
       16        2   short     point ref (from POSL)
   * the entries from offset 12 to 14 repeat, index( offset 8 ) times.
   * some software uses this info, but it will munge and play without it.
     (I think 3d Converter uses it to determine polys instead of the STRP.)
   * I don't know if they go clockwise or counter-clockwise.
   * needs more study

[STRP] (child of SEGM)
   offset   length   type      contents
        0        4   char(4)   STRP
        4        4   long      byte length of section from next byte
        8        4   long      an index. not certain about what it's indexing
       12        2   short     POSL vertex reffrence
   * the entry (offset 12) repeat, index( offset 8 ) times.
   * This tag lists triangle strips.
     The tringles are defined by refs to vertecies in the POSL tag.
     The strips are seperated into groups of triangle strips.
     The start of a strip is indicated by 2 entries with a high bit set.
     Basicly the vertex ref has 32768 added to it. Subtracting 32768 gives the vertex index.
     They should be listed in clockwise, counter-clockwise, clockwise order, I haven't confirmed this.
     They should be listed in   counter-clockwise,   clockwise,   counter-clockwise
     order, I haven't confirmed this.
   * needs more study

ENVL (child of GEOM)
   offset   length   type      contents
        0        4   char(4)   ENVL
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 4 byte entries to follow
       12        4   long      bone MNDX reference
   * the entry offset 12 repeats, index( offset 8 ) times.

CLTH (child of GEOM)
   offset   length   type      contents
        0        4   char(4)   CLTH
        4        4   long      byte length of section from next byte

CTEX (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   CTEX
        4        4   long      byte length of section from next byte
        8        ?   ascii     string with the end padded with 00 bytes

[CPOS] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   CPOS
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12       12   float(3)  ( x , y , z )
   * the entry at offset 12 repeats, index( offset 8 ) times.
   *same as POSL
   
[CUV0] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   CUV0
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 8 byte entries to follow
       12        8   float(2)  ( tx , ty )
   *same as UV0L

[FIDX] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   FIDX
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 4 byte entries to follow
       12        4   long      CPOS vertex indexes
   * haven't looked into this yet

[FWGT] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   FWGT
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12        4   ascii     bone names
   *assuming these are the names of the bones that the CPOS vertexes refered to in FIDX, are weighted to.

[CMSH] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   CMSH
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12       12   long(3)   CPOS vertex refferences defining a counter-clockwise triangle

[SPRS] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   SPRS Stretch Constraint
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12        4   short(2)  points sharing an edge
   *ANDEWEGET's GT post

[CPRS] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   CPRS Cross Constraint
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12        4   short(2)  points sharing a quad
   *ANDEWEGET's GT post

[BPRS] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   BPRS Bend Constraint
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12        4   short(2)  Two border-points sharing a "straight" edge over the complete cloth.
   *ANDEWEGET's GT post

[COLL] (child of CLTH)
   offset   length   type      contents
        0        4   char(4)   COLL
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12        ?   ascii     Collision object name
       ??        ?   ascii     Object this collision moves with(parent)
       ??        4   long      possible FIDX refference
       ??        4   float     for type sphere   = radius
                               for type cylynder = radius
                               for type cube     = size x/2
       ??        4   float     for type sphere   = n/a
                               for type cylynder = height
                               for type cube     = size y/2
       ??        4   float     for type sphere   = n/a
                               for type cylynder = n/a
                               for type cube     = size z/2
   *seems to work like SWCI, but for cloth
   
[SWCI] (child of MODL)
   offset   length   type      contents
        0        4   char(4)   SWCI
        4        4   long      byte length of section from next byte
        8        4   long      collision primitive type
                               0 = sphere
                               2 = cylynder
                               4 = cube
       12        4   float     for type sphere   = radius
                               for type cylynder = radius
                               for type cube     = size x (might be scaled)
       16        4   float     for type sphere   = n/a
                               for type cylynder = height
                               for type cube     = size y (might be scaled)
       20        4   float     for type sphere   = n/a
                               for type cylynder = n/a
                               for type cube     = size z (might be scaled)
   * needs more study

all of the following always appear after the last MODL chunk and their size is not included in the MSH2 value.

[SKL2] (child of HEDR)
   offset   length   type      contents
        0        4   char(4)   SKL2
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 20 byte entries to follow
       12        4   byte(4)   CRC used to refer to bone_ entries
       16        4   long      type
       20        4   float     Constrain
       24        4   float     bone length 1
       28        4   float     bone length 2
   * the entries from offset 12 to 28 repeat, index( offset 8 ) times.
   * needs more study

[BLN2] (child of HEDR)
   offset   length   type      contents
        0        4   char(4)   BLN2
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of 8 byte entries to follow
       12        4   byte(4)   CRC used to refer to bone_ entries
       16        4   float?    blend factor

ANM2 (child of HEDR)
   offset   length   type      contents
        0        4   char(4)   ANM2
        4        4   long      byte length of section from next byte (Including sub section sizes)

[CYCL] (child of ANM2)
   offset   length   type      contents
        0        4   char(4)   CYCL
        4        4   long      byte length of section from next byte
        8        4   long      an index - number of entries to follow
       12       64   char(64)  animation name
       76        4   float     frame rate
       80        4   long      play Style
       84        4   long      first Frame
       88        4   long      last Frame
   * needs more study
   ** each entry is typically 80 bytes. I'm guessing they left a long space to accomodate animation names?

[KFR3] (child of ANM2)
   offset   length   type      contents
        0        4   char(4)   KFR3
        4        4   long      byte length of section from next byte
        8        4   long      an index? - number of x byte entries to follow
       12        4   byte(4)   CRC used to refer to bone_ entries
       16        4   long      keyframe type
       20        4   long      translation count
       24        4   long      rotation count
   * all the translation entries will be listed first
       28        4   long      frame position translation index (starts at 0)
       xx        4   float     translation x
       xx        4   float     translation y
       xx        4   float     translation z
   * then all the rotation entries
       xx        4   long      frame joint rotation index (starts at 0)
       xx        4   float     quaternion qx
       xx        4   float     quaternion qy
       xx        4   float     quaternion qz
       xx        4   float     quaternion qw
   * needs more study
   * In animation msh's, this is the biggest tag,having several translation and rotation entries.

[CL1L] (child of HEDR)
   offset   length   type      contents
        0        4   char(4)   CL1L
        4        4   ?         00 00 00 00
   * I wonder if this could mean anything other than EOF

   ** the Quaternions are in the order qx,qy,qz,qw
      qw is the scalar
      qx, qy, qz are the ... vector? (I don't know how to name it. read up on quats and you will discover that everyone has a different name for them. It's no wonder we don't have a single, unified field theory. We can't even agree on how to count to 'turn left'.)

   ** ALL the CLTH tag info is directly from Ande's work. I have no way to test is as I don't have BF2.
      It's solid though, and should be easy to validate.
     
[/spoiler]

I believe the above is pretty complete for the purposes of recognizing and translating the msh data. As I run across errors, I will correct them. The code I am building is drawing directly from the info in this document.
I am now moving on to explore the next level of abstraction the data represents. Envelopes, smoothing groups, IK chains and such.

The attatched tools will br removed when I make suitable replacements for them.
They are all being re-written from the ground up to include everything I have learned in the last several months.
I haven't decided if they will be several small tools, or a single tool with several options.
I do know that none will have a GUI. All input and output will take place through the console or via txt files.

I don't think there will be much more that I can do with the raw format, utillity-wise. But I am looking into various ascii formats. Perhaps I can takea swing at, say, a msh to obj converter that doesn't loose half the msh data (like the entire skeleton.)

-next update : re-tooling the tools. (testing)

Regarding the names of assets.

Through the process of learning how to localize weapon and unit names, I learned how important it is to stick with a naming convention.

faction_weap_inf_WeaponName
faction_inf_UnitName

It gets confusing real fast if you don't.

My question is: Is the "_weap_" in weapon names connected to anything in the game other than as part of a naming convention?

I've hexed the tga entries on some BF1 stock models and have found the number of characters to be limiting.  So I overcame this by using a shorter string in place of "_weap_".  I know you can use whatever name and get it working as long as it's referenced correctly. I'm just worried about breaking something I haven't learned about yet.

A second (and probably better) question.

Is it possible to hex the tga entries(and adjust the file/entry size/offsets) in a msh without lossing the msh?

Long time player (ps2) first time mapping.

Hope there is room for one more.