How to isolate pieces of a msh p1 Piece Removal

[Phobos]

When this epic tutorial series thread is complete I'd like to convert it to PDF format if thats okay with you. Amazing work here, hopefully I get some time to study it some more soon. Great job.

[tirpider]

When this epic tutorial series thread is complete I'd like to convert it to PDF format if thats okay with you.

That's fine. I'd like to go back over it all with a spellcheck at some point.
It seems epic, but I hope that is because I am over explaining things.

[tirpider]

WIP

EXAMPLE 4 - Piece Extraction
Preperations

INITIAL SETUP
[spoiler]As before, we need to have a project folder.
In that folder we need a place to keep the source materials and a place to work on the msh.
The folder hierarchy here isn't explicitly important, but it makes keeping track of a project easier.

Here is a screenshot of my initial project folder:

_tool is the folder from the msh kit archive. It needs to keep it's folder name.
sources is for source material, and is empty at the moment.

Now we need to gather the assets of the msh we are going to edit.
I chose to remove the earpiece from the rebel hoth soldier for this example.
While this is not the most useful thing to do, it is an easier one to explain.
So, make a folder in sources to recieve the assets and copy them from the bfbuilder assets folder.

Just the stuff for the snowtrooper,  his lowres would be in it's own project folder if we were going to go that far with this.

Time to move on to tearing the msh down and creating a build environment.[/spoiler]

ENVIRONMENT SETUP
[spoiler]Run MSH2chunks.exe in the _tool folder and direct it to the all_inf_snowtrooper.msh in the sources folder.
Press ok and it will cut the msh up for us, leaving a _chunks folder in the same folder as the msh.

Now move that _chunks folder to the root of our project folder.
Then move the _tool folder inside the _chunks folder.


Copy the BuildIt.bat file to the root of the chunks folder.
Run it once. (It will ask you to press a key when it is done)


There should be some new files in the _chunk folder now.
_recycle - think of this as a temp folder. Chunks2MSH ignores this folder and it's contents when rebuilding the msh.
modelmunge.log - the log file produced by ModelMunge.exe. ModelMunge isn't required for all this, but it is sometimes convienient to have this for troubleshooting wierd issues. (wierd is common with this process)
tst_inf_tst.msh - this is the rebuilt msh file.
tst_inf_tst.msh_Report.txt - a report of all the changes made by the size validation tool. Sometimes it's useful, most of the time I never even look at it.

Copy the tga files for the msh, from the sources folder to the _chunks folder.
This is so that the model will be textured when testing in the viewer.
It can be easier to spot things with the tga's in place.

Now, open the msh with the MSH viewer.
It should look the same as the original msh.

Ok, thats the build environment.
We have the msh torn down, and can rebuild it.
Any changes you make to the chunk files in the folders will be reflected in the rebuilt msh after buildit.bat is run.

Next we make sure that the model is ready for this specific edit.[/spoiler]

MODEL SETUP
[spoiler]Sometimes a msh needs to be adjusted a bit to work with all the tools.
The most common I have run into involve getting the msh to load in 3D Object Converter correctly.
I use the Unwrap feature in 3DOC to produce bitmaps of the uv's later.
This process completely doesn't work without those bitmaps, so getting the msh into 3DOC is critical.

Sometimes there is an error in the msh data that prevents it from loading into 3DOC. The next step can help identify what MODL chunk contains the error. The errors range from a trailing null in the STRP, to an absent NDXL chunk. Fixing those errors is another topic that uses the same tools and techniques in these examples. Right now, I'll demonstrate a solid workaround that helps identify them though.

Side Note:[spoiler]( The errors usually don't prevent the model from munging properly. This just proves that different programmers have different ways of loading assets. It turns out that 3DOC relies on a part of the msh that was only used to build the ps2 version (NDXL) to generate polygons. ModelMunge for pc uses the STRP only and completely ignores the NDXL and NDXT. But if we want the msh to work with 3DOC, we need to have a working NDXL.)[/spoiler]

Without going into messy repair details, the most common offender is the shadow volume (the chunk whose name starts with sv_ )
We just pull it out while working on the msh, then put it back in after our final build.

Another reason to temporarily remove the sv_ it to help in visualizing the msh.


The grey stuff poking out of the model is the shadow.
You can see how much cleaner the second one is, without it.

To remove it, just move the sv_ chunk folder to _recycle.


BuildIt.bat and chunks2msh will ignore all the stuff in _recycle when rebuildng the msh.

To put it back in, just move it back to the _chunk folder and run buildit.bat.

-
The model we are using doesn't have any problems loading in 3DOC, but it still helps to get the sv_ out of sight for visualization purposes.

<this space reserved for further repair explination or a link>
If you have a project that throws errors, or won't load at all, and I haven't got a topic started on how to fix those, then pm or ask and I'll try to help out.
It may help to know that passing a troublesome SEGM through this process fixes some (not all) of the errors.[/spoiler]


Setup is done.
Next we will get the uv map and use Paint.Net to define (discover?) the UV regions (smoothing groups?) of the msh.

[tirpider]

WIP

EXAMPLE 4 - Piece Extraction
Define UV Regions

PREPARE REIGON MAP
[spoiler]UV's are the coordinates on a texture that a vertex is mapped to. If you connect all the coordinates together in the same way the vertecies are connected, you get a UV "map'.

You will notice on most textures ( not just SWBF msh's,) the connected UV's tend to form islands or groups when layed out. Those are what I am calling "UV Regions.

There may be a proper term for them, but that is the term I am sticking to for these examples.

What we are about to do is create a Paint.Net file (pdn) with layers for the UV's, texture and a working layer for marking regions. All this could be done in Gimp, Photoshop, or any image editor that handles layers. I am going to use Pain.Net because it is free and accesable to me. (Gimp confuses me and photoshop is out of my price range.)

For future reference, it is most helpfull to only use one tga per pdn. Basicly, work with one MODL or SEGM at a time, to keep things from getting confusing.

Also, I try to keep track of the pdn files, in case I need the numbers from it again.

Onward..

Getting the UV map is easy.
Load the tst_inf_tst.msh into 3D Object Converter. (Shareware or registered, it doesn't matter)
Press the Unwrap UV button to bring up the Unwrap paramaters dialog.


Make sure the output resolution is set to 1024 x 1024.

Select the object name of the piece we are working with (in this case, it is override_texture)

Then check the Save the selected objects' UV map checkbox and press ok.

Before closing 3D Object converter, this is a great time to find the name of the MODL that contains the earpiece.
Press the Object and Material Selector button to open the Object and Material Selector.

Clicking on each of the items under 3D Scene will highlight the piece associated with it in red.

You can see that override_texture is the one that contains the earpiece. (and a LOT of other things as well)
That is the name of the MODL chunk we will be working with.

Close 3DOC.

Copy all_inf_soldier_hoth.tga and tst_inf_tst.msh_override_texture.bmp to _recycle.


The tga will probably still have the read only attribute set on it if you got it from the bfbuilder assets.
Right click on it and select properties.
Then uncheck the read only box and press ok.

Next we prep the files to be imported into Paint.net
They need to both be 1024x1024 and flipped verticaly.
I'm sure there is a way to do this inside the program, but this is how I do it.

Open the tga in IrfanView and press ctrl r to bring up the Resize dialog.

Change it's width and height to 1024 and press ok.


Press v once, to flip it.

Press ctrl s to save it (make sure it is being saved in the _recycle folder and not the _chunks folder.)
Confirm it's overwrite request.

Next, open the bmp with irfanview.

Press v to flip it, and ctrl s to save.

Close IrfanView.

The files are prepped.
The size is for compatability with my spreadsheet. 1024 gives a prety detailed resolution for trapping uv points.
We flipped it because of the way Paint.Net indexes coordinates. We need the 0,0 to be in the same corner the game sees it. It's easier to flip the textures than to code around it in the spreadsheet.

Next, we import them into Paint.net

Open Paint.net

We will need 3 of it's utility windows open.
Tools, Layers, and Colors.
They can be toggled on and off with the function keys F5, F7, and F8, or can be selected on the Window menu.


Select "Import From File" on the Layers menu.
Then browse to the _recycle folder where the 2 images we preped are located, and open the all_inf_soldier_hoth.tga.
This imports the image as a new layer.

Do it again but this time, open the _override_texture.bmp.

There should be 3 layers in the program now.


Click once on the "Background" layer and press the Duplicate Layer button on the bottom of the Layer window.


Now we are going to rename some stuff so you know what layer I am talking about later.

Double Click on one of the layers named Background and rename it to "Regions".

Rename the layer with all the UV lines to "UV"

Rename the layer with all the tga texture image to "Texture"

Now, use the arrow buttons to reorder the layers.
From top to bottom, they should be, Regions, UV, Background, Texture.


Now to prepare the layers.

Regions need to be empty, so:
Click once on the Regions layer to select it.
Press ctrl a to select everything on the layer.
Press delete once on your keyboard.

The white space in UV needs to go away.
Click once on the UV layer to select it.
Click on the "Magic Wand" tool on the toolbar.
Ensure that flood mode is set to global.

Click once on any white space to select everything but the black lines.
Press delete once on your keyboard.

Uncheck the box next to the Background layer to hide it. You should see the lines on top of the texture.

If not, then you need to re-do the layer preperation.
(ctrl z (Undo) is your friend)

Save the file.
Make sure you save as a pdn file so the layers stay intact.
I like to name it after the tga texture that I'm working on, so call it
"all_inf_soldier_hoth.pdn"[/spoiler]

DEFINE REGIONS
[spoiler]You see all the islands of uv's? Those are what I was talking about earlier.
We are going to be surrounding most all of them with a basic box.
Later on, during the data collection part, we will be getting the corner information from each of those boxes and recording them in the spreadsheet.

What's happening with the black lines is that each UV reference in the msh refers to a point somewhere on this texture. When you connect them in the order the msh connects them (via triangle or polygon definitions found in the NDXL, NDXT, and STRP chunks) you end up with the lines seen here. The boxes we are about to draw are 'traps' that define a region that contains the UV points we want, and through that, we identify the vertecies we want in the spreadsheet.

The reason I say we will surround most with boxes is that we don't want any to intersect at all.
Some of the shapes (the torso and arm part, for example) defy being surrounded by a single rectangle without intersecting another box. The solution for that is to use several boxes to surround pieces of it. or be very clever with "acceptable intersects".
I won't cover that method here, as we aren't chasing a torso, and I want to keep it as straight forward as possible.

We don't actually need to surround each and every one, but it can help troubleshoot issues later on if we do.

To get started, we need to have the Background layer (the solid white one) enabled, so check it's box in the layer window.
This makes it easier to identify 'island' seperations.

Click once on the 'Regions' layer and close the layers window.
When you have a layer selected, all edits only happen to that layer. We close it to prevent accidentally switching layers and to get the window out of the way.

Now, left click once on red on the color window and close it.
Red will stand out nicely against the white background. You can use any color, I choose red. And, again, we close it to prevent accidentally switching colors and to get the window out of the way.

Left click on the line tool
Adjust the line width to 1
Turn off Antialiasing


Next, click the grid enable button.

This will give us a grid when zoomed in. It makes it easier to keep the lines straight.

We will begin with the piece in the upper left corner.
Don't worry so much about identifying pieces yet. Right now we are just getting the bits isolated.

Zoom in on the upper left corner till the grid spaces become nice and big.
If you are using a scroll mouse, holding ctrl while you scroll the wheel will zoom in and out.


Start laying down an outline around the shape, with a 1 pixel gap between your lines and the shape.

Work in small sections and surround the entire shape.
The 1 pixel gap is to accomodate rounding errors that might appear in the spreadsheet.

The outline has to be a rectangle. No bumps at all. So if you encounter a bit that falls out side the outline you are drawing, you have to erase that line all the way back to it's starting corner and start over to accomodate the bump in the uv island.

In instances where you cannot include a 1 pixel gap (like the edge of the image) it is ok to go ahead and draw the line on the edge. This is an area that I haven't encountered much, but it does come up. If it introduces an error, then we will have to cover it then.

Repeat this for all the pieces that can be outlined without intersecting other outlines.
Here's what I ended up with:


Save and exit Paint.net

Here is a cheat. At the end of this post is the a zip containing the above pdn.
You can use it for the rest of the walkthrough or as a reference to what needs to be outlined and how.
I encourage you to try to outline it yourself to become familiar with Paint.NET's controls.

all_inf_soldier_hoth_pdn.zip (2.6mb zip)[/spoiler]

That's it for defining the regions.
Next we will load the spreadsheet.

[tirpider]

WIP

EXAMPLE 4 - Piece Extraction
Gathering data and loading the spreadsheet

LOAD THE SPREADSHEET
In the _tool folder there is a file named part.ods. That is the spreadsheet that will hold all the data from the SEGM and produce the code needed to let the AutoIT scripts rebuild the piece.

I should probably mention this earlier, but here is a good place to state that you can only work with one SEGM at a time. What that means is that the spreadsheet and scripts are only set up to work with one set of SEGM data. This isn't really relevent to this example, but as you pursue other edits, you will see what I mean. I may make an additional example that shows how to split the folder of a MODL that has more than one SEGM chunk.

Back to the project.

Copy the part.ods to the folder of the MODL chunk we are working on.
(the 110_GEO_2_override_texture folder.)

Open the part.ods in the 110_GEO_2_override_texture folder.
(It takes a bit to load. I didn't include macros or anything like that, just cell formulas.)

Now go back to the _tool folder and run SWBF_MSH_INFO.exe (if you are on a 64-bit system, you will need to run the SWBF_MSH_INFO.exe in the x64 folder)

Select File>Open and browse to  tst_inf_tst.msh (it should be one folder up)

Once it it opened, browse to the override_texture MODL chunk on the left part of the program.
Expand it out to see all the pieces under it.


Click once on SEGM in the tree view on the left.
Select Tool>Copy selected SEGM vertex Summery

That copied all the POSL, WGHT, NRML, and UV0L data onto the windows clipboard.

No go back to the spreadsheet. (leave the MSH_INFO_tool open, we need a few more things from it.)

Make sure the vPaste page is selected

and select the first cell under idx.


Press ctrl v to paste the vertex data.
Calc will prompt you for formatting instructions.
The only thing you should have to adjust is to make sure Tab is checked.
Press ok to finish importing.


The sheet should fill with numbers

If it doesn't, then go back to the info tool and select the copy tool and try again.
The Info tool has not been tested a lot, so it is possible for it to do unexpected things.
PM/email me or post if it gives you grief, and I will try to fix what I can.


We need to extend the formulas for the vertex count in 2 other sheets.
To do this, we need to copy the vertex index column.

Select the filled cells in the idx column on the vPaste sheet.
I do this by selecting the idx cell, then press ctrl shift down
Press ctrl c to copy it to the clipboard
<screenshot>

Click on the vBuilder tab to open that sheet.
Select the idx cell and press ctrl v to paste the column of vertex indecies
<screenshot>

Now, click the General tab to open that sheet.
Again, select the idx cell and press ctrl v to paste the column of vertex indecies
<screenshot>

<break>
(I realized how rediculous daily edits to something this big is. So I am writing the rest offline and will update when it's done. )

[tirpider]

Sorry for the necro-post, and the double post... and all the other "I'm sorry's".. heh

I have to cancel the last two parts of this series. I've indicated this in the first post of the thread and at the begining of the fourth part.

It's a time intense process, that was eating at the time I could dedicate to actual projects.  I (like most of the folks here) can't advocate dedicating youself to a hobby task that isn't fun. 

I will gladly answer direct questions about any of all this. And if you would like to try walking through the last portions via chat or other correspodence, message me and we can arrange a time.  Might take anywhere from a couple of hours to many hours.  It's an involved process and it might just be easier to learn the XSI Mod Tool interface and ZETools.

I'm sorry for dropping this guide. I really want to share the process, but it's become a real drain on me.  Actually making things that are going into the game is much more rewarding.

btw, I'm just dropping this project, not anything else.

[Unit 33]

Well, it's been a really useful tutorial, though some of it has been beyond my current comprehension I have still learnt a lot about .msh crafting.

Thanks tirpider!