lua4dec - Lua 4.0 Decompiler

[{AR}MetalKiller]

Hi,

I have been working on a Lua 4.0 decompiler for some time and I think it's ready enough to be shown.
What was originally planned to be a weekend project turned out to take almost half a year of off/on development.
lua4dec was initiated by my interested in SleepKillers unmunge tool which extracts lua bytecode (among other formats).
This lua bytecode can be fed into lua4dec which converts it back to human-readable lua scripts.

The project is around 90% done.
Some uncommon instructions are not implemented, and 3 or 4 instructions are missing some more attention.
But overall I am pleased with the results.
The decompiler even spawned a little side project that shows the AST of a converted script.
In the long run, I could imagine it being used to modify the bytecode and recompiling it back into the SWBF files.

Here is the link to the project: https://github.com/styinx/lua4dec

I will still continue to work on it and finish it someday.
Maybe you want to test it, leave some feedback, or improve it yourself.

Feel free to reach out.

Cheers

[Dark_Phantom]

Hey Metalkiller! This is a great project. I doubt I have implemented anywhere near as much as you but if there's anything you can glean from my bad implementation of bytecode parsing, go ahead:
https://github.com/phantom567459/BF1LuaDC/blob/master/LuaDC1/Program.cs

I'll take a look in the near future and see if I can get some of the more obscure implementations documented and maybe fork your code instead to go further back in the process. I thought I tried Lua4Dec at one point but maybe I need to try again.  I know my project is a little bit different but tries to target the bytecode more directly for BF1

Thanks!

[{AR}MetalKiller]

Hey Metalkiller! This is a great project. I doubt I have implemented anywhere near as much as you but if there's anything you can glean from my bad implementation of bytecode parsing, go ahead:
https://github.com/phantom567459/BF1LuaDC/blob/master/LuaDC1/Program.cs

Glad you like it.
I stumbled across your code a couple of times. 
It seems to be the only other project that tries to target Lua 4.0 (besides an unfinished Python implementation).

I'll take a look in the near future and see if I can get some of the more obscure implementations documented and maybe fork your code instead to go further back in the process. I thought I tried Lua4Dec at one point but maybe I need to try again.  I know my project is a little bit different but tries to target the bytecode more directly for BF1

The trick for me, was to mimic the stack-based behaviour.
But you are right, the documentation/implementation of the interpreter is really something.

Unless you are familiar with CMake and C Code, I would recommend to wait for a first "developer" build.
There are a couple of switches that have to be changed to build the code for 32-bit lua bytecode.
The default build uses the architecture you are on which is most certainly 64-bit.
The tests also run with the 64-bit version.
However, reading through the code should be straight forward.
The important parts are in the parser.cpp file.

I will try to prepare something for usage in the next month and report back in this thread.

[{AR}MetalKiller]

I just created a test release with prebuilt binaries (64-bit Windows 10): https://github.com/styinx/lua4dec/releases/tag/vP.0.0 (includes some test files)

While building, I re-remembered that SWBF's bytecode does not contain names of local variables, so added a prefix "local" to those.

[vonfrank]

I've been looking for a way to decompile mission.lvl files for really old addon maps in order to increase reinforcement count, unit count, and potentially change specific units altogether. Would your tool allow this?

I'm not too familiar with command-line programs, so I'm not sure if the test release is capable of that already.

[Led]

I've been looking for a way to decompile mission.lvl files for really old addon maps in order to increase reinforcement count, unit count, and potentially change specific units altogether. Would your tool allow this?

I'm not too familiar with command-line programs, so I'm not sure if the test release is capable of that already.

It is not difficult to make a new mission.lvl file.  You just need the name of the map.  Refer to the SWBFupdate project I was working on. 

[{AR}MetalKiller]

I've been looking for a way to decompile mission.lvl files for really old addon maps in order to increase reinforcement count, unit count, and potentially change specific units altogether. Would your tool allow this?

I'm not too familiar with command-line programs, so I'm not sure if the test release is capable of that already.

The test release contains the tat3a.out sample which is a compiled script file.
To decompile the mission.lvl file you would need SleepKillers unmunge tool first: https://github.com/PrismaticFlower/swbf-unmunge

The level format is structured something like this:

Code Select Expand

- ucbf header
- file size
- some meta info
- chunk type "script"
  - chunk size
  - some meta info
  - the actual compiled script    <-- this is what you would want
- other chunks type
  - ...
- ...

The unmunge tool extracts the known chunk types into their respective formats (like meshes, textures, ...).
For unknown chunks, it just dumps the binary to a folder with a dummy extension.
Scripts have the ".script" extension and contain the meta information and chunk size besides the compiled script.
If you remove this additional information from the file you can pass it to the "luadec_32_deb.exe" program and it should print you the original script.

A compiled script starts with the following binary signature:

Code Select Expand

1B 4C 75 61 40    -> .Lua@    (. is a non printable character, @ = 40 which means Lua 4.0)
You can hex edit the file and look out for this sequence of bytes.
Remove everything infront and you are good to go.

If you need assistance feel free to reach out.

[vonfrank]

The test release contains the tat3a.out sample which is a compiled script file.
To decompile the mission.lvl file you would need SleepKillers unmunge tool first: https://github.com/PrismaticFlower/swbf-unmunge

The level format is structured something like this:

Code Select Expand

- ucbf header
- file size
- some meta info
- chunk type "script"
  - chunk size
  - some meta info
  - the actual compiled script    <-- this is what you would want
- other chunks type
  - ...
- ...

The unmunge tool extracts the known chunk types into their respective formats (like meshes, textures, ...).
For unknown chunks, it just dumps the binary to a folder with a dummy extension.
Scripts have the ".script" extension and contain the meta information and chunk size besides the compiled script.
If you remove this additional information from the file you can pass it to the "luadec_32_deb.exe" program and it should print you the original script.

A compiled script starts with the following binary signature:

Code Select Expand

1B 4C 75 61 40    -> .Lua@    (. is a non printable character, @ = 40 which means Lua 4.0)
You can hex edit the file and look out for this sequence of bytes.
Remove everything infront and you are good to go.

If you need assistance feel free to reach out.

Thank you very much for the reply!

I downloaded UnMunge, unmunged the mission.lvl into the 2 .script files, HEX edited the extra data out of the front of the files, and got as far as attempting to pass the script to the luadec_32_deb.exe. However, doing so I get multiple errors:

MSVCP140D.dll is missing
VCRUNTIME140D.dll was not found
ucrtbased.dll was not found

These errors occur when double clicking luadec_32_deb or when running it through command prompt.

I assume I'm missing certain drivers required for the program to run.

[{AR}MetalKiller]

Damn. These problems only show up outside the development environment. Sorry for the inconvenience.
I will fix it when I come home.

Side note: A double click on the executable will have no effect.
You have to pass it a file to decompile, which only works through a console (cmd, powershell).

[{AR}MetalKiller]

The dependency to the DLLs should now be removed.
You should only require the exe.

While testing, I found out that there is a bug that only occurs in the release build, so the debug build is included as well: https://github.com/styinx/lua4dec/releases/tag/vP.0.1
The difference is the size of the binary and that you get a dump of the byte code.

[vonfrank]

The dependency to the DLLs should now be removed.
You should only require the exe.

While testing, I found out that there is a bug that only occurs in the release build, so the debug build is included as well: https://github.com/styinx/lua4dec/releases/tag/vP.0.1
The difference is the size of the binary and that you get a dump of the byte code.

Thank you very much!

Unfortunately, I don't know what I need to type in command prompt to get it to work.

I tried

```E:\GAMES\LucasArts\LUA_Decompile>luadec_32_deb.exe -cours1a.script```

and

```E:\GAMES\LucasArts\LUA_Decompile>luadec_32_deb.exe cours1a.script```

with no results. (my anti-virus went crazy while attempting this btw)

[{AR}MetalKiller]

Your feedback is really appreciated!
Perhaps I should add some more error messages if something goes wrong.

The only thing that you need to pass to the program is the file (no flags or anything else required):

Code Select Expand

lua4dec_32_deb.exe tat3a.outThe file ending does also not matter, as long as the content of the file contains valid Lua 4.0 compiled code.

Any anti-virus software is always unhappy about executing unsigned Software.
I try to look for signing tools, but I don't think that there are any free one available that get you around this problem.
In the meantime you have to trust me, that the exe does not contain malicious code.

I am assuming that you extracted the cours1a.script file directly from the unmunge tool?
If so, you still need to remove the metadata from it.
Here is a picture that might help me to get my point across:

The blue part is what you want, the rest you don't care about.

Here is a sample script file from the unmunge tool inspected with a hex editor:

Code Select Expand

ucfbh...scr_`...
NAME....bes1a...
INFO........BODY
;....Lua@.......    <- This is the part where the actual compiled script starts.
................       So, you need to remove anything in front of ".Lua@".

---

To speed up your progress I could offer to decompile the script for you.

[vonfrank]

Your feedback is really appreciated!
Perhaps I should add some more error messages if something goes wrong.

The only thing that you need to pass to the program is the file (no flags or anything else required):

Code Select Expand

lua4dec_32_deb.exe tat3a.outThe file ending does also not matter, as long as the content of the file contains valid Lua 4.0 compiled code.

Any anti-virus software is always unhappy about executing unsigned Software.
I try to look for signing tools, but I don't think that there are any free one available that get you around this problem.
In the meantime you have to trust me, that the exe does not contain malicious code.

I am assuming that you extracted the cours1a.script file directly from the unmunge tool?
If so, you still need to remove the metadata from it.
Here is a picture that might help me to get my point across:

The blue part is what you want, the rest you don't care about.

Here is a sample script file from the unmunge tool inspected with a hex editor:

Code Select Expand

ucfbh...scr_`...
NAME....bes1a...
INFO........BODY
;....Lua@.......    <- This is the part where the actual compiled script starts.
................       So, you need to remove anything in front of ".Lua@".

---

To speed up your progress I could offer to decompile the script for you.

I'm happy to help test this for you! And I appreciate the assistance with it.

So think I managed to get it working. I didnt realize the program doesnt actually save to a file, but it just optputs the lines of LUA code within command prompt itself.

Is there a way to get it to save to a file? or is that not possible?

[{AR}MetalKiller]

Glad to hear that 
Great that it worked now!

Is there a way to get it to save to a file? or is that not possible?

I can make it an option.
What would you prefer?
A second argument, or using the input filename as output filename with the ".lua" extension?

Code Select Expand

lua4dec_32.exe compiled.script                                  ->   compiled.lua
lua4dec_32.exe compiled.script myscript.lua                     ->   myscript.lua
lua4dec_32.exe --input compiled.script --output myscript.lua    ->   myscript.lua

Alternatively, you can copy the text from the command prompt, but I think you knew that already.

[vonfrank]

Glad to hear that 
Great that it worked now!

I can make it an option.
What would you prefer?
A second argument, or using the input filename as output filename with the ".lua" extension?

Code Select Expand

lua4dec_32.exe compiled.script                                  ->   compiled.lua
lua4dec_32.exe compiled.script myscript.lua                     ->   myscript.lua
lua4dec_32.exe --input compiled.script --output myscript.lua    ->   myscript.lua

Alternatively, you can copy the text from the command prompt, but I think you knew that already.

I think simply having it output with the same filename but using the .lua extension would make most sense. Although yes, copy and pasting the text also works.

That being said, I've attempted to re-munge the file output by the decompiler, and it crashes the mission when it attempts to load in-game. I'm not sure if I'm doing something wrong with the re-munge attempt, or if there is an error somewhere in the way it decompiles?