Hello everyone,

I’m Mgamerz, owner of ME3Tweaks.com, and the lead developer of the Mass Effect Trilogy editor toolset, ME3Explorer. I also write end user installation tools ME3Tweaks Mod Manager and ALOT Installer. If you’ve modded the series recently as an end user or a developer there is a high likelyhood you’ve used my software to do some or all of it.

Recently, BioWare announced Mass Effect Legendary Edition, which is a remastered version of the original three games. I’d like to share with you what I know about the new games, what they mean for modding, and what my plans are going forward, as I have had many people ask or tell me things that need to be cleared up.

 

Will existing mods work on Legendary Edition?

No. To be clear, when I say ‘existing mods’, I mean package file based mods, which is the majority of mods. Anything that ends in .pcc, .upk, .u, .sfm are package files, and are what comprise of the majority of each game.

Our files are tied to certain engine versions of BioWare’s fork of Unreal Engine 3. On top of this, each game in the trilogy is on a different build of Unreal Engine 3 with more and more BioWare changes on top, and that spans 5 years of engine development by Epic Games as well. Unless BioWare changes absolutely nothing in their game, things will not work.

Complicating matters is that while our modding tools have improved greatly recently, many mods were built with older, buggier tools, and it is more a bug that they actually work in the game, rather than them actually being properly supported. Many of my older mods even have issues, but the game somehow still works with them.

Essentially, you can think of it this way: Will vanilla package files from the trilogy games work on legendary edition? I’m extremely doubtful, there are too many dependencies on core files that are nearly guaranteed to be changed.

 

But BioWare says they’re working with the modding community?

BioWare may be working with some people, but I’m not sure who. I’m curious, but I also understand whoever they are, they are under NDA, and asking people things who are under NDA just makes their life more difficult.

However, I’m not under NDA, so I can tell you what I know. Nobody in the toolset development group has been contacted, which consists of three people: Myself (Mgamerz), SirCxyrtyx, and Kinkojiro. We are the ones who build and maintain the tools (ME3Explorer) that are used to make mods, and I personally am the one who also builds the tools that installs the majority of said mods (ME3Tweaks Mod Manager).

These tools are practically required to mod the trilogy (excluding texture-only mods), and all three of the developers know the engine internals in ways that many mod developers don’t – for example, the flags in package headers, various file formats, how property serialization works in our games, Unreal’s memory system, etc.

At this point in development of Legendary Edition, there is not a lot that BioWare can internally change to help with modding. You cannot bolt proper mod support onto a game, it must be done from the start, and nobody reasonable expected BioWare to do such a thing.

 

What are my concerns?

Ever since Mac Walters tweeted about working with the modding community, I have had dozens of people message me directly about how they’re excited to have existing mods work on legendary edition. Many mod authors are also excited to work on legendary edition.

I do feel that there is an expectation of modding Legendary Edition that is not actually based in reality, and it’s going to cause issues if the game turns out to not be so easy to work with. End users will blame both us, the tool developers, and BioWare, as they feel they were mislead.

They will also blame mod developers because BioWare’s statements made them think this was easy – we have users already come for assistance asking why these games are not easy to mod like Skyrim.

 

But can BioWare do anything to help?

BioWare for nearly a decade has seemingly taken a ‘do not acknowledge mods’ policy, so for them to say they’re working with the mod community out of the blue is a real whiplash for the scene. I would love to have a conversation with BioWare to discuss things that could help the modding scene that should not take a lot of effort to implement, but I’ll detail some of them below.

 

1. We need a way to debug the game

ME1 on PC included the application log functionality, which wrote a bunch of useful information to the Documents/BioWare/Mass Effect/Logs folder, including crash information (such as loading an unused mip).

This log functionality was removed in ME2, and all internal debug code also was stripped out, so debugging ME2 is nearly impossible.

ME3 included a blank stub for ‘appErrorF’, which we hooked to get a basic level of debugging, but it’s still pretty weak and often doesn’t have any useful information. Complicating things was the use of TOC files which for years added extra steps to modding that were not required for ME1/ME2.

If you want us to be able to more easily mod the game, we need a way to know what the game engine is doing, or is unhappy about. Can you imagine writing software where the only way you know something is wrong is when the application unexpectedly closes? Because that’s what modding is often like.

We may have advanced tools these days, but when it comes to debugging and fixing a mistake you made, it can be such a frustrating experience that you just give up. If Legendary Edition removes the logs feature and strips out debugging tools we can leverage, it’s going to seriously cripple the ability to mod these games. Even if logging functionality required another edit to enable (such as through the configuration files) it would go a LONG way in helping us.

 

2. WE NEED A WAY TO LOAD FILES IN AN OVERRIDE FASHION

In the current scene, we load files primarily through the use of the DLC loading mechanism for ME2 and ME3. We recently (about 2 years ago) uncovered some tricks in ME1 to also load DLC overrides in ME1, but it’s weaker and has some caveats, such as not working with movie files and the TLK files being embedded into a billion files (though I doubt this is an issue, as the PS3 version of ME1 did not use this system).

There are some core files that can’t load from DLC, such as SFXGame, which is a real problem because it has a lot of crucial edits. One system that I personally leverage is TESTPATCH in ME3, which can override classes, but we don’t understand how it works well enough to generate new entries.

If the DLC system is removed – which it very well could be, because if this is the ‘best’ version of the games, they won’t need to have post release content, and thus there’s no need for DLC loading – it’s going to greatly complicate modding. Replacing files in the main game is extremely easy to do wrong, leads to wide incompatibility, and is frustrating to support; I know this because for years I wrote multiplayer mods, which required file replacement.

DLC mod loading allowed us to override and remove mods fairly easily. There are plenty of downsides but it is a huge improvement to direct file replacement.

 

3. Moral support? (optional)

For nearly a decade, modders have built loads of mods – Mass Effect Happy Ending Mod, controller support, bugfix mods, new levels, same gender romances, texture overhauls, etc…

Personally I’ve built about a dozen mods, made many programs, even built an entire web application that can be used to build mods – but I can count on two fingers the amount of times I’ve ever had my work acknowledged by someone at BioWare in any sort of way that actually meant something, beyond essentially them saying ‘cool’.

This is a somewhat weird thing to bring up, but if you look at the creative scene for Mass Effect, there are many areas you look at – artwork, cosplay, streamers, etc, but the modding community is by and large completely ignored, and it is somewhat demoralizing.

We build these mods because we love these games, even if we at times are very critical of your work. This modding scene was actually created because of the critical reception to the end of Mass Effect 3, but it’s because those people loved the series, not because they hated it.

There are likely a lot of legal ramifications that bar you from coming out and endorsing mods, plus a lot of things like ‘is this a reputable person’, etc. that are not really present when you post on a picture on Twitter that someone made. I also know that by tweeting or coming out saying ‘What a nice mod this is!’, people would almost assuredly misinterpret it as you ‘supporting’ mods, rather than just ‘supporting the idea of them’.

That said, it would be nice if the work we do could at least have a way of being acknowledged. For years we have been mostly ignored, and while I’m sure some developers are fine with this, it would greatly improve modding morale if we at least knew someone at BioWare actually cared, or at least thought it was interesting, even off the record. I’m sure there are some people that make these games that do enjoy our work, but you would be hard pressed to find anyone who develops mods that knows that.

 

4. WE NEED A WAY TO IDENTIFY WHICH FILE BELONGS TO WHICH GAME

Yes, this is a real nitpick, but it’s actually come up when trying to parse files from PS3 and Xbox assets of ME games. If package file headers could have some way to identify their game (something unique, like package tag) it would greatly reduce complexity. It seems small but as someone who works on tools, knowing what game a file belongs to is VERY IMPORTANT.

This is of course assuming that you are unifying the engine versions, which I assume you are, since this was already done for the PS3 versions of the games.

 

Future plans

I plan on working on legendary edition, but I want to make it clear to end users AND mod developers that things are going to take time. One of the things that greatly concerns me is how news sites are saying how great this game will be to mod based on just a few tweets and an interview that did not really talk about modding.

I want to make sure users and developers understand that modding will take time, will not work on day one, and likely will not work on month one, and will likely have serious teething issues possibly for years.

As someone who works at the top of the modding chain, a lot of the development efforts have fallen to me – a developer needs a feature in Mod Manager for their mod? I have to implement it. Need an easier way to edit game files? One of the toolset developers typically has to do it – and we do this for free on our own time.

The other developers I work with (SirCxyrtyx and Kinkojiro) are great people who have all done years of research and development, and together we’ve built lots of great tools and features, such as Pathfinding Editor, entry porting and relinking, dialogue editing tools, and more.

I personally spent YEARS developing and refining Mod Manager and have spent years completely reworking ME3Explorer into a much more professional grade tool. Our scene is much healthier than it was back in 2013-2015 when our tools were still immature, making and using mods is far easier these days. Some of this progress is going to be reset with the release of Legendary Edition.

 

WILL EXISTING TOOLS WORK ON LEGENDARY EDITION?

Short answer, no. We will have to wait and see. I do not have any plans to have ME3Tweaks Mod Manager work on LE – it will require me to build a new tool, as M3 is very tailored to the current games, tools and codebase. ALOT Installer will not be necessary (hopefully!), and ME3Explorer may be forked into a new project depending on how much has changed internally, but we don’t really know anything useful yet. 

Personally, I am not a fan of trying to build a conversion tool between the trilogy games and legendary edition, as it would be a monumental amount of work, especially if we don’t understand changes to the internals of the games (including many things we currently don’t know about the existing games). The toolset already supports about 7 or 8 games across 4 different platforms, adding another 3 will be significantly complicate the codebase. But it really depends on how much is changed.

 

Anything else?

Yeah, the last thing I’d like to mention, is that a lot of this post is about the developer side of things. Myself, SirCxyrtyx, and Kinkojiro have all done absolutely masochistic things in the name of research and development (but especially Kinkojiro), but the flip side is that there are far more users who have little to no patience even installing mods.

If installing mods is difficult (such as a complicated system without a DLC loader), many users will not even attempt it, and this leads to diminished returns on the development side. I work on both sides of the coin, writing the tools for installing mods, as well as developing them, and over time  I’ve adjusted my software to make it easier for users.

I’ve noticed that as it becomes easier to install mods, more developers have appeared, and part of it may have to do with decreased support time for mods. My end-user tools for example have a very detailed diagnostic system that can be used to identify a wide variety of issues, which means users don’t have to ask developers how to fix things.

 

CONCLUSION

I believe our modding tools for legendary edition could be greatly improved if we had someone we (the modding toolset developers) could talk to at BioWare about, as there are still many gaps in our knowledge of BioWare’s proprietary changes to the engine that significantly hinder our efforts. With the release of Legendary Edition, those knowledge gaps will likely significantly widen. 

Want to have a talk about some other things the scene could use BioWare? Drop me a line at femshep AT me3tweaks.com, there are many small things that would greatly reduce the friction of modding that would really help set up the scene for success.

Hi all, it’s me again, your favorite modder who publishes a single research blog post a year. Welcome to my new blog, where I will also post maybe once a year! I got fed up with blogger’s endless unfixed bugs. I’m going to leave the content there though for historical sake.

I just finished a hardcore crunch to ship ALOT Installer V4, which is a complete rewrite of ALOT Installer. ALOT Installer is the Mass Effect modding scene’s main texture installation tool, built on top of aquadran’s MassEffectModder program, which can be used to install textures in a more advanced fashion. In V4 of ALOT Installer, I split the main ‘core’ features into a cross-platform .NET Core library so I can also write a frontend that works on Linux. But that’s not why I’m here today – I’m here to follow up on how I fixed Mass Effect on PC to not require elevation for good.

Mass Effect on PC: About what you’d be expect from a mid 2000’s console port

For those of you not in the know, Mass Effect came out on PC back in 2008, and was ported from the Xbox 360 by a studio named Demiurge, who also developed Pinnacle Station for Mass Effect. It’s… a really meh port that has not aged very well. It’s passable as a game but it has a lot of problems, even when it came out. Particle LODs not working properly, texture LODs being read backwards, ini settings being randomly reset to their defaults, the problems are pretty numerous, just to name a few. But nothing completely game breaking.

Well, kind of. There is one, but it’s not specifically due to Mass Effect. The big issue is that Mass Effect requires administrator rights to run, because Demiurge seems to have assumed everyone would run the game as administrator – which might have been OK if the game was only really developed when Windows XP existed, but Windows Vista had already been out for over a year by the time the game had released. Even back then though, Windows XP had a concept of LUA (Least User Access) with separated user accounts. For more information on this, you should check out the original post I wrote, Why Mass Effect on PC requires administrator. It describes a lot of backstory to this post.

Oh boy, PhysX, my favorite physics library!

I may have a slight beef with this SDK.

Mass Effect for PC runs on a lightly modified version of Unreal Engine 3, which appears to be dated around late 2006. According to some former BioWare developers, this version of Unreal Engine was not very mature yet, to put it lightly. According to some stories from these developers, it was really difficult to work with because Epic Games was focused on Gears of War and not dedicating much time to their partners who were also using the engine.

Unreal Engine 3 uses PhysX for physics interactions, so Epic Games built a dll that interfaces PhysX to Unreal Engine data formats through a file named PhysXLoader.dll, which loads the PhysX libraries from both parties. PhysX is a physics simulation library that was acquired by AGEIA Technologies in the mid 2000s before AGEIA was sold to Nvidia in early 2008. If you remember Physics Processing Unit cards, or PPU, they were using PhysX before Nvidia promptly killed that idea.

PhysXLoader.dll, PhysXCore.dll, and NxCooking.dll make up the PhysX dlls for Mass Effect.

All three Mass Effect games use PhysX, but Mass Effect 2 and Mass Effect 3 use the system’s install of PhysX, while Mass Effect uses the local game’s PhysX. Mass Effect 2 and Mass Effect 3 also use the ‘modern’ version of PhysX, rather than the legacy one that was shipped by AGEIA. Nvidia changed some paths under the hood when it took over, which separates Legacy out from it’s ‘modern’ versions.

But that doesn’t seem to stop Legacy PhysX’s uninstaller from deleting modern PhysX’s files/registry keys, so during the course of testing this fix, my other copies of Mass Effect 2/3 didn’t work, even after installing the ‘modern’ PhysX redistributable. It’s really annoying how BioWare couldn’t just ship a 8MB library with the game – they already shipped the installer for PhysX with the game, so it’s not like it saved space!

But anyways…

The issue with Epic Games’ PhysXLoader.dll is that it can load PhysXCore.dll locally, or from the system’s installed version

Err… wait, how is that an issue? Can’t you just load the local dll, and if that doesn’t exist, load the system one? How is that an issue exactly?

OH BOY HERE WE GO
You won’t believe how many facepalms there were as I making this fix.

On boot, Mass Effect writes two values to the Windows HKEY_LOCAL_MACHINE registry:

REG_BINARY HKLM\SOFTWARE\AGEIA Technologies enableLocalPhysXCore [mac address, 6 bytes]
REG_DWORD HKLM\SOFTWARE\AGEIA Technologies EpicLocalDllHack [1]

*Mass Effect is a 32-bit program, so on 64-bit systems it goes into HKLM\SOFTWARE\WOW6432Node\AGEIA Technologies instead, if you’re looking for yourself.

Remember these registry values, they’re going to be important later!

These registry values are why Mass Effect requires administrative permissions. In my previous blog post linked above, we explored why these writings were enough to make Microsoft put Mass Effect into it’s compatibility database, which forces it to run as admin when matching on certain executable criteria, which we worked around by modifying the executable criteria to no longer match.

We have to modify the executable to enable Large Address Aware, so the game could load higher resolution textures without running out of memory, so there was no way to avoid breaking the signature. This in turn caused Origin to no longer run the game as it would not elevate games without a valid EA signature. But if the game cannot write these registry keys on boot, the game may crash…

So it’s already a big fun chain of problems, but we worked around Mass Effect needing administrative rights by simply giving the user account permissions to that specific AGEIA Technologies registry key. This would let the game process write the values it needed, and would we could go on our merry way. I assumed the game crashed because it was denied write permissions and Demiurge couldn’t be bothered to write a try/catch around the registry writing code.

You probably shouldn’t name your registry values as a hack if you want me to think this is a good idea

Our solution to this problem did not change Mass Effect’s behavior – the values it wanted to write to the registry were going to be written one way or another, so we were just letting it do the thing it’s always done, just without administrative rights. There wasn’t really any change in application behavior.

The two registry values that Mass Effect writes.

mirh, a moderator for PC Gaming Wiki, sounded the alarm for years that somehow we were breaking other games in ALOT Installer – even though our application didn’t actually change how Mass Effect was behaving writing these values, so there’s no way our change would break other games.

After many months, he wrote a fairly detailed reason why ALOT Installer (when, in reality, it was Mass Effect) is breaking other games: enableLocalPhysXCore being in the registry is used by other games using Epic Game’s PhysXLoader.dll. When I was writing V4 of ALOT Installer, I told mirh I would take a more serious look into his idea of a solution that would not break other games, even though at the time I did not really understand how a registry key with the system’s MAC address would break other games – or why it even used a MAC address to begin with.

mirh seems to have determined this enableLocalPhysXCore lets Mass Effect use the local directory’s PhysXCore.dll/NxCooking.dll, instead of loading the one from the installed PhysX redistributable. Mass Effect doesn’t install the PhysX redistributable, so it could not rely on it existing, so it needed to use the local libraries.

Hope you’re strapped in because this is where it gets really dumb:

The MAC address stored in in the registry by MassEffect.exe is read by PhysXLoader.dll and compared against your system’s MAC address to determine if it should load the local directory’s PhysX libraries or the system’s.

Which MAC address?

¯\_(ツ)_/¯

So the way Mass Effect works:

  1. Very early in the boot process of MassEffect.exe, your MAC address is read and written to the registry as enableLocalPhysXCore (along with EpicLocalDllHack)
  2. MassEffect.exe loads PhysXLoader.dll
  3. PhysXLoader.dll reads the value of enableLocalPhysXCore and compares your system’s MAC address against it
  4. If it matches, it uses the local folder’s PhysX, if not, it uses the system’s redistributable version of PhysX

Yes, you read that right.

It turns out that other games, such as Mirror’s Edge, have a PhysXLoader.dll that also reads these values (as they’re based on the same code), but they don’t include local PhysX libraries. So those games boot up, see enableLocalPhysXCore, and try to load the local library, which fails, and the game doesn’t start. This information is second hand from mirh – I have not tested other games broken by this registry value.

Normally that value wouldn’t exist, and it should use the system PhysX. This behavior can be tested in Mass Effect by denying it write permissions to the registry key, deleting the values, and having Legacy PhysX installed – it will use the system libraries instead. If system PhysX is not installed, the application will not boot – this is why we originally had to let Mass Effect write these keys, otherwise it could appear that the installer broke Mass Effect, when it actually was a terrible implementation by Epic Games.

Facepalm
It’s hard to imagine any possible scenario where this was a good idea.

If you’re interfacing with a library that has exports you can call to initialize/load the PhysX SDK… couldn’t you just, you know, pass a boolean to tell it to locally load? Why does it not locally look to begin with? And what’s up with the MAC address? Why is this in the registry, where it behaves LIKE A GLOBAL SETTING???

All of these seem like terrible design decisions – and after disassembling the PhysXLoader.dll, it seems like that’s all they were – terrible design decisions. Let’s take a deeper look at Mass Effect and walk through the process of fixing this, from start to finish.

Finding a starting point

WARNING: I’m a super novice reverse engineer. I’ve assembly modded Megaman Battle Network games (and wrote a pretty neat guide to how to design hooks), designed mods in ActionScript2 P-Code, and worked with UnrealScript bytecode, but never really got far into x86 assembly. I’ve opened IDA numerous times, and could kind of find what I was looking for, but never really could understand it. I’m sure this process is much easier for more experience reverse engineers.

Scary wall of IDA
It’s hard to get excited for reverse engineering when you have almost no idea where to start. This is IDA’s graph view, which helps a lot to visualize the assembly… but is still very hard to understand in a large 20MB binary.

Recently (as in the past 2 years), the National Security Agency of the USA (the NSA) released Ghidra, a free open-source reverse engineering toolkit that can reverse assembly back into somewhat-readable C code, which is infinitely easier to read than IDA assembly graphs. Both IDA and Ghidra have their strengths; IDA has a debugger that lets you step through the assembly and see which code paths are about to execute, and can find Unicode strings (which Mass Effect games use), while Ghidra can recompile assemblies from it’s decompiled C code (sometimes), has an assembly-to-C converter (sorry, I don’t know the name of this), is open source, and works on tons of platforms, binaries, you name it.

Ghidra logo
Ghidra is a great tool if you’re getting into reverse engineering as it lets you see the assembly as C code, though without variable names.

So the information I knew starting out was that Mass Effect was writing enableLocalPhysXCore and EpicLocalDllHack. Let’s start by looking at MassEffect.exe, finding these strings, and seeing what references them. Using a hex editor, I know these are unicode strings, so I’m going to look for them in IDA, since Ghidra doesn’t seem to support this.

IDA Strings Window
IDA Strings Window. I finally learned that Shift + F12 opens this useful tab.

Inside of IDA’s Strings window, searching for enableLocalPhysXCore shows the string. Double clicking it takes us to the data section of the executable it’s defined in:

IDA strings for enableLocalPhysXCore
You can see above where enableLocalPhysXCore, EpicLocalDLLHack, and even the registry key path is defined, all right next to each other.

Above we can see the definitions of the strings that all seem related to our quest. Above the text definition, we can see there’s a DATA XREF, which means something directly references this data – probably what’s writing it. Let’s double click the XREF and see where we go.

Shown in IDA View, rather than Graph View.

Looking at this, we can see it’s writing RegSetValueExW. I’m a very weak C developer, so after doing some googling, I can see that this is the stack being setup to invoke a method call from the Windows API in C, which you can somewhat see with the parameter name that IDA shows, such as lpData and dwType. We know that enableLocalPhysXCore’s value is set to your MAC address – lets see where that gets set. Let’s change to Graph View for a more logical look at this.

enableLocalPhysXCore's 'set' method

We can see in the third block that eax is pushed onto the stack for lpData, and it’s also pushed onto the stack for this mysterious ‘sub_10929393’ call. There are no other calls in this subroutine that don’t have defined names, so this is probably where the MAC is obtained. Let’s jump into it.

This seems to be some sort of wrapper subroutine, or perhaps something IDA does, but it just points to another subroutine. Let’s go there instead.
IP Adapter code

This subroutine has names that are pulled in from the Windows API that show us that this has to do with networking. We don’t really care about the MAC address, but this lets us define the name of this subroutine. We’ll call it GetMacAddress. We’ll go back to our original subroutine we were looking at, and rename that too – this seems to be something like SetupPhysXSDKLoad, so we’ll name it that.

Our renamed subroutines

This is a relatively small subroutine, and all it does is write the two aforementioned registry values, and that’s it. There are no other references to this subroutine beyond one early in the boot process – so at this point I’m fairly certain that Mass Effect’s executable never actually reads these values, and at this point I’m still not sure what EpicLocalDllHack does.

Cracking open PhysXLoader.dll

Now we know that Mass Effect’s executable never reads this key, it must be in one of it’s dll’s. While I won’t show it here, using ProcMon (a great tool for modding and doing things like this in general), I could see that the registry value was read right before the library was loaded in the MassEffect.exe process, and the local dll was loaded. I observed it reading the system one when I denied Mass Effect write permissions to this directory, and the game not loading at all when there was no system version of Legacy PhysX installed.

The first dll loaded is PhysXLoader, after which PhysXCore.dll loads, so that is our logical dll to analyze. Let’s crack this open in IDA and see where enableLocalPhysXCore gets used there. I’m also going to pop this dll open in Ghidra so I can get a better idea of what’s going on. Doing the same procedure as above for finding where enableLocalPhysXCore’s string is used, we find this subroutine:
PhysXCore.dll subroutine

This is not too hard of a subroutine to read, especially in graph view – we can see there’s a loop from the left box going to the box above it. Still, not the easiest thing for a novice to read, so let’s see what this looks like in Ghidra. I use the location of this subroutine to jump to it in Ghidra (0x10001640).
Ghidra at enableLocalPhysXCore usage

This gives us a bit more of an idea what’s going on – the subroutine calls, the loop, the return type of the subroutine. We can identify things in one tool and label them in both to help us figure out what we’re doing. Immediately we can see that there’s a subroutine call that passes a registry key path, a value name, as well as two variables. In the loop we can see it’s comparing 6 items by index to determine if they’re the same.

Just to make this a post a bit shorter, that registry lookup subroutine is indeed, a simple registry fetch wrapper. Its parameters after reverse engineering (which mostly was just mapping subroutine inputs to what they are for the windows api calls) are subkey, valuename, returned data pointer, returned data size.

You can see it has a loop that runs for 6 iterations, checking each value is identical. Using the information we’ve gained, we can rename some variables to get a better picture of what this subroutine is doing.
Partially reverse engineered subroutine

We know that Mass Effect wrote a 6 byte mac address to the registry, and that PhysXLoader.dll just read that value from the registry, and the subroutine is comparing something byte by byte 6 times. Logically, we can assume that in the above picture local_14 is the MAC address. Knowing this as well, we can guess that FUN_10001580 is fetching the MAC address and setting it, so we’ll rename some more items in this subroutine.

Now, this subroutine call seems to not do the actual loading – it is just checking for the key, and if the MAC address matches. Given the name and what we know about this key’s activity, we can give this subroutine an educated guess of a name of ‘ShouldUseLocalPhysX’. Comparing IDA and Ghidra’s decompilation of this subroutine however lead to slightly different results, with Ghidra’s seemingly being wrong:

IDA showing extra data that doesn't appear in Ghidra decomp
IDA shows that al is being set to 1 if the loop exits normally, and 0 (the xor al,al) if any bytes don’t match – Ghidra doesn’t show this, in fact it shows the return type is void, which seems to be wrong.

After doing some googling for this part of the post, I learned that EAX is typically used as the return register for x86, and the ‘al’ register is the bottom 8 bits of EAX. I’m not experienced enough with Ghidra to know how to retype the signature for this kind of lower 8 bits being returned, it may be just something Ghidra does not support currently, or I am missing a setting I was supposed to use.

Code for loading PhysXCore.dll
Disassembly of the subroutine that calls the one that is looking at enableLocalPhysXCore.

However, if we look at the references to this subroutine (there are two – likely one for each library) in IDA and Ghidra, we can see that ShouldUseLocalPhysX is being called, it’s checking if al is not zero. If it’s not zero, it loads the local PhysXCore.dll. If it is, it appears to look it up through the system PhysX installation, which is identified by another registry value in the AGEIA Technologies key named PhysXCore Path. We’re not really interested in that, since we’re looking to force PhysX to always load locally, regardless of the enableLocalPhysXCore being set or not.

Looking at the other cross reference, we can indeed see it’s loading the NxCooking library, using ShouldUseLocalPhysX in the same way:
NxCookingLoader

Armed with this knowledge, there are a few ways we could fix this. I’ve done a lot of function modding in UnrealScript bytecode, which for a very long time could not be extended or shrunk in size, so figuring out ways to make logic checks fail or pass without changing the size of the script was a challenge.

For example, if I needed an if check to essentially be removed, I’d need to find a way to change the comparisons to always be true or false. One way I could make an if check fail would be to modify the object references and comparison bytecode tokens to produce a condition such as if (objectA != objectA), will always produce false (assuming they are not null). We need to find a way in ShouldUseLocalPhysX here to always produce a true result.

When I was writing a symbol table for Megaman Battle Network 3, I learned to comment everything I figured out about the disassembly. I would work on something for hours, totally forgetting what I did, but could come back to my comments and understand it again.

Sometimes my symbols/comments on certain lines, such as subroutine names or pointer tables I’d identified, would come up in other subroutines, giving me useful context that I would not have seen otherwise. I’ve commented and labeled several items below, which make reading this subroutine easier.

Patching out the world’s worst boolean check

Notated ShouldUseLocalPhysX
We need to ensure the bottom left block is always reached – while also making sure we don’t touch the stack (which can EASILY nuke the whole program). Technically, we could just rewrite some of this subroutine, but I’d like to make the least amount of change possible here. We need to essentially make sure the jump to the bottom right block never happens.

Conveniently, x86 has a 1 byte instruction named ‘nop’, which does literally nothing but take up one byte. Also conveniently, the jump instruction to that block is 2 bytes, comprising of a 0x75 (jnz rel8) and a 0x19 (relative offset).
Hex view of the instructions

[FUN SIDE STORY] Seeing a 1-byte offset brings me back to my Megaman Battle Network modding days, where jump/branch instructions would make or break moddability for certain sections of the ROM. When writing a hook (which redirects the program counter to your own code), you had to find a jump or branch instruction, which you would modify it’s relative offset to point to your code. You would then push registers onto the stack, run your code, then put the stack back together so the subroutine exited like it should.

ARM (more specifically, THUMB) had limited branch instructions that used different sizes for their relative offsets, which were not able to always jump to any point in the ROM due to their location in the ROM. Since the game was written in assembly, finding free space at times could be difficult – sometimes you’d have to chain multiple hooks until you could get the program counter into to a free area to write new assembly code. This jnz uses opcode 0x75, which is jnz rel8, so it can only jump up to 128 bytes (or is it forward only, so 255 bytes?) away, which would be a real problem if I was doing assembly modding the way we used to back before we had fancy tools like IDA and Ghidra. [END OF FUN SIDE STORY]

Anyways, after we nop out that jnz to the not-matching block of assembly, our ShouldUseLocalPhysX subroutine now looks like this:
Patched subroutine

Now the not equal block cannot be reached. The ‘check’ still occurs, but it never can return false. The local PhysX core dll will always be used.

What are the downsides?

The PhysXLoader.dll file is signed by Epic Games, so this obviously breaks the signature since we modified the file. The game is not checking signatures on load, so that’s not an issue. Some antivirus programs may complain about broken signatures, but over time that typically improves. Outside of writing an in-memory patch (like we do with the asi mod loader our scene has), we would have to modify the binary of the library.

Final behavior

Using this patched dll, the game now works with or without the registry value, which means Mass Effect no longer needs administrative rights at all to run. Disassembling this was a real rant-fest because I could not get over the asinine way this check was implemented, not just a registry value but a match on MAC address too. While I was debugging and stepping through the instructions, I actually broke the game because I turned on my VPN and my MAC address it used changed.

This was a good learning experience, I’ve learned more about Ghidra and IDA, and about more problems in the PC version of Mass Effect. This patch is automatically applied as part of the install step of ALOT Installer, so users will never need to worry about having the enableLocalPhysXCore key set after it’s patched. We also modify the Mass Effect executable to write a value named enableLocalPhysXCor_, so our patched versions won’t write the value that breaks games. Vanilla Mass Effect executables will still break other games, but it is beyond the scope of our program to try and protect people’s software from poorly written PhysX loaders.

Essentially, this is how I felt afterwards:
Yeah...

Oh, and what about EpicLocalDllHack? Well… it does literally nothing. Absolutely worthless, it’s never read. The only thing I can think of that it could maybe exist for is to keep the registry key around if a PhysX redistributable is uninstalled, as it’s not part of the PhysX redistributable’s list of values – but that’s just a guess.

Was having a parameter ‘PreferLocalSDK’ for PhysXLoader.dll too much to ask for Epic Games?

This post originally appeared on my now deprecated ME3Tweaks Blog that was hosted on Blogger. It’s been ported here for consistency. This post was published originally on January 30, 2018 and has only been slightly modified to make it easier to read here. The contents remain unchanged.

———————

Hi all. It’s been some time since I’ve posted. No I am not dead. I have been doing a lot of work behind the scenes. Been working on my own ME3Explorer fork which has many quality of life improvements and even some new tools. I’ve also been working on Mod Manager 5.1 which has some slick new importing of third party mods, but it’s been on the back burner since I’ve been working on the new ALOT Installer front end.

ALOT Installer 2017
ALOT Installer using the 2017 manifest

I have been working with CreeperLava and Aquadran to implement this, which should make life way easier for end users installing ALOT and its Addon (textures by others). One issue I have had is that Origin will not launch the game once ALOT is installed unless run as admin. And since you cannot have origin startup as admin on boot it is really annoying. This also affects MEUITM. So I looked into figuring out why. It’s actually a perfect storm of security, bad coding, and people trying to make others lives easier.

Lets break down how Mass Effect works with Origin in a non-modified state, on Windows 10.

  • You run MassEffect.exe. It immediately asks for elevation to administrator.
  • At the end of the MassEffect.exe image there is code to call Origin to run the game, as a form of DRM. It calls origin, and then exits.
  • Origin checks your entitlement to the game and will then run MassEffect.exe as specified by the registry (not the one you booted specifically) and attempts to run the executable.
  • Origin is unable to run the executable because it requires elevation. In order for the DRM to work it must be able to interface with the process, so it elevates one of its internal services so it can communicate with the game for DRM purposes.
  • MassEffect.exe is run as administrator. Origin communicates with MassEffect.exe and game execution continues like it does from the DVD version.

Now, this all works (though two UAC prompts) on an unmodified game. But install MEUITM or ALOT and you won’t be able to run the game anymore through Origin as a standard user. What gives?

File signatures

Both MEUITM and ALOT modify the MassEffect.exe executable to use Large Address Aware. This allows the 32-bit Mass Effect process to use up to 4GB of ram instead of the usual 32-bit limit of 2GB. By modifying the LAA flag, the digital signature on MassEffect.exe is broken – the signature is used to verify the file is not modified. Once the file is modified, the signature is no longer valid.

Origin, when running an elevated process, checks to see if the EXE is signed by EA and is valid. If it is not signed by EA, it doesn’t elevate its DRM communication module. Mass Effect boots up, and then immediately closes as the DRM unlock doesn’t work because there is nothing to talk to it from Origin’s side as it refused to elevate.

So by modifying the EXE, origin will refuse to run an elevated game executable. But we need LAA, so we must work around this issue. Our only hope is to prevent MassEffect.exe from running as an administrator. We should figure out how it is set to run as administrator first.

Looking in the EXE’s manifest I can see it runs as the invoker – the user who is running the EXE. Which means this executable should not require administrator. I check my compatibility settings, nothing there either. Somehow, this is being elevated at boot but not through the exe itself nor my own doing. The culprit? The Microsoft Windows Compatibility Database.

mirh (I’ve seen him in some of the modding circles) did some sleuthing to figure out why Mass Effect is being forced to run as administrator. It meets the criteria in the database – it specifically has an entry for Mass Effect – which has its own always-forced compatibility settings. It makes sense – users shouldn’t have to configure the settings if MS already knows ones that (technically) work.

As you can see there are two entries for this game – MassEffect.exe (the game) and the launcher (which is not included in origin’s version sadly). The compatibility fix is RunAsHighest – which means administrator – and the criteria are:

  • EXE is named MassEffect.exe
  • Company name in manifest is BioWare
  • Product name in manifest is Mass Effect
  • Product version is equal to or less than 1.2.0.0.

This criteria matches all known versions of the game – I believe including pirated versions. So by matching all of these criteria, the exe is forced to run as an administrator. You can test this out easily by simply renaming MassEffect.exe to anything else and it will no longer require admin to run. (Origin won’t be happy).

The fix

So now we have an idea of how to fix this – but why is this entry here? Well, due to Demiurge/Bioware not following the idea of Least User Access (LUA), Mass Effect on its very first boot requires administrative privilege to write to the HKEY_LOCAL_MACHINE\SOFTWARE\AGEIA Technologies registry key. If this key does not exist, it will attempt to make it – without admin, it doesn’t have permission to, and the game simply crashes. This key seems to hold some sort of information about what is now known as PhysX. This could possibly have been done by the installer, but it seems they made it be done in the game instead.

This is why Microsoft forces the game to run as admin always – for this one single item. It makes sense – make it run as admin, user does not need to worry about compatibility settings. However due to this combo of all three – LAA breaking the signature, MS forcing it to run as admin, and Origin refusing to work with elevated processes that have a broken EA signature – Mass Effect does not run with Origin and LAA.

So what did we do to fix it? We simply modify in the EXE the product name from Mass Effect to Mass_Effect. Really, that’s it. It fails the criteria check and the game no longer needs admin and Origin is happy (except for that update nagging it always does). In both MEUITM and ALOT Installer we added code to make it create this registry key with write privileges for the current user, so if Mass Effect needs to make the keys (maybe its never been run somehow) it will be happy.