-
Notifications
You must be signed in to change notification settings - Fork 233
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Game Pass save support #306
Comments
Spent a day trying to do something with these files as well.. Think you can pretty much forget it.. If they bothered to compress the files I'm sure they also encrypted them. I see references to AES etc in the loggings... Looking how well tight shut everything around store apps is and the inability to mod anything, I'm pretty sure MS closed down this things as well.. Good luck trying though. |
Figured out the container stuff on my own for my own save editor. The data and meta files are different from Steam and that's the reason they cannot be loaded in any editor. For Steam the data file is completely plaintext. So, the most crucial part to support the Windows Store is to decipher the data (and meta?) file into plaintext (and back) Edit: As far as I know for the saves it's completely up to the devs what they do with it. But as far as my own research goes, it seems to be the same encryption/compression as for the PS4 |
@SubsterNL since I was able to take someone else's save and load it into my game, there's no MS-specific encryption. I was actually pretty surprised by this, I was expected them to be locked down as well but it doesn't seem like that's the case. @cengelha do you have any idea what that compression is? have you gotten ahold of any compressed PS4 saves, and do they look like the XGP ones (chunk of uncompressed text interleaved with chunks of compressed text)? |
@snoozbuster Unfortunately no idea. Tried a few things but I'm not expert in it... About the PS4 saves: I have a few from SaveWizard (which includes plaintext JSON) but if you look in this issue (#258) it looks like it's not the default and the output of the PS4 Savemounter contains something that looks a lot like what we have here. Another clue for me are the PS4 meta. Couldn't figure out one entry but I assume it's the compressed size of the JSON and roughly the same as for the Windows Store for a freshly started game. |
@cengelha if the meta files are the same 24-byte files that I described in the post, I think it's playtime and save date in there. I think that file is used to display the info on the save select screen (if you swap out the meta file from another save, it shows the other's save's difficulty, playtime, and save date). Now, it might also be used in the decompression process, since the game will fail to load the save data if the meta and data don't match. but as long as they match, the game will load the save successfully, even if you got it from another container and renamed the hex files on disc to match - so there's nothing in the container files that's used on the save data itself. I'd love it if you could attach some of those PS4 saves you have - I'm definitely interested in seeing how similar they are. |
@snoozbuster Sorry. Forgot to mention that the meta differs on all 3 platforms. As the PS4 saves are not mine I will ask if it's okay to post them. But it's one big file that contains a byte header with a size of about 4kb including the meta data among other. The meta data have a size of 48 byte for each save and the most interesting part here are the last save timestamp and the size if the JSON (data) and its offset in the file. After that header the JSON for each save is appended with a separator string between them. The meta for Steam is a 104 byte file but only 56 byte are used. It contains a SpookyHash and a SHA256 of the data file. If they don't match, the save cannot be loaded. Timestamp here is the last write time of the data file itself. Sounds reasonable what you wrote about the Windows Store meta. I will have a look after work. But even if it's a part of the decompession/decryption, there is still the question what algorithm is used... |
@snoozbuster Ran my tests and here's the result (converted it to an int32 array):
Save timestamp is probably like Steam the last write of the file itself.
It's a fresh save, just ran to the ship and saved. As you see the last value doesn't grow with the play time. Don't know what the first jump in the last value was but for the second bigger jump I interacted with the emergency beacon at the crash site. |
@cengelha fascinating... if the last number is the size of the decompressed JSON, are fresh saves on Steam usually 900kb? cause that seems awfully large. that last number is definitely mysterious though... When I mixed a meta file from one save with a data file from another it wouldn't load the data file anymore. what if we mix the meta file from one save with a data file from another, but transfer the last 8 bytes from the matching meta file as well? (basically just changing the first 16 bytes that we are pretty sure we understand). if doing that successfully loads the game then that could point to that last number being used in the decompression or validation of the save data. |
@snoozbuster Mixing the meta data is definitely worth a try. |
@snoozbuster Good news! Mixing the meta data works, btw, but I also got the algorithm for the compression and have the first step already running. Edit: Found. |
In case you haven't noticed: I finished it for my editor and you can use it now with any platform you like |
@cengelha Thank you! -- For anyone wanting the link: https://github.com/cengelha/NomNom/ |
As an aside, since it is pretty new, I recommend to anyone to ensure you back up your wgs folder. This is a common action to take on the state of decay 2 community editor "just in case". |
MS Game Pass is now supported in NMSSaveEditor. |
Just a note in regards to the container index file, "$00F5 (red) wchar[int32]" may be a timestamp, for cloud syncing purposes. |
tl;dr - if you're here to ask for support, thumbs up this issue. If you're interested in gory technical details, read on.
I've spent some time the last few days reverse-engineering the save format that the XGP version of No Man's Sky uses. Since the code for this tool isn't open source, I can't open a PR to add support myself, so instead I'll describe the file formats as best I can in the hopes that @goatfungus can use it to add the support relatively easily.
Part 1: Microsoft's piece
Since the XGP version of NMS is a UWP app, it saves the games using some MS SDK, presumably an Xbox one to enable cloud saves. These saves are stored at
where
XYZ
is some long hexadecimal string which appears to be unique per person. The number after the underscore, however, appears to be the same across everyone.Inside this directory are a lot more folders with all-hex names. At its root is a
containers.index
file which can be read to understand the save slots and which folders they live in.I've been able to identify these sections of the file so far:
container index
All offsets are in reference to this particular file (since there's a lot of variable width strings in here). This file holds the save slots - for any given save folder on disk, if it's not in this index file, it won't show up in game.
$0000
$0004
numSaves
; it's the number of save files (including autosaves). I'm not sure if this number is a 32-bit or 64-bit one; the extra 4 bytes were always 0 for me.$000C
int32
denoting the string length, followed by that many 16-bit characters. It's alwaysHelloGames.NoMansSky_bsl90hzglsesy!NoMansSky
, which is the UWP package name.$0066
$0074
a512e4cc-80fd-4c57-a96d-15166f42a9f0
. for me. It's not the same for other people though.$00C0
00 00 00 10 00 00 00 00
, even across people.$00C8
SlotNAuto
orSlotNManual
, for auto and manual saves respectively).$00F5
"0x8D83E8BC3970A7F"
). I have a theory that this is the memory address that the save was at when it was written to disk, but I don't know what it would be used for. Sometimes it's a 0-length string, as well.$011E
containerNum
. It's used to lookup this save file in one of the hex-string folders we saw earlier.$011F
$0124
$0134
$014B
numSaves - 1
times. There is a final null terminating byte at the end of the file."Filename format"
The folder name for each save is stored as
[int32, int16, int16, int16, int8, int8, int8, int8, int8, int8]
. Parse each of these integers (little endian), then convert them to hex strings in big endian format and join them to get the folder name for the save slot. As an example, when this is applied to the purple section highlighted above, it looks like:As we can see, we end up with the second folder name from the first screenshot. Now we can open that folder to see what's in the
"Slot1Auto"
save.Inside this folder we have three more files. This is where the
containerNum
from before comes in - we need to opencontainer.[containerNum]
in this folder to figure out which of these two holds the save file data and save file meta. (Of course, the meta is the tiny one, and the save data is the big one, so you could probably skip parsing this file and use that assumption intead.)container contents
This file is a little confusing to me.
I won't color this one, cause it's easy enough to look at. There's two int32s - I don't know what these mean here. In all my saves, they're 4 and 2 respectively. Then, "data" as a wchar[], followed by a bunch of 0s and then two copies of the "data" filename in the Filename format described above. Immediately after those two, there's "meta" followed by more 0s and the "meta" filename (again twice).
I know that at one point I had one of these containers that only had the filename entries once each for data and meta - but none of them are like that anymore, and I don't remember if it changed the first two bytes. The filenames here might be constant length
wchar[160]
s as well - they do appear to be controlled by the game, because once we get to those files we are out of Microsoft-land and into the NMS part of it.Part 2: NMS save files
The save is broken up into two pieces, the "data" and the "meta". Part 1 described how to figure out which was which, but what are these pieces?
The meta file
The meta file is extremely small:
It turns out, the game uses this to display the summary of the save file on the save select screen without needing to load the whole save data. The different pieces are:
$0000
0x101E
in every file I opened (even from other saves I downloaded).$0004
0x01
for Normal,0x03
for Survival,0x05
for Permadeath, and presumably0x07
for Ambient, although I didn't try it. As far as I'm aware, this doesn't need to match the save data itself.$0008
The data file
And now, the final piece: I think the "data" section is a totally normal NMS PC save file.
(truncated for length)
Parts of the JSON structure are visible with mangled key names. However, parts of this file are garbled entirely - I'm not sure if normal NMS saves have this weird psuedo-encryption applied to them as well, although I could see from other tools that something like this was at least present in some older versions of NMS. If not, it's possible that a different piece of the "meta" or container.index files could be the key - the encryption kinda looks like it's XOR-based with a key that has large chunks of 0s in it.
I was hoping I could load this file directly into NMSSE, but since it doesn't seem to support loading arbitrary files and I don't actually know what a normal NMS save looks like on disc, I couldn't try to copy to the normal NMS format and location to try it. My hope is that it's a totally normal NMS save, and this extra information about the container file formats can be used to auto-detect and load the different save slots into the editor, just like for the Steam version.
Last notes
With this information, I was able to:
containerNum
incontainers.index
and the corresponding file suffix on disc, and the game still recognized and loaded the saveInterestingly, when I mixed a "meta" from one save and a "data" from another, the save select screen displayed the correct metadata for the corresponding save, but loading up the save actually just acted as if I was starting a new game (white Initialize screen). It's possible that the meta and data files are tied together in another way I don't understand (maybe that file holds the encryption key?).
Anyway, that's all I've found. Even if this doesn't help get XGP support into NMSSE, I hope it helps some other poor shmuck with a hex editor tweak their file (all I wanted when I started this was to change the difficulty of my save so I could play with a friend).
Attachments
Here are my saves for reference:
NMS_XGP_saves.zip
There's four saves in this zip:
The text was updated successfully, but these errors were encountered: