Option to append PT_LOAD segments to ELFs

[matheusmoreira]

It would be useful to have a command line option or plugin for the linker that appends an empty PT_LOAD program header table entry to ELF executables. This will greatly facillitate patching executables with new data after linking.

The Linux kernel automatically loads those segments onto memory and passes a pointer to the program header table via the auxiliary vector. This would be the perfect mechanism to allow executables to easily and efficiently access data embedded into the executable itself, even data patched in after the the binary has been compiled.

Current solutions are insufficient. objcopy can add new sections but they do not get loaded by the kernel without a PT_LOAD segment and those can only be created at link time since adding new program headers would change all offsets in the file. Linker scripts support a PHDRS command but using that disables the linker's default behavior and forces users to specify all the segments and map all the sections to them instead of letting the linker do it.

A simple --append-program-header that just adds an empty program header to the end of the table would be ideal. With that feature in place, custom tools can be written to copy arbitrary data into the ELF and then edit the placeholder's offset and size to match.

Links:

Related StackOverflow question
Binutils mailing list discussion
Equivalent LLVM linker issue

[rui314]

We have the --spare-dynamic-tags=N option, where N is an integer. This option appends N additional null dynamic tags to .dynamic, serving as placeholders for adding more dynamic tags during post-processing. It seems you are seeking something similar. Did I understand your request correctly?

[matheusmoreira]

Yes! That's absolutely correct. The difference in this case is the generation of program headers instead of sections. A --spare-program-headers=N command line option would be very useful.

The program header type has a reserved OS-specific range:

0x60000000    PT_LOOS
0x6FFFFFFF    PT_HIOS

Could types in this range be used to temporarily mark the sections so they could be targeted by post-processing tools?

[rui314]

PT_NULL is defined to be the type that is ignored, so we should probably use that type.

[brenoguim]

Why don't you just mmap the program in runtime and read the data?

[matheusmoreira]

@brenoguim That is my last resort in case all other alternatives fail.

My rationale for the program header solution:

1. Reduces system call usage
   • Increases program performance
   • Decreases program initialization/startup time
   • Cleaner strace output
2. Reduces complexity
   • More convenient
     • All I need to do is find my program header
       • Pointer and size to my data, ready for use
   • No need to parse the ELF myself
     • The kernel does it for me while loading the program
     • Input layers are sources of bugs and security holes
   • Opening /proc/self/exe has edge cases
     • The file might be deleted
       • Impossible if the kernel maps the data at load time
         • Program header solution is therefore more robust
3. Gets the Linux kernel to do more
   • With a PT_LOAD segment, the kernel will mmap the data
     • It will already be available when the program starts executing
       • No extra work needed
   • The kernel already passes an AT_PHDR auxiliary value to all programs
     • All I have to do is get that pointer and dereference it
4. I'm writing a small freestanding nolibc Linux executable
   • More self-contained
   • Reduces the number of Linux system calls I need to write stubs for

[brenoguim]

Fair enough. Even though I disagree with some of arguments, others make up for it.
If linker vendors refuse your idea, you may want to take a look at implementing it in patchelf. It should be pretty simple to do it.

[Alcaro]

Opening /proc/self/exe has edge cases
The file might be deleted

No, it's a magic link. Opening it doesn't dereference the filename; it always leads to the relevant file, even if it's been moved or deleted. /proc/self/fd/1 is also a magic link.

That said, there are possible problems in opening /proc/self/exe. /proc isn't always mounted (afaik this happens mostly in containers).

[matheusmoreira]

@brenoguim

If linker vendors refuse your idea, you may want to take a look at implementing it in patchelf. It should be pretty simple to do it.

Certainly.

Correct me if I'm wrong but it's my understanding that appending program headers would require re-linking the whole executable. Since it's located at the beginning of the file, increasing the size of the program header table would result in all addresses having to be adjusted. I assume that's the reason objcopy can add new sections but not new segments. This is the reason why I sought a linker solution.

@Alcaro

No, it's a magic link. Opening it doesn't dereference the filename; it always leads to the relevant file, even if it's been moved or deleted. /proc/self/fd/1 is also a magic link.

I see. I must have misunderstood things when I read the documentation:

Under Linux 2.2 and later, this file is a symbolic link containing the actual pathname of the executed command.
This symbolic link can be dereferenced normally; attempting to open it will open the executable.
If the pathname has been unlinked, the symbolic link will contain the string '(deleted)' appended to the original pathname.

I thought it was a normal link that pointed to the file.

Reading the manual again, I noticed a couple more edge cases:

In a multithreaded process, the contents of this symbolic link are not available if the main thread has already terminated
Permission to dereference or read this symbolic link is governed by a ptrace access mode PTRACE_MODE_READ_FSCREDS check

So there might be multithreading and permission problems.

That said, there are possible problems in opening /proc/self/exe. /proc isn't always mounted (afaik this happens mostly in containers).

You're completely correct. My program will definitely be running in those conditions.

My goal is to be able to boot Linux directly into it and bring up the system myself, so I'll have to mount /proc myself and can't rely on such a mechanism for early program initialization.

[brenoguim]

Patchelf is able to change things that objcopy doesn't. Specifically, patchelf is designed to operate on linked programs/libraries.
Sometimes there is "empty space" in the virtual address layout such that you can insert things without disrupting the whole binary. Some other times it needs to move data around like string sections to make room for your changes.
It has its defects, but in general it works pretty well and is relied upon by lots of users.

[rui314]

Correct me if I'm wrong but it's my understanding that appending program headers would require re-linking the whole executable. Since it's located at the beginning of the file, increasing the size of the program header table would result in all addresses having to be adjusted. I assume that's the reason objcopy can add new sections but not new segments. This is the reason why I sought a linker solution.

Technically, you can add a new program header without re-linking the entire executable by copying the existing PHDR to the end of the file with some spare slots. You can then edit the PHDRs as needed and rewrite the ELF header to refer to the new PHDR instead of the old one.

That said, improving the linker to add a few spare slots at the end of the PHDRs is quite straightforward, and the feature request is convincing, so I'll try to do that.

[matheusmoreira]

Technically, you can add a new program header without re-linking the entire executable by copying the existing PHDR to the end of the file with some spare slots. You can then edit the PHDRs as needed and rewrite the ELF header to refer to the new PHDR instead of the old one.

That's awesome, I didn't think of that. It looks like a patchelf feature is certainly possible. I'll try to contribute a patch.

That said, improving the linker to add a few spare slots at the end of the PHDRs is quite straightforward, and the feature request is convincing, so I'll try to do that.

Thank you so much. I will integrate mold immediately upon release.

[matheusmoreira]

I've already integrated this feature into my tools and will be using it as soon as it is released. Thanks again!

@brenoguim I've just created a dedicated ELF patching tool for my own project and it's already working nicely. I'll make a pull request to patchelf as well.

I've also requested that this feature be added to ld on the GNU binutils mailing list.