-
Notifications
You must be signed in to change notification settings - Fork 289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorrect EFI_FILE_PROTOCOL->Open() usage in fallback #382
Comments
@vathpela Hello Peter -- I've been looking into this (I've had a bit of free time near the end of my Christmas vacation). The usage of file handles (EFI_FILE_PROTOCOL-pointers) in Some things that stand out, as of commit 657b248:
Thanks. |
Also, it's not clear why Connected to that, the Even if we eliminate the directory scanning from |
BTW, related UEFI spec ticket: https://mantis.uefi.org/mantis/view.php?id=2367 |
@frozencemetery Hi Robbie, could you comment please? My main question is whether the shim maintainers would like me to
Thanks. |
Furthermore, directory scanning code, with arguably overlapping functionality (and also with its own bugs, unfortunately), already exists in |
I wish ppl would use code blocks when posting code... 🤕 |
I originally reported this issue at https://bugzilla.redhat.com/show_bug.cgi?id=1966973 ; there it was entirely obvious where the code started/ended and where I, the poster, started again; refer to https://bugzilla.redhat.com/show_bug.cgi?id=1966973#c0. The formatting got lost with the re-filing of the RHBZ ticket in the upstream issue tracker. What's frustrating is that github.com went with this ridiculous markdown syntax, when plain ASCII was 100% sufficient for reporting bugs with careful visual layout. |
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket tianocore#2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
worked around in edk2 commit 8abbf6d87e68 ("OvmfPkg/VirtioFsDxe: tolerate opening an abs. pathname rel. to a reg. file", 2023-10-19), after filing a ticket for the UEFI spec as well (Mantis#2367) |
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
… file Referring to a file relative to a regular file makes no sense (or at least it cannot be implemented consistently with how a file is referred to relative to a directory). VirtioFsSimpleFileOpen() has enforced this strictly since the beginning, and a few months ago I reported USWG Mantis ticket #2367 [1] too, for clearing up the related confusion in the UEFI spec. Unfortunately, the shim boot loader contains such a bug [2] [3]. I don't believe the shim bug is ever going to be fixed. We can however relax the check in VirtioFsSimpleFileOpen() a bit: if the pathname that's being opened relative to a regular file is absolute, then the base file is going to be ignored anyway, so we can let the caller's bug slide. This happens to make shim work. Why this matters: UEFI-bootable Linux installer ISOs tend to come with shim and grub in the embedded (ElTorito) FAT image (ESP). Sometimes you want to build upstream shim/grub binaries, but boot the same ISO otherwise. The fastest way for overriding the ESP for this purpose is to copy its original contents to a virtio filesystem, then overwrite the shim and grub binaries from the host side. Note that this is different from direct-booting a kernel (via fw_cfg); the point is to check whether the just-built shim and grub are able to boot the rest of the ISO. [1] https://mantis.uefi.org/mantis/view.php?id=2367 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1966973 [3] rhboot/shim#382 Cc: Ard Biesheuvel <[email protected]> Cc: Gerd Hoffmann <[email protected]> Cc: Jiewen Yao <[email protected]> Cc: Jordan Justen <[email protected]> Signed-off-by: Laszlo Ersek <[email protected]> Message-Id: <[email protected]> Reviewed-by: Gerd Hoffmann <[email protected]> Tested-by: Gerd Hoffmann <[email protected]>
Laszlo reported this in a RHEL bugzilla ( https://bugzilla.redhat.com/show_bug.cgi?id=1966973 ), but I need to reference it here, so here's the story:
*** Description of problem:
In "fallback.c", in the following call tree:
find_boot_csv()
try_boot_csv()
read_file()
fh->Open()
the "fh" base handle, relative to which Open() is supposed to open a
file, is not a directory, but a regular file. This works with some
EFI_SIMPLE_FILE_SYSTEM_PROTOCOL implementations (notably
FatPkg/EnhancedFatDxe from edk2), but not with others (for example,
OvmfPkg/VirtioFsDxe).
The passage of the UEFI spec that governs the expected behavior of
EFI_FILE_PROTOCOL (aka (*EFI_FILE_HANDLE)) in the above context is
itself buggy.
*** Version-Release number of selected component (if applicable):
*** How reproducible:
100%
*** Steps to Reproduce:
Install an OVMF binary no earlier than edk2-stable202102, on the virt
host.
Install libvirtd packages no earlier than v7.1.0, on the virt host.
Install a reasonably recent QEMU, on the virt host.
Define a new libvirt domain with the following XML fragment
(customize the host-side location of the virtio-fs root directory as
needed):
No other bootable device should be present (disk, cd-rom, network
card).
Using an installed (virtual or physical) RHEL-8.5 system as origin,
recursively copy the "EFI" directory from "/boot/efi" to
".../virtio-fs-root" on the virt host:
ssh rhel85 tar -C /boot/efi -c EFI | tar -C .../virtio-fs-root -x -v
Launch the domain.
*** Actual results:
shim logs:
** Expected results:
grub should be reached -- shim should launch grub.
*** Additional info:
starting with find_boot_csv() in "fallback.c", we find:
798 efi_status = fh->Open(fh, &fh2, bootarchcsv,
799 EFI_FILE_READ_ONLY, 0);
800 if (EFI_ERROR(efi_status) || fh2 == NULL) {
801 console_print(L"Couldn't open \EFI\%s\%s: %r\n",
802 dirname, bootarchcsv, efi_status);
803 } else {
804 efi_status = try_boot_csv(fh2, dirname, bootarchcsv); ----------+
805 fh2->Close(fh2); |
806 if (EFI_ERROR(efi_status)) |
807 console_print(L"Could not process \EFI\%s\%s: %r\n", |
808 dirname, bootarchcsv, efi_status); |
|
|
645 try_boot_csv(EFI_FILE_HANDLE fh, CHAR16 *dirname, CHAR16 *filename) <-------------------+
646 {
647 CHAR16 *fullpath = NULL;
648 UINT64 pathlen = 0;
649 EFI_STATUS efi_status;
650
651 efi_status = make_full_path(dirname, filename, &fullpath, &pathlen);
652 if (EFI_ERROR(efi_status))
653 return efi_status;
654
655 VerbosePrint(L"Found file "%s"\n", fullpath);
656
657 CHAR16 *buffer;
658 UINT64 bs;
659 efi_status = read_file(fh, fullpath, &buffer, &bs); -------------------+
660 if (EFI_ERROR(efi_status)) { |
661 console_print(L"Could not read file "%s": %r\n", |
662 fullpath, efi_status); |
|
|
134 read_file(EFI_FILE_HANDLE fh, CHAR16 *fullpath, CHAR16 **buffer, UINT64 *bs) <-+
135 {
136 EFI_FILE_HANDLE fh2;
137 EFI_STATUS efi_status;
138
139 efi_status = fh->Open(fh, &fh2, fullpath, EFI_FILE_READ_ONLY, 0);
140 if (EFI_ERROR(efi_status)) {
141 console_print(L"Couldn't open "%s": %r\n", fullpath, efi_status);
On line 798, the file "\EFI\redhat\BOOTX64.CSV" is opened successfully,
the new file handle is output in "fh2". "fh2" is then passed
try_boot_csv() as parameter "fh" (line 804 to line 645).
On line 659, "fh" (still the handle for "\EFI\redhat\BOOTX64.CSV") is
passed to read_file() as parameter "fh" (line 134).
On line 139, shim tries to open "fullpath" (also
"\EFI\redhat\BOOTX64.CSV") relative to the file handle "fh", which
stands for "\EFI\redhat\BOOTX64.CSV".
This is wrong: open regular files (as opposed to open directories)
cannot be used as base locations for opening new filesystem objects.
This is what the virito-fs driver reports with EFI_INVALID_PARAMETER.
Note that this problem originates from a bug in the UEFI specification.
The UEFI spec (v2.9) says, under section "13.5 File Protocol",
Now consider the following pathname:
\EFI\FOO
Assume that we have an open file handle (an EFI_FILE_PROTOCOL) that
refers to this filesystem object.
Now further assume that we open the following pathname:
BAR
relative to the open file handle. What is the expectation for the full
(absolute) pathname of the resultant object? Two choices:
\EFI\FOO\BAR [1]
\EFI\BAR [2]
The first option makes sense if the original file handle, \EFI\FOO,
stands for a directory. (That is, "FOO" is a subdirectory of "EFI".) And
in that case, the second option is wrong.
The second option makes sense, perhaps, if the original file handle,
\EFI\FOO, stands for a regular file. (That is, "FOO" is a regular file
in the "EFI" directory.) In that case, the first option is simply
undefined -- \EFI\FOO identifies a regular file, so it has no child
directory entries, such as \EFI\FOO\BAR. This means that when we attempt
to open a pathname relative to a regular file, what we could mean in the
best case would be a sibling, and not a child, of the last pathname
component.
To stress it again: the UEFI spec language implies a child relationship
for the relative pathname when the base file handle stands for an open
directory, and -- at best -- a sibling relationship when the base file
handle stands for an open regular file.
This inconsistency is the bug in the UEFI specification. The language I
quoted above, namely
is ill-defined for regular files. Even if we attempt to retrofit the
intended meaning from directories to regular files, the behavior will
not be consistent -- see the child vs. sibling interpretations,
respectively.
To put differently, given "fh->Open()", the expression
"the location of fh"
is ill-defined in the UEFI spec. If "fh" is a directory, then its
"location" is defined as "itself", in the usual sense of pathnames. But
if "fh" stands for a regular file, then its "location" is -- at best --
defined as its parent directory. Because of this, filesystem objects
relative to the "location of fh" are also ill-defined. It only becomes
well-defined if we exclude regular files as base handles.
I expressly investigated this UEFI spec bug while developing the
virtio-fs driver for OVMF, and the EFI_INVALID_PARAMETER return value is
intentional. For opening filesystem objects with the
EFI_FILE_PROTOCOL.Open() member function, the base handle must refer to
a directory; it must not refer to a regular file.
Regarding a shim fix: the try_boot_csv() function should be invoked with
a file handle to the root directory (= the volume root). Actually, the
"fh" handle in find_boot_csv() already stands for an open directory, so
passing that to try_boot_csv() should work too (given that the pathname
is an absolute one, so it doesn't matter what directory it is relative
to -- but it still cannot be relative to a regular file).
The text was updated successfully, but these errors were encountered: