Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not return entry point offset if it's not backed up by disk data #975

Merged
merged 12 commits into from
Aug 30, 2021

Conversation

HoundThe
Copy link
Member

The idea to fix #962

I am not completely sure about the correct solution to this, I have created a simple fix, but this has few problems that need to be addressed if this is the way to go:

  • The EP offset is used to detect anomaly if EP is outside the mapped section - so this PR creates an output "Entry point is outside of mapped sections", but it can be part of memory only section. This output can be easily fixed or we can create a separate anomaly in a case entry point isn't backed by any actual file data.
  • If any of the two - EP offset/addresses are missing, FileInfo will give a warning "Warning: Invalid address of entry point." I guess it's good, to have the warning, to bring attention to the weird entry point, but that wording isn't exactly correct, it could be a valid address of entry point that could be created by TLS callbacks before EP is executed, but again, the message can be deleted in this case or there can be another specific message created for this situation as well

@PeterMatula PeterMatula self-requested a review July 12, 2021 06:06
@PeterMatula
Copy link
Collaborator

This is a good fix - I think exactly what @metthal wanted in #962. So few things to finalize this:

  1. Anomalies - add a separate entry(s) for this - something like "Entry point in pure in-memory section" (not sure about the correct terminology here), "Entry point offset cannot be computed", etc. Please try to use as accurate terminology as possible. Also, keep the format/text similar to other anomalies. If I'm correct, multiple anomalies can be set at once, so I see no problem to add even 2 (or more) separate, atomic, anomalies for offset/address and combine them if applicable. It would be better than creating some super anomaly that would be just for this case and would have some conjunction inside it - KISS.
  2. Warnings - the same as before, feel free to modify it as you see fit to make sense. Either add more warning messages that would detect these cases by some condition combo (i.e. ep bad, offset bad, both bad, etc.) or simply make warning similar to anomalies - i.e. separate, atomic, simple, and print all the applicable warnings at the same time.
  3. Add tests for this issue, but also, if you add anomalies/warnings, don't forget to exhaustively tests these separate from this bug - just search for another anomaly/warning tests in the regression tests suite and expand these with what you add.

@HoundThe
Copy link
Member Author

I've changed some unsigned long long types to std::uint64_t, but god a little bit carried away by the number of changes that were necessary 😅 I should probably remove the b2c3089 commit right?

…doesn't give warning about invalid entry point due to the memory-only entry points
@HoundThe
Copy link
Member Author

I've removed the warning in case that the getEpOffset() returns false. Offset outside of the file doesn't have to mean it's invalid.

I've also changed the anomaly flow, that if the address (instead of offset like before) isn't part of any section, then the warning "Entry point is outside of mapped sections" gets used, which makes more sense to me.

I've added an anomaly for the offset outside of the physical file when the address is part of an actual section: "Entry point in memory-only part of a section". For the file linked in the issue the EP anomalies look like this after these changes:

0     EpInWritableSection    Entry point in writable section                             
1     EpInMemoryOnly         Entry point in memory-only part of a section 

Which makes sense to me

But I've noticed, the getEpAddress() function seems to not check that the address is part of the existing section, unlike the previous code fo getEpOffset(), which traversed all the sections and checked if the offset belongs to any of them. I think such check should be added?

Current code looks like this:

	bool getEpAddress(std::uint64_t & epAddress) const
	{
		std::uint64_t imageBase = peFile->imageLoader().getImageBase();
		std::uint32_t entryPoint = peFile->imageLoader().getOptionalHeader().AddressOfEntryPoint;

		// Do not report zero entry point on DLLs
		epAddress = imageBase + entryPoint;
		return (entryPoint != 0 || isDll() == false);
	}

The last thing is I'm not sure about the difference between the warnings and anomalies. The warnings seem to be specific for CompilerDetector. But I don't think that makes anything different about the made change.

…ger. Separate the ignore of invalid offset just for PE.
@HoundThe
Copy link
Member Author

HoundThe commented Jul 12, 2021

Maybe actually it is not useful to check for EP address validity there, I think we would like to export the information anyway and the anomalies already check and export information that the EP is outside of the mapped section.

But that brings me to a failing regression test (bugs/invalid-address-of-entry-point-2 test_invalid_entry_point) that expects the warning, but that information is already part of anomalies. Maybe remove the test case for the PE files that solve this more accurately with the anomalies, or should I keep the warnings there?

@metthal
Copy link
Member

metthal commented Aug 5, 2021

If the test no longer makes sense then please remove it. The point here is to not return EP offset if it can't be mapped back from any address and if this is the case for that file then that test can be removed.

If this is ready to be merged please let us know.

@HoundThe
Copy link
Member Author

HoundThe commented Aug 6, 2021

I feel like this could be finished

@@ -483,6 +484,50 @@ uint32_t PeLib::ImageLoader::getFileOffsetFromRva(uint32_t rva) const
return rva;
}

// similar to getFileOffsetFromRva, but the offset is within the real file and not memory image
uint32_t PeLib::ImageLoader::getValidOffsetFromRva(uint32_t rva) const
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same review notes as I given you in #982.

@PeterMatula
Copy link
Collaborator

lets run TC tests

@PeterMatula
Copy link
Collaborator

@HoundThe At the moment, there are merge conflicts.

@PeterMatula PeterMatula merged commit d895715 into avast:master Aug 30, 2021
PeterMatula added a commit that referenced this pull request Aug 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Do not provide entry point offset in case it doesn't exist
3 participants