Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for pixelwise EDS live time / real time into digital micrograph file reader #124

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

nem1234
Copy link
Contributor

@nem1234 nem1234 commented May 25, 2023

Draft

Not ready to merge

Description of the change

  • Add suport for pixel-wise live time / real time for EDS image
  • Should these maps be converted to images or not?

ToDo:

  • sum / average of pixel-wise live-time and real-time.
  • What is EDS.Acquisition.Live-time multiplier ?

Progress of the PR

  • Change implemented (can be split into several points),
  • update docstring (if appropriate),
  • update user guide (if appropriate),
  • add an changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
  • Check formatting changelog entry in the readthedocs doc build of this PR (link in github checks)
  • add tests,
  • ready for review.

Minimal example of the bug fix or the new feature

import hyperspy.api as hs
s = hs.load("eds-si.dm4")
print(s.metadata.EDS.Real_time_map)
print(s.metadata.EDS.Live_time_map)
print(s.metadata.EDS.Count_rate_map)

@nem1234 nem1234 marked this pull request as draft May 25, 2023 03:36
@codecov
Copy link

codecov bot commented May 25, 2023

Codecov Report

Patch coverage: 35.71% and project coverage change: -0.39 ⚠️

Comparison is base (1c1baae) 85.00% compared to head (5285b01) 84.61%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #124      +/-   ##
==========================================
- Coverage   85.00%   84.61%   -0.39%     
==========================================
  Files          73       73              
  Lines        9054     9119      +65     
  Branches     2049     2060      +11     
==========================================
+ Hits         7696     7716      +20     
- Misses        892      936      +44     
- Partials      466      467       +1     
Impacted Files Coverage Δ
rsciio/digitalmicrograph/_api.py 70.52% <35.71%> (-4.80%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@nem1234 nem1234 changed the title Add support for pixelwise EDS live time / real time Add support for pixelwise EDS live time / real time into digital micrograph file reader May 25, 2023
@sem-geologist
Copy link
Contributor

sem-geologist commented May 25, 2023

I am really interested into outcomes of this PR. During EMAS workshop I had an opportunity to talk with few different engineers from different vendors of EDS. Different vendors have different approaches to the same problem. In case of Bruker, live time is encoded inside EDS spectra (known as 0eV peak). Just as a funny thing, from one of other vendor representative when discussing how dead times are handled in other vendor EDS, 0eV peak was commented by one as the most brilliant way, and other as most ******-way (Both engineers was not from Bruker, just to clarify).

So for some time I was wondering if live_time should be returned as static array, or as a function, or property (which would trigger catched getter)... @nem1234 what you would suggest? Also how to deal with fact that Hyperspy expects a single value instead of array as live time?

@nem1234
Copy link
Contributor Author

nem1234 commented May 30, 2023

Umm.....

We want to compare the quantity of elements between EDS and EELS.
In such case, pixel time is very important and want to correct the signal intensity using pixel live time.
But, most users just want a mapping of element, and they don't care pixel-wise live-time.

And in our case, (may be depend on the combination of TEM and EDS), overall live-time is not recorded into digital micrograph tags.
To calculate overall live/dead time, we need pixel-wise live time table.

In my opinion,
"static array, or as a function, or property" these are good enough, but should be accessible via common method not depend on equipments / io plugins.

My current problems....

  1. real_time is not same as pixel time setting. Pixel time in metadata should be overwritten with real/live time?
  2. Only one live time map is recorded when multi cycle scanning is selected.
  3. Overall live/dead-time is not recorded. They need to be calculated from pixel-wise data.
  4. real/live_time is fluctuated in 20-30% of setting value when software sync is selected.

Hyperspy related problems:

  1. In hyperspy/io.py, post_process functions run before mapping functions.
    Calculation of overall live-time can not be placed into post_process.
  2. Mapping and post_process functions should be placed in rosettasciio,
    they should not depends on hyperspy.

@nem1234
Copy link
Contributor Author

nem1234 commented May 30, 2023

In definition of EDS metadata,

  1. Parameter about number of frames is not defined
  2. Total exposure time is not defined

if _notes is None:
_notes = ""
_num_cycle = _om_si.get("Number of cycles")
_pi = _om_si.get("Pixel iterator")

Check notice

Code scanning / CodeQL

Unused local variable

Variable _pi is not used.
if _pt * 1.1 < _rta:
_logger.warning("Pixel time ({:.1f} ms) / EDS real time ({:.1f} ms) mismatch".format(_pt * 1000, _rta * 1000))
if _num_cycle != 1:
_warn = "Part of pixel-wise real-time/live-time data was lost when number of cycles != 1"

Check notice

Code scanning / CodeQL

Unused local variable

Variable _warn is not used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants