-
-
Notifications
You must be signed in to change notification settings - Fork 428
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A way to get report data in detail #1793
Comments
Take a look at the json report: it has all of the data in a machine-readable format. |
I used the json report and the results were the same as the conventional report. Just a summary of the file, it does not have a report by functions like html does. However, extracting this information from an HTML file is quite laborious. |
Ah sorry, you are right: the per-function and per-class information is not yet in the other reports. That won't be hard to add. |
Can you give me some guidance on how I can do this? |
You could look at the commit that added them to the HTML report. There's a lot of extra stuff happening in that commit, but that's the meat of it. |
Also the commit that added regions helps to understand the approach: 68c5f93 If you're not planning to, I can try to add regions to other kinds of report. |
I've implemented a simple way to output region stats in the basic report, but would like to get some input about the report design before going further. Basically, we can opt to print stats about files, classes and functions in a single report, or we can do it like the html version and report only on files, or classes, or functions in each report. What's the preferred way? Here's what I have so far:
|
Thanks for checking in. My gut reaction is that it will be too much information for a plain text report, but I'd like to hear from others. I like the interleaved lines of files/classes/functions, it kind of provides a drill-down right in one place. Now I wonder if that's a better way to organize the HTML report. If I had to choose a place to start, it would be the JSON report, because it's not space constrained, and it gives access to the data for other programmatic uses. |
Thanks, I'll try to get it working for the JSON report first then. It should be simple to add FWIW I like the current layout of the HTML report, maybe I'm biased because I stared at it for too long, but having a page for each kind of region plus one for files is simple and works for me. It might make sense to add yet another page with interleaved lines, for a quick glimpse of the coverage situation, but retaining the ability to look at only functions or files at a time. We could also add a |
Here's what a JSON report looks like with regions enabled by calling {
"meta": {
"format": 2,
"version": "7.5.4a0.dev1",
"timestamp": "2024-06-13T06:49:42.776728",
"branch_coverage": true,
"show_contexts": false
},
"files": {
"c.py": {
"executed_lines": [
1,
2,
5,
6,
9,
10,
13,
14
],
"summary": {
"covered_lines": 8,
"num_statements": 11,
"percent_covered": 72.72727272727273,
"percent_covered_display": "73",
"missing_lines": 3,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
3,
7,
11
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": [],
"class": {
"A": {
"executed_lines": [],
"summary": {
"covered_lines": 0,
"num_statements": 1,
"percent_covered": 0.0,
"percent_covered_display": "0",
"missing_lines": 1,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
3
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
},
"B": {
"executed_lines": [],
"summary": {
"covered_lines": 0,
"num_statements": 1,
"percent_covered": 0.0,
"percent_covered_display": "0",
"missing_lines": 1,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
7
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
},
"B.Inner": {
"executed_lines": [],
"summary": {
"covered_lines": 0,
"num_statements": 1,
"percent_covered": 0.0,
"percent_covered_display": "0",
"missing_lines": 1,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
11
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
}
},
"function": {
"A.test": {
"executed_lines": [],
"summary": {
"covered_lines": 0,
"num_statements": 1,
"percent_covered": 0.0,
"percent_covered_display": "0",
"missing_lines": 1,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
3
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
},
"B.test": {
"executed_lines": [],
"summary": {
"covered_lines": 0,
"num_statements": 1,
"percent_covered": 0.0,
"percent_covered_display": "0",
"missing_lines": 1,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
7
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
},
"B.Inner._in": {
"executed_lines": [],
"summary": {
"covered_lines": 0,
"num_statements": 1,
"percent_covered": 0.0,
"percent_covered_display": "0",
"missing_lines": 1,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [
11
],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
},
"c": {
"executed_lines": [
14
],
"summary": {
"covered_lines": 1,
"num_statements": 1,
"percent_covered": 100.0,
"percent_covered_display": "100",
"missing_lines": 0,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
}
}
},
"d.py": {
"executed_lines": [
1,
2,
3
],
"summary": {
"covered_lines": 3,
"num_statements": 3,
"percent_covered": 100.0,
"percent_covered_display": "100",
"missing_lines": 0,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
},
"missing_lines": [],
"excluded_lines": [],
"executed_branches": [],
"missing_branches": []
}
},
"totals": {
"covered_lines": 11,
"num_statements": 14,
"percent_covered": 78.57142857142857,
"percent_covered_display": "79",
"missing_lines": 3,
"excluded_lines": 0,
"num_branches": 0,
"num_partial_branches": 0,
"covered_branches": 0,
"missing_branches": 0
}
} We might want to add something to |
I don't see a reason to add to meta, since you can see the region types based on the actual content. Though, we have I'm wondering about the need for |
Maybe we don't need I'm happy to implement it any way you prefer, and also to iterate on different designs. Should I create a couple of PRs with different designs so you can test them and maybe have others opine? Or is it better to wait and decide on a design before submitting a PR? |
I think regions in the text report is too much. Let's focus on the JSON report for now, since it's effectively a convenient data export mechanism. |
This data is now available in the JSON report as of commit ab609ef. |
This is now released as part of coverage 7.6.0. |
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [coverage](https://github.com/nedbat/coveragepy) | `==7.5.4` -> `==7.6.0` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/coverage/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/coverage/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/coverage/7.5.4/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/coverage/7.5.4/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>nedbat/coveragepy (coverage)</summary> ### [`v7.6.0`](https://github.com/nedbat/coveragepy/blob/HEAD/CHANGES.rst#Version-760--2024-07-11) [Compare Source](https://github.com/nedbat/coveragepy/compare/7.5.4...7.6.0) - Exclusion patterns can now be multi-line, thanks to `Daniel Diniz <pull 1807_>`*. This enables many interesting exclusion use-cases, including those requested in issues `118 <issue 118_>`* (entire files), `996 <issue 996_>`\_ (multiple lines only when appearing together), `1741 <issue 1741_>`\_ (remainder of a function), and `1803 <issue 1803_>`\_ (arbitrary sequence of marked lines). See the :ref:`multi_line_exclude` section of the docs for more details and examples. - The JSON report now includes per-function and per-class coverage information. Thanks to `Daniel Diniz <pull 1809_>`\_ for getting the work started. This closes `issue 1793`\_ and `issue 1532`\_. - Fixed an incorrect calculation of "(no class)" lines in the HTML classes report. - Python 3.13.0b3 is supported. .. \_issue 118:[nedbat/coveragepy#118 .. \_issue 996[nedbat/coveragepy#996 .. \_issue 153[nedbat/coveragepy#1532 .. \_issue 17[nedbat/coveragepy#1741 .. \_issue 1[nedbat/coveragepy#1793 .. \_issue [nedbat/coveragepy#1803 .. \_pull[nedbat/coveragepy#1807 .. \_pul[nedbat/coveragepy#1809 .. \_changes\_7-5-4: </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/rustymotors/server). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [coverage](https://github.com/nedbat/coveragepy) | `==7.5.4` -> `==7.6.0` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/coverage/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/coverage/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/coverage/7.5.4/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/coverage/7.5.4/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>nedbat/coveragepy (coverage)</summary> ### [`v7.6.0`](https://github.com/nedbat/coveragepy/blob/HEAD/CHANGES.rst#Version-760--2024-07-11) [Compare Source](https://github.com/nedbat/coveragepy/compare/7.5.4...7.6.0) - Exclusion patterns can now be multi-line, thanks to `Daniel Diniz <pull 1807_>`*. This enables many interesting exclusion use-cases, including those requested in issues `118 <issue 118_>`* (entire files), `996 <issue 996_>`\_ (multiple lines only when appearing together), `1741 <issue 1741_>`\_ (remainder of a function), and `1803 <issue 1803_>`\_ (arbitrary sequence of marked lines). See the :ref:`multi_line_exclude` section of the docs for more details and examples. - The JSON report now includes per-function and per-class coverage information. Thanks to `Daniel Diniz <pull 1809_>`\_ for getting the work started. This closes `issue 1793`\_ and `issue 1532`\_. - Fixed an incorrect calculation of "(no class)" lines in the HTML classes report. - Python 3.13.0b3 is supported. .. \_issue 118:[nedbat/coveragepy#118 .. \_issue 996[nedbat/coveragepy#996 .. \_issue 153[nedbat/coveragepy#1532 .. \_issue 17[nedbat/coveragepy#1741 .. \_issue 1[nedbat/coveragepy#1793 .. \_issue [nedbat/coveragepy#1803 .. \_pull[nedbat/coveragepy#1807 .. \_pul[nedbat/coveragepy#1809 .. \_changes\_7-5-4: </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/allenporter/flux-local). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [coverage](https://github.com/nedbat/coveragepy) | `>=7.5,<7.6` -> `>=7.6,<7.7` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/coverage/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/coverage/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/coverage/7.5.4/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/coverage/7.5.4/7.6.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>nedbat/coveragepy (coverage)</summary> ### [`v7.6.0`](https://github.com/nedbat/coveragepy/blob/HEAD/CHANGES.rst#Version-760--2024-07-11) [Compare Source](https://github.com/nedbat/coveragepy/compare/7.5.4...7.6.0) - Exclusion patterns can now be multi-line, thanks to `Daniel Diniz <pull 1807_>`*. This enables many interesting exclusion use-cases, including those requested in issues `118 <issue 118_>`* (entire files), `996 <issue 996_>`\_ (multiple lines only when appearing together), `1741 <issue 1741_>`\_ (remainder of a function), and `1803 <issue 1803_>`\_ (arbitrary sequence of marked lines). See the :ref:`multi_line_exclude` section of the docs for more details and examples. - The JSON report now includes per-function and per-class coverage information. Thanks to `Daniel Diniz <pull 1809_>`\_ for getting the work started. This closes `issue 1793`\_ and `issue 1532`\_. - Fixed an incorrect calculation of "(no class)" lines in the HTML classes report. - Python 3.13.0b3 is supported. .. \_issue 118:[nedbat/coveragepy#118 .. \_issue 996[nedbat/coveragepy#996 .. \_issue 153[nedbat/coveragepy#1532 .. \_issue 17[nedbat/coveragepy#1741 .. \_issue 1[nedbat/coveragepy#1793 .. \_issue [nedbat/coveragepy#1803 .. \_pull[nedbat/coveragepy#1807 .. \_pul[nedbat/coveragepy#1809 .. \_changes\_7-5-4: </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/hetznercloud/hcloud-python). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
The report method only returns the total coverage amount. However, sometimes we need this information in detail. For example, I need to know the function-by-function coverage of a file, and I can't do this using the report method. I know it's possible to do this using the html report method, but having the value inside a variable would make it much easier for a more detailed analysis of where the coverage is bad.
So if the report method returned a dictionary, for example, with the names of the functions and their coverage, it would be very useful.
The text was updated successfully, but these errors were encountered: