Any way to get install statistics?

[JanDeDobbeleer]

The title says it all, I’m looking for a way to understand how many time a package is downloaded, installed, searched,… Couldn’t find anything about this, might not yet exist.

[OfficialEsco]

This should probably get moved to https://github.com/microsoft/winget-cli (@denelon)
I agree that this should be a feature as it would show activity and look a bit more professional (Chocolatey)

AFAIK winget cli does contact the database everytime you run winget search/winget install(there is clearly some Telemetry), however it is probably not something that they would have implemented since winget is a command line tool

There have however been a request for a logo/icon in the manifest microsoft/winget-cli#1253, so maybe we'll get a different front-end in the future

[denelon]

We do have some telemetry (assuming the OS and the client are configured to provide it). We use it primarily for troubleshooting and identifying errors and failure scenarios. This hasn't been vetted from a privacy or legal standpoint for external use. I'll start the discussions to see if there is some data we can provide.

[JanDeDobbeleer]

thanks @denelon. I would expect anonymous data can be possible as you could look at how many times a manifest is requested per install/search rather than do this from the CLI client side.

[denelon]

Those results can be deceiving as the winget show command also grabs a manifest. I'll look into how well things correlate to see what that looks like.

[rcmaehl]

Any update on this?

[denelon]

Not yet. I'll ping the e-mail thread to see if we can make some progress.

[taozuhong]

Any update on this?

[denelon]

We don't get "all" install statistics. We only sample the data. As we've been discussing the mechanisms and governance concerns around sharing this kind of data, we've identified a few areas of concern. Publishers would need to have access to something to get this kind of data and should be able to decide if they are OK sharing the data. This data is available to Microsoft Store publishers today via Partner Center.

As we continue to work on verified developer, this topic will be a part of the discussion. The main concern is how we verify developers and provide mechanisms for them to manage which GitHub aliases are authorized to make changes to the "allowed" list of users. That would also potentially be a good place for publishers to acquire the data and make the decision if they are comfortable with making the sampled data available to the public.

[MrHinsh]

Why would this data not be public, like it is for other package managers?

Downloads / installs is available for Azure DevOps Extensions, Chocolaty packages, and more, to anyone.

[Lockszmith-GH]

Is it about the publisher's data though?
Seems like this isn't their data to control in the first place. Since this is exclusively a popularity ranking within winget alone.
This is about embracing not only open-source, but open-data

It's a mixure of user data and publisher data.

Give both users and publishers an opt-out option, and make the data public.
Make it configurable in the package description whether stats are collected and whether they are presented publicly or not. And when acknowleding winget's EULA, allow to opt-out of statistics.

Seeing how many downloads, successful installs, unsuccessful installs, uninstalls - would benefit the community of end-users tremendously, and provide a fantastic insight for the publisher who actually wants to improve their product.

[MaverickMartyn]

How is this still an issue?
If every other package manager on the market exposes these statistics, what is keeping you form doing the same?
It's non-sensitive information, and it would help various frontends (like UnigetUI) to guide the user as to which packages to choose.

[Lockszmith-GH]

literally came here with same exact thought

[denelon]

Any statistics we could publish would be based on samples, so the results would not necessarily be accurate/complete.

Who would the data be for?

Publishers:

I've had a few discussions with @JanDeDobbeleer about the data for Oh-My-Posh as a publisher. I've also talked with a few other publishers, so I believe I understand what is being asked for from that perspective. The main challenge is a business process challenge to ensure we're only providing that kind of detailed data to an authorized party with some kind of agreement (or to support a proper mechanism for opt-in/opt-out). That's why I mentioned Partner Center earlier. I'm not aware of any "official" way to validate the identity of another publisher/organization who choose not to participate in the GitHub ecosystem where the community repository "lives" without integrating with some other identity provider(s).

WinGet Users:

Is this about helping a user find "the best (success/failure rates/percentages)" or "the most popular (search, install, upgrade, uninstall rates/percentages/counts)" package?

What questions does the data help a customer answer?

What kind of UX makes sense for the CLI experience?

UniGetUI (@marticliment) and Dev Home as two example UIs integrating with WinGet would likely leverage COM APIs to get the data from WinGet rather than a separate web service.

Other Concerns
This could also lead into additional complexity for WinGet attempting to get similar data from private sources or even the Microsoft Store. Unfortunately, this isn't a trivial feature to enable. Beyond data collection, aggregation and distribution, there are also the global requirements around security, privacy, compliance, etc.

Is this work more important/valuable than other features/bug fixes?

I'm happy to break this down into some smaller bite sized pieces of work we can start digging into. I just need to understand the cost/benefit equation(s) associated with doing this work vs. other work. I also want to make sure we build the right thing. If I understand the questions/needs, then I can also look to see if there are other ways to provide enough data/information to help users make informed decisions.

[denelon]

Thanks for the detailed response!

You're looking for a more nuanced picture than something a "first" building block would be able to answer.

That also comes with much more complexity and operational cost as well because of the histogram. That doesn't mean it isn't worth building the "baby steps" along the way to get to that end goal, but it does help me understand to some extent when there would be enough value for you to be "delighted".

There is also value in my mind as the product manager of WinGet looking to have publishers keep things updated in the community repository themselves and in the ideal working state for their customers. I'll make up some contrived numbers as an example:

If 1,000,000,000 people use WinGet to install/update/manage their software, then publishers might be more inclined to support it (WinGet) vs. 100 people using WinGet regularly. If customers could see that data, it could also provide motivation to the publishers to make those improvements given reliable data to back up the "real world" results.

Unfortunately, there are many of those "other concerns" types of reasons I don't even have the perfectly accurate number of how many users regularly use the product.

---

Given the reality that any data "measure" I could provide is sampled, and I can't know the population size to determine a proper statistical confidence interval... (lots of technical disclaimers...)

What is the best single measure (number, statistic, percentage) I could give if I could only give one?

There are some very subtle distinctions in my mind about the difference between the term "downloads" in the form of a count, as opposed to an install success percentage as measured by the number of WinGet install commands for a given package that succeeded with a return of a "0" treated as success by WinGet, and other return values mapped to success codes.

I'm also aware it's possible to "game" the system by setting up a device in the "perfect" state (or a collection of devices) and running the equivalent of a for loop to game the number. Or someone nefariously doing the opposite to manipulate a measure in some way. In order to give users "high quality" measures, we also have to be concerned with the things that could be done to manipulate a measure and the reasonable amount of work needed to invest in having the ability to detect abuses and try to stop/block them or adjust the results to "account" for them.

I hope my intent is clear. I do want to be able to provide meaningful data to help customers make informed decisions, there are just a bunch of constraints I need to deal with in order to be transparent about the methods of collection and the data itself given the assumption that today I cannot just make all the data available. Maybe an opt-in mechanism would help overcome some of the limitations, and it would be something that could be exposed transparently. I'll have to look into that with the experts here say in order to see what's explicitly not allowed vs. what is allowed. Maybe there is an acceptable approach for everyone with a vested interest. I'm happy to see what that would be and how we can work towards that solution. Microsoft has very strict policies regarding the collection and dissemination of data, and a majority of the WinGet terms, conditions, agreements, etc. is covered under the Windows terms, conditions, agreements, etc. I'm not the expert in that area so I have to defer to others with more knowledge. I'm often wrong, so I don't want to incorrectly make assumptions or assertions about any of the specifics.

[Lockszmith-GH]

Thank you for taking the time to read my long winded responses. And thank you for providing such a detailed well thought off answer.

I also really appreicate that you seem to have completely groked my line ot thinking.
You are completely right about the distance we are from the vision I painted. And I agreee that getting stats like these, especially public stats, is not a trivial task. (Nobody should care whether I'm 'delighted' or not, but I know that dreaming big before setting out on a task is key to find successful goals - aiming high and missing the mark by a bit can be much more rewarding than aiming low)

Whatever you end up doing, the fact that you are activley engaging a nobody in the community about this - to me personally, is heart warming.
Thinking about 'how to approach' a problem, in the open, is what I really wanted to see. So that any conclusion made, would be something with breadcrumbs that show how the decision was made.

As for your question:
At this point, because you have zero visibility to the size of the audience, I would say - that is probably the first piece of data that needs to be collected. (I mean comparing that number with the github activity you see, might provide even more insights)

So I would start with: How many are using winget.

You still need to figure out how to distinguish between separate installs, and how to make sure:

• Nobody can game the system (at least in any meaningful way without much effort)
• Pepole are ok with what you're collecting about the installation (especially being a Microsoft project, this can be a very touchy subject)

I think when it comes to analytics collection, it's probably ok to use a combination of a 'very anonymous', but innacurate method, as well as an opt-in option to provide much more accurate, yet still non-identifying stats.

I think this detail on its own, would be very helpful in communicating the importance of winget as a successful project.
If you manage to get the infrastructure of that working, as well as a method of sharing the reports with the community, that should help with gaining the trust of the community and to start cooperating with the users and software publishers on the next data-points you might be able to collect.

A wild thought - find a way to let the publishers host a public facing end-point (signed by winget while collecting the data) - so that the infrastructure for the collection will be in the control of the publishers (but that's a bit too far fetched, as distributed data systems tend to be VERY, VERY very very complex).

Next data-point should probably be about how many installs with winget per machine (just installs, not upgrades - that's a separate stats that can come up next) - here determining if repeated installs and how to count them - because the reason behind the uninstall/install would be completely missed (In my mind, the only 'legitemate' reason for a repeated unistall/install would be failure of an installation)

In anyway, I hope others chime in here with their own thoughts.

[denelon]

We don't tend to get a large number of users engaged here at GitHub, so I use individuals as data points for "perception". I do have fairly detailed anonymous information about percentages for a fairly large number of WinGet command success/failure rates across a few geographies for a "reasonable" number of devices. I have no data for users who opt-out in WinGet settings and for some geographies it's essentially no data for me. I do have a reasonably unique pseudo device ID to associate results across unique devices to help with the deduplication for the abuse scenarios.

I'm personally concerned about privacy, so I also put my own constraints on requirements for data sampling/collection in terms of things that might be identifiable. I'm far more concerned about overall product health and I do distinguish the difference between "the code works as designed" and "the user got what they wanted". There are certainly a few scenarios that have come out of the data that I didn't anticipate which is to be expected. You can clearly look in the source code to see exactly what we include in sampled data, and when path is included, we filter out user/profile names etc...

That means I have enough samples to know reasonable percentage rates for install success and failure for package identifiers in the community repository and the Microsoft Store, but nothing about private repositories. I'd share the numbers if I was allowed, but unfortunately today, I'm not. I would hazard a guess that publishers might not want to have their failure rates disclosed to the public before they had a chance to address any "known" issues. In many cases where I've had a chance to dig into individual scenarios, it's not always just the installer causing the failures.

I could certainly pursue an opt-in for publishers to be willing to allow us to share that data, but who knows how that will resolve. Raw install attempt counts scaled towards the population could be used to provide some kind of "this is the most popular package users attempted to install via WinGet for sampled data" might help with an indication of popularity. It's more of a rank than anything else, but I think that is most likely to get approval consensus as a "starting point". If that makes sense, I'm willing to pursue that.

Something like xx% of users attempted to install <publisher.package> might give an indication of authenticity or popularity as a starting point. It essentially would provide some form of stack ranking for popularity. Initially at least, I'd expect that to be in the form of a report a user would need to "view" on a website before I could integrate that into the client or the COM APIs we use to expose external data. I'd expect that to be reasonably accurate and reasonably not worth the expense to game initially.

[Lockszmith-GH]

...I think that is most likely to get approval consensus as a "starting point".

You got my vote on this starting point.

As for tracking private repositories, that indeed needs to be an explicitly opt-in, or a design decision that needs to go into the API between the library and repository, and not something that is always reported "to the mothership".

I know I'll track this with a lot of interest.

[Lockszmith-GH]

BTW, in the interested of keeping the balance between public data vs safety and allowing publishers to respond.
Data publication which is delayed by an X amount of days (could part of the opt-in agreement / API) might be a good counter/balance, as long as that infomration is public as well.

Scenario I have in mind:

• A product update is issued
• new version is made available via winget-pkgs
• default auto install calls: attempt to update, and a high rate of failure is detected.
• publishers, with an opt-in API private key, have access to their sliver of analytics data as close to real-time as the winget project can allow. Giving them the X days window to address any issues in their control (or publish instructions about what is going on)

This will allow cautios environments to know when publisher's grace period starts before making a decision to update.