Support for private feeds requiring authentication

[sandorfr]

I might be wrong but I have the feeling that when it comes to PackageReference Intellisense it does not support private feed which are requiring authentication.

[Error - 12:39:05] Failed to provide completions.
NuGet.Protocol.Core.Types.FatalProtocolException: The source does not have a Search service!
   at async Task<IEnumerable<string>> NuGet.Protocol.AutoCompleteResourceV3.IdStartsWith(string packageIdPrefix, bool includePrerelease, ILogger log, CancellationToken token)
   at async Task<SortedSet<string>> MSBuildProjectTools.LanguageServer.Utilities.NuGetHelper.SuggestPackageIds(IEnumerable<AutoCompleteResource> autoCompleteResources, string packageIdPrefix, bool includePrerelease, ILogger logger, CancellationToken cancellationToken) in D:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Common\Utilities\NuGetHelper.cs:line 142
   at async Task<SortedSet<string>> MSBuildProjectTools.LanguageServer.Documents.ProjectDocument.SuggestPackageIds(string packageIdPrefix, bool includePrerelease, CancellationToken cancellationToken) in D:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\Documents\ProjectDocument.cs:line 377
   at async Task<List<CompletionItem>> MSBuildProjectTools.LanguageServer.CompletionProviders.PackageReferenceCompletion.HandlePackageReferenceAttributeCompletion(ProjectDocument projectDocument, XSAttribute attribute, CancellationToken cancellationToken) in D:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\CompletionProviders\PackageReferenceCompletion.cs:line 155
   at async Task<CompletionList> MSBuildProjectTools.LanguageServer.CompletionProviders.PackageReferenceCompletion.ProvideCompletions(XmlLocation location, ProjectDocument projectDocument, CancellationToken cancellationToken) in D:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\CompletionProviders\PackageReferenceCompletion.cs:line 92
   at async Task<CompletionList> MSBuildProjectTools.LanguageServer.Handlers.CompletionHandler.OnCompletion(TextDocumentPositionParams parameters, CancellationToken cancellationToken) in D:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\Handlers\CompletionHandler.cs:line 193

[tintoy]

Hi! I haven't tried this myself, but since I'm just using the NuGet client libraries to parse nuget.config and connect to the feed it should work in theory.

Are your credentials stored in the config file? If so, are they encrypted? I think NuGet uses DPAPI to protect credentials and this probably doesn't work on .NET Core. You could test this by putting plaintext credentials in there and seeing if completions work then...

[tintoy]

NuGet/Home#1851

[tintoy]

Looks like this may have been fixed (for Windows only) in newer versions of the NuGet client. I'll check how recent the version I'm using is.

[sandorfr]

Yes credentials are encrypted in the config files.

[tintoy]

Does it work if you use plaintext credentials?

[tintoy]

Ok @sandorfr, I've tried using the latest version of the NuGet client libraries but doing so breaks access to v2 NuGet feeds. I'm a little worried that this might cause issues for people who are still using v2 feeds so I'll have to do some research to see how common that still is (it's been a while, so maybe everyone's on v3 feeds by now).

I'll also look into whether it's possible to pull in a new version of only the configuration parser library and leave the rest of the client libraries at their existing version (although based on the NuGet team's track record for backward-compatibility, I'm not all that confident).

I'll let you know what I find; in the meanwhile, would you mind trying to store the credentials in plain-text (just temporarily) and see if you can then get package completion to work? It would help me verify that this is the the problem you're having.

[tintoy]

I've pulled in the latest version of their config library but I need to do some testing to make sure nothing is broken.

[sandorfr]

Sorry, I forgot to update you, indeed, cleartextpassword works.

[tintoy]

Thanks! I can try building a new version of the VSIX with a newer version of the NuGet client if you want to try it out, though!

[tintoy]

Ok @sandorfr - if you have time would you be willing to try out an experimental build?

msbuild-project-tools-0.2.41.zip

(you'll need to extract the .vsix file from the .zip file and install it using the instructions specified here).

[tintoy]

(you may need to uninstall the existing extension, first)

[ijabit]

Where is this feature at? Your experimental build above is an older release number than what is currently out there, so is this deployed? Does this updated only work with plain text credentials?

Thanks!

[tintoy]

Hi - I'd kinda forgotten about it because I don't have an authenticated feed to test this with. Would you be willing to try it out to see if it works for you? You can just try the latest version (from memory, you can use encrypted credentials on Windows but only plain-text ones on other platforms since .NET Core on those platforms doesn't have DPAPI).

[ijabit]

Actually if it's just looking for an encrypted nuget.config file I don't think it will work for me. I'm using the MicrosoftCredentialProvider for my dotnet core application. The credentials it stores are to log in to VSTS (now called Azure DevOps) and get Nuget packages from that source. It stores the credentials encrypted in a binary file.

Here is a link to the documentation and this is where the credentials are stored:

The Credential Provider will save session tokens in the following locations:
Windows: $env:UserProfile\AppData\Local\MicrosoftCredentialProvider
Linux/MAC: $HOME/.local/share/MicrosoftCredentialProvider/

[tintoy]

Sorry, that link just takes me back to this issue - could you try re-posting it?

[tintoy]

BTW unless I'm mistaken, I don't think I can use the very latest version of the NuGet client libraries because, as of some version a while back, they no longer support v2 feeds.

I'm not sure whether we're ready to drop v2 feed support yet (I'd need to figure out what percentage of our uses are still using them before making a decision).

[ijabit]

Sorry, here is the correct link.

[ijabit]

If you can't read directly from the encrypted file, maybe you can call into the credential provider and use the interactive login feature, just like dotnet restore does? Or maybe there's a function you can call to go get the credentials and decrypt them automagically. I haven't looked at the code in this project yet to see if it would be possible to call into a DLL or not, I'll poke around a bit...

[ijabit]

It looks like the microsoft credential provider is actually a nuget plugin, so if if you use nuget it will just work, no effort needed. But it needs nuget version 4.8.0.5385 or later. According to this the nuget 4.x client is no longer compatible with v2 feeds, which must be what you're talking about. Can you use both versions? Try to restore with the older nuget client and if it fails try the newer client?

[tintoy]

Unfortunately, no, I don't think we can build against both versions at the same time (not unless we launched a separate process to retrieve completions, which would probably be too slow to be useful).

I think for now we can only support plain-text credentials (or DPAPI-protected ones on Windows?) until we're willing to drop support for v2 feeds. At this stage I'm planning to drop support for v2 feeds at the end of the year (I'll be spending some time in the next 2 months working out how common v2 feeds still are) at which point we'll pull in the latest NuGet packages and you'll get support for the Microsoft credential provider.

Sorry for the delay on this, but I want to make sure I don't break existing users any more than I have to 😁

[tintoy]

Thanks for the link to the credential provider, BTW - this looks like it will be useful once we upgrade.

[ijabit]

Okay, great to know this will come in time! The NuGet Package Manager extension also doesn't support authenticated package sources so you'll have a leg up on them :)

[sandorfr]

Sorry, I've not found the time to do extensive testing. But the public version was working fine with cleatextpassword in the nuget.config. The problem with that approach is that other tools (including visual studio) tend to mess with that. @ijabit seem to have nailed the core issues.

When it comes to private authenticated feed for testing, vsts artifacts is completely free for 5 users. So you should be covered :) https://marketplace.visualstudio.com/items?itemName=ms.feed

[tillig]

This appears to persist into the 0.3.12+ era. Here's from a recent hand-rolled 0.3.13 release (testing for another issue here)...

[Error - 10:43:56 AM] Error configuring NuGet package sources for MSBuild project '"/Users/tillig/dev/projectpath/default.proj"'.
NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/MYORG/_packaging/MYFEEDNAME/nuget/v3/index.json.
 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
   at HttpResponseMessage System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at void NuGet.Protocol.HttpSource+<>c__DisplayClass12_0<T>+<<GetAsync>b__0>d.MoveNext()
   at async Task<T> NuGet.Common.ConcurrencyUtilities.ExecuteWithFileLockedAsync<T>(string filePath, Func<CancellationToken, Task<T>> action, CancellationToken token)
   at async Task<T> NuGet.Protocol.HttpSource.GetAsync<T>(HttpSourceCachedRequest request, Func<HttpSourceResult, Task<T>> processAsync, ILogger log, CancellationToken token)
   at async Task<ServiceIndexResourceV3> NuGet.Protocol.ServiceIndexResourceV3Provider.GetServiceIndexResourceV3(SourceRepository source, DateTime utcNow, ILogger log, CancellationToken token)
   --- End of inner exception stack trace ---
   at async Task<ServiceIndexResourceV3> NuGet.Protocol.ServiceIndexResourceV3Provider.GetServiceIndexResourceV3(SourceRepository source, DateTime utcNow, ILogger log, CancellationToken token)
   at async Task<Tuple<bool, INuGetResource>> NuGet.Protocol.ServiceIndexResourceV3Provider.TryCreate(SourceRepository source, CancellationToken token)
   at async Task<T> NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync<T>(CancellationToken token)
   at async Task<Tuple<bool, INuGetResource>> NuGet.Protocol.AutoCompleteResourceV3Provider.TryCreate(SourceRepository source, CancellationToken token)
   at async Task<T> NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync<T>(CancellationToken token)
   at async Task<List<AutoCompleteResource>> MSBuildProjectTools.LanguageServer.Utilities.NuGetHelper.GetAutoCompleteResources(IEnumerable<PackageSource> packageSources, CancellationToken cancellationToken) in /Users/tillig/dev/projectpath/C:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Common\Utilities\NuGetHelper.cs:line 105
   at async Task<bool> MSBuildProjectTools.LanguageServer.Documents.ProjectDocument.ConfigurePackageSources(CancellationToken cancellationToken) in /Users/tillig/dev/projectpathC:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\Documents\ProjectDocument.cs:line 389
[Error - 10:43:57 AM] Error scanning NuGet package references for MSBuild project '"/Users/tillig/dev/projectpath/default.proj"'.
System.ArgumentNullException: Value cannot be null. (Parameter 'keyValuePairs')
   at void MSBuildProjectTools.LanguageServer.Utilities.CollectionExtensions.AddRange<TKey, TValue>(IDictionary<TKey, TValue> dictionary, IEnumerable<KeyValuePair<TKey, TValue>> keyValuePairs) in /Users/tillig/dev/projectpath/C:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\Utilities\CollectionExtensions.cs:line 62
   at async Task<bool> MSBuildProjectTools.LanguageServer.Documents.ProjectDocument.UpdatePackageReferences(CancellationToken cancellationToken) in /Users/tillig/dev/projectpath/C:\Development\github\tintoy\msbuild-project-tools-vscode\lib\server\src\LanguageServer.Engine\Documents\ProjectDocument.cs:line 426

Seems there are a few issues related to the Azure Artifacts Credential Provider and .NET 5, unsure if this overlaps. dotnet restore does use the credential provider as does dotnet add for package references, so I know it's working, at least from the dotnet CLI.

[tintoy]

Hmm - this was a PITA back in the day (very hacky), but if the NuGet feed API actually supports / respects credentials then we can probably manage to do the same (I’ll have to see how they do it before knowing the effort involved)....

[tintoy]

https://docs.microsoft.com/en-us/nuget/reference/extensibility/nuget-cross-platform-plugins#protocol-messages-index

Glad I don’t have to build a plug-in myself or this would be a non-starter!

[tintoy]

Implemented in v0.4.0.