Implement the RazorLight.Precompile tool #492

MarkKharitonov · 2022-07-25T04:02:46Z

This method allows to implement a persistent cache as opposed to the existing in-memory cache.

I do not include an implementation of such a cache. But an example of how this can be done is:

public class FileSystemCachingProvider : ICachingProvider
{
    private MemoryCachingProvider m_cache = new();
    private readonly string m_root;

    public FileSystemCachingProvider(string root)
    {
        m_root = root;
    }

    public void PrecreateAssemblyCallback(IGeneratedRazorTemplate generatedRazorTemplate, byte[] rawAssembly, byte[] rawSymbolStore)
    {
        var srcFilePath = Path.Combine(m_root, generatedRazorTemplate.TemplateKey[1..]);
        var asmFilePath = srcFilePath + ".dll";
        File.WriteAllBytes(asmFilePath, rawAssembly);
        if (rawSymbolStore != null)
        {
            var pdbFilePath = srcFilePath + ".pdb";
            File.WriteAllBytes(pdbFilePath, rawSymbolStore);
        }
    }

    public void CacheTemplate(string key, Func<ITemplatePage> pageFactory, IChangeToken expirationToken)
    {
    }

    public bool Contains(string key)
    {
        var srcFilePath = Path.Combine(m_root, key);
        var asmFilePath = srcFilePath + ".dll";
        if (File.Exists(asmFilePath))
        {
            var srcLastWriteTime = new FileInfo(srcFilePath).LastWriteTimeUtc;
            var asmLastWriteTime = new FileInfo(asmFilePath).LastWriteTimeUtc;
            return srcLastWriteTime < asmLastWriteTime;
        }
        return false;
    }

    public void Remove(string key)
    {
        var srcFilePath = Path.Combine(m_root, key);
        var asmFilePath = srcFilePath + ".dll";
        var pdbFilePath = srcFilePath + ".pdb";
        if (File.Exists(asmFilePath))
        {
            File.Delete(asmFilePath);
        }
        if (File.Exists(pdbFilePath))
        {
            File.Delete(pdbFilePath);
        }
    }

    public TemplateCacheLookupResult RetrieveTemplate(string key)
    {
        var srcFilePath = Path.Combine(m_root, key);
        var asmFilePath = srcFilePath + ".dll";
        if (File.Exists(asmFilePath))
        {
            var srcLastWriteTime = new FileInfo(srcFilePath).LastWriteTimeUtc;
            var asmLastWriteTime = new FileInfo(asmFilePath).LastWriteTimeUtc;
            if (srcLastWriteTime < asmLastWriteTime)
            {
                var res = m_cache.RetrieveTemplate(key);
                if (res.Success)
                {
                    return res;
                }
                var rawAssembly = File.ReadAllBytes(asmFilePath);
                var pdbFilePath = srcFilePath + ".pdb";
                var rawSymbolStore = File.Exists(pdbFilePath) ? File.ReadAllBytes(pdbFilePath) : null;
                return new TemplateCacheLookupResult(new TemplateCacheItem(key, CreateTemplatePage));

                ITemplatePage CreateTemplatePage()
                {
                    var templatePageTypes = Assembly
                        .Load(rawAssembly, rawSymbolStore)
                        .GetTypes()
                        .Where(t => typeof(ITemplatePage).IsAssignableFrom(t) && !t.IsAbstract && !t.IsInterface)
                        .ToList();
                    if (templatePageTypes.Count != 1)
                    {
                        throw new ApplicationException($"Code bug: found {templatePageTypes.Count} concrete types implementing {nameof(ITemplatePage)} in the generated assembly.");
                    }
                    m_cache.CacheTemplate(key, CreateTemplatePage);
                    return CreateTemplatePage();

                    ITemplatePage CreateTemplatePage() => (ITemplatePage)Activator.CreateInstance(templatePageTypes[0]);
                }
            }
        }
        return new TemplateCacheLookupResult();
    }
}

And then it can be used like this:

string root = Path.GetDirectoryName(m_razorTemplateFilePath);
var provider = new FileSystemCachingProvider(root);
var engine = new RazorLightEngineBuilder()
    .UseFileSystemProject(root, "")
    .UseCachingProvider(provider)
    .AddPrecreateAssemblyCallbacks(provider.PrecreateAssemblyCallback)
    .Build();

MarkKharitonov · 2022-07-25T04:04:42Z

This Pull Request provides an answer to my question raised in the issue #491

jzabroski

Sorry, but this isn't happening, at least not in the format you proposed. It is especially bad that you're just swallowing exceptions - that is a non-starter. The interface complexity of the callback is also very high - I don't think anyone would know what this does without reading the source code.

There is a Precompile project for compiling views, but it hasn't been finished. Would you be interested in building a .NET Global Tool that can do that?

jzabroski · 2022-07-25T14:29:15Z

src/RazorLight/Compilation/RoslynCompilationService.cs

+						{
+							callback(razorTemplate, rawAssembly, rawSymbolStore);
+						}
+						catch (Exception)


Swallowing exceptions is bad. How will the end user know if there is any problems with the callback?

In my defense I followed the exact same scheme as used for the already existing prerenderCallbacks -

RazorLight/src/RazorLight/TemplateRenderer.cs

Lines 223 to 239 in 22381db

private void ExecutePageCallbacks(ITemplatePage page, IList<Action<ITemplatePage>> callbacks)

{

if (callbacks?.Count > 0)

{

foreach (var callback in callbacks)

{

try

{

callback(page);

}

catch (Exception)

{

//Ignore

}

}

}

}

I figured if that is good enough for them, why should it be different for these? However, I am will gladly change it, although I do not see any regression as opposed to the prerender callbacks.

It's a fair point, but very few people actually utilize those callbacks because they're not easily discoverable. I've found most people follow a really old tutorial from medium.com for how to use RazorLight that is woefully out of date and contains some rather questionable decisions, that resulted in me spending way over a month figuring out why people were reporting bugs just "standing up RazorLight" that I couldn't fathom happening until I found that online tutorial's faulty guide.

jzabroski · 2022-07-25T14:32:06Z

src/RazorLight/Compilation/RoslynCompilationService.cs


-		public RoslynCompilationService(IMetadataReferenceManager referenceManager, Assembly operatingAssembly)
+		public RoslynCompilationService(IMetadataReferenceManager referenceManager, Assembly operatingAssembly, List<Action<IGeneratedRazorTemplate, byte[], byte[]>> precreateAssemblyCallbacks = null)


The interface here is really not "discoverable" - I have worked on this code base for over 4 years now, and just looking at this callback makes my brain hurt. The lack of a default implementation also further increases the odds people will use it incorrectly or copy-paste some bad code from somewhere on the Internet, and we will get a lot of support Issues dealing with lazy programming. - This is the top maintenance cost of this project thus far.

Why are you calling it precreate vs. precompile?

If your goal is to precompile assemblies, why not contribute a PR to the RazorLight.Precompile csproj?

I think you're trying to do too many things in one pass - using a separate tool to precompile the views makes each tool quadratically less difficult to test, as you are testing A + B services area rather than the square of the number of states in A and B.

My reply would be this:

How is it different from the prerender callbacks?

My rationale is that it is called before the Assembly is created. The code has already been compiled, so it does not happen before the compilation. But if precompile is more recognizable, then it makes sense to rename. I guess this has to do with discoverability. precompile is a known buzz word, precreate is not. So, makes sense to rename. Unless, we want to keep it obscured :-).

I do not know. I was not aware of this tool. I did notice the project, but since it was not mentioned in the documentation (or I missed it) I just ignored it. What is the state of this tool?

Makes sense in general.

TBH, I am acting on my selfish reasons - I need to implement something in Powershell yesterday and as it is either I drop RazorLight entirely (in favour of what exactly? as far as I know there is nothing being as popular and actively maintained) or complicate a straightforward and conceptually simple scripting code to move all the rendering to one executable call.

If RazorLight.Precompile is already usable, then definitely I would prefer using it. But it may be a longer engagement and in the meantime - the ability to cache generated assemblies is needed now. At least I need it now.

Can we find a compromise that would allow us to release something quickly yet satisfying the quality requirements? I will gladly implement all your recommendations, if at the end I will be able to achieve my goal in a timely fashion.

Hope it makes sense.

If RazorLight.Precompile is already usable, then definitely I would prefer using it. But it may be a longer engagement and in the meantime - the ability to cache generated assemblies is needed now. At least I need it now.

I'm not trying to create work for you, but it seems straight forward to lift your PR into a tool vs. something that caches everything on startup. The RazorLight.Precompile project is just a stub right now.

if at the end I will be able to achieve my goal in a timely fashion.

I'm not trying to create work, but I do think most people would rather precompile this stuff vs. create a persistent cache. It makes the entire flow a lot more debuggable since you have to use the compiled DLLs on disk vs. use a persistent cache that could get turned off via misconfiguration or load a different projects cached items by accident.

…e tool.

MarkKharitonov · 2022-08-25T04:54:25Z

I have changed the PR to actually implement the RazorLight.Precompile tool.
What do you think?

MarkKharitonov · 2022-08-25T04:56:13Z

I think it makes sense to abandon this PR and replace it with a new one.
?

MarkKharitonov · 2022-08-31T17:54:41Z

I have just added an ability to render the given model by the precompile command.

Please, run precompile -h and render -h to get help.

jzabroski · 2022-08-31T20:58:02Z

Hi Mark. I hope you are well. I have an immediate family member in the hospital so this has been on hold for now. I promise it's not simply being ignored

MarkKharitonov · 2022-08-31T21:07:02Z

@jzabroski - sorry to hear that. Hope things get better. Thanks for letting me know.

jzabroski · 2022-09-19T19:40:55Z

Getting to this now. Expect a reply in the next 3 days. I do think it looks good though

jzabroski · 2022-10-10T18:50:11Z

Thanks, Mark. Thank you as well for your patience while I worked through some things personally.

MarkKharitonov · 2022-10-10T18:51:25Z

No problem. Thank you for merging. Does it going to publish to nuget.org automatically?

jzabroski · 2022-10-10T19:01:51Z

No, I have to do that but I will do it now assuming I can remember how I set it up on GitHub & the API key is not expired. :-)

jzabroski · 2022-10-10T19:05:21Z

@MarkKharitonov It looks like there is one failing test after I merged. Can you check? I am not immediately clear why it happened but I think its due to the backslash separator is not compatible with Ubuntu Linux?

Path separator are platform dependent :

For windows, it’s \ and for unix it’s /.

  Failed RenderFolderRecursive("FullMessage.cshtml","folder\\MessageItem.cshtml",RazorLight.Caching.FileHashCachingStrategy) [93 ms]
  Error Message:
   RazorLight.RazorLightException : The razor template file /home/runner/work/RazorLight/RazorLight/tests/RazorLight.Precompile.Tests/bin/Debug/net6.0/Samples/folder\MessageItem.cshtml does not exist.
  Stack Trace:

MarkKharitonov · 2022-10-10T19:07:42Z

I will have a look

jzabroski · 2022-10-10T19:09:23Z

RazorLight/tests/RazorLight.Precompile.Tests/Render2Tests.cs

Line 58 in c7f97d6

    
           T("FullMessage.cshtml", "folder\\MessageItem.cshtml", FileHashCachingStrategy.Instance, EXPECTED),

I think you just need a platform check. We use xunit, which has some platform specific features built-in via plugins. But if the test is only for .NET 6, you can also just use the .NET 6 platform abstraction APIs to detect the platform elegantly.

jzabroski · 2022-10-10T19:19:50Z

Actually you can probably just use https://learn.microsoft.com/en-us/dotnet/api/system.io.path.pathseparator?view=netframework-4.8 since it works across all versions

MarkKharitonov · 2022-10-10T19:37:28Z

I need to setup a local linux environment to test, because it all passes on Windows.

jzabroski · 2022-10-10T19:44:55Z

I honestly dont think you need to go that far :-) You can fork GitHub repo and set-up actions locally in your repo using our template and just make the one line change i suggested to fix the issue. Have you ever just hit . on a repo after forking it? You get a VS code editor in your browser.

MarkKharitonov · 2022-10-11T14:18:02Z

I think there should be a PR workflow on this repository to prevent future inconveniences.

Anyway, I am able to run the workflow from my branch in my fork and I can see the problem. Will fix it.

MarkKharitonov force-pushed the 491 branch 3 times, most recently from b02afbc to 30a7cd4 Compare July 25, 2022 04:19

jzabroski self-requested a review July 25, 2022 14:28

jzabroski requested changes Jul 25, 2022

View reviewed changes

Kharitonov, Mark added 6 commits August 24, 2022 09:16

Suppress most of the compile time warnings.

aba6461

Expose the NormalizeKey logic to the outside.

1833f89

Implement the precompile command of the RazorLight.Precompile tool.

1e66312

Add unit tests for the precompile command of the RazorLight.Precompil…

bd7f102

…e tool.

Implement the render command of the RazorLight.Precompile tool.

5346e57

Add unit tests for the render command of the RazorLight.Precompile tool.

260f3d1

MarkKharitonov force-pushed the 491 branch from 30a7cd4 to 260f3d1 Compare August 25, 2022 04:52

MarkKharitonov changed the title ~~Add RazorLightEngineBuilder.AddPrecreateAssemblyCallbacks method.~~ Implement the RazorLight.Precompile tool Aug 25, 2022

MarkKharitonov requested a review from jzabroski August 31, 2022 17:17

Add optional rendering ability to the precompile command

34e15b5

jzabroski merged commit c7f97d6 into toddams:master Oct 10, 2022

MarkKharitonov deleted the 491 branch October 10, 2022 18:50

jzabroski added this to the 2.2.0 milestone Oct 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement the RazorLight.Precompile tool #492

Implement the RazorLight.Precompile tool #492

MarkKharitonov commented Jul 25, 2022

MarkKharitonov commented Jul 25, 2022

jzabroski left a comment

jzabroski Jul 25, 2022 •

edited

Loading

MarkKharitonov Jul 25, 2022

jzabroski Jul 26, 2022

jzabroski Jul 25, 2022 •

edited

Loading

MarkKharitonov Jul 25, 2022

jzabroski Jul 26, 2022

MarkKharitonov commented Aug 25, 2022

MarkKharitonov commented Aug 25, 2022

MarkKharitonov commented Aug 31, 2022

jzabroski commented Aug 31, 2022

MarkKharitonov commented Aug 31, 2022

jzabroski commented Sep 19, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 10, 2022

jzabroski commented Oct 10, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 10, 2022

jzabroski commented Oct 10, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 10, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 11, 2022 •

edited

Loading

	private void ExecutePageCallbacks(ITemplatePage page, IList<Action<ITemplatePage>> callbacks)
	{
	if (callbacks?.Count > 0)
	{
	foreach (var callback in callbacks)
	{
	try
	{
	callback(page);
	}
	catch (Exception)
	{
	//Ignore
	}
	}
	}
	}


		public RoslynCompilationService(IMetadataReferenceManager referenceManager, Assembly operatingAssembly)
		public RoslynCompilationService(IMetadataReferenceManager referenceManager, Assembly operatingAssembly, List<Action<IGeneratedRazorTemplate, byte[], byte[]>> precreateAssemblyCallbacks = null)

Implement the RazorLight.Precompile tool #492

Implement the RazorLight.Precompile tool #492

Conversation

MarkKharitonov commented Jul 25, 2022

MarkKharitonov commented Jul 25, 2022

jzabroski left a comment

Choose a reason for hiding this comment

jzabroski Jul 25, 2022 • edited Loading

Choose a reason for hiding this comment

MarkKharitonov Jul 25, 2022

Choose a reason for hiding this comment

jzabroski Jul 26, 2022

Choose a reason for hiding this comment

jzabroski Jul 25, 2022 • edited Loading

Choose a reason for hiding this comment

MarkKharitonov Jul 25, 2022

Choose a reason for hiding this comment

jzabroski Jul 26, 2022

Choose a reason for hiding this comment

MarkKharitonov commented Aug 25, 2022

MarkKharitonov commented Aug 25, 2022

MarkKharitonov commented Aug 31, 2022

jzabroski commented Aug 31, 2022

MarkKharitonov commented Aug 31, 2022

jzabroski commented Sep 19, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 10, 2022

jzabroski commented Oct 10, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 10, 2022

jzabroski commented Oct 10, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 10, 2022

jzabroski commented Oct 10, 2022

MarkKharitonov commented Oct 11, 2022 • edited Loading

jzabroski Jul 25, 2022 •

edited

Loading

jzabroski Jul 25, 2022 •

edited

Loading

MarkKharitonov commented Oct 11, 2022 •

edited

Loading