-
Notifications
You must be signed in to change notification settings - Fork 38k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Efficient webjars version resolution via webjars-locator-lite
#27619
Comments
Here's an implementation (with no caching or any optimizations): public class WebJarsVersionResourceResolver extends AbstractResourceResolver {
private static final String PROPERTIES_ROOT = "META-INF/maven/";
private static final String NPM = "org.webjars.npm/";
private static final String PLAIN = "org.webjars/";
private static final String POM_PROPERTIES = "/pom.properties";
@Override
protected Resource resolveResourceInternal(@Nullable HttpServletRequest request, String requestPath,
List<? extends Resource> locations, ResourceResolverChain chain) {
Resource resolved = chain.resolveResource(request, requestPath, locations);
if (resolved == null) {
String webJarResourcePath = findWebJarResourcePath(requestPath);
if (webJarResourcePath != null) {
return chain.resolveResource(request, webJarResourcePath, locations);
}
}
return resolved;
}
@Override
protected String resolveUrlPathInternal(String resourceUrlPath,
List<? extends Resource> locations, ResourceResolverChain chain) {
String path = chain.resolveUrlPath(resourceUrlPath, locations);
if (path == null) {
String webJarResourcePath = findWebJarResourcePath(resourceUrlPath);
if (webJarResourcePath != null) {
return chain.resolveUrlPath(webJarResourcePath, locations);
}
}
return path;
}
@Nullable
protected String findWebJarResourcePath(String path) {
String webjar = webjar(path);
if (webjar.length() > 0) {
String version = version(webjar);
// A possible refinement here would be to check if the version is already in the path
if (version != null) {
String partialPath = path(webjar, version, path);
if (partialPath != null) {
String webJarPath = webjar + File.separator + version + File.separator + partialPath;
return webJarPath;
}
}
}
return null;
}
private String webjar(String path) {
int startOffset = (path.startsWith("/") ? 1 : 0);
int endOffset = path.indexOf('/', 1);
String webjar = endOffset != -1 ? path.substring(startOffset, endOffset) : path;
return webjar;
}
private String version(String webjar) {
Resource resource = new ClassPathResource(PROPERTIES_ROOT + NPM + webjar + POM_PROPERTIES);
if (!resource.isReadable()) {
resource = new ClassPathResource(PROPERTIES_ROOT + PLAIN + webjar + POM_PROPERTIES);
}
// Webjars also uses org.webjars.bower as a group id, so we could add that as a fallback (but not so many people use those)
if (resource.isReadable()) {
Properties properties;
try {
properties = PropertiesLoaderUtils.loadProperties(resource);
return properties.getProperty("version");
} catch (IOException e) {
}
}
return null;
}
private String path(String webjar, String version, String path) {
if (path.startsWith(webjar)) {
path = path.substring(webjar.length()+1);
}
return path;
}
} and here's how to install it in a Spring Boot application: @Bean
public WebMvcConfigurer configurer() {
return new WebMvcConfigurer() {
@Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:META-INF/resources/webjars/").resourceChain(true).addResolver(new WebJarsVersionResourceResolver());
}
};
} (See it work in the Petclinic here: https://github.com/dsyer/spring-petclinic/blob/webjars/src/main/java/org/springframework/samples/petclinic/system/WebJarsVersionResourceResolver.java.) |
I gave this solution a spin in one of my projects and so far the impressions are good. Is it OK if I refine it a bit and provide a PR to replace the existing (WebJars Locator based) |
@vpavic I don't think we should implement the resolution mechanism in Spring Framework directly, as this is webjars-locator's purpose in the first place. As far as I know there is no concrete specification here to follow, so we might derive from current or future behavior of the official library. GraalVM native becoming an important topic in the Java community, I think we should instead think about proper native support in webjars. Frameworks could instead drive webjars-locator at build time; the library could resolve all available resources and dump them in an index and possibly GraalVM resource configuration, since those resources will be needed at runtime. At runtime, Frameworks could then query the locator library for resources and it could use its own index (no scanning involved!) to resolve resources. I'm seeing that other frameworks are working on custom-made solutions to tackle this problem and I think it would be nice to consider that as a community. I'm closing this issue as a result since this is not the path we're choosing, but we can keep using this to discuss possible plans. Thanks! |
I'm a little bit disappointed as I don't think it's true that "this is webjars-locator's purpose in the first place" - that library has intentionally given itself a much larger surface area than we need for simply locating a version for a webjar, and it's a huge waste of resources to use webjars-locator if that's all you need (which I believe is 99-100% of Spring users). |
The tagline of "webjars-locator-core" is "locate assets within WebJars" and that's exactly how we're using it. If the approach described here is much more efficient and compatible with native apps, this should be considered as the default in the library itself. I don't think that re-implementing this feature in Spring directly is doing much good to the webjars community. |
I agree with @dsyer. The implementation outlined here does things Spring way and is therefore simpler than anything 3rd party could be, with added benefits of not requiring any 3rd party dependencies (or its own transitive dependencies), and not having to manage dependency at Spring Boot level. @bclozel you might want to also take a look at spring-projects/spring-boot#31560 (which is how I got here in the first place) - over there even @jamesward expressed preference for Spring Framework itself not having to rely on |
Spring Framework is mostly about integration with 3rd party libraries. We usually roll our own implementations for well-known specs or when there's no support in the Java community. This is not the case here.
Thanks @vpavic I was very much aware of the Spring Boot issue. This decision is backed by both teams.
In the official Webjars documentation I'm seeing that many projects don't support version agnostic resolution. This is the feature shipped by the webjars-locator-core library. If this feature is not considered useful after all, we could use the existing infrastructure entirely and drop any webjar-specific implementation. Would this solve the problem? |
The main purpose of |
As far as I understand, this is the only feature we're using in Spring.
But how this library would know about which version string to use, doesn't that require looking into the classpath?
Finding out about the version string for each JAR is really the important part and this doesn't depend on Spring. As for the performance cost, there isn't any if the scanning is performed at build time. During the AOT phase, Spring could trigger the scanning and the resolution of the "webjar name"/"version string" pairs required for runtime resolution.
With the mechanism I've just described, not only GraaVM native support could be achieved for all consumers, but this could also be used for the vanilla JVM case and bring significant improvements as scanning would not be required during startup anymore. Should I open an issue against the webjars library to discuss this? |
Yes, but to get the version you only need to read a classpath resource and it's always in the same place - there's no need for scanning, which is where the extra baggage comes in webjars. Scanning isn't really required at all (as shown by the code snippet I provided originally), so it's a distraction. If @jamesward is open to make the scanning features optional in webjars (either by extracting another jar, or by making the existing dependencies optional), I can definitely help with that. Spring users just want those version-free resource paths. |
One of the core goals of WebJars is making the assets easily cachable which is why the artifacts include the versions in the paths. The usage of
And that finds the resource and renders something like:
In Play there is a whole JS / CSS pipeline that WebJars fits into as well and I'm not sure if there is something similar in Spring. All that to say... Ultimately I'd rather help users move away from serving versionless paths to WebJar artifacts but I'm not sure how feasible that is in Spring. |
This is exactly what we've been doing since Spring Framework 4.1 - see the reference documentation on static content. We also support appending a content hash to the path for immutable resources with CDNs. The code snippet I pointed out above uses webjars-locator-core to resolve the correct version and rewrite links (resolutions are of course cached). As far as I understand, Play's I think @dsyer 's point still stands. |
Just to make sure everyone's on the same page in terms of the current state of |
@dsyer said "It is fairly inefficient", which is very polite of him but I want to stretch on that a bit: I was profiling a test-suite the other day whose allocations flame-graphs have ~63% Now of course this doesn't translate to CPU 1:1 where it's only ~10%, but notice how much is spent in G1 garbage collection on top of that (unsurprisingly). A test-suite is obviously not a production environment where this isn't as noticeable. But tests usually start several contexts and the general startup routine is executed more often usually. So working on improving that inside Spring directly might be a tremendous boost in developer productivity for certain projects, because it will directly impact test suites, startups etc... |
@dreis2211 If you have time and are willing to, maybe you could do the same profiling session against the I still can't understand downside of having this natively in Spring as:
|
In order to improve efficiency (see spring-projects/spring-framework#27619) and allow native image compatibility, this commit uses WebJars versioned URLs which are supported out of the box on Spring Boot via /META-INF/resources default resource location configuration, removing the need to use webjars-locator-core dependency and WebJarsResourceResolver. I have been able to measure a consistent 5% startup time improvement on the JVM with that simple change on my local machine.
As discussed in webjars/webjars-locator-lite#1, I have reopening this issue with the goal to provide an efficient and native-friendly support for WebJars in Spring Framework 6.2 leveraging |
I've put together a |
Thanks @vpavic for the offer but I have already something locally so I should be good. But I will welcome early feedback on snapshots when pushed. |
Waiting for feedback on webjars/webjars-locator-lite#1 (comment) before merging the feature. |
Merged, feedback welcomed. Please use Until Spring Boot has updated |
webjars-locator-lite
In order to improve efficiency (see spring-projects/spring-framework#27619) and allow native image compatibility, this commit uses WebJars versioned URLs which are supported out of the box on Spring Boot via /META-INF/resources default resource location configuration, removing the need to use webjars-locator-core dependency and WebJarsResourceResolver. I have been able to measure a consistent 5% startup time improvement on the JVM with that simple change on my local machine.
This is a follow-up to spring-projects/spring-framework#27619 This commit adds support for "org.webjars:webjars-locator-lite" for enabling the statis resources chain. As of this commit, support for "org.webjars:webjars-locator-core" is deprecated for obvious performance reasons. Closes gh-40146
In order to improve efficiency (see spring-projects/spring-framework#27619) and allow native image compatibility, this commit uses WebJars versioned URLs which are supported out of the box on Spring Boot via /META-INF/resources default resource location configuration, removing the need to use webjars-locator-core dependency and WebJarsResourceResolver. I have been able to measure a consistent 5% startup time improvement on the JVM with that simple change on my local machine.
Sorry for the late feedback, but I only now realized I didn't post anything here. It feels to me that support for In this case the default constructor does change the behavior since it will work with lite locator, but I'd argue this is in practice isn't an issue since I'll open a draft PR soon to clarify what I mean with some code. |
Spring (MVC and Webflux) has a
ResourceResolver
abstraction that can be used to resolve the versions in webjars, avoiding the need to maintain the version explicitly in 2 or more places (build file and HTML source). E.g. (from Petclinic):Resolves to
classpath:/META-INF/resources/webjars/jquery/<version>/jquery.min.js
at runtime.Spring Boot carries the responsibility of configuring the resource resolver, and currently it uses the
webjars-locator-core
(https://github.com/webjars/webjars-locator-core) library to do that, so version resolution only works if that library is on the classpath. TheWebJarsAssetLocator
from that library has a very broad and powerful API for locating files inside webjars, but there are some issues, namely:/META-INF/resources/webjars
classpath on startup (in a constructor!).But we don't need
webjars-locator-core
to just do version resolution, which is all Spring Boot offers, because webjars have a very well-defined structure. They all have apom.properties
with the version in it, and they only use a handful of well-known group ids, so they are easy to locate. It might be a good idea to implement it in Framework, since it is so straightforward and only depends on reading resources from the classpath.All of the issues above could be addressed just by providing a simpler version resolver natively (and configuring the resource config in a native image with a hint).
The text was updated successfully, but these errors were encountered: