Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for URL in "packageManager" #359

Merged
merged 20 commits into from
Feb 20, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,17 @@ along with the SHA-224 hash of this version for validation.
recommended as a security practice. Permitted values for the package manager are
`yarn`, `npm`, and `pnpm`.

You can also provide a URL to a `.js` file (which will be interpreted as a
CommonJS module) or a `.tgz` file (which will be interpreted as a package, and
the `"bin"` field of the `package.json` will be used to determine which file to
use in the archive).

```json
{
"packageManager": "yarn@https://registry.npmjs.org/@yarnpkg/cli-dist/-/cli-dist-3.2.3.tgz#sha224.16a0797d1710d1fb7ec40ab5c3801b68370a612a9b66ba117ad9924b"
aduh95 marked this conversation as resolved.
Show resolved Hide resolved
}
```

## Known Good Releases

When running Corepack within projects that don't list a supported package
Expand Down Expand Up @@ -223,6 +234,10 @@ same major line. Should you need to upgrade to a new major, use an explicit
not to lookup on the remote registry for the latest version of the selected
package manager.

- `COREPACK_ENABLE_URL_VERSION_FOR_KNOWN_PM` can be set to `1` to allow use of
custom URLs to load a package manager known by Corepack (`yarn`, `npm`, and
`pnpm`).

aduh95 marked this conversation as resolved.
Show resolved Hide resolved
- `COREPACK_ENABLE_NETWORK` can be set to `0` to prevent Corepack from accessing
the network (in which case you'll be responsible for hydrating the package
manager versions that will be required for the projects you'll run, using
Expand Down
27 changes: 25 additions & 2 deletions sources/Engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import defaultConfig from '../config.js
import * as corepackUtils from './corepackUtils';
import * as folderUtils from './folderUtils';
import * as semverUtils from './semverUtils';
import {Config, Descriptor, Locator} from './types';
import {Config, Descriptor, Locator, PackageManagerSpec} from './types';
import {SupportedPackageManagers, SupportedPackageManagerSet} from './types';

export type PreparedPackageManagerInfo = Awaited<ReturnType<Engine[`ensurePackageManager`]>>;
Expand All @@ -34,7 +34,23 @@ export class Engine {
return null;
}

getPackageManagerSpecFor(locator: Locator) {
getPackageManagerSpecFor(locator: Locator): PackageManagerSpec {
if (!corepackUtils.isSupportedPackageManagerLocator(locator)) {
const url = `${locator.reference}`;
return {
url,
bin: {},
registry: {
type: `url`,
url,
fields: {
tags: ``,
versions: ``,
},
},
};
}

const definition = this.config.definitions[locator.name];
if (typeof definition === `undefined`)
throw new UsageError(`This package manager (${locator.name}) isn't supported by this corepack build`);
Expand Down Expand Up @@ -143,6 +159,13 @@ export class Engine {
}

async resolveDescriptor(descriptor: Descriptor, {allowTags = false, useCache = true}: {allowTags?: boolean, useCache?: boolean} = {}) {
if (!corepackUtils.isSupportedPackageManagerDescriptor(descriptor)) {
return {
name: descriptor.name,
reference: new URL(descriptor.range),
};
}

const definition = this.config.definitions[descriptor.name];
if (typeof definition === `undefined`)
throw new UsageError(`This package manager (${descriptor.name}) isn't supported by this corepack build`);
Expand Down
9 changes: 5 additions & 4 deletions sources/commands/Up.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import {Command, UsageError} from 'clipanion';
import semver from 'semver';
import {Command, UsageError} from 'clipanion';
import semver from 'semver';
import type {SupportedPackageManagerLocator} from 'sources/types';

import {BaseCommand} from './Base';
import {BaseCommand} from './Base';

export class UpCommand extends BaseCommand {
static paths = [
Expand Down Expand Up @@ -39,7 +40,7 @@ export class UpCommand extends BaseCommand {
if (!resolved)
throw new UsageError(`Failed to successfully resolve '${descriptor.range}' to a valid ${descriptor.name} release`);

const majorVersion = semver.major(resolved?.reference);
const majorVersion = semver.major((resolved as SupportedPackageManagerLocator)?.reference);
const majorDescriptor = {name: descriptor.name, range: `^${majorVersion}.0.0`};

const highestVersion = await this.context.engine.resolveDescriptor(majorDescriptor, {useCache: false});
Expand Down
94 changes: 68 additions & 26 deletions sources/corepackUtils.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
import {createHash} from 'crypto';
import {once} from 'events';
import fs from 'fs';
import type {Dir} from 'fs';
import Module from 'module';
import path from 'path';
import semver from 'semver';

import * as debugUtils from './debugUtils';
import * as folderUtils from './folderUtils';
import * as fsUtils from './fsUtils';
import * as httpUtils from './httpUtils';
import * as nodeUtils from './nodeUtils';
import * as npmRegistryUtils from './npmRegistryUtils';
import {RegistrySpec, Descriptor, Locator, PackageManagerSpec} from './types';
import {createHash} from 'crypto';
import {once} from 'events';
import fs from 'fs';
import type {Dir} from 'fs';
import Module from 'module';
import path from 'path';
import semver from 'semver';

import * as debugUtils from './debugUtils';
import * as folderUtils from './folderUtils';
import * as fsUtils from './fsUtils';
import * as httpUtils from './httpUtils';
import * as nodeUtils from './nodeUtils';
import * as npmRegistryUtils from './npmRegistryUtils';
import {RegistrySpec, Descriptor, Locator, PackageManagerSpec, SupportedPackageManagerDescriptor, SupportedPackageManagerLocator, URLLocator} from './types';

export function getRegistryFromPackageManagerSpec(spec: PackageManagerSpec) {
return process.env.COREPACK_NPM_REGISTRY
Expand Down Expand Up @@ -102,17 +102,46 @@ export async function findInstalledVersion(installTarget: string, descriptor: De
return bestMatch;
}

export function isSupportedPackageManagerDescriptor(descriptor: Descriptor): descriptor is SupportedPackageManagerDescriptor {
return !URL.canParse(descriptor.range);
}

export function isSupportedPackageManagerLocator(locator: Locator): locator is SupportedPackageManagerLocator {
return typeof locator.reference === `string`;
}

function parseURLReference(locator: URLLocator) {
const {hash, href} = locator.reference;
if (hash) {
return {
version: encodeURIComponent(href.slice(0, -hash.length)),
build: hash.slice(1).split(`.`),
};
}
return {version: href, build: []};
}
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

export async function installVersion(installTarget: string, locator: Locator, {spec}: {spec: PackageManagerSpec}) {
const {default: tar} = await import(`tar`);
const {version, build} = semver.parse(locator.reference)!;
const locatorIsASupportedPackageManager = isSupportedPackageManagerLocator(locator);
const {version, build} = locatorIsASupportedPackageManager ? semver.parse(locator.reference)! : parseURLReference(locator);

const installFolder = path.join(installTarget, locator.name, version);
const corepackFile = path.join(installFolder, `.corepack`);

let corepackContent;
try {
if (locatorIsASupportedPackageManager) {
corepackContent = await fs.promises.readFile(corepackFile, `utf8`);
aduh95 marked this conversation as resolved.
Show resolved Hide resolved
}
} catch (err) {
if ((err as nodeUtils.NodeError)?.code !== `ENOENT`) {
throw err;
}
}
// Older versions of Corepack didn't generate the `.corepack` file; in
// that case we just download the package manager anew.
if (fs.existsSync(corepackFile)) {
const corepackContent = await fs.promises.readFile(corepackFile, `utf8`);
if (corepackContent) {
const corepackData = JSON.parse(corepackContent);

debugUtils.log(`Reusing ${locator.name}@${locator.reference}`);
Expand All @@ -123,13 +152,18 @@ export async function installVersion(installTarget: string, locator: Locator, {s
};
}

const defaultNpmRegistryURL = spec.url.replace(`{}`, version);
const url = process.env.COREPACK_NPM_REGISTRY ?
defaultNpmRegistryURL.replace(
npmRegistryUtils.DEFAULT_NPM_REGISTRY_URL,
() => process.env.COREPACK_NPM_REGISTRY!,
) :
defaultNpmRegistryURL;
let url: string;
if (locatorIsASupportedPackageManager) {
const defaultNpmRegistryURL = spec.url.replace(`{}`, version);
url = process.env.COREPACK_NPM_REGISTRY ?
defaultNpmRegistryURL.replace(
npmRegistryUtils.DEFAULT_NPM_REGISTRY_URL,
() => process.env.COREPACK_NPM_REGISTRY!,
) :
defaultNpmRegistryURL;
} else {
url = decodeURIComponent(version);
}

// Creating a temporary folder inside the install folder means that we
// are sure it'll be in the same drive as the destination, so we can
Expand Down Expand Up @@ -158,6 +192,14 @@ export async function installVersion(installTarget: string, locator: Locator, {s
const hash = stream.pipe(createHash(algo));
await once(sendTo, `finish`);

if (!locatorIsASupportedPackageManager) {
if (ext === `.tgz`) {
spec.bin = require(path.join(tmpFolder, `package.json`)).bin;
} else if (ext === `.js`) {
spec.bin = [locator.name];
}
}

const actualHash = hash.digest(`hex`);
if (build[1] && actualHash !== build[1])
throw new Error(`Mismatch hashes. Expected ${build[1]}, got ${actualHash}`);
Expand Down Expand Up @@ -229,7 +271,7 @@ export async function runVersion(locator: Locator, installSpec: { location: stri
// Node.js segfaults when using npm@>=9.7.0 and v8-compile-cache
// $ docker run -it node:20.3.0-slim corepack [email protected] --version
// [SIGSEGV]
if (locator.name !== `npm` || semver.lt(locator.reference, `9.7.0`))
if (locator.name !== `npm` || semver.lt((locator as SupportedPackageManagerLocator).reference, `9.7.0`))
// @ts-expect-error - No types
await import(`v8-compile-cache`);

Expand Down
48 changes: 27 additions & 21 deletions sources/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ function getPackageManagerRequestFromCli(parameter: string | undefined, context:
return null;

const [, binaryName, binaryVersion] = match;
const packageManager = context.engine.getPackageManagerFor(binaryName);
if (!packageManager)
return null;
const packageManager = context.engine.getPackageManagerFor(binaryName)!;

if (packageManager == null && binaryVersion == null) return null;

return {
packageManager,
Expand All @@ -47,28 +47,34 @@ function getPackageManagerRequestFromCli(parameter: string | undefined, context:
}

async function executePackageManagerRequest({packageManager, binaryName, binaryVersion}: PackageManagerRequest, args: Array<string>, context: Context) {
const defaultVersion = await context.engine.getDefaultVersion(packageManager);
const definition = context.engine.config.definitions[packageManager]!;

// If all leading segments match one of the patterns defined in the `transparent`
// key, we tolerate calling this binary even if the local project isn't explicitly
// configured for it, and we use the special default version if requested.
let fallbackLocator: Locator = {
name: binaryName,
reference: undefined as any,
};
let isTransparentCommand = false;
for (const transparentPath of definition.transparent.commands) {
if (transparentPath[0] === binaryName && transparentPath.slice(1).every((segment, index) => segment === args[index])) {
isTransparentCommand = true;
break;
if (packageManager != null) {
const defaultVersion = await context.engine.getDefaultVersion(packageManager);
const definition = context.engine.config.definitions[packageManager]!;

// If all leading segments match one of the patterns defined in the `transparent`
// key, we tolerate calling this binary even if the local project isn't explicitly
// configured for it, and we use the special default version if requested.
for (const transparentPath of definition.transparent.commands) {
if (transparentPath[0] === binaryName && transparentPath.slice(1).every((segment, index) => segment === args[index])) {
isTransparentCommand = true;
break;
}
}
}

const fallbackReference = isTransparentCommand
? definition.transparent.default ?? defaultVersion
: defaultVersion;
const fallbackReference = isTransparentCommand
? definition.transparent.default ?? defaultVersion
: defaultVersion;

const fallbackLocator: Locator = {
name: packageManager,
reference: fallbackReference,
};
fallbackLocator = {
name: packageManager,
reference: fallbackReference,
};
}

let descriptor: Descriptor;
try {
Expand Down
40 changes: 32 additions & 8 deletions sources/specUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,40 @@ export function parseSpec(raw: unknown, source: string, {enforceExactVersion = t
if (typeof raw !== `string`)
throw new UsageError(`Invalid package manager specification in ${source}; expected a string`);

const match = raw.match(/^(?!_)([^@]+)(?:@(.+))?$/);
if (match === null || (enforceExactVersion && (!match[2] || !semver.valid(match[2]))))
throw new UsageError(`Invalid package manager specification in ${source} (${raw}); expected a semver version${enforceExactVersion ? `` : `, range, or tag`}`);
const atIndex = raw.indexOf(`@`);

if (atIndex === -1 || atIndex === raw.length - 1) {
if (enforceExactVersion)
throw new UsageError(`No version specified for ${raw} in "packageManager" of ${source}`);

const name = atIndex === -1 ? raw : raw.slice(0, -1);
if (!isSupportedPackageManager(name))
throw new UsageError(`Unsupported package manager specification (${name})`);

return {
name, range: `*`,
};
}

const name = raw.slice(0, atIndex);
const range = raw.slice(atIndex + 1);

const isURL = URL.canParse(range);
if (!isURL) {
if (enforceExactVersion && !semver.valid(range))
throw new UsageError(`Invalid package manager specification in ${source} (${raw}); expected a semver version${enforceExactVersion ? `` : `, range, or tag`}`);

if (!isSupportedPackageManager(name)) {
throw new UsageError(`Unsupported package manager specification (${raw})`);
}
} else if (isSupportedPackageManager(name) && process.env.COREPACK_ENABLE_URL_VERSION_FOR_KNOWN_PM !== `1`) {
throw new UsageError(`Illegal use of URL for known package manager. Instead, select a specific version, or set COREPACK_ENABLE_URL_VERSION_FOR_KNOWN_PM=1 in your environment (${raw})`);
}

if (!isSupportedPackageManager(match[1]))
throw new UsageError(`Unsupported package manager specification (${match})`);

return {
name: match[1],
range: match[2] ?? `*`,
name,
range,
};
}

Expand All @@ -43,7 +67,7 @@ export function parseSpec(raw: unknown, source: string, {enforceExactVersion = t
*/
export async function findProjectSpec(initialCwd: string, locator: Locator, {transparent = false}: {transparent?: boolean} = {}): Promise<Descriptor> {
// A locator is a valid descriptor (but not the other way around)
const fallbackLocator = {name: locator.name, range: locator.reference};
const fallbackLocator = {name: locator.name, range: `${locator.reference}`};

if (process.env.COREPACK_ENABLE_PROJECT_SPEC === `0`)
return fallbackLocator;
Expand Down
22 changes: 20 additions & 2 deletions sources/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ export interface Config {
* A structure containing the information needed to locate the package
* manager to use for the active project.
*/
export interface Descriptor {
export type Descriptor = SupportedPackageManagerDescriptor | URLDescriptor;
export interface SupportedPackageManagerDescriptor {
/**
* The name of the package manager required.
*/
Expand All @@ -107,11 +108,24 @@ export interface Descriptor {
*/
range: string;
}
interface URLDescriptor {
/**
* The name of the package manager required.
*/
name: string;

/**
* The range of versions allowed.
*/
range: string;
}

/**
*
*/
export interface Locator {
export type Locator = SupportedPackageManagerLocator | URLLocator;

export interface SupportedPackageManagerLocator {
/**
* The name of the package manager required.
*/
Expand All @@ -122,3 +136,7 @@ export interface Locator {
*/
reference: string;
}
export interface URLLocator {
name: string;
reference: URL;
aduh95 marked this conversation as resolved.
Show resolved Hide resolved
}
Loading
Loading