Rime: Nix URI libs and utils

Rime

rime fully implements a spec compliant URI parser in pure Nix.

Additionally this repo provides type abstractions for parsing Flake Input URIs ( flake refs ) to attributes, and can convert them attrs to URIs.

Finally you’ll find a nice collection of scripts that expose these routines as executables for use in scripts.

If you’re processing URIs with Nix don’t do it from scratch, let rime do the heavy lifting for ya.

“Rime”

Thin crystal layer of ice and snow formed by fog; particularly on branches.

Quick Start

nix-repl> :lf github:aakropotkin/rime

nix-repl> lib.liburi.flakeRefAttrsToString { type = "github"; owner = "aakropotkin"; repo = "floco"; dir = "bar"; }
"github:aakropotkin/floco?dir=bar"

nix-repl> lib.liburi.parseFlakeRefFT "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz?dir=foo"
{ dir = "foo"; type = "tarball"; url = "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz"; }

libs and types

Given that the Nix flakes interfaces revolve around URI parsing, I’ve defined some URI parsers and types ( using YANTS ).

I initially took a shot at exposing Nix’s builtin parseFlakeRef or fetchTree routines, but they were rooted in the actual fetching of flakes and trees to port as standalone routines. It could be done, but it’s less effort for me to just write that stuff in Nix anyway and avoid fooling with plugins.

Templates

inputs.nix

A shim exposing flake.lock contents to CLI tooling and non-flake Nix expressions.

This should have a dramatic effect on performance of “dirty tree” operations in large repositories when used in combination with nix (build|eval) -f ./build.nix;.

# Add `inputs.nix' to a flake
$ nix flake init -t rime#inputs;

# Example: Print URIs usable by `builtins.getFlake' and Nix CLI tools.
$ nix eval --json -f inputs.nix|jq;
{
  "ak-nix": "github:aakropotkin/ak-nix/123fc484c3c68e354704c86a88d88e343e6371f3",
  "at-node-nix": "github:aameen-tulip/at-node-nix/ef7cc7ef572f50e537739706668aadb9029d5d1e",
  "laika": "github:aakropotkin/laika/5b8eb6aab5f9eb2b4f9d93f40bb75ab7d0f5127c",
  "nixpkgs": "github:NixOS/nixpkgs/95aeaf83c247b8f5aa561684317ecd860476fcd6",
  "nixpkgs_2": "github:NixOS/nixpkgs/95aeaf83c247b8f5aa561684317ecd860476fcd6",
  "rime": "github:aakropotkin/rime/b7440e737b5e43b99a88dd192f469d4d9d419b7f"
}

# Example: Print URI of `at-node-nix' ( unquoted ) for setting an env var.
$ FLAKE_REF="$( nix eval --raw -f inputs.nix; )";
$ echo "$FLAKE_REF";
github:aameen-tulip/at-node-nix/ef7cc7ef572f50e537739706668aadb9029d5d1e
$ nix run "$FLAKE_REF#genMeta" -- --help;
Generate a Floco metaSet for an NPM registry tarball
...

# Example: Extended attrs
$ nix eval --json -f inputs.nix ak-nix.locked|jq;
{
  "lastModified": 1670101191,
  "narHash": "sha256-q9D0cEHvvfTVK+f6hjrEnLtXAiP0cVfrzlOduD/Lz58=",
  "owner": "aakropotkin",
  "repo": "ak-nix",
  "rev": "123fc484c3c68e354704c86a88d88e343e6371f3",
  "type": "github"
}

Flake Ref Types

These YANTS types can be used to assert the validity of “flake refs”, like those used as flake inputs, or arguments to builtins.fetchTree.

nix-repl> :p map ( builtins.getFlake "github:aakropotkin/rime/main" ).lib.ytypes.FlakeRef.Structs.flake_ref_github [
  { url = "github:aakropotkin/rime/main"; }
  { type = "github"; url = "github:aakropotkin/rime/main"; }
  { type = "github"; owner = "aakropotkin"; repo = "rime"; ref = "main"; }
]
=>
[ { url = "github:aakropotkin/rime/main"; }
  { type = "github"; url = "github:aakropotkin/rime/main"; }
  { owner = "aakropotkin"; ref = "main"; repo = "rime"; type = "github"; }
]

These types will throw on failure.

nix-repl> lib.ytypes.FlakeRef.Structs.flake_ref_file { type = "file"; url = "file:./foo"; }
=>
{ type = "file"; url = "file:./foo"; }

nix-repl> lib.ytypes.FlakeRef.Structs.flake_ref_file { type = "file"; path = "./foo"; }
=>
error: {
         path = "./foo";
         type = "file";
       } does not conform to restriction 'flake_ref[file]'

Available Checks

• flake_ref abstract form, allows any combination of fields used by children.
• flake_ref_indirect an inputs.foo.follows = "bar"; style ref.
• flake_ref_path
• flake_ref_file
• flake_ref_tarball
• flake_ref_git
• flake_ref_github
• flake_ref_sourcehut ( basically an alias of github, not tested exhaustibly )
• flake_ref_mercurial ( basically an alias of github, not tested exhaustibly )

Utilities and Scripts

Rime provides a handful of useful scripts that help extract info from Nix. These are generally pretty simple, and most anyone could cook these up as shell aliases; but exposing them in this flake allows me to lock and consume them in other projects

nix-outputs

Given a package name or installable URI, print the available outputs for the derivation. This simply dumps <DERIVATION>.outputs as a JSON list.

Use this to quickly look up extras that might hold headers, libs, docs, etc for a package.

If no URI is given, arg is interpreted as an attr on nixpkgs.

Options

• -H,--no-header

Suppress log-line ( normally printed to STDERR ).

nix2json

Naively convert a Nix file to JSON. Faster than nix-serialize, but will fail on input that contains functions or store paths.

nix-serialize

Best effort conversion from Nix to a flat representation. Intended to take a URI as an arg for an attrset/installable, which gets processed through nix eval "$@" --json --impure --apply ....

Similar to nixpkgs#lib.generators.toPretty but supports __serial functors from github:aameen-tulip/at-node-nix, and will completely remove fields that cannot be converted to JSON.

nix-prefetch-tree

Prefetch flake ref URI ( it doesn’t need to point to a flake ) and print the “locked” attribute representation.

This can be used to lookup narHash for URLs quickly. I basically made this entire script to try catch to create narHash locks on large numbers of tarballs.

By default we assume you assume you are prefetching a tarball, but adding --type ARG allows you to prefetch any type of ref.

Options

• -K, --fallback

When fetching tarballs, if unpacking fails ( due to archive permissions ) fall back to fetching as a regular file.

nix-parse-uri

Parse URIs and dump as JSON. Multiple arguments will produce multiple one JSON record per line of output.

This uses Nix’s internal parseURL and parseUrlScheme functions and is more lightweight than nix flake prefetch.

$ nix run github:aakropotkin/rime#nix-parse-uri "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz?foo=bar&baz=1#hey"|jq;
{
  "application-layer": "",
  "authority": "registry.npmjs.org",
  "base": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz",
  "fragment": "hey",
  "path": "/lodash/-/lodash-4.17.21.tgz",
  "query": {
    "baz": "1",
    "foo": "bar"
  },
  "scheme": "https",
  "translport-layer": "https"
}

resolve

Resolve a URL using the nix registry without necessarily fetching or locking the given tree. This differs from nix flake prefetch in that resolve is able to print unlocked URLs and runs significantly faster because it doesn’t fetch.

$ nix run github:aakropotkin/rime#resolve nixpkgs;
github:NixOS/nixpkgs/nixpkgs-unstable;

$ nix registry pin nixpkgs;

$ nix run github:aakropotkin/rime#resolve nixpkgs;
github:NixOS/nixpkgs/ea11a3977f4cba013d8680667616be827c967ac0

$ nix registry remove nixpkgs;

$ nix run github:aakropotkin/rime#resolve nixpkgs;
github:NixOS/nixpkgs/nixpkgs-unstable;