Trying to create my first package variant - How do I add a patch?

Continuation from here, which in short, was me attempting to create my first package variant, with xkeyboard-config as the package I’m trying to add something to, but very likely took the complete wrong approach in the first thread.

I now believe I understand what I need to do, which is to write in the system config

that I would like to use a different package for the keyboard config,

and replace it with the same one, but with a patch added.

What I’ve got so far is this:

/etc/config.scm

...
(define xorg-server-yr
  (package
    (inherit xorg-server)
    (inputs (modify-inputs (package-inputs xorg-server)
              (replace "xkeyboard-config" xkeyboard-config-yr)))))

(define xkeyboard-config-yr
  (package
    (inherit xkeyboard-config)
    (source
     (origin
       (inherit (package-source xkeyboard-config))
       (patches (list xkeyboard-config-yr-patch))))))

(define xkeyboard-config-yr-patch
  (origin
    (method url-fetch)
    (uri "https://codeberg.org/folaht/xkeyboard-config-yr/src/branch/main/sxrs/xkeyboard-config-yr.patch")
    (sha256 (base32 "1jn9zg7cqpmxbdisklbl8iblwy8q1wxhp5p2zav8vwmr94kjjci7"))))

(operating-system
...
(set-xorg-configuration
             (xorg-configuration (keyboard-layout keyboard-layout)
                                 (server xorg-server-yr))))

This however does not compile

$ sudo guix system reconfigure /etc/config.scm
...
guix/ui.scm:1030:18: Throw to key match-error' with args `("match" "no matching pattern" #<origin "mirror://xorg/individual/data/xkeyboard-config/xkeyboard-config-2.44.tar.xz" #<content-hash sha256:0aillh6pmx5ji5jbqviq007vvg69ahz5832rz941s0xvxqzc7ljl> #<origin "" #<content-hash sha256:1jn9zg7cqpmxbdisklbl8iblwy8q1wxhp5p2zav8vwmr94kjjci7> () 7f9aa6434420> 7f9aa6434480>)'.

Is my hash wrong or am I totally going into the wrong direction?

Alright, I went from (patches xkeyboard-config-yr-patch) to (patches (list xkeyboard-config-yr-patch)) and that compiles.

However, I don’t know which package my system is using now and it doesn’t seem to be applied, even though it applies fine from the source code.
But when I look up the package ?I think my system is using? through realpath $(guix build xkeyboard-config) and guix package -I | grep xkeyboard-config it does show the patch applied however.

[edit]

It looks like the patch is nowhere to be seen when I try a guix upgrade.

$ guix upgrade
guix upgrade: error: canonicalize-path: No such file or directory: "xkeyboard-config-yr.patch"

?

The system is using which ever package you defined in your system config, so xorg-server-yr.

I can’t recall what other check I did to determine whether or not the patch was applied, but it was missing at that moment.

And I don’t know what I’ve done, but currently both

realpath $(guix build xkeyboard-config) and guix package -I | grep xkeyboard-config

now point towards the package without the patch applied.

The system is using which ever package you defined in your system config, so xorg-server-yr.

And that package I cannot find with realpath $(guix build xorg-server-yr) or ls /gnu/store | grep xorg-server-yr.

To test something like this, it’s really helpful to break the problem into smaller parts. This is kinda a long response with a few tangents, but hopefully there’s something that works or helps. it gets difficult to keep going if you can’t break things into smaller parts…

There are a few packages that I’ve created to add various files like this to systems, but usually these compile some program/tool that I want.

I also used to add my own ISO language code to X11 in KDE where I used the laughing man pic from ghost in the shell as the flag lol. The language code was io and the XKB layout that it enabled typed symbols with the digit keys (without shift) and shift was used to type the digits.

TLDR:

  • These aren’t the best ways to solve these problems. I would recommend avoiding “literate programming” and org-babel, unless you have experience to use them sparingly. If you copy my approaches, keep an open mind because there are better workflows!
  • Running guix install xorg-server-yr or guix install xkeyboard-config-yr won’t make it available on the running system, obviously. but guix shell xkeyboard-config-yr and ls $GUIX_ENVIRONMENT , etc helps a lot. I don’t think the correct way to solve this is a full xorg-server-yr package. One of the main strengths of guix is that you can mix pieces together – & you get true “multiple inheritance” patterns for packages!
  • If you do use emacs, guix.el takes a bit of setup, but helps you browse derivations & installed profiles. It’s not really compatible with arei.el
  • Use watchexec for packages. It gets messy with systems. This can combine well with a tmux profile or something that opens the shells you need. Change one .scm file here, watch the build fail there.

watchexec doesn’t work for systems, since the last step creates an actual disk image, takes forever and eats 5-10GB for each distinct derivation!

In any case, there are a few approaches to building & testing these packages. See below.

I strongly recommend the section in the Guix manual on The Perfect Setup. This is a bit advanced, since it assumes you may edit files in a local checkout of guix. However, you also avoid a lot of issues. You may ask around on IRC a bit. There’s also the online Guix Social meetup. Sometimes it’s just 100x faster if you can watch others solve problems or show them your setup. idk.

The guix cookbook also contains some critical techniques.

Testing derivation output while build is failing

TLDR: you need to run a command like this

# can be guix build
guix shell -L $loadpath xkeyboard-config

Then use some tools like these functions (or realpath) to find the derivation’s output path in /gnu/store. Then cd into that path.

Then look around and check that the files are there as you expect. If everything looks alright (i.e. the list of patches were applied correctly), then you’re more ready to put all the pieces together in the system.

Reentering the failed derivation environment

For a failing Go package, I was able to use --keep-failed then effectively enter the same build environment by using guix shell --container. This helped a ton! Otherwise the error was being swallowed in the guix-daemon’s build environment.

The environment probably won’t match the guix-daemon environment 100%, even with guix shell --container. Still, this helps you run that last failing make task or see whether it’s worth it to keep tests.

Approaches to structuring Guile modules

When evaluating packages to build:

  1. you can use the -e (@@ (...)) syntax which is a bit confusing as a noob
  2. you can use -e "$(cat script.scm) and evaluate a script like this in pico.scm in my notes on raspberry-pico SDK. here the script usually starts with use-modules.
  • I use this approach to tangle the .scm files with emacs, then I move towards pulling the files into the guile modules in my dotfiles.
  • IMO: avoid tangling and org-babel. You can instead run a command like watchexec -p -e *.scm – guix build -f ./sourcetrail.scm in a terminal so any change to .scm files there will rerun that guix build command
  1. you can define a full module namespace with directories/files/etc and get the guix command to evaluate them with -L $modules_root. this is the approach i use when I’ve mostly completed the code.
  2. you can instead evaluate a module as a script – or load a module into the -e (@@ (...)) syntax – if it returns a value at the end of the script.
    zettelkasten/dcguix/packages/golang-crypto/foxboron-tpm.org at master · dcunited001/zettelkasten · GitHub

With a full channel, you need to manage the keyring branch and ensure the master branch has a linear series of commits where PGP signatures are never disrupted. It’s prone to failure even for super-leet hacker experts lol.

Guile module structure

For my dotfiles, i have some guile modules that I use to develop package variants and define my systems. This avoids the need to deal with a full channel – it’s like a lightweight “half-channel” approach.

I have some more notes on this here:

The modules in my dotfiles look like this:

.
├── dc
│   └── dc
│       ├── home
│       │   └── services
│       ├── packages
│       ├── services
│       └── systems
├── ellipsis
│   └── ellipsis
│       ├── home
│       │   └── services
│       ├── packages
│       ├── services
│       └── systems
├── env
│   └── dc
│       ├── channels.scm
│       └── packages.scm
├── Makefile

Adding custom packages to a custom ISO

The dc module namespace covers stuff that’s personal, where the ellipsis module was intended to be shared in a full channel… which i never got to. Anyways, I have a usb-gpg-tools system in ./ellipsis/ellipsis/systems` and it opens like below.

When I build the system, i use -L ./ellipsis -L ./dc to share code from my dotfiles – without need to worry about a personal channel. This impacts reproducibility, but yeh… works for me.

Then i have a Makefile task that runs guix system. Make requires a bit of setup, so you may want to skip that for now. just is also preferred by some users.

;;; Module
(define-module (ellipsis system usb-gpg-tools)
  #:use-module (srfi srfi-1)
  #:use-module (gnu)
  #:use-module (gnu system)
  #:use-module (gnu system nss)
  #:use-module (gnu system pam)

  #:use-module (ellipsis packages emacs-xyz)
  #:use-module (ellipsis services security-token)
  #:use-module (ellipsis system common)
  ;; #:use-module (gnu services certbot)

  #:use-module (nongnu packages linux)
  #:use-module (nongnu system linux-initrd)

  #:export (usb-gpg-tools
            usb-gpg-tools-amd))

(use-modules (guix utils))

;; ....

Also, once you have a wayland window manager, then you can add stuff to ~/.config/xkb/rules

If you really need X11, that doesn’t help unfortunately.

If using hyprland, if it parse the xkeyboard-config correctly, you can modify the input in ~/.config/hypr to set layout and variant.

I can’t quite remember whether my full io keyboard is defined as a variant or a set of options. I had always wanted to set up i3 to load the keyboard like this, by following a similar approach to what you’re describing above … but i never really got around to it.

  • I would definitely want to try that on a VM or that isn’t intangled with my development environment.
  • or rebuild while iterating without a windowing system and find some way to “startx” that didn’t involve logging in & out.
  • or something else…

everything x11 was pulled into display managers a long time ago, so it’s not well documented any more. sway and hyprland can run multiple instances. IIRC, sway can run inside sway.

Another possibility is trying to get xwayland to eat your xkeyboard-config-yr … somehow. i have no idea how to configure xwayland … it just “magically works”

oh and @Folaht you should definitely check out toys.whereis.social.

  • it maintains an index of other people’s channels/packages/services.
  • Also, the HTTP API methods are well documented, so getting JSON results is pretty simple.
  • there’s links to help browse files in the backing git repositories

The authors of that page may be willing to help you out or something idk

adding channels to guix is a tough decision, since guix-hpc or whatever can require an older version of the main guix channel.

(edit)

Like, for example, here’s someone’s channel where (at first glance) they seem to have solved your problem:

i already have a local clone of the rrr channel and, sadly, there aren’t any other references to setxkbmap-rrr, so i’m not sure how a system would use it.

grep -re 'setxkbmap' $guixcheckout shows usage in ibus xorg, desktop, enlightenment, gnome, etc. the desktop module defines xmodmap-shepherd-service which calls setxkbmap.

IIRC, in my early linux days, i would just edit the files in /etc/X11/xkb and reload the server. So i dont think you need everything X11 to fully build with xkeyboard-config-yr. instead, you just need the correct package files to be merged into the system’s profile, then replace the instance of xmodmap-shepherd-service in your desktop-services … but i may be wrong. you’ll want to cross validate that.

I really wish X11 and free-desktop wasn’t such a mess. IMO they always, always made it very difficult to use anything related to XKB. even the newer libxkbcommon… but idk what i’m talking about.

but guix shell xkeyboard-config-yr and ls $GUIX_ENVIRONMENT , etc helps a lot.

I’m not sure how that would help. The problem is that my `xkeyboard-config-yr` is an unknown package.

$ guix shell xkeyboard-config-yr
guix shell: error: xkeyboard-config-yr: unknown package
$ ls $GUIX_ENVIRONMENT

If you really need X11, that doesn’t help unfortunately.

I use XFCE. They plan to switch to wayland at end of year.

For deeper dependencies like this, you really want to make sure the package or derivations are building as you expect before you try to add them to your system. For keyboard stuff, there are wierd workflow requirements – sometimes you may need to restart X11, sometimes you don’t; you can’t easily test everything. There are things like that here (and with the QEMU package) that make it tough to do all at once.

I’m not sure which parts your aware of here, so I tried to include everything anyways. Create a directory tree somewhere that looks like this:

echo "tmp-guix-modules/folaht/folaht/packages/xkb.scm" | tree --fromfile .
.
└── tmp-guix-modules
    └── folaht
        └── folaht
            └── packages
                └── xkb.scm

I named it as tmp-something because you’ll probably change how you structure the files as you learn more. I follow the directory structure in the guix codebase bc it’s just easier. they probably made better design decisions.

Including two ./fohlat/folaht levels gives you one extra layer of modules: you could have two or more separate guile module roots inside the same project. That was important for me later on when using the ares repl. Many channels do place them at the root, as do most scheme projects.

Then add the scheme module below into xkb.scm. once that’s complete, you should be able to run Guix commands. Using define-module below is a little heavier – it requires the directory tree for guile modules. Writing scripts starting with use-modules is a bit lighter, but requires a bit more CLI scripting. You have to know more about Guile’s module system.

mkdir -p tmp-guix-modules/folaht/folaht/packages; cd tmp-guix-modules
# edit ./folaht/folaht/packages/xkb.scm
guix build -L ./folaht --keep-failed xkeyboard-config-yr
# when a build fails, check the /tmp path. guix keeps it there

Once it succeeds:

guix shell -L ./folaht xkeyboard-config-yr
ls -al $GUIX_ENVIRONMENT # here's the package derivation root
tree -d -L2 $GUIX_ENVIRONMENT # here's the directory tree

Look around and check it out. Here’s the xkb.scm source

;; if you're still using (use-modules ...) at the top, this needs to be in a 
;; module declared like below
(define-module (fohlat packages xkb)

you’ll almost certainly need these modules for guix functionality

  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix gexp)
  #:use-module (guix utils)
  #:use-module (guix download)
  #:use-module (guix git-download)
  #:use-module (guix packages)

And these modules for package dependencies

  #:use-module (gnu packages)
  #:use-module (gnu packages freedesktop)
  #:use-module (gnu packages xdisorg)
  #:use-module (gnu packages pkg-config)


  ;; occasionally you'll need srfi-n modules. you almost always want srfi-1
  #:use-module (srfi srfi-1)

This module gives you the (pretty-print “some scheme thing”)` function which can help debugging a ton. It probably won’t be available inside “G-expressions” though, once you get to those.

  #:use-module (ice-9 pretty-print)
)

You almost always remove the pretty-print module and lines before committing. It’s a bit slow, but it’s easier to use early on… The output will appear in the logs.

Now that the module is declared, public symbols inside it can be made available to other guile scheme modules. The -L ./fohlat option to guix commands makes Guix aware of the modules in that directory tree. The modules must be named according to the directory structure.

;; symbols must be public or guix won't interpret them as packages 
;; that can be installed from the CLI
(define-public xkeyboard-config-yr
  (package
    (inherit xkeyboard-config)
    (source
     (origin
       (inherit (package-source xkeyboard-config))
       (patches (list xkeyboard-config-yr-patch))))))

Many applications will read evdev.xml. how they handle errors depends on what dependencies read it (usually it’s compiled in; these days its libxkbcommon). For me, hyprland will freak out a little if it goes to query the evdev.xml database and can’t read it. The built-in dependency libxkbcommon then reads the RMVLO and KcCGST (I think…).

RVMLO is the rules file format: the one with the ! lines. It’s really confusing. If you already know how to work with that and the KcCGST files (with the C-like syntax), then you’ll probably alright.

;; this is pulling in the patch. it seems that xkb's files should be patched correctly.
;;
;; xkb's directory tree conventions are very, very strict!
(define xkeyboard-config-yr-patch
  (origin
    (method url-fetch)
    (uri "https://codeberg.org/folaht/xkeyboard-config-yr/src/branch/main/sxrs/xkeyboard-config-yr.patch")
    (sha256 (base32 "1jn9zg7cqpmxbdisklbl8iblwy8q1wxhp5p2zav8vwmr94kjjci7"))))

since evdev.xml is not a generated file (old/bad design on X11’s part), then a patch is an easier way to deal with it. there may be other ways (idk). you actually may want to split the patch into two: one for added files only then a separate one for evdev.xml only. This won’t help right now though.

To use Guile with Guix, you need to read chapters 1,2,3,4 from the Guile Manual, but especially chapters 3 and 4. Sections 4.2, 4.3, 4.4 are critical.

Later on, chapters 6 and 7 are useful, but more advanced. Ch. 7 helped clarify the SRFI’s for me.

Don’t mess with ~/.guile or it screws with how Guix loads the repl and may break editor integration with geiser. I’m the kind of user that reads way ahead for REPL configuration and that really set me back.

Geiser is frustrating and IMO basically broken for Guix. I’d love to see other users’ setups. It works fine for small projects like mcron, where i’m able to run tests from emacs.

A repl started using guix repl -L ./folaht will let you evaluate symbols in xkb.scm, but everything in that file so far is a value, not a procedure. So by the time the module is included, everything is already evaluated. From the repl, if you type ,h all you get help. with ,m (folaht packages xkb), statements will be evaluated from that module’s namespace.

It’s very close to clojure. So close.

This sounds as though I don’t need to create a package variant. I just need to wait until XCFE upgrades to Wayland.

Yeh I would also wait.

Those config files are supported by apps that use libxkbcommon which helps bridge X11’s keyboard functionality over to Wayland.

Idk whether X11 supports them. It may.

Either way, it’s tough to edit those files, esp if you’re guessing around. The CLI tools to validate configs aren’t easy to use.

A conlang sounds pretty cool though, but why does it need an alternate keyboard layout?

A conlang sounds pretty cool though, but why does it need an alternate keyboard layout?

Well it started with me not having learned to type blind until I was an adult. I decided that if I’m going to learn how to type blind, I might as well have the best keyboard and keyboard layout for it. So I went with Typematrix 2030 USB “us dvorak” and learned to type blindly this way.

Then a lot of things came together for a conlang.

  1. Politics.
    1. Politics interests me and even as a child I’ve been a fan of a United Europe, because the country I live in feels unnecessarily small, but even more so culturally weak and I wanted a strong cultural nation, not one that depends a lot on another country.
    2. I do not like the two-party system of the US and already felt alienated of the direction its culture was going in decades ago, had a feeling that this would not change except for the worse and this has continued for two or three decades.
  2. Keyboard layout & Politics - So I was using the us dvorak layout and wanted to learn a bit more Japanese. I wanted to type Japanese and so had to switch from one layout to the other. XFCE was nice enough to have these two languages be represented in the menu corner by a flag. One Japanese, one US. I’m not from the US. So I wanted an EU flag and for that I needed an EU keyboard layout. So why not create a keyboard layout for it?
  3. Internet arguments & Science & Politics - I saw people using arguments in English that did not make sense or sounds off in translation because of equivocations and other language tricks. I noticed it so much, that I even started noticing when it doesn’t sound off in my own language but should when it still contains the same equivocations or other language tricks.

Now apart of the EU flag when switching layouts, I first looked at Esperanto and wonder what happened to it. Then I discovered Esperanto was never meant to be the language for the EU, but a much earlier Euro-centric attempt for a “world unifying language” and that’s not what I’m aiming for. What if that ‘one-world government’ fails? I’d like our planet earth to have a large spare governments just in case until we can create nearby artificial planets to flee to. Also, Esperanto is supposed to have a very Spanish accent and I listened to all EU languages and the world and settled for French with a hint of Japanese as my basis, so an accent that is cute, intelligent & sexy.

I wanted an easy “one letter one sound” setup and even go as far as “one syllable one letter”, like Hangul, but creating that is much more difficult, so I went with three systems:

  1. Aoi - the Hangul-like one
  2. Afeuro/Afyro (Alphabet phonemique europeenne) - One that uses a lot of diacritics.
  3. Anglabet (Anglo alphabet) - The one you can type with qwerty/dvorak, but very limited.

So my focus has been on #2 and I deviced a system where I could use all diacritics with the “Alt Gr” and “Alt Gr shift” button.