#2025-05-01

Attendees: hexa, mightyiam, mic92, jfly, picnoir, mightyiam

• hexa:

  • hydra-server abuse management
  • loki for nginx analyzing the access logs
  • internal grafana instance for access to sensitive data (e.g. PII)
  • looking at whether go-away can better reflect our needs
  • hydra-queue-runner work
  • runnables are steps that can be sent to builder to realize
  • we have far more linux capacity than darwin capacity and the queue runner often stacks up darwin runnables, but cannot satisfy linux runnables
    • effectively preventing us from increasing linux capacity
  • Every two weeks meeting with Simon
  • hydra is modular, components use the database to synchronize

• jfly

  • can we put our meetings on the nixos cal? https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com
  • TODO: jeremy will add
  • mailserver
  • mxlogin.com (mxroute) Deliverability: unclear what's going on with Valentin's email (https://github.com/NixOS/infra/issues/668)
  • Bounces from GMail: does anyone have any ideas for this? https://github.com/NixOS/infra/issues/650
  • Leaking email: https://github.com/NixOS/infra/issues/649
    • TODO: try BCC-ing the people we're forwarding to (recipient_bcc_maps instead of virtual alias maps)
  • ARC: are we OK to enable it? https://github.com/NixOS/infra/pull/634
    • Test it on another mailserver first

• Mic92:

  • Get rid of nix/hydra overlay to make sure we are not mixing nix from unstable with nixpkgs from stable
  • Would likes to reproduce cgroups build issues in NixOS infra

• picnoir:

  • Cache key rotation https://github.com/NixOS/rfcs/pull/149
  • Minimal approach, no HSM etc.
    • Generate new key on hydra machine
    • Sign everything with two keys
    • PR in Nix has been merged, not in a stable release yet, but cherry-pickable
    • Rollover is easy for NixOS, but not so for Nix users on other Distros
    • Nix already hardcodes the current signing key for cache.nixos.org
    • We could investigate if we can also ship the new one that way
    • Set up and maintain public information about keys, the period in which they were used to sign packages
    • Blocked on social issue, needing to take charge
    • Testing on staging hydra
  • Remote-Signing (PR by Raito exists, https://github.com/NixOS/nix/pull/9076) would be nice, not super trivial, but also new failure point
    • Can fail in a lot of ways, since the channel between the signing infra and the queue-runner/nix is undefined, and e.g. over the network is not trivial
  • Next steps:
  • Staging Hydra setup to validate the setup/migration script.
  • Investigate Nix upgrade path.

• mightyiam/jfly:

  • Code: https://github.com/molybdenumsoftware/pr-tracker
  • Demo: https://pr-tracker.snow.jflei.com/
  • Alternatives: Replicate the Github Webhooks via pub/sub for anyone
  • Demonstrate the need for this, then we may consider making it an official nixos.org deployment