@cryptotaxi247 / infra / commits / a97dee63

meeting-notes: reformat

Wow, this means we can't just copy notes into here. Kinda surprising.

Martin Weinelt committed Apr 17, 2025 at 19:23 UTC a97dee639ef2f7c7141556575175701decfc358e
2 files changed +104 -84
docs/meeting-notes/2025-04-03.md
+45 -35
@@ -2,42 +2,52 @@
2
3 Attendees: dgrig/erethon, mic92, vcunat
4
5 -- erethon:
6 - - Tested umbriel email server -> works https://github.com/NixOS/infra/pull/600
7 - - Security bug tracker: no news, still running and ingesting CVEs.
8 - - Want to work on deprecating go-neb for matrix-alertmanager tomorrow (4/4) https://github.com/NixOS/infra/issues/549#issuecomment-2764778573
9 -- mic92:
10 - - Our Nixos infra hydra patches needs to be fixed or merged into master: https://github.com/NixOS/hydra/pull/1456
11 - - staging-hydra:
12 - - Works and trial-migration worked.
13 - - We still need to figure out how to copy old store path from old evaluations to the new cache (worked with shivaraj and m1-s)
14 - - maybe we can use https://releases.nixos.org/nixpkgs/nixpkgs-17.03pre96825.497e6d2/store-paths.xz instead
15 - - Glacier can be actually also quite expensive or very slow access
16 - - small objects need to be excluded through filters
17 - - migration costs from s3
5 +- erethon:
6 + - Tested umbriel email server -> works https://github.com/NixOS/infra/pull/600
7 + - Security bug tracker: no news, still running and ingesting CVEs.
8 + - Want to work on deprecating go-neb for matrix-alertmanager tomorrow (4/4)
9 + https://github.com/NixOS/infra/issues/549#issuecomment-2764778573
10 +- mic92:
11 + - Our Nixos infra hydra patches needs to be fixed or merged into master:
12 + https://github.com/NixOS/hydra/pull/1456
13 + - staging-hydra:
14 + - Works and trial-migration worked.
15 + - We still need to figure out how to copy old store path from old
16 + evaluations to the new cache (worked with shivaraj and m1-s)
17 + - maybe we can use
18 + https://releases.nixos.org/nixpkgs/nixpkgs-17.03pre96825.497e6d2/store-paths.xz
19 + instead
20 + - Glacier can be actually also quite expensive or very slow access
21 + - small objects need to be excluded through filters
22 + - migration costs from s3
23 - vcunat:
19 - - Hydra:
20 - - Runs stable
21 - - No big issues, xz compression bottlenecks less worse because of more CPU power
22 - - Build-ingestions of queue-runner is the new bottleneck (maybe a large latency between s3 and the server)
24 + - Hydra:
25 + - Runs stable
26 + - No big issues, xz compression bottlenecks less worse because of more CPU
27 + power
28 + - Build-ingestions of queue-runner is the new bottleneck (maybe a large
29 + latency between s3 and the server)
30 - Jeremy:
24 - - Mailserver:
25 - - Everything prepared and we just need to update DNS and sunset the old service
26 - - Maybe need another dump of the user
27 - - Saturday: 16:00 UTC / 18:00 Berlin Time
31 + - Mailserver:
32 + - Everything prepared and we just need to update DNS and sunset the old
33 + service
34 + - Maybe need another dump of the user
35 + - Saturday: 16:00 UTC / 18:00 Berlin Time
36 - Arian:
29 - - binary cache:
30 - - looked into moving parts of the binary cache into instant access glacier tier
31 - - phased approach. E.g. start with `nars/a*` then `nars/b*` etc ...
32 - - rule of thumb: access is twice as expensive but storage twice as cheap compared to infrequent access. but same latency guarantees
33 - - TODO: Please give Mic92 the s3 cost sheet
34 - - current storage (not bandwidth cost):
35 - - Want to enable object versioning on narinfos
37 + - binary cache:
38 + - looked into moving parts of the binary cache into instant access glacier
39 + tier
40 + - phased approach. E.g. start with `nars/a*` then `nars/b*` etc ...
41 + - rule of thumb: access is twice as expensive but storage twice as cheap
42 + compared to infrequent access. but same latency guarantees
43 + - TODO: Please give Mic92 the s3 cost sheet
44 + - current storage (not bandwidth cost):
45 + - Want to enable object versioning on narinfos
46 - edef:
37 - - https://releases.nixos.org/nixpkgs/nixpkgs-17.03pre96825.497e6d2/store-paths.xz
38 - - narinfos are always in standard tier. due to small size. (Except for some pathalogically large narinfos)
39 - - rules
40 - - recent retrieval
41 - - recent upload
42 - - or in releases.nixos.org
43 -
47 + - https://releases.nixos.org/nixpkgs/nixpkgs-17.03pre96825.497e6d2/store-paths.xz
48 + - narinfos are always in standard tier. due to small size. (Except for some
49 + pathalogically large narinfos)
50 + - rules
51 + - recent retrieval
52 + - recent upload
53 + - or in releases.nixos.org
docs/meeting-notes/2025-04-17.md
+59 -49
@@ -3,66 +3,76 @@
3 Atttendees: zimbatm, arian, erethon, hexa, Mic92, jfly
4
5 - zimbatm:
6 - - Official leadership rotation to hexa
7 - - Rotation permissions for zimbatm and eelco in various places
8 - - Gandi
9 - - GitHub
10 - - others...
6 + - Official leadership rotation to hexa
7 + - Rotation permissions for zimbatm and eelco in various places
8 + - Gandi
9 + - GitHub
10 + - others...
11
12 - erethon:
13 - - Security bug tracker: Development is restarting
14 - - https://tracker.security.nixos.org/
15 - - Want to restart makemake.ngi.nixos.org, who has access to the Hetzner Robot account if things go south?
16 - - Remote KVM via Infra
17 - - Infra-Build holds Hetzner Accesss
18 - - @Mic92 mentioned Hetzner supporting Subaccounts, let's investigate that
19 - - Go-neb Deprecation
20 - - Still WIP
21 - - https://github.com/NixOS/infra/issues/549#issuecomment-2782452767
22 - - Links to Grafana/Prometheus/Alertmanager would be useful
13 + - Security bug tracker: Development is restarting
14 + - https://tracker.security.nixos.org/
15 + - Want to restart makemake.ngi.nixos.org, who has access to the Hetzner Robot
16 + account if things go south?
17 + - Remote KVM via Infra
18 + - Infra-Build holds Hetzner Accesss
19 + - @Mic92 mentioned Hetzner supporting Subaccounts, let's investigate that
20 + - Go-neb Deprecation
21 + - Still WIP
22 + - https://github.com/NixOS/infra/issues/549#issuecomment-2782452767
23 + - Links to Grafana/Prometheus/Alertmanager would be useful
24
25 - hexa:
25 - - Onboarding US Macs (delegated to Arian)
26 - - Winter has worked on getting Apple Business and Mosyle
27 - - DNS migration
28 - - Prompted by shared access with Marketing to Netlify
29 - - Adopted hopeful-rivest (RX170)
30 - - Mailserver
31 - - Authenticated Receive Chain
32 - - Will get Commit Access to nixos-mailserver
33 - - nixcon.org Mail Migration
26 + - Onboarding US Macs (delegated to Arian)
27 + - Winter has worked on getting Apple Business and Mosyle
28 + - DNS migration
29 + - Prompted by shared access with Marketing to Netlify
30 + - Adopted hopeful-rivest (RX170)
31 + - Mailserver
32 + - Authenticated Receive Chain
33 + - Will get Commit Access to nixos-mailserver
34 + - nixcon.org Mail Migration
35
36 - arian:
36 - - AWS Account cleanup and audit
37 - - Did an audit of all access to the Logicblox account
37 + - AWS Account cleanup and audit
38 + - Did an audit of all access to the Logicblox account
39
39 - - Can somebody copy over the messages I sent in infra-internal about my research there? I lost access to my Matrix history due to deleting it because of the spam issue.
40 - - Going to disable unused high-privilege IAM roles and users that are probably from the Logicblox days
41 - ```
42 - I am going to disable the roles accessible by that account now. And I suggest we delete them in a few weeks if nobody complains?
40 + - Can somebody copy over the messages I sent in infra-internal about my
41 + research there? I lost access to my Matrix history due to deleting it
42 + because of the spam issue.
43 + - Going to disable unused high-privilege IAM roles and users that are
44 + probably from the Logicblox days
45 + ```
46 + I am going to disable the roles accessible by that account now. And I suggest we delete them in a few weeks if nobody complains?
47
44 - There is also some other external account ids that are in use:
48 + There is also some other external account ids that are in use:
49
46 - * 297794765570 (has read only access. Seems to be from the same time as 33233536009 which has access to the same read only role).
47 - * CrowdStrike (has access to audit logs. hasn't accessed our account on like 500 days. Delete?)
48 - * CloudCheckr (accessed some metadata a few hours ago. Seems to be an AWS cost management tool)
49 - * Fastly logs (this makes sense)
50 - * Duckbill Group (makes sense but maybe we can delete now?)
51 - ```
50 + * 297794765570 (has read only access. Seems to be from the same time as 33233536009 which has access to the same read only role).
51 + * CrowdStrike (has access to audit logs. hasn't accessed our account on like 500 days. Delete?)
52 + * CloudCheckr (accessed some metadata a few hours ago. Seems to be an AWS cost management tool)
53 + * Fastly logs (this makes sense)
54 + * Duckbill Group (makes sense but maybe we can delete now?)
55 + ```
56
53 - - Do we want to terraform the AWS management account?
54 - - Want to enable CloudTrail audit logging for all our accounts in our management account.
55 - - Working on moving AMI builds out of Hydra to GHA. Almost done. Uses qemu emulation for the aarch64 build as opposed to KVM but works fine. https://github.com/NixOS/amis/pull/262
56 - - Mic and I had the plan to do the same for ISOs at https://github.com/NixOS/images - do we still want to work on that?
57 - - Planning to meet up with edef semi-regularly regarding s3 stuff
58 - - Wanted to look into the Glacier migration or Intelligent Tiering
59 - - 60 EUR worth of access to "old paths"
60 - - Intelligent Tiering Transitions might easily become more expensive than that
57 + - Do we want to terraform the AWS management account?
58 + - Want to enable CloudTrail audit logging for all our accounts in our
59 + management account.
60 + - Working on moving AMI builds out of Hydra to GHA. Almost done. Uses qemu
61 + emulation for the aarch64 build as opposed to KVM but works fine.
62 + https://github.com/NixOS/amis/pull/262
63 + - Mic and I had the plan to do the same for ISOs at
64 + https://github.com/NixOS/images - do we still want to work on that?
65 + - Planning to meet up with edef semi-regularly regarding s3 stuff
66 + - Wanted to look into the Glacier migration or Intelligent Tiering
67 + - 60 EUR worth of access to "old paths"
68 + - Intelligent Tiering Transitions might easily become more expensive than
69 + that
70
71 - Mic92
63 - - Idea: Fastly Pull-Through Cache for GitHub Releases
64 - - To have stable URLs and allowing us to move them in the backend as needed
72 + - Idea: Fastly Pull-Through Cache for GitHub Releases
73 + - To have stable URLs and allowing us to move them in the backend as needed
74
75 - jfly
67 - - Does a cache hit by fastly prevent an access log entry at AWS S3?
68 - - Cache at fastly can be hot, while things are in low priority tier at AWS S3
76 + - Does a cache hit by fastly prevent an access log entry at AWS S3?
77 + - Cache at fastly can be hot, while things are in low priority tier at AWS
78 + S3