update eb_hooks.py for memory expensive builds of jax by laraPPr · Pull Request #25 · EESSI/software-layer-scripts

laraPPr · 2025-06-27T14:05:13Z

No description provided.

Signed-off-by: laraPPr <[email protected]>

laraPPr · 2025-06-27T14:51:59Z

bot: help

eessi-bot-aws · 2025-06-27T14:52:03Z

Updates by the bot instance eessi-bot-mc-aws (click for details)

received bot command help from laraPPr
- expanded format: help
handling command help resulted in:
How to send commands to bot instances
- Commands must be sent with a new comment (edits of existing comments are ignored).
- A comment may contain multiple commands, one per line.
- Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
- Currently supported COMMANDs are: help, build, show_config, status
For more information, see https://www.eessi.io/docs/bot

eessi-bot-deucalion · 2025-06-27T14:52:03Z

Updates by the bot instance eessi-bot-deucalion (click for details)

received bot command help from laraPPr
- expanded format: help
handling command help resulted in:
How to send commands to bot instances
- Commands must be sent with a new comment (edits of existing comments are ignored).
- A comment may contain multiple commands, one per line.
- Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
- Currently supported COMMANDs are: help, build, show_config, status
For more information, see https://www.eessi.io/docs/bot

eessi-bot-surf · 2025-06-27T14:52:03Z

Updates by the bot instance eessi-bot-surf (click for details)

received bot command help from laraPPr
- expanded format: help
handling command help resulted in:
How to send commands to bot instances
- Commands must be sent with a new comment (edits of existing comments are ignored).
- A comment may contain multiple commands, one per line.
- Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
- Currently supported COMMANDs are: help, build, show_config, status
For more information, see https://www.eessi.io/docs/bot

eessi-bot-jsc · 2025-06-27T14:52:04Z

Updates by the bot instance eessi-bot-jsc (click for details)

received bot command help from laraPPr
- expanded format: help
handling command help resulted in:
How to send commands to bot instances
- Commands must be sent with a new comment (edits of existing comments are ignored).
- A comment may contain multiple commands, one per line.
- Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
- Currently supported COMMANDs are: help, build, show_config, status
For more information, see https://www.eessi.io/docs/bot

laraPPr · 2025-06-27T14:52:35Z

bot: build repo:eessi.io-2023.06-software arch:aarch64/neoverse_v1 instance:eessi-bot-mc-aws
bot: build repo:eessi.io-2023.06-software arch:zen4 instance:eessi-bot-mc-azure

eessi-bot-aws · 2025-06-27T14:52:41Z

New job on instance eessi-bot-mc-aws for CPU micro-architecture aarch64-neoverse_v1 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2025.06/pr_25/72376

date	job status	comment
Jun 27 14:52:41 UTC 2025	submitted	job id `72376` awaits release by job manager
Jun 27 14:53:15 UTC 2025	released	job awaits launch by Slurm scheduler
Jun 27 14:59:23 UTC 2025	running	job `72376` is running
Jun 27 15:05:33 UTC 2025	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-72376.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-neoverse_v1-17510363870.tar.gz` size: 0 MiB (17444 bytes) entries: 1 modules under 2023.06/software/linux/aarch64/neoverse_v1/modules/all no module files in tarball software under 2023.06/software/linux/aarch64/neoverse_v1/software no software packages in tarball other under 2023.06/software/linux/aarch64/neoverse_v1 `2023.06/init/easybuild/eb_hooks.py`
Jun 27 15:05:33 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:aarch64_neoverse_v1+default P: perf: 985.161 timesteps/s (r:0, l:None, u:None) [ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:aarch64_neoverse_v1+default P: perf: 978.355 timesteps/s (r:0, l:None, u:None) [ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:aarch64_neoverse_v1+default P: latency: 3.12 us (r:0, l:None, u:None) [ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 3.16 us (r:0, l:None, u:None) [ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 4.3 us (r:0, l:None, u:None) [ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 4.41 us (r:0, l:None, u:None) [ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 0.42 us (r:0, l:None, u:None) [ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:aarch64_neoverse_v1+default P: latency: 0.39 us (r:0, l:None, u:None) [ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:aarch64_neoverse_v1+default P: bandwidth: 36419.58 MB/s (r:0, l:None, u:None) [ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:aarch64_neoverse_v1+default P: bandwidth: 33671.4 MB/s (r:0, l:None, u:None) [ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-72376.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

laraPPr · 2025-06-27T15:00:26Z

@ocaisa why is that eb_hooks check failing?

ocaisa · 2025-06-27T15:41:17Z

The check will only pass once the hooks have been ingested (similar to how missing installations works)

laraPPr · 2025-06-30T07:48:50Z

bot: build arch:x86_64/amd/zen3 repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws

eessi-bot-aws · 2025-06-30T07:49:01Z

New job on instance eessi-bot-mc-aws for CPU micro-architecture x86_64-amd-zen3 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2025.06/pr_25/73001

date	job status	comment
Jun 30 07:49:00 UTC 2025	submitted	job id `73001` awaits release by job manager
Jun 30 07:49:49 UTC 2025	released	job awaits launch by Slurm scheduler
Jun 30 07:54:51 UTC 2025	running	job `73001` is running
Jun 30 08:01:58 UTC 2025	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-73001.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-amd-zen3-17512701510.tar.gz` size: 0 MiB (17445 bytes) entries: 1 modules under 2023.06/software/linux/x86_64/amd/zen3/modules/all no module files in tarball software under 2023.06/software/linux/x86_64/amd/zen3/software no software packages in tarball other under 2023.06/software/linux/x86_64/amd/zen3 `2023.06/init/easybuild/eb_hooks.py`
Jun 30 08:01:58 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:x86_64_amd_zen3+default P: perf: 508.806 timesteps/s (r:0, l:None, u:None) [ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:x86_64_amd_zen3+default P: perf: 523.952 timesteps/s (r:0, l:None, u:None) [ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:x86_64_amd_zen3+default P: latency: 3.34 us (r:0, l:None, u:None) [ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:x86_64_amd_zen3+default P: latency: 1.16 us (r:0, l:None, u:None) [ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:x86_64_amd_zen3+default P: latency: 2.21 us (r:0, l:None, u:None) [ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:x86_64_amd_zen3+default P: latency: 2.28 us (r:0, l:None, u:None) [ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:x86_64_amd_zen3+default P: latency: 0.31 us (r:0, l:None, u:None) [ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:x86_64_amd_zen3+default P: latency: 0.33 us (r:0, l:None, u:None) [ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:x86_64_amd_zen3+default P: bandwidth: 13657.53 MB/s (r:0, l:None, u:None) [ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:x86_64_amd_zen3+default P: bandwidth: 13826.73 MB/s (r:0, l:None, u:None) [ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-73001.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

laraPPr · 2025-06-30T07:58:20Z

This was the error on neoverse_v1

ERROR: Failed to import easyblock easybuild.easyblocks.generic.easyblock_specific from /tmp/eb-2jbmecey/eb-kwi3p08c/included-easyblocks-f5qdyad3/easybuild/easyblocks/generic: No module named 'test.easyblocks'

easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-5.1.0-2023a.yml

This commit was used in the original pr in the software-layer maybe this was for a reason.

laraPPr · 2025-06-30T08:02:09Z

bot: build repo:eessi.io-2023.06-software arch:aarch64/neoverse_v1 instance:eessi-bot-mc-aws

eessi-bot-aws · 2025-06-30T08:02:16Z

New job on instance eessi-bot-mc-aws for CPU micro-architecture aarch64-neoverse_v1 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2025.06/pr_25/73002

date	job status	comment
Jun 30 08:02:15 UTC 2025	submitted	job id `73002` awaits release by job manager
Jun 30 08:03:01 UTC 2025	released	job awaits launch by Slurm scheduler
Jun 30 08:09:04 UTC 2025	running	job `73002` is running
Jun 30 09:20:21 UTC 2025	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-73002.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-neoverse_v1-17512744410.tar.gz` size: 6 MiB (6841536 bytes) entries: 1192 modules under 2023.06/software/linux/aarch64/neoverse_v1/modules/all `absl-py/2.1.0-GCCcore-12.3.0.lua` `ml_dtypes/0.3.2-gfbf-2023a.lua` `pytest/7.4.2-GCCcore-12.3.0.lua` `pytest-xdist/3.3.1-GCCcore-12.3.0.lua` software under 2023.06/software/linux/aarch64/neoverse_v1/software `absl-py/2.1.0-GCCcore-12.3.0` `ml_dtypes/0.3.2-gfbf-2023a` `pytest/7.4.2-GCCcore-12.3.0` `pytest-xdist/3.3.1-GCCcore-12.3.0` other under 2023.06/software/linux/aarch64/neoverse_v1 `2023.06/init/easybuild/eb_hooks.py`
Jun 30 09:20:21 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:aarch64_neoverse_v1+default P: perf: 984.574 timesteps/s (r:0, l:None, u:None) [ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:aarch64_neoverse_v1+default P: perf: 975.834 timesteps/s (r:0, l:None, u:None) [ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:aarch64_neoverse_v1+default P: latency: 3.11 us (r:0, l:None, u:None) [ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 2.92 us (r:0, l:None, u:None) [ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 4.21 us (r:0, l:None, u:None) [ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 4.39 us (r:0, l:None, u:None) [ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:aarch64_neoverse_v1+default P: latency: 0.42 us (r:0, l:None, u:None) [ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:aarch64_neoverse_v1+default P: latency: 0.38 us (r:0, l:None, u:None) [ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:aarch64_neoverse_v1+default P: bandwidth: 35312.23 MB/s (r:0, l:None, u:None) [ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:aarch64_neoverse_v1+default P: bandwidth: 36448.11 MB/s (r:0, l:None, u:None) [ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-73002.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

laraPPr · 2025-06-30T12:03:02Z

It seems like this, easybuilders/easybuild-easyblocks#3486, needs som more work. To get it working.

boegel · 2025-08-19T10:06:44Z

easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-5.1.0-2023a.yml

@@ -0,0 +1,5 @@
+easyconfigs:
+  - jax-0.4.25-gfbf-2023a.eb:


To be clear: this is only used for testing, it should be removed before final build & deploy?

laraPPr · 2025-09-18T10:26:58Z

@boegel are you planning to return to this easybuilders/easybuild-easyblocks#3486 at some point or should we close these prs for now?

update eb_hooks.py for memory expensive builds of jax

b42a7a2

Signed-off-by: laraPPr <[email protected]>

laraPPr added the 2023.06-software.eessi.io 2023.06 version of software.eessi.io label Jun 27, 2025

This was referenced Jun 27, 2025

{2023.06}[foss/2023a] AlphaFold 2.3.2 (CPU-only) EESSI/software-layer#785

Closed

{2023.06}[foss/2023a] AlphaFold 2.3.2 (CPU-only) EESSI/software-layer#1134

Open

test eb_hooks.py with test easystack

7769117

Signed-off-by: laraPPr <[email protected]>

laraPPr commented Jun 30, 2025

View reviewed changes

easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-5.1.0-2023a.yml Outdated Show resolved Hide resolved

Use older commit of easyblock

fb79b5b

This commit was used in the original pr in the software-layer maybe this was for a reason.

boegel reviewed Aug 19, 2025

View reviewed changes

Conversation

laraPPr commented Jun 27, 2025

Uh oh!

laraPPr commented Jun 27, 2025

Uh oh!

eessi-bot-aws bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-deucalion bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-surf bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-jsc bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laraPPr commented Jun 27, 2025

Uh oh!

eessi-bot-aws bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laraPPr commented Jun 27, 2025

Uh oh!

ocaisa commented Jun 27, 2025

Uh oh!

laraPPr commented Jun 30, 2025

Uh oh!

eessi-bot-aws bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laraPPr commented Jun 30, 2025

Uh oh!

Uh oh!

laraPPr commented Jun 30, 2025

Uh oh!

eessi-bot-aws bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laraPPr commented Jun 30, 2025

Uh oh!

boegel Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

laraPPr commented Sep 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

eessi-bot-aws bot commented Jun 27, 2025 •

edited

Loading

eessi-bot-deucalion bot commented Jun 27, 2025 •

edited

Loading

eessi-bot-surf bot commented Jun 27, 2025 •

edited

Loading

eessi-bot-jsc bot commented Jun 27, 2025 •

edited

Loading

eessi-bot-aws bot commented Jun 27, 2025 •

edited

Loading

eessi-bot-aws bot commented Jun 30, 2025 •

edited

Loading

eessi-bot-aws bot commented Jun 30, 2025 •

edited

Loading