Reduce the level of parallelism when building SciPy-bundle (for RISC-V) by julianmorillo · Pull Request #156 · EESSI/software-layer-scripts

julianmorillo · 2026-02-02T15:43:57Z

We had problems with the number of opened files in EESSI/dev.eessi.io-riscv#25

bedroge · 2026-02-02T16:10:15Z

Let's see if the bot is configured to listen to this repo:

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-02T16:10:23Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263311

date	job status	comment
Feb 02 16:10:21 UTC 2026	submitted	job id `263311` awaits release by job manager
Feb 02 16:10:56 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 02 16:12:01 UTC 2026	running	job `263311` is running
Feb 02 16:13:13 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-263311.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-riscv64-generic-layer-1770048715.tar.zst` size: 0 MiB (22 bytes) entries: 0 modules under 2025.06-001/software/linux/riscv64/generic/modules/all no module files in tarball software under 2025.06-001/software/linux/riscv64/generic/software no software packages in tarball reprod directories under 2025.06-001/software/linux/riscv64/generic/reprod no reprod directories in tarball other under 2025.06-001/software/linux/riscv64/generic no other files in tarball
Feb 02 16:13:13 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263311.test` does not exist in job directory, or parsing it failed.

bedroge · 2026-02-02T16:26:46Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-02T16:26:55Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263314

date	job status	comment
Feb 02 16:26:53 UTC 2026	submitted	job id `263314` awaits release by job manager
Feb 02 16:27:07 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 02 16:28:15 UTC 2026	running	job `263314` is running
Feb 02 22:11:45 UTC 2026	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-263314.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-riscv64-generic-layer-1770070080.tar.zst` size: 0 MiB (67990 bytes) entries: 15 modules under 2025.06-001/software/linux/riscv64/generic/modules/all `gfbf/2025a.lua` software under 2025.06-001/software/linux/riscv64/generic/software `gfbf/2025a` reprod directories under 2025.06-001/software/linux/riscv64/generic/reprod no reprod directories in tarball other under 2025.06-001/software/linux/riscv64/generic no other files in tarball
Feb 02 22:11:45 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263314.test` does not exist in job directory, or parsing it failed.

julianmorillo · 2026-02-03T15:48:52Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-03T15:49:01Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263457

date	job status	comment
Feb 03 15:48:59 UTC 2026	submitted	job id `263457` awaits release by job manager
Feb 03 15:49:58 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 03 15:51:04 UTC 2026	running	job `263457` is running
Feb 03 21:34:42 UTC 2026	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-263457.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-riscv64-generic-layer-1770154261.tar.zst` size: 0 MiB (66186 bytes) entries: 15 modules under 2025.06-001/software/linux/riscv64/generic/modules/all `gfbf/2025a.lua` software under 2025.06-001/software/linux/riscv64/generic/software `gfbf/2025a` reprod directories under 2025.06-001/software/linux/riscv64/generic/reprod no reprod directories in tarball other under 2025.06-001/software/linux/riscv64/generic no other files in tarball
Feb 03 21:34:42 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263457.test` does not exist in job directory, or parsing it failed.

bedroge · 2026-02-04T10:34:44Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T10:34:52Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263544

date	job status	comment
Feb 04 10:34:50 UTC 2026	submitted	job id `263544` awaits release by job manager
Feb 04 10:35:42 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 10:36:48 UTC 2026	running	job `263544` is running
Feb 04 10:41:05 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263544.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 10:41:05 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263544.test` does not exist in job directory, or parsing it failed.

…r-scripts into SciPy-bundle

easystacks/riscv/2025.06-001/eessi-2025.06-eb-5.2.0-2025a.yml

bedroge · 2026-02-04T11:01:40Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T11:01:48Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263553

date	job status	comment
Feb 04 11:01:46 UTC 2026	submitted	job id `263553` awaits release by job manager
Feb 04 11:02:47 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 11:03:52 UTC 2026	running	job `263553` is running
Feb 04 11:08:10 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263553.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 11:08:10 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263553.test` does not exist in job directory, or parsing it failed.

bedroge · 2026-02-04T11:08:27Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T11:08:35Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263556

date	job status	comment
Feb 04 11:08:33 UTC 2026	submitted	job id `263556` awaits release by job manager
Feb 04 11:09:16 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 11:10:22 UTC 2026	running	job `263556` is running
Feb 04 11:14:39 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263556.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 11:14:39 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263556.test` does not exist in job directory, or parsing it failed.

bedroge · 2026-02-04T11:58:22Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T11:58:30Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263565

date	job status	comment
Feb 04 11:58:28 UTC 2026	submitted	job id `263565` awaits release by job manager
Feb 04 11:59:03 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 12:00:09 UTC 2026	running	job `263565` is running
Feb 04 12:10:50 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263565.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 12:10:50 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263565.test` does not exist in job directory, or parsing it failed.

bedroge · 2026-02-04T12:26:51Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T12:26:59Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263569

date	job status	comment
Feb 04 12:26:57 UTC 2026	submitted	job id `263569` awaits release by job manager
Feb 04 12:27:27 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 12:28:35 UTC 2026	running	job `263569` is running
Feb 04 12:33:54 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263569.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 12:33:54 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263569.test` does not exist in job directory, or parsing it failed.

julianmorillo · 2026-02-04T16:00:06Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T16:00:16Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263600

date	job status	comment
Feb 04 16:00:14 UTC 2026	submitted	job id `263600` awaits release by job manager
Feb 04 16:01:16 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 16:02:22 UTC 2026	running	job `263600` is running
Feb 04 16:21:39 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263600.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 16:21:39 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263600.test` does not exist in job directory, or parsing it failed.

julianmorillo · 2026-02-04T16:24:08Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T16:24:16Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263629

date	job status	comment
Feb 04 16:24:14 UTC 2026	submitted	job id `263629` awaits release by job manager
Feb 04 16:24:48 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 16:25:54 UTC 2026	running	job `263629` is running
Feb 04 16:48:23 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263629.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 04 16:48:23 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263629.test` does not exist in job directory, or parsing it failed.

julianmorillo · 2026-02-04T16:49:33Z

bot: build repo:dev.eessi.io-riscv-2025.06-001 instance:eessi-bot-riscv for:arch=riscv64/generic

riscv-eessi-io-bot · 2026-02-04T16:49:42Z

New job on instance eessi-bot-riscv for repository dev.eessi.io-riscv-2025.06-001
Building on: generic
Building for: riscv64/generic
Job dir: /home/eessibot/shared/jobs/2026.02/pr_156/263669

date	job status	comment
Feb 04 16:49:40 UTC 2026	submitted	job id `263669` awaits release by job manager
Feb 04 16:50:30 UTC 2026	released	job awaits launch by Slurm scheduler
Feb 04 16:51:36 UTC 2026	running	job `263669` is running
Feb 05 09:12:44 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job263669.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Feb 05 09:12:44 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job263669.test` does not exist in job directory, or parsing it failed.

julianmorillo · 2026-02-05T09:49:30Z

Looking into the first test failure:

=================================== FAILURES ===================================
________________________ test_public_modules_importable ________________________
scipy/_lib/tests/test_import_cycles.py:18: in test_public_modules_importable
    assert pid.wait() == 0, f'Failed to import {PUBLIC_MODULES[i]}'
E   AssertionError: Failed to import scipy.cluster
E   assert 1 == 0
E    +  where 1 = wait()
E    +    where wait = <Popen: returncode: 1 args: ['/cvmfs/dev.eessi.io/riscv/versions/2025.06-001...>.wait
        i          = 0
        pid        = <Popen: returncode: 1 args: ['/cvmfs/dev.eessi.io/riscv/versions/2025.06-001...>
        pids       = [<Popen: returncode: 1 args: ['/cvmfs/dev.eessi.io/riscv/versions/2025.06-001...>, <Popen: returncode: None args: ['/c....io/riscv/versions/2025.06-...>, <Popen: returncode: None args: ['/cvmfs/dev.eessi.io/riscv/versions/2025.06-...>, ...]

Looks similar to the ones reported in scipy/scipy#24163. There is no mention, however, to: OSError: [Errno 24] Too many open files.
Also, there the Popen only appears once while we have the Popen message 4 times... it looks like crucial to somehow run the tests sequentially with just 1 thread.

Reduce the level of parallelism when building SciPy-bundle (for RISC-V)

f34c2c7

Add easystack for testing purposes

64375ac

Set the maximum parallelism to 1 for SciPy-bundle (for RISC-V)

4f866a5

bedroge added 2 commits February 4, 2026 11:34

use hooks file from software-layer-scripts repo

f78a7b2

Merge branch 'main' into SciPy-bundle

1551b8d

bedroge and others added 3 commits February 4, 2026 11:42

revert change

d302a5c

Use very last version of the easyconfig

11bcf6e

Merge branch 'SciPy-bundle' of github.com:julianmorillo/software-laye…

045322d

…r-scripts into SciPy-bundle

bedroge reviewed Feb 4, 2026

View reviewed changes

easystacks/riscv/2025.06-001/eessi-2025.06-eb-5.2.0-2025a.yml Outdated Show resolved Hide resolved

fix from-commit

0677a47

Conversation

julianmorillo commented Feb 2, 2026

Uh oh!

bedroge commented Feb 2, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Feb 2, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julianmorillo commented Feb 3, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bedroge commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julianmorillo commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julianmorillo commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julianmorillo commented Feb 4, 2026

Uh oh!

riscv-eessi-io-bot bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julianmorillo commented Feb 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

riscv-eessi-io-bot bot commented Feb 2, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 2, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 3, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading

riscv-eessi-io-bot bot commented Feb 4, 2026 •

edited

Loading