Skip to content

libs/libc/elf: Keep a module registry per task group. - #20492

Open
casaroli wants to merge 2 commits into
apache:masterfrom
casaroli:libelf-per-task-registry
Open

casaroli wants to merge 2 commits into
apache:masterfrom
casaroli:libelf-per-task-registry

Conversation

@casaroli

@casaroli casaroli commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Each task group now has its own registry of the ELF objects that dlopen() loads, as suggested in #20132. The registry is in struct task_info_s and not in struct tls_info_s, because POSIX makes a dlopen() handle valid in the whole process, not only in the thread that opened it.

Kernel modules from insmod() stay on one registry for the system, and /proc/modules shows only them.

A library that a module names in DT_NEEDED goes on the registry of that module. In a flat build, exec() gives the libraries of the program to the new task group, so dlopen() in the program finds the same copy.

When a task group exits, the libraries that it did not close are closed. A flat build does this in group_release(), after the last thread. A protected build does it in exit(), because the kernel cannot reach the user registry.

Impact

Two processes that use one library now each have their own copy of its data, and its constructors and destructors run once per process. An FDPIC library executed in place from xipfs still shares its text.

A library that is not executed in place is now loaded once per process, so its text uses RAM in each process.

A program loaded by exec() no longer resolves symbols from libraries that other processes loaded, or from insmod() modules.

dlopen() handles that a process does not close are now freed when it exits. Before, they stayed loaded.

The xipfs C++ test in apps checks the old shared behaviour. apache/nuttx-apps#3813 changes it.

Testing

Arm GNU Toolchain 13.2.rel1, QEMU mps2-an500.

test master this PR
xipfs_test fdpic (FDPIC, mps2-an500:xipfs) not run on the current master 34 passed, 0 failed, with the apps PR
fdpicxip solib library pinned once, totals 9 and 9 pinned twice, totals 3 and 6, 0 pins after exit
fdpicxip cxx one library destructor one destructor per process
sotest (plain ELF) same output same output
sotest that exits with its libraries open the library destructor never runs it runs at exit
mps2-an500:knsh (protected) with ELF and dlfcn: ostest exits 0 exits 0
qemu-armv7a:knsh (kernel) with dlfcn builds builds
esp32s3-devkit:kernel_n8r2 (kernel, CONFIG_LIBC_ELF), kernel and user programs builds builds
esp32s3-devkit:kernel_oct (kernel, CONFIG_LIBC_ELF) on an ESP32-S3-WROOM-2-N32R8V: ostest and sandbox not run ostest exits 0, fork() passes; sandbox reports every check passed

tools/checkpatch.sh -c -u -m -g passes.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

MemBrowse Memory Report

qemu-armv8a

  • Code: .text.init_builtin_run +60 B (+0.0%, 347,480 B)
  • Data: .bss.g_rr_values -12 B (-0.0%, 77,462 B)

qemu-intel64

  • Code: .text +74 B (+0.0%, 8,682,514 B)

s698pm-dkit

@casaroli
casaroli force-pushed the libelf-per-task-registry branch from 69e3fe6 to 52e93c8 Compare October 7, 2026 19:26
@casaroli

casaroli commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@xiaoxiang781216 i think this is breaking change. Should I mark it and bring a vote to the list?

@xiaoxiang781216

Copy link
Copy Markdown
Contributor

@xiaoxiang781216 i think this is breaking change. Should I mark it and bring a vote to the list?

the old behavior doesn't compliant to POSIX, this patch fix the problem. So, I think it's a bug fix.:)

One global registry held every ELF object that dlopen(), insmod() and
the DT_NEEDED loader loaded.  Thus a library was one instance for the
whole system, and all task groups shared its data.

Put the objects that dlopen() loads on a registry in struct task_info_s,
so each task group records the libraries it loaded.  POSIX makes a dlopen()
handle valid in the whole process, so the registry is per task group and
not per thread.  Kernel modules of insmod() stay on the global registry,
and /proc/modules shows only them.

A DT_NEEDED library goes on the registry of the module that needs it.  A
program that exec() loads collects its libraries on its own registry.  In
a flat build, exec_module() moves them to the registry of the new task
group, so dlopen() in the program finds the same copy.  In the protected
build the program is loaded by the kernel libc, which has another lock
and heap, so its libraries stay on its own registry.

Close the libraries that a task group did not close when it exits.  In a
flat build, group_release() does it after the last thread, before it
unloads the program.  In the protected build the kernel cannot reach the
user registry, so exit() does it.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Describe the per task group registry: each task group loads its own copy
of a library, constructors and destructors run once per task group, and
a task group closes what it left open when it exits.  Kernel modules stay
on one list for the system.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
@casaroli
casaroli force-pushed the libelf-per-task-registry branch from 52e93c8 to a04d4c0 Compare October 8, 2026 06:24
@casaroli
casaroli marked this pull request as ready for review October 8, 2026 06:50
@casaroli

casaroli commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

ok, i agree

@github-actions

Copy link
Copy Markdown

❌ Cross-repo dependency could not be applied

The Build report says the declared dependency PR(s) could not be applied, so CI did not run against the combined code:

Reason: cherry-pick failed (if your PR has merge commits, rebase instead)

CI run: https://github.com/apache/nuttx/actions/runs/38088049757

@casaroli casaroli closed this Oct 11, 2026
@casaroli casaroli reopened this Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: BINFMT Size: M The size of the change in this PR is medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants