gh-98894: Fix DTrace test_check_probes for shared builds

[stratakis]

When building with --enable-shared, the SystemTap/DTrace notes live in libpython. Add detection logic to be used by readelf.

• Issue: Some DTrace probes are broken in 3.11 #98894

[stratakis]

Easily reproducible traceback by:

./configure --with-dtrace --enable-shared && make -j && LD_LIBRARY_PATH=. ./python -m test -vv test_dtrace

This should cover also the install builds, the out of tree builds and their combination. Tested them all and it works.

I don't think a news entry is warranted here.

cc @pablogsal

[vstinner]

./configure --with-dtrace --enable-shared && make -j && LD_LIBRARY_PATH=. ./python -m test -vv test_dtrace

I tested on Fedora 44. I confirm that currently, test_check_probes() of test_dtrace fails.

With the change, test_dtrace pass (test_check_probes() pass).

But 6 tests are skipped:

setUpClass (test.test_dtrace.BPFTraceNormalTests) ... skipped "bpftrace not available: [Errno 2] No such file or directory: 'bpftrace'"
setUpClass (test.test_dtrace.BPFTraceOptimizedTests) ... skipped "bpftrace not available: [Errno 2] No such file or directory: 'bpftrace'"
skipped 'dtrace(1) failed: /usr/bin/dtrace invalid option -q\nUsage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]'
setUpClass (test.test_dtrace.DTraceOptimizedTests) ... skipped 'dtrace(1) failed: /usr/bin/dtrace invalid option -q\nUsage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]'
setUpClass (test.test_dtrace.SystemTapNormalTests) ... skipped "stap(1) failed: [Errno 2] No such file or directory: 'stap'"
setUpClass (test.test_dtrace.SystemTapOptimizedTests) ... skipped "stap(1) failed: [Errno 2] No such file or directory: 'stap'"
OK (skipped=6, expected failures=1)
Total tests: run=3 skipped=6

Note: my locale is French (LANG=fr_FR.UTF-8) and test_dtrace fails with my locale. I had to run the test with LC_ALL=C.

[vstinner]

Ah, /usr/bin/stap was not installed. I had to run sudo dnf install systemtap-client to install it.

I also had to install Linux kernel header files: dnf install kernel-devel.

After these installations, 6 tests are still skipped even if I run test_dtrace as root:

setUpClass (test.test_dtrace.BPFTraceNormalTests) ... skipped "bpftrace not available: [Errno 2] No such file or directory: 'bpftrace'"
setUpClass (test.test_dtrace.BPFTraceOptimizedTests) ... skipped "bpftrace not available: [Errno 2] No such file or directory: 'bpftrace'"
test_filter_probe_rows_ignores_warnings (test.test_dtrace.BPFTraceOutputTests.test_filter_probe_rows_ignores_warnings) ... ok
readelf version: 2.46
test_check_probes (test.test_dtrace.CheckDtraceProbes.test_check_probes) ... ok
test_missing_probes (test.test_dtrace.CheckDtraceProbes.test_missing_probes) ... expected failure
skipped 'dtrace(1) failed: /usr/sbin/dtrace invalid option -q\nUsage /usr/sbin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]'
setUpClass (test.test_dtrace.DTraceOptimizedTests) ... skipped 'dtrace(1) failed: /usr/sbin/dtrace invalid option -q\nUsage /usr/sbin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]'
setUpClass (test.test_dtrace.SystemTapNormalTests) ... skipped 'stap(1) failed: Checking "/lib/modules/7.0.9-204.fc44.x86_64/build/.config" failed with error: No such file or directory\nIncorrect version or missing kernel-devel package, use: dnf install kernel-devel-7.0.9-204.fc44.x86_64'
setUpClass (test.test_dtrace.SystemTapOptimizedTests) ... skipped 'stap(1) failed: Checking "/lib/modules/7.0.9-204.fc44.x86_64/build/.config" failed with error: No such file or directory\nIncorrect version or missing kernel-devel package, use: dnf install kernel-devel-7.0.9-204.fc44.x86_64'

[vstinner]

DTraceOptimizedTests is skipped because it runs dtrace with the -q option which doesn't exist (in my dtrace version, systemtap-sdt-dtrace-5.5-1.fc44.x86_64).

[vstinner]

LGTM.

The code is a little bit complicated (I expected shorter code), but it works as expected. So I'm fine with it.

[vstinner]

Can you apply the following code to use the C locale and so disable localization?

        # Force the C locale to disable localization
        env = dict(os.environ, LC_ALL='C')
        stdout, _ = subprocess.Popen(
            command,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            universal_newlines=True,
            env=env,
        ).communicate()

To reproduce my issue with the French locale, you can use:

LC_ALL=fr_FR.UTF-8 LD_LIBRARY_PATH=$PWD ./python -m test -v test_dtrace

[vstinner]

I also installed sudo dnf install bpftrace, but even if I run test_dtrace as root, bpftrace tests fail with the timeout of 10 seconds. In practice, the tests are marked as "skipped" (which is good).

I ran the bpftrace test manually as root. I had to stop it with CTRL+C, it does not display anything.

bpftrace -e 'usdt:/home/vstinner/python/main/python:python:function__entry { printf("probe: success\\n"); exit(); }' '-c' '/usr/bin/true'