Skip to content

gh-158540: Add the profiled script's directory to sys.path - #158548

Open
hetaozdh wants to merge 5 commits into
python:mainfrom
hetaozdh:fix-sampling-script-path-main
Open

hetaozdh wants to merge 5 commits into
python:mainfrom
hetaozdh:fix-sampling-script-path-main

Conversation

@hetaozdh

@hetaozdh hetaozdh commented Oct 1, 2026

Copy link
Copy Markdown

Fixes gh-158540.

When a script is executed with python -m profiling.sampling run script.py
from another directory, the script cannot import modules placed next to it:
_sync_coordinator._execute_script() runs it with the working directory
(added by _setup_environment() for the -m case) as sys.path[0]
instead of the script's own directory.

This change adds the script's directory to sys.path in _execute_script(),
matching the behavior of python script.py.

Added a regression test in
Lib/test/test_profiling/test_sampling_profiler/test_sync_coordinator.py
that runs the coordinator on a script importing a sibling module, plus a
Misc/NEWS.d entry.

Tests:

  • ./python -m test test_profiling
  • ./python Tools/patchcheck/patchcheck.py

@hetaozdh
hetaozdh requested a review from pablogsal as a code owner October 1, 2026 05:01
@python-cla-bot

python-cla-bot Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

When a script is profiled with ``python -m profiling.sampling run
script.py`` from another directory, the script cannot import modules
placed next to it, because ``_sync_coordinator._execute_script()``
executes it with the working directory (added by
``_setup_environment()`` for the module case) as ``sys.path[0]``
instead of the script's own directory.

Make the script's directory importable in ``_execute_script()``,
matching the behavior of ``python script.py``.

Add a regression test that runs the coordinator on a script importing
a sibling module.
@hetaozdh
hetaozdh force-pushed the fix-sampling-script-path-main branch from 6c4dd14 to 16ff1ec Compare October 1, 2026 05:40
Comment thread Lib/profiling/sampling/_sync_coordinator.py Outdated
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
Comment on lines +171 to +174
# Put directory containing the script (not the current working directory) on sys.path so that modules next to it can be imported. Similar to ``python script.py``
script_dir = os.path.dirname(script_path)
if script_dir not in sys.path:
sys.path.insert(0, script_dir)

@maurycy maurycy Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about this edge case?

2026-10-01T12:57:30.615506000+0200 maurycy@gimel /Users/maurycy/work/cpython-pr158548 (pr158548 91c5ab3?) % tree w
w
├── link.py -> sub/where.py
└── sub
    ├── helper.py
    └── where.py

2 directories, 3 files
2026-10-01T12:57:32.453682000+0200 maurycy@gimel /Users/maurycy/work/cpython-pr158548 (pr158548 91c5ab3?) % sudo ./python.exe -m profiling.sampling run w/link.py
Traceback (most recent call last):
  File "<frozen runpy>", line 201, in _run_module_as_main
  File "<frozen runpy>", line 87, in _run_code
  File "/Users/maurycy/work/cpython-pr158548/Lib/profiling/sampling/_sync_coordinator.py", line 261, in <module>
    main()
    ~~~~^^
  File "/Users/maurycy/work/cpython-pr158548/Lib/profiling/sampling/_sync_coordinator.py", line 254, in main
    _execute_script(script_path, script_args, cwd)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/maurycy/work/cpython-pr158548/Lib/profiling/sampling/_sync_coordinator.py", line 201, in _execute_script
    exec(code, main_module.__dict__)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/maurycy/work/cpython-pr158548/w/link.py", line 1, in <module>
    import helper
ModuleNotFoundError: No module named 'helper'
2026-10-01T12:57:43.707720000+0200 maurycy@gimel /Users/maurycy/work/cpython-pr158548 (pr158548 91c5ab3?) % ./python.exe -VV
Python 3.16.0a0 (heads/pr158548:91c5ab3e096, Oct  1 2026, 10:22:50) [Clang 21.0.0 (clang-2100.3.34.2)]

I'm wondering if this might be an approach to mimic:

cpython/Lib/pdb.py

Lines 192 to 197 in 763b6ed

self._target = self._safe_realpath(target)
# If PYTHONSAFEPATH (-P) is not set, sys.path[0] is the directory
# of pdb, and we should replace it with the directory of the script
if not sys.flags.safe_path:
sys.path[0] = os.path.dirname(self._target)

@hetaozdh hetaozdh Oct 1, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment. Fixed in 264a36c

@maurycy maurycy Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A note for the reviewer is that there are two approaches in the stdlib.

pdb does not insert but replace:

sys.path[0] = os.path.dirname(self._target)

Same as trace:

sys.path[0] = os.path.dirname(opts.progname)

While profile does insert:

sys.path.insert(0, os.path.dirname(progname))

… sys.path

``python script.py`` resolves symlinks when computing ``sys.path[0]``, so
a script reached through a symlink (``link.py -> sub/where.py``) imports
modules from the real script's directory.  Apply ``os.path.realpath()``
before taking the directory name, and make sure the result is placed at
the front of ``sys.path`` even if it was already listed.

Add a regression test for a symlinked script.
@hetaozdh
hetaozdh force-pushed the fix-sampling-script-path-main branch from d9d8e40 to c8242dd Compare October 1, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

profiling.sampling: run script.py does not put the script's directory on sys.path, so sibling imports fail

3 participants