Skip to content

fix: refuse restart after stop() - #1336

Open
davidberenstein1957 wants to merge 3 commits into
masterfrom
fix/stop-idempotent
Open

davidberenstein1957 wants to merge 3 commits into
masterfrom
fix/stop-idempotent

Conversation

@davidberenstein1957

@davidberenstein1957 davidberenstein1957 commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Stop idempotency itself (a repeat stop() writing only one CSV row) is covered by the already-merged #1408. This PR adds a _stopped guard, set by stop(), and checked by both start() and start_task(), so calling either after stop() logs the same clear "already stopped, cannot be restarted" error instead of running into whatever the now-torn-down state produces. On master, start_task() after stop() hits self._scheduler_monitor_power.start() on None, since _scheduler is None post-stop, and raises AttributeError; start() after stop() was not previously guarded either. _stopped is initialized in _initialize_runtime_state() and only ever set in stop().

Also adds a regression test, test_stop_releases_the_lock_only_once, covering that a repeat stop() does not release the lock a second time (the lock by then may belong to a different tracker instance).

Related Issue

None. Stop idempotency itself (one CSV row on a double stop()) is covered by the already-merged #1408.

Motivation and Context

start() and start_task() are two separate entry points that a caller can invoke after stop(). Before this PR, neither refused to run after a stop, so a caller could hit stale or torn-down state; in the case of start_task(), that meant an unrelated AttributeError from calling .start() on the now-None _scheduler_monitor_power, instead of a clear error saying the tracker cannot be restarted.

How Has This Been Tested?

tests/test_emissions_tracker.py::TestRestartAfterStop::test_start_after_stop_is_refused calls start() after stop() and asserts the "cannot be restarted" error is logged.

tests/test_emissions_tracker.py::TestRestartAfterStop::test_start_task_after_stop_is_refused calls start_task() after stop() and asserts the same error is logged, with no active task started, instead of an AttributeError. It fails on master (AttributeError: 'NoneType' object has no attribute 'start') and passes with this change.

tests/test_emissions_tracker.py::TestCarbonTracker::test_stop_releases_the_lock_only_once asserts Lock.release() is called exactly once across two stop() calls.

Full file: 38 passed (tests/test_emissions_tracker.py + tests/test_offline_emissions_tracker.py).

Screenshots (if appropriate):

N/A

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

AI Usage Disclosure

  • 🟥 AI-vibecoded
  • 🟠 AI-generated
  • ⭐ AI-assisted
  • ♻️ No AI used

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the docs/how-to/contributing.md document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@codecov

codecov Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.78%. Comparing base (e5e46ab) to head (044c0d6).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1336      +/-   ##
==========================================
+ Coverage   91.70%   91.78%   +0.08%     
==========================================
  Files          49       49              
  Lines        5157     5160       +3     
==========================================
+ Hits         4729     4736       +7     
+ Misses        428      424       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 12, 2026 19:14
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code owner August 12, 2026 19:14
stop() used the schedulers as its state flag, so a second call re-ran the
final measurement, wrote a second row and released the lock twice -- by
then the lock may already belong to another tracker.

Add `_stopped_at` as the single state flag: `_start_time is None` means
never started, `_stopped_at is not None` means stopped. A second stop()
returns the memoised emissions; a start() after stop() is refused with an
error instead of half-restarting a tracker whose output handlers, lock and
schedulers are already gone.

Folds in #1337, which inferred the same state from `self._scheduler`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
davidberenstein1957 added a commit that referenced this pull request Aug 19, 2026
Handlers were installed before acquire() could fail. On the "another
instance is already running" path acquire() raises, the tracker sets
_another_instance_already_running and stop() returns at its early guard
without ever reaching release() -- so the host application's SIGINT and
SIGTERM stayed hijacked for the life of the process.

Install them after open(LOCKFILE, "x") succeeds instead, and drop the
_atexit_hook indirection: atexit.unregister() compares with ==, not
identity, so a bound method unregisters fine.

Also make release() idempotent (moved here from #1336): it edits the same
few lines of release() this branch already rewrites.

The deadlock test now unregisters its atexit hook, so a reverted lock.py
fails the suite instead of wedging the interpreter at exit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
davidberenstein1957 added a commit that referenced this pull request Aug 20, 2026
`Lock` installed SIGINT/SIGTERM handlers and threw away the previous ones, so
the host application's handlers were destroyed and Ctrl-C stopped raising
KeyboardInterrupt. Save the previous handlers, chain to them from
`_handle_exit`, and restore them in `release()`, unregistering the atexit hook
so a released lock is not pinned.

Handlers are installed in `acquire()` after `open(LOCKFILE, "x")` succeeds,
not in `__init__`. On the "another instance is already running" path
`acquire()` raises, the tracker sets `_another_instance_already_running`, and
`stop()` returns at its early guard without ever reaching `release()` -- so
handlers installed in the constructor stayed hijacked for the life of the
process.

The thread lock is reentrant: `_handle_exit` calls `release()`, which takes
`_thread_lock`, so a signal delivered while the same thread was inside
`acquire()`/`release()` deadlocked on a plain `Lock`. `release()` is also
idempotent now (moved here from #1336, since it edits the same few lines this
branch already rewrites), and the `_atexit_hook` indirection is dropped:
`atexit.unregister()` compares with `==`, not identity, so a bound method
unregisters fine.

Tests cover the default and ignored signal dispositions, and the deadlock test
unregisters its atexit hook so a reverted lock.py fails the suite instead of
wedging the interpreter at exit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@benoit-cty

Copy link
Copy Markdown
Contributor

🤖 This review comment was written and posted by Claude Opus 5.5 (AI assistant), at the request of @benoit-cty. Findings were checked by reading the code and running tests locally (merged with current master where relevant), but please double-check before acting on them.

Verdict: 🔧 Request changes (rebase down, or close)

Mostly superseded by #1408 (8171d13, now on master). stop() already sets _start_time = None and returns the cached final_emissions, and issue #1307 is closed. The only new behaviour left here is "refuse to restart after stop".

Must fix:

  1. Conflicts with master, and the remaining logic won't work after a rebase.

    • The _stopped_at guard duplicates fix: make tracker.stop() idempotent #1408, and conflicts with it in stop() (emissions_tracker.py ~L911-925).
    • The restart refusal sits inside if self._start_time is not None: in start(). After fix: make tracker.stop() idempotent #1408 that value is None once stopped, so the refusal would never fire.
    • If you keep this PR, rebase it down to the restart refusal only, and use a dedicated flag (e.g. self._stopped = True set in stop()).
  2. Behaviour choice to confirm with maintainers. On master today, start → stop → start → stop "works", noisily:

    • The second start() logs a suppressed AttributeError: 'NoneType' object has no attribute 'start', because the scheduler is None.
    • The second stop() writes a second CSV row whose energy includes the first run's.

    With this PR, the second run writes nothing and only logs an error. Refusing is defensible, but the error message should say so clearly and suggest creating a new tracker. The alternative is to support restarting properly: re-create the scheduler and reset the counters.

Nit:

Merge origin/master and reduce the PR to the part #1408 does not cover:
a dedicated `_stopped` flag set in stop() makes start() log an error and
return, instead of half-restarting with a None scheduler. Keep the
lock-release-once regression test; drop the idempotent-stop test that
duplicates #1408.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@davidberenstein1957

Copy link
Copy Markdown
Collaborator Author

Made the changes in ee7ff84: merged master, reduced to the restart refusal only.

Refusing vs. properly supporting restart is still your call.

@davidberenstein1957 davidberenstein1957 changed the title fix: make tracker.stop() idempotent fix: refuse restart after stop() Sep 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants