Skip to content

Fix: NotebookProgressCallback crash when evaluating with the Trainer#44949

Open
Charly21r wants to merge 3 commits intohuggingface:mainfrom
Charly21r:fix-evaluate-after-train
Open

Fix: NotebookProgressCallback crash when evaluating with the Trainer#44949
Charly21r wants to merge 3 commits intohuggingface:mainfrom
Charly21r:fix-evaluate-after-train

Conversation

@Charly21r
Copy link

@Charly21r Charly21r commented Mar 23, 2026

What does this PR do?

Fixes #44936

This PR fixes an issue with NotebookProgressCallback in the Trainer where calling evaluate() before or after training would crash due to the training tracker being None. The callback now properly handles evaluation even if training has not yet started or if it has already finished, ensuring metrics can be computed and displayed.

Previously, the on_evaluate method assumed that self.training_tracker was always initialized, but:

  • Before training: self.training_tracker has not being initialised by on_train_begin yet.
  • After training: on_train_end sets self.training_tracker to None, so calling on_evaluate afterwards would fail.

Fix: on_evaluate now checks whether self.training_tracker exists before using it, and safely handles cases where it is None. This prevents crashes and ensures evaluation can run regardless of training state.

Additionally, new unit tests were added to ensure that evaluation works in this scenario, and existing notebook callback tests were updated to cover this case. This improves robustness of notebook-based workflows, especially in Jupyter or Colab environments.

Code Agent Policy

  • I confirm that this is not a pure code agent PR.

Before submitting

Who can review?

@SunMarc

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Charly21r Charly21r force-pushed the fix-evaluate-after-train branch from e61e591 to e25f2a6 Compare March 23, 2026 17:20
@SunMarc
Copy link
Member

SunMarc commented Mar 24, 2026

does this fix your issue @HenrikEilers ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

trainer.evaluate() fails after trainer.train()

2 participants