Add callback_source to context to differentiate DAG vs task callbacks#61643
Add callback_source to context to differentiate DAG vs task callbacks#61643vasanthrpjan1-boop wants to merge 1 commit intoapache:mainfrom
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
|
|
Amazing! Exactly the thing I was planning to do. |
|
Maybe it's worth wrapping What if we want to add more meta information, like the |
| class Context(TypedDict, total=False): | ||
| """Jinja2 template context for task rendering.""" | ||
|
|
||
| callback_source: NotRequired[CallbackSource] |
There was a problem hiding this comment.
Why is it NotRequired? Doesn't every callback have a source?
There was a problem hiding this comment.
Or is it because we inject it just before the execution?
|
@vasanthrpjan1-boop This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack. |
c9fe979 to
26b49f7
Compare
|
@vasanthrpjan1-boop This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack. |
Currently a callback defined at the DAG level receives a context that
is nearly identical to a task-level callback context, making it
impossible to tell which level triggered the call.
This adds a `CallbackSource` enum ("dag" / "task") and injects it as
`context["callback_source"]` in every code path that invokes callbacks:
- task_runner._run_task_state_change_callbacks (in-worker task path)
- processor._execute_dag_callbacks (DAG processor path)
- processor._execute_task_callbacks (DAG processor path)
- DagRun.handle_dag_callback (dag.test() path)
Closes: apache#61119
Co-authored-by: Cursor <cursoragent@cursor.com>
26b49f7 to
7f05f97
Compare
Summary
Adds a
callbackkey to the callback context that carries aCallbackMetanamed tuple, letting shared callbacks tell whether they were triggered at the DAG level or the task level.CallbackSourcestr enum ("dag"/"task") andCallbackMetanamed tuple inairflow.sdk.definitions.contextcontext["callback"] = CallbackMeta(source=...)in all four callback invocation pathsContextTypedDict field isNotRequiredbecause it is only set during callback execution, not during normal task template renderingairflow.sdkfor user accessUsage example
Using a
CallbackMetanamed tuple (instead of a bare enum) makes it easy to add more metadata fields in the future without breaking existing callbacks.Test plan
test_task_callback_context_has_callback_metacovers success and failure task callbackscontext_from_server)Gen-AI disclosure
This PR was developed with the assistance of a generative AI coding tool (Cursor). All code was reviewed, understood, and validated by me before inclusion. I take full responsibility for the changes.
Closes #61119