[workload] Remove Workload object

[tkarna]

• The abstract Workload class is removed. Examples continue to use a local class for convenience.
• The runner utilities execute and benchmark now take low-level abstractions as input.
• Runner utilities take optional MemoryManager class type to handle allocations.
• The runner utilities take an optional callback(memory_manager, inputs) function that can be used to copy device buffers to host, for example.
• Lighthouse does not have any support for correctness testing now but the runner api allows adding one on the user side.
• Renamed directories:
  • lighthouse/workload -> lighthouse/execution
  • examples/workload -> examples/execution

The execute/benchmark interface is not very nice at the moment but we can clean it up once we have more concrete use cases, for example, by binding some of the arguments together.

[Copilot]

Pull request overview

This PR removes the high-level Workload abstraction and shifts the execution/benchmarking utilities to operate on lower-level inputs (payload MLIR module + schedule modules), adding optional memory-manager support and callback hooks for device/host transfers. It also renames/relocates workload-related code into lighthouse/execution and updates examples accordingly.

Changes:

• Removed Workload and the old workload/runner.py; introduced lighthouse/execution/runner.py with execute/benchmark taking low-level abstractions.
• Added optional memory-manager integration (GPUMemoryManager/ShardMemoryManager) and callback support in runner APIs.
• Updated examples to the new API and moved “workload” examples to “execution”.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
lighthouse/workload/workload.py	Deletes the abstract Workload base class.
lighthouse/workload/runner.py	Deletes old workload runner utilities in favor of new execution runner.
lighthouse/schedule/xegpu/mlp_schedule.py	Adds a new parameter presence assertion in schedule generation.
lighthouse/execution/runner.py	Adds new low-level execution/benchmark runner with optional memory management + callbacks.
lighthouse/execution/memory_manager.py	Adjusts ShardMemoryManager.get_input_buffers signature/validation.
lighthouse/execution/init.py	Updates exports for the new execution package API.
examples/xegpu/xegpu_workload.py	Removes example-specific XeGPUWorkload base class.
examples/xegpu/tune_matmul_gridsearch.py	Migrates tuning to new benchmarking/execution helpers.
examples/xegpu/mlp.py	Migrates MLP example to new execution API and adds local correctness checking + verbosity flag.
examples/xegpu/matmul.py	Migrates matmul example to new execution API and adds local execute/check + run_benchmark helpers.
examples/xegpu/lit.local.cfg	Removes exclusion for deleted xegpu_workload.py.
examples/workload/example_mlir.py	Removes the old workload-based MLIR allocation example.
examples/feed-forward-mpi/feed-forward-mpi.py	Migrates distributed FF example to new execution API with callback-based correctness flow.
examples/execution/example.py	Updates CPU elementwise example to use new execution runner and performs correctness checking inline.
examples/cpu/x86/matmul.py	Migrates x86 matmul example to new execution runner API and adds inline correctness checking.

Comments suppressed due to low confidence (1)

lighthouse/execution/memory_manager.py:245

• get_input_buffers now gives required parameters default None values and then enforces them via assert. This makes the signature misleading and the checks can be skipped with python -O. Prefer keeping kinds_and_ranks/elem_type required (no defaults) or raising ValueError when they are missing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rengolin]

So, I don't mind it being a bunch of helpers for now. As you say, this needs to evolve organically.

Just checking: the idea is that the "lower_payload" would be either the compiler driver or the tuner driver (so removed from here later) and then this would become a class of its own that both drivers would use to run/benchmark stuff, right?

[rengolin]

Do we need a slightly different example than the two existing matmul / mlp ones (x86, xe)?