Skip to content

feat: add session-profiling protocol and profile-session template per…#72

Draft
themechbro wants to merge 2 commits intomicrosoft:mainfrom
themechbro:feat/session-profiler-template
Draft

feat: add session-profiling protocol and profile-session template per…#72
themechbro wants to merge 2 commits intomicrosoft:mainfrom
themechbro:feat/session-profiler-template

Conversation

@themechbro
Copy link

This is a fresh draft for the Session Profiler Template (Issue #44), scoped strictly to the template architecture as discussed in the previous PR.

Changes in this Draft:

  • Added protocols/reasoning/session-profiling.md to define the analysis methodology.
  • Added templates/profile-session.md as the execution layer.
  • Made minimal, additive-only updates to manifest.yaml (placed under the investigation category).
  • Ensured all files comply with CONTRIBUTING.md standards (SPDX headers, YAML frontmatter, root directory placement).

@Alan-Jowett - Tagging you here for early feedback as requested! Let me know if the directory placement and YAML frontmatter look perfectly aligned with the engine's expectations before I build out the markdown/JSON formatting logic.

Alan-Jowett pushed a commit to Alan-Jowett/PromptKit that referenced this pull request Mar 24, 2026
Add profile-session template and session-profiling reasoning protocol
for analyzing completed LLM session logs to identify token inefficiencies
and structural waste.

Built on the structure from PR microsoft#72 by @themechbro — keeps the 5-phase
methodology and adds full PromptKit conventions:

Protocol (session-profiling):
- 5 phases: segment log, map to components, detect inefficiencies,
  quantify impact, produce recommendations
- 7 inefficiency types: redundant reasoning, false starts, re-derivation,
  protocol loops, unused context, verbose compliance, persona drift
- Each recommendation tied to a specific PromptKit component file

Template (profile-session):
- Full frontmatter: persona (specification-analyst), protocols
  (anti-hallucination, self-verification, session-profiling),
  format (investigation-report), params, contracts
- Params: session_log, assembled_prompt, focus_areas
- Non-goals: not a quality audit, not a guardrail remover

Manifest: additive-only changes (protocol + template entries, no
reformatting of existing content).

Co-authored-by: themechbro <109350438+themechbro@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add profile-session template and session-profiling reasoning protocol
for analyzing completed LLM session logs to identify token inefficiencies
and structural waste.

Protocol (session-profiling):
- 5 phases: segment log, map to components, detect inefficiencies,
  quantify impact, produce recommendations
- 7 inefficiency types: redundant reasoning, false starts, re-derivation,
  protocol loops, unused context, verbose compliance, persona drift
- Each recommendation tied to a specific PromptKit component file

Template (profile-session):
- Full frontmatter: persona (specification-analyst), protocols
  (anti-hallucination, self-verification, session-profiling),
  format (investigation-report), params, contracts
- Params: session_log, assembled_prompt, focus_areas
- Non-goals: not a quality audit, not a guardrail remover

Manifest: additive-only changes (protocol + template entries, no
reformatting of existing content).

Co-authored-by: Alan-Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Alan-Jowett Alan-Jowett force-pushed the feat/session-profiler-template branch from b20c0d9 to eeabaca Compare March 24, 2026 16:26
@Alan-Jowett
Copy link
Member

@themechbro — I pushed a co-engineering commit to your branch that builds on your 5-phase structure. Here's what changed:

What I kept from your PR

  • The 5-phase methodology (segment → map → detect → recommend) — this was the right structure
  • The inefficiency categories (redundant reasoning, false starts, protocol loops, unused context)
  • The file locations (protocols/reasoning/session-profiling.md, templates/profile-session.md)

What I added

Protocol — expanded each phase with the depth PromptKit protocols need (comparable to traceability-audit.md):

  • Concrete instructions for each phase (what to extract, how to record it)
  • 7 classified inefficiency types with IDs (RE-REDUNDANT, RE-BACKTRACK, etc.)
  • Phase 4 (quantify impact) added — token cost estimation and ranking
  • Recommendations must name the specific component file to change

Template — added full PromptKit frontmatter so the assembly engine can compose it:

  • persona: specification-analyst (systematic, evidence-driven — fits profiling)
  • protocols: [anti-hallucination, self-verification, session-profiling]
  • format: investigation-report (F-NNN findings with severity)
  • params with {{placeholder}} substitution
  • input_contract / output_contract
  • Non-goals section (not a quality audit, won't recommend removing guardrails)

Manifest — reverted the whitespace reformatting and kept only the 2 additive entries (protocol + template). The reformatting was causing 12 validation failures.

For reference

Compare templates/investigate-bug.md for the template pattern, and protocols/reasoning/traceability-audit.md for the protocol depth we target.

Let me know if you have questions or want to adjust anything!

@Alan-Jowett
Copy link
Member

@themechbro — stepping back, I want to apologize. You asked for early feedback on your draft, and instead of reviewing and letting you iterate, I force-pushed a rewrite directly to your branch. That wasn't the right call — it's your PR and your learning opportunity, and I should have respected that.

If you'd prefer to revert to your original commit and work from review feedback instead, that's completely fine. You can restore your version with:

git reflog  # find your original commit hash
git reset --hard <your-original-hash>
git push --force

Either way — your original structure (5 phases, the inefficiency categories, the file locations) was the right foundation. The main things that need to happen for the assembly engine to work with it are:

  1. Template frontmatter needs persona, protocols, format, params, input_contract, output_contract — see templates/investigate-bug.md as the reference pattern
  2. Protocol frontmatter needs type: reasoning and applicable_to — see protocols/reasoning/traceability-audit.md
  3. Manifest changes should be additive only — just the 2 new entries, no reformatting of existing lines (the YAML style changes were causing validation failures)
  4. SPDX headers at the top of both files
  5. {{param}} placeholders in the template body so the assembly engine can substitute user inputs

Happy to answer questions or do a more detailed review — whichever you'd find more helpful. Again, sorry for overstepping.

@themechbro
Copy link
Author

@Alan-Jowett - I've pulled down your co-engineering commit and applied the fixes flagged by the Copilot review.

Changes included in this push:

  • Fixed the YAML frontmatter indentation in profile-session.md that was causing parsing risks.
  • Added the explicit severity levels to the instruction set.
  • Rephrased the Phase 4 protocol to focus purely on "efficiency tradeoffs" rather than output quality.
  • Appended the missing CONTRIBUTING.md Quality Checklist to the bottom of the template.

Let me know if this is ready to drop the "Draft" status!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants