[observability] Observability Coverage Report - 2026-03-24

[github-actions[bot]]

Executive Summary

Using the agentic-workflows MCP logs tool with a 7-day window (start_date: -7d), I analyzed all runs returned in the downloaded dataset and inspected run artifacts under /tmp/gh-aw/aw-mcp/logs. The current dataset contained 14 runs, with 12 completed runs carrying agent artifacts and 2 runs still in progress.

Observability coverage for completed agent runs is strong: all firewall-enabled runs included access.log, and all MCP-enabled runs included MCP telemetry via rpc-messages.jsonl (canonical fallback). No run in scope was missing critical observability artifacts.

Overall health is healthy for the analyzed dataset, with one warning-level data quality anomaly in parsed MCP summary metadata (telemetry file present, but zero parsed MCP call entries in one run summary).

Key Alerts and Anomalies

No critical issues detected.

🔴 Critical Issues:

• None.

⚠️ Warnings:

• Run §23517034683 contains rpc-messages.jsonl (telemetry present) but has 0 parsed MCP tool-call entries in run_summary.json; this is a summary quality gap, not a telemetry absence.
• Two runs are currently in_progress and were excluded from firewall/MCP coverage denominators because their artifact sets are not finalized: §23517602275, §23517606316.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	12	12	100.0%	✅
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	12	12	100.0%	✅

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Daily Semgrep Scan	23517572286	✅	26	14	0	✅
Auto-Triage Issues	23517561053	✅	14	9	0	✅
Auto-Triage Issues	23517533736	✅	14	9	0	✅
Auto-Triage Issues	23517451249	✅	14	9	0	✅
Auto-Triage Issues	23517380081	✅	15	9	0	✅
Daily Project Performance Summary Generator (Using MCP Scripts)	23517034683	✅	53	17	0	✅
Daily Safe Output Integrator	23516276215	✅	55	22	0	✅
AI Moderator	23515904639	✅	52	12	3	✅
Daily Secrets Analysis Agent	23514841196	✅	58	23	0	✅
Draft PR Cleanup	23514299027	✅	31	17	0	✅
Daily Safe Outputs Conformance Checker	23514266321	✅	10	4	0	✅
AI Moderator	23514144845	✅	41	11	3	✅

Missing Firewall Logs (access.log)

None.

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Daily Semgrep Scan	23517572286	rpc-messages.jsonl	2	2	2	0	✅
Auto-Triage Issues	23517561053	rpc-messages.jsonl	8	2	8	0	✅
Auto-Triage Issues	23517533736	rpc-messages.jsonl	8	2	8	0	✅
Auto-Triage Issues	23517451249	rpc-messages.jsonl	8	2	8	0	✅
Auto-Triage Issues	23517380081	rpc-messages.jsonl	8	2	8	0	✅
Daily Project Performance Summary Generator (Using MCP Scripts)	23517034683	rpc-messages.jsonl	0	0	0	0	⚠️
Daily Safe Output Integrator	23516276215	rpc-messages.jsonl	1	1	1	0	✅
AI Moderator	23515904639	rpc-messages.jsonl	9	2	9	0	✅
Daily Secrets Analysis Agent	23514841196	rpc-messages.jsonl	6	2	6	0	✅
Draft PR Cleanup	23514299027	rpc-messages.jsonl	3	2	3	0	✅
Daily Safe Outputs Conformance Checker	23514266321	rpc-messages.jsonl	1	1	1	0	✅
AI Moderator	23514144845	rpc-messages.jsonl	6	2	6	0	✅

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

None.

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 383
• Domains accessed: 5 unique
• Blocked requests: 6 (1.57%)
• Most accessed domains: api.githubcopilot.com, api.openai.com, ab.chatgpt.com

Gateway/MCP Telemetry Quality

• Telemetry source observed: rpc-messages.jsonl (canonical fallback)
• Total parsed MCP telemetry entries (from run summaries): 60
• MCP servers used: safeoutputs, github, semgrep
• Total tool calls: 60
• Error rate: 0.0%
• Average response time: N/A (duration fields not available from accessible summary data)

Additional Artifact Coverage

• agent-stdio.log present in 12/12 completed agent runs
• safeoutputs.jsonl present in 12/12 completed agent runs
• aw_info.json present in 14/14 analyzed runs

Healthy Runs Summary

• Runs with complete firewall observability: 12/12 firewall-enabled runs
• Runs with complete MCP telemetry coverage: 12/12 MCP-enabled runs

Recommended Actions

1. Add a post-processing validation check that fails or warns when rpc-messages.jsonl exists but parsed MCP call count is zero in run_summary.json.
2. Add explicit gateway.jsonl generation where feasible to improve structured duration and status analytics; keep rpc-messages.jsonl as fallback.
3. Re-run this report after the two in-progress runs complete to confirm they maintain the same 100% observability coverage.

📊 Historical Trends

The current downloaded dataset (from the MCP logs pull in this run) includes 14 runs, concentrated in the latest execution window. Trend analysis across multiple days is limited by available fetched runs in this snapshot; recommend persisting daily coverage metrics in a dedicated artifact/discussion index for week-over-week comparisons.

Context

• Analysis window requested: last 7 days
• Runs analyzed in downloaded dataset: 14
• Completed runs with full artifact sets: 12
• Generated at: 2026-03-24 (UTC)

References:

• §23517602275
• §23517572286
• §23515904639

AI generated by Daily Observability Report for AWF Firewall and MCP Gateway · history

• expires on Mar 25, 2026, 11:52 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #23002.