Integration test failure: azure-deploy – containerized web application Terraform [Assertion mismatch]

[github-actions]

Prompt

Create a containerized web application and deploy to Azure Container Apps using Terraform infrastructure in my current subscription in swedencentral region.

Summary

Run: Integration Tests - all #112
Test: azure-deploy_ - Integration Tests › terraform-azure-container-apps-deploy › creates containerized web application with Terraform
Result: Fail
Duration: 16m 50s

Root Cause Category

Assertion mismatch

Diagnosis

What was expected

The test expects containsDeployLinks to be true — the agent's response must include deployment endpoint URLs. It also expects Terraform infrastructure files to be created (infra/*.tf, no .bicep files).

What actually happened

The agent completed within the timeout (1009891 ms ≈ 16 min 50s) but containsDeployLinks returned false. The assertion at integration.test.ts:575 failed:

Expected: true
Received: false
expect(containsDeployLinks).toBe(true);

Note: The same test passed in the combined azure-container-apps-deploy job (1312615 ms ≈ 21 min 53s), indicating intermittent behavior for this test case.

Why it failed

The agent deployed to Azure Container Apps with Terraform in swedencentral but did not surface the deployment URL in its response. This could mean: (1) deployment succeeded but deploy links weren't included in the output, (2) deployment failed partway through and the agent completed without links, or (3) the deploy link format wasn't recognized.

Suggested fix

• Review agent-metadata to determine if azure-deploy was invoked and whether deployment links were generated
• Check if the Container Apps URL output format from Terraform differs from expected patterns in hasDeployLinks()
• Verify that ACA app URLs are included in the azure-deploy skill output

azure-deploy Skill Invocation

Skill	Invoked
azure-deploy	Unknown (artifact download unavailable in sandbox)

In azure-deploy integration tests, the full skill chain is azure-prepare → azure-validate → azure-deploy. The test completed without timeout, suggesting the chain ran but deploy links were not surfaced in the output.

Skill Report Context

Per-test section from SKILL-REPORT.md (click to expand)

Artifact ZIP download was blocked by the AWF sandbox network firewall. Skill report content unavailable for this run. See the workflow run artifacts for full details.

Environment

• Runner OS: ubuntu-latest
• Node.js: (from workflow)
• Model: claude-sonnet-4.5
• Run URL: https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23479553631
• Commit: 482be0e

Generated by Analyze Test Run · ◷

Warning

⚠️ Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• productionresultssa15.blob.core.windows.net

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "productionresultssa15.blob.core.windows.net"

See Network Configuration for more information.

[JasonYeMSFT]

Agent hit a resource group name collision and stopped asking user for next step. This is the right behavior but it's not friendly for automated testing. We should consider adding instructions to tell the agent to use psudo-random values in resource group names.