Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 26, 2026

The CLI agent fails in msbench evaluation environments where authentication is handled server-side by a CAPI proxy. The agent validates tokens client-side via getAuthInfo() before the SDK respects proxy URL overrides (overrideCapiUrl, overrideProxyUrl), causing failures when no client-side token exists.

Changes

  • Modified CopilotCLISDK.getAuthInfo() to check for proxy URL configuration before attempting token validation

    • When overrideProxyUrl or overrideCapiUrl is set, return empty token and defer to proxy
    • Otherwise, proceed with normal GitHub session authentication flow
  • Added IConfigurationService dependency to CopilotCLISDK to access proxy configuration settings

public async getAuthInfo(): Promise<NonNullable<SessionOptions['authInfo']>> {
    // Check if proxy URLs are configured - if so, skip client-side token validation
    // as the proxy will handle authentication server-side
    const overrideProxyUrl = this.configurationService.getConfig(ConfigKey.Shared.DebugOverrideProxyUrl);
    const overrideCapiUrl = this.configurationService.getConfig(ConfigKey.Shared.DebugOverrideCAPIUrl);

    if (overrideProxyUrl || overrideCapiUrl) {
        this.logService.info('[CopilotCLISession] Proxy URL configured, skipping client-side token validation');
        return {
            type: 'token',
            token: '',
            host: 'https://github.com'
        };
    }

    const copilotToken = await this.authentService.getGitHubSession('any', { silent: true });
    return {
        type: 'token',
        token: copilotToken?.accessToken ?? '',
        host: 'https://github.com'
    };
}

Test Coverage

Added unit tests covering:

  • Proxy URL bypass behavior
  • CAPI URL bypass behavior
  • Normal authentication flow
  • Fallback when authentication unavailable
Original prompt

Summary for vscode-copilot-chat Change
The Problem
When using the CLI/background agent (openAgentInBackground: true) in the msbench evaluation environment, the agent fails with:

This happens because:

The evaluation harness configures a CAPI proxy that handles authentication server-side
The harness sets VS Code settings to route requests through this proxy:
github.copilot.advanced.debug.overrideCapiUrl
github.copilot.advanced.debug.overrideProxyUrl
But the CLI agent's SDK calls getToken() to validate auth before any network request is made
Since no client-side token exists (proxy handles auth), this early validation fails
The Root Cause
In the Copilot SDK (@github/copilot/sdk), the flow for the CLI agent appears to be:

The auth check happens before the SDK respects the proxy URL override settings.

Proposed Fix in vscode-copilot-chat
The CLI agent should skip the client-side token validation when proxy URLs are configured. The proxy will handle authentication.

Option A: Check if overrideCapiUrl/overrideProxyUrl settings are present before calling getToken():

Option B: Make getToken() failures non-fatal when proxy is configured - allow the request to proceed and let the proxy handle it.

Why This Matters
The regular chat agent works fine because it uses VS Code's built-in Copilot auth flow
The CLI/background agent (copilotcli) has its own auth path that doesn't respect proxy settings early enough
This blocks testing background agent functionality in msbench CI environments

The user has attached the following file paths as relevant context:

  • .github\copilot-instructions.md

Created from VS Code.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits January 26, 2026 18:29
Co-authored-by: amunger <2019016+amunger@users.noreply.github.com>
Co-authored-by: amunger <2019016+amunger@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix client-side token validation in CLI agent Skip client-side auth validation when proxy URLs are configured Jan 26, 2026
Copilot AI requested a review from amunger January 26, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants