Add Xtab275EditIntent prompting strategy with edit intent filtering #3181
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a new prompting strategy that extends Xtab275 with edit intent tag parsing. The model returns an edit_intent tag indicating confidence level (no_edit, low, medium, high), which is then filtered against the user's aggressiveness level. Filtering logic: - no_edit: Never show - low intent: Show for all aggressiveness levels - medium intent: Show for low/medium aggressiveness - high intent: Only show for low aggressiveness Changes: - Add EditIntent enum and namespace with parsing and filtering logic - Add EditWindowWithEditIntent response format - Add Xtab275EditIntent prompting strategy - Implement parseEditIntentFromStream in xtabProvider - Add editIntent telemetry field - Add comprehensive unit tests for EditIntent Co-authored-by: GitHub Copilot <noreply@github.com>
…parsing the rest of the response
…t short-circuits to skip the rest of the stream
Member
Author
Manual testUser config "github.copilot.chat.advanced.inlineEdits.xtabProvider.modelConfiguration": {
"modelName": "copilot-nes-oct",
"promptingStrategy": "xtab275EditIntent",
"includeTagsInCurrentFile": false,
},Traces from manual test# NoEdit (filtered out -- rough speedup from 1947ms to 311-426 ms, assuming NoEdit would consume an equal number of tokens and not accounting for speculative decoding)Medium confidence + Medium aggressiveness (applied)Low confidence + Med aggressiveness (filtered out) |
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds a new prompting strategy Xtab275EditIntent that extends the Xtab275 strategy with edit intent tag parsing and filtering. The model returns an <|edit_intent|> tag indicating its confidence level (no_edit, low, medium, high), which is then filtered against the user's aggressiveness level to determine whether to show the edit suggestion.
Changes:
- Introduces
EditIntentenum and namespace with parsing logic and confidence-based filtering - Adds
EditWindowWithEditIntentresponse format for the new prompting strategy - Implements
parseEditIntentFromStreamfunction to extract edit intent tags from model responses - Extends telemetry with
editIntentandeditIntentParseErrorfields - Provides comprehensive unit tests covering all edge cases and integration scenarios
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/platform/inlineEdits/common/dataTypes/xtabPromptOptions.ts |
Adds EditIntent enum with filtering logic and Xtab275EditIntent prompting strategy |
src/platform/inlineEdits/common/statelessNextEditProvider.ts |
Adds telemetry fields for edit intent and parse errors |
src/extension/xtab/node/xtabProvider.ts |
Implements parseEditIntentFromStream parsing function and integrates filtering logic into provider |
src/extension/xtab/common/promptCrafting.ts |
Updates postscript generation to handle Xtab275EditIntent strategy |
src/extension/xtab/test/node/editIntent.spec.ts |
Adds comprehensive unit tests for all edit intent functionality |
src/platform/inlineEdits/common/inlineEditLogContext.ts |
Fixes trailing comma for consistency |
…ptions.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…/vscode-copilot-chat into ben/edit-intent-parsing
Member
Author
|
CC @ulugbekna, could you please review this PR? 🙏 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a new prompting strategy that extends Xtab275 with edit intent tag parsing. The model returns an edit_intent tag indicating confidence level (no_edit, low, medium, high), which is then filtered against the user's aggressiveness level.
See Personalized XTab: edit intent (#3144) for details. This PR is required in order to flight the edit intent model.
Model can output tags (
<|edit_intent|>no_edit|low|medium|high<|/edit_intent|>) or short names (N|L|M|H) to save tokens, which will be parsed strictly from the first line of the response. The first model we've trained will output tags.Changes: