Instagram phone
github copilot

Picking the right GitHub Copilot model

GitHub recently released documentation on all the large language models available in GitHub Copilot. The list is ever expanding, especially when you have previews turned on for your account. But its still hard for many people to pick the right model for their task.

Jesse Houwing

• 2 min read
Picking the right GitHub Copilot model
Photo by Daniel Gonzalez / Unsplash

You can find the guidance here:

Choosing the right AI model for your task - GitHub Docs
Compare available AI models in Copilot Chat and choose the best model for your task.
GitHub Docs

I wanted to make model selection even easier, so I asked GitHub Copilot to read the doc for me and suggest the right model:

#fetch https://docs.github.com/en/copilot/using-github-copilot/ai-models/choosing-the-right-ai-model-for-your-task

What model would I use to plan a large refactoring involving multiple files for which I do not yet have a working example?

Select the best 2 and explain the pros and cons between them

And it actually came back with a pretty good suggestion:

To plan a large refactoring involving multiple files without a working example, the two best models to consider are GPT-4.5 and GPT-4o. Here's a comparison of their pros and cons:

GPT-4.5

Pros:

  • Deep Reasoning and Contextual Understanding: GPT-4.5 excels at handling complex logic and multi-step reasoning, making it ideal for large-scale refactoring tasks.
  • Cross-File Context: It can retain and process context across multiple files, which is crucial for understanding dependencies and relationships in a large codebase.
  • Reliable and Consistent Results: Produces high-quality, structured suggestions for improving maintainability and modularity.
  • Nuanced Problem-Solving: Handles ambiguous or incomplete requirements effectively, which is helpful when no working example exists.

Cons:

  • Higher Latency: Slightly slower response times compared to smaller models like GPT-4o.
  • Higher Cost: More resource-intensive, which may not be ideal for cost-sensitive scenarios.

GPT-4o

Pros:

  • Faster Response Times: Provides quicker iterations, which is beneficial for rapid back-and-forth during refactoring.
  • Balanced Performance: Offers a good mix of reasoning and speed, making it suitable for lightweight tasks or smaller refactoring steps.
  • Cost-Effective: Less expensive than GPT-4.5, making it a better choice for budget-conscious projects.

Cons:

  • Limited Depth for Complex Tasks: While capable, it may struggle with deeply nuanced or highly interconnected refactoring tasks compared to GPT-4.5.
  • Reduced Cross-File Context: May not handle dependencies across multiple files as effectively as GPT-4.5.

Recommendation:

  • Use GPT-4.5 if the refactoring involves complex logic, dependencies across multiple files, or requires high-quality, structured suggestions.
  • Use GPT-4o if you need faster iterations, are working on smaller or less complex refactoring tasks, or are constrained by cost.

Both models are strong candidates, but the choice depends on the complexity of the refactoring and your priorities (speed vs. depth).