GitHub Copilot can now interact with desktop apps. It can read content, click controls, enter text, press keys, scroll and drag. This works in Copilot CLI and the Copilot app. Copilot asks for approval before controlling an app. Users can review or reset always-allowed apps. Organization admins can disable the feature. It is in public preview on macOS and Windows. Users enable it via commands or settings. It works best with clear outcome descriptions and app constraints.
What changed
Copilot could only interact with code, terminals, and supported integrations.
What it unlocks
Automating workflows in desktop apps without APIs, CLIs, or MCP support.
What you need to act on it
- public preview access
- macOS or Windows
- Accessibility and Screen Recording permissions on macOS
- user approval for app control
Sources