Utilizing ‘Click to Do’ for AI-Driven Actions on Text and Images in Windows 11

Utilizing ‘Click to Do’ for AI-Driven Actions on Text and Images in Windows 11

Unlocking the Power of the Click to Do Feature on Windows 11

Windows 11 introduces an innovative AI feature known as “Click to Do”, crafted to enhance your productivity by providing swift actions tailored to content on your screen. This feature intelligently analyzes the text and images you select, presenting a range of contextual actions that enhance your workflow. In this detailed guide, we’ll explore how to enable and effectively utilize this feature on your Copilot+ PC for a seamless user experience.

What is Click to Do?

“Click to Do” is a sophisticated tool within Windows 11 that streamlines your interaction with on-screen content. Designed to function alongside the Windows Recall feature, it takes a local screenshot and employs AI to recognize actionable text and images, maximizing your efficiency. However, it’s worth noting that it does not interpret the content of the text or images.

It recognizes only the applications visible on your screen, omitting any that are minimized. Rest assured, all analysis occurs locally, keeping your privacy intact, as data is only sent to the internet when necessary actions, such as web searches or inquiries via Copilot, are initiated.

It’s important to be aware that the actions you can leverage through Click to Do depend on the specific elements selected, your device’s capabilities, and regional availability. Currently, this feature is accessible through the Windows Insider Program’s Release Preview, Beta, and Dev Channels, with a full rollout anticipated with the Windows 11 May 2025 Update.

System Requirements for Click to Do

The Click to Do feature is exclusive to Copilot+ PCs equipped with a Neural Processing Unit (NPU) capable of over 40 TOPS, alongside essential specs that include:

  • A minimum of 16GB of RAM
  • Eight-core processor
  • At least 256GB of SSD storage

This functionality is supported exclusively on cutting-edge processors from Qualcomm, such as the Snapdragon X series, and the latest AI chips from AMD and Intel, all of which meet the NPU criteria. Regular PCs will not support this feature.

Getting Started: Using Click to Do on Windows 11

Before engaging with Click to Do, ensure you manually confirm and enable the feature if necessary. Once activated, you can harness Click to Do to execute various screen-based actions effectively.

Enabling Click to Do

To enable or disable Click to Do on your Copilot+ PC, simply follow these steps:

  1. Launch the Settings app on Windows 11.

  2. Select Privacy & Security.

  3. Navigate to the Click to Do section.

    Windows 11 open Click to Do settings

  4. To enable the feature, toggle the “Click to Do” switch to the on position.

    Enable Click to Do

  5. If you wish to disable it, toggle the “Click to Do” switch to the off position.

    Disable Click to Do

After completing these steps, you’ll be ready to start using Click to Do for various actions informed by on-screen content.

Accessing Click to Do Outside Recall

There are several convenient methods to access the Click to Do feature:

  • Locate it within the Start menu as a dedicated app.
  • Utilize the Windows key + mouse click keyboard shortcut.
  • Press Windows key + Q to launch it directly.
  • Open Click to Do from the Snipping Tool’s toolbar.

Once you launch the app, an animation will appear, and a toolbar will manifest at the top of your screen, providing a search box for natural language input. You also have the option to execute searches using voice commands. Additionally, the “Recall” button allows you to access snapshots using other innovative AI tools.

Accessing Click to Do Within Recall

To utilize Click to Do from within the Recall app, simply do the following:

  1. Open the Start menu.

  2. Search for and select Recall from the results.

  3. Select the desired snapshot.

  4. Enable the “Show Click to Do” button if it isn’t already activated.

    Recall Show Click to Do option

After these steps, you can select text or images and unlock the Click to Do actions.

Interacting with Text Using Click to Do

Upon selecting text within Click to Do, you’ll be presented with a variety of options:

  • Copy: Duplicates the selected text to your clipboard.
  • Open with: Launches the chosen text in a compatible text editor.
  • Search the web: Quickly searches for the selected text using Microsoft Edge and Bing.
  • Ask Copilot: Directly sends the text to the Copilot app’s text prompt.
  • Summarize: Produces a concise summary of the selected text.
  • Create a bulleted list: Converts the selected text into a bulleted format.
  • Rewrite: Alters the tone or format of the selected text, customizing it to your preference.

Additionally, if you select a web link, it automatically converts into a clickable format. Right-clicking this link allows access to the “Open website” option, facilitating navigation.

Click to Do with text selection actions
Click to Do with text selection actions / Image: Mauro Huculak

The Summarize, Create a bulleted list, and Rewrite functionalities become available when selecting ten or more words, necessitating your sign-in to a Microsoft account to utilize AI tokens for these features.

Click to Do with text selection rewrite options
Click to Do with text selection rewrite options / Image: Mauro Huculak

Interacting with Images Using Click to Do

When working with images in Click to Do, you’ll have a variety of tools at your disposal:

  • Copy: Copies the selected image to your clipboard.
  • Save as: Downloads the image to your computer.
  • Share: Opens the interface for sharing files.
  • Open with: Launches the image in apps like Paint, Snipping Tool, or Photos.
  • Ask Copilot: Sends the image to the Copilot app text prompt.
  • Visual search with Bing: Executes a visual search based on the image.
  • Blur background with Photos: Allows manipulating the image background using the Photos app.
  • Erase objects with Photos: Lets you remove unwanted items in the image with Photos.
  • Remove background with Paint: Facilitates background removal for images using Paint.

For those missing options, ensure your device has the latest updates for the Photos and Paint applications to access the full suite of features.

Click to Do image selection actions
Click to Do image selection actions / Image: Mauro Huculak

Understanding Click to Do’s Privacy Measures

Your privacy is a top priority when utilizing Click to Do. The initial AI processing is conducted locally, ensuring no data is shared with Microsoft during analysis. If you opt to send images from Click to Do to an app, a temporary file is created in the %HOMEPATH%\AppData\Local\Temp directory, which is deleted once the task is complete.

The Summarize, Create a bulleted list, and Rewrite features utilize a small language model known as Phi Silica, which operates entirely on your machine. However, should you decide to search the web, open a website, or conduct a visual search using Bing, your data will be transmitted to the cloud. Similarly, the “Ask Copilot” option ensures that while the text or image remains secure, information may still be processed once submitted to Copilot.

Frequently Asked Questions

Frequently Asked Questions

1. What devices support the Click to Do feature?

The Click to Do feature is exclusive to Copilot+ PCs with specific hardware requirements, such as a Neural Processing Unit (NPU) with over 40 TOPS and advanced processors from Qualcomm, AMD, or Intel.

2. Is my data safe while using Click to Do?

Your initial data analysis happens locally on your device, ensuring privacy. Data is only sent to the cloud when necessary features, like web searching, require internet access.

3. Can I use Click to Do without signing in to a Microsoft account?

While basic functionality may be available, using advanced options such as Summarization and Rewrite requires a Microsoft account sign-in due to the use of AI tokens.

Source & Images

Leave a Reply

Your email address will not be published. Required fields are marked *