Google DeepMind researchers have introduced a new type of AI-powered cursor concept. This technology transforms the simple act of pointing with a mouse into a fully interactive dialogue. Instead of writing long text prompts, users can simply point at an object and give short commands like "compare this," "explain," or "move here." This is reported by Ixbt.com reports .
According to company experts, the principle of interacting with a computer has remained almost unchanged for the last half-century, with the cursor serving only to mark coordinates on the screen. DeepMind proposes teaching the system not just to track cursor location, but to understand what the user is looking at and their intent. This approach allows AI to be seamlessly integrated into daily interface workflows.
The new concept relies on four key principles: freeing the user from switching to separate chatbots, automatic visual context awareness, the use of natural language, and transforming interface elements from mere pixel collections into "meaningful objects" that AI can interact with directly. Modern multimodal models like Gemini are advanced enough to implement this mechanism.
Google is gradually integrating this technology into its products. For example, in the Google Chrome browser, the ability to ask questions about specific elements on web pages using Gemini has appeared. The company also announced the Magic Pointer feature for future Googlebook laptops. These changes are an important step toward adapting the PC interface to the era of generative AI.
Read “Zamin” on Telegram!
