The AI Tea
At a glance
- What changed
- DeepMind shared interaction principles and demos for an AI-enabled mouse pointer, and says Gemini in Chrome can answer questions about the exact part of a webpage you point to.
- Why it matters
- Most AI interfaces still make users copy context into a chat window. Pointing-and-speaking can keep work inside the app you are already using, reducing prompt friction and making “show me this” requests more reliable.
- Who is affected
- knowledge workers, developers building AI UX, product teams
- What to do next
- Watch what data is captured from the screen, how permissions and privacy controls work across apps, and whether “point-to-context” interactions hold up in messy real-world pages.
What changed
On May 12, 2026, Google DeepMind described an AI-enabled pointer prototype and said Gemini in Chrome can answer questions about the specific part of a webpage you point to.
Why it matters
Most AI interfaces still make users copy context into a chat window. Pointing-and-speaking can keep work inside the app you are already using, reducing prompt friction and making “show me this” requests more reliable.
In plain English
Instead of pasting text into an AI tool, you point at the thing you mean and speak. The system uses that on-screen context to answer or take the next step.
What this means for you
Who is affected: knowledge workers, developers building AI UX, product teams
Next move: Watch what data is captured from the screen, how permissions and privacy controls work across apps, and whether “point-to-context” interactions hold up in messy real-world pages.
- DeepMind outlined four interaction principles: maintain the flow, show and tell, use “this/that” shorthand, and turn pixels into actionable entities.
- The post shows demos like summarizing a PDF into an email, turning a table into a chart, and using pointing plus speech for tasks in Google AI Studio.
- DeepMind says these ideas are being integrated into products, including Gemini in Chrome for page-specific questions and a planned “Magic Pointer” rollout in its new laptop experience.
What remains uncertain
Watch what data is captured from the screen, how permissions and privacy controls work across apps, and whether “point-to-context” interactions hold up in messy real-world pages.