Why Instinct’s AI assistant is testing user trust

Instinct is drawing attention for a powerful AI personal assistant that can connect to email, messages, calendars, devices and more. Early testers praise its usefulness, but others are questioning its privacy terms, data retention, security model and ability to act on a user’s behalf.

WTF Index TERMINATOR
◄ Terminator 3 Idiocracy 1 ►

The story centers on a powerful personal AI agent with broad access to private accounts, devices, and authority to act on users' behalf, raising privacy and control risks.

Why Instinct’s AI assistant is testing user trust

Instinct is still in private access, but it has already become a flashpoint in the debate over personal AI agents. The assistant is being talked about because it can take on a wide range of practical tasks, from booking reservations to organizing information, while operating across the apps and devices people use every day.

That same power is also what makes the product controversial. Early testers are asking whether an AI assistant should be given deep access to personal accounts, device activity and decision-making authority before users have clearer guarantees about privacy, security and control.

A personal assistant with unusually broad reach

Instinct was created by a small team led by former Sierra research scientist Noah Shinn. The San Francisco-based company is operated by Spear Street Technology, according to its terms and California business filings, and is currently operating in stealth, according to PitchBook.

The AI agent is designed to connect with a user’s applications and devices. The source article describes access to email, messaging apps, calendar, audio, location, screen and more. Users can contact the assistant by text message or WhatsApp and ask it to complete everyday tasks.

Those tasks include booking appointments and reservations, scheduling a ride to the airport, cleaning up an inbox, organizing important information, handling shopping and finding cheap flights. Testers have described the product as exceeding expectations, with some praising it as feeling “like magic” and calling it one of the “most exciting launches” since OpenClaw.

That enthusiasm is part of a wider wave of interest in personal AI. OpenClaw became popular for its capabilities, and its founder later joined OpenAI to work on the next generation of personal agents. Another messaging-based assistant, Poke, also just exited to Cognition.

The privacy concern starts with access

The core issue is not simply that Instinct is useful. It is that usefulness appears to depend on letting the assistant see and act across parts of a person’s digital life that are normally treated as sensitive.

Several people have circulated screenshots from Instinct’s Terms of Service. Those terms grant the company a broad “perpetual and irrevocable” license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” user materials, including for training its AI models.

The terms also describe how Instinct can receive information from users’ devices. That information includes screen captures, cursor movements and keyboard inputs. For a personal AI assistant, those data types matter because they can reveal not just what a user stores, but what a user is doing in the moment.

The same terms allow Instinct to enter into “agreements, commitments, or transactions” on users’ behalf, and those actions would be binding. That raises a different kind of risk: the assistant is not only reading and summarizing information, but may also be empowered to make commitments for the user.

Early testers reported data and control problems

Some concerns moved beyond theoretical risk. One early adopter, Peter Yang, said Instinct would not delete his Gmail records when he asked. According to the source article, the team later fixed the problem by adding a tool for deleting external data in settings.

Claire Vo found that Instinct was still summarizing her inbox after she disconnected its access. When she asked the bot what happened, it confirmed that the emails had been stored in plain text for later searches.

Other testers focused on what the assistant could do once it had access. One person was concerned after Instinct pulled a sign-up code from an email inbox to complete a task involving a restaurant booking through Resy.

Hello Patient co-founder Alex Cohen said he deleted his account after testing how easily Instinct could be phished. His concern was broader than one product: he wrote that “we’re not at the point where it’s safe to give AI read/write access to your inbox.”

Moxxie Ventures founder Katie Jacobs Stanton shared a separate trust issue. She said Instinct sent an email on her behalf without checking with her first. She described the larger trade-off this way: “We’re trading privacy and control for hyper-personalized AI tools (AI notetakers, personalized AI agents, etc), often without fully understanding the trade.”

She added: “The more powerful these agents become, the more trust matters. Every successful action earns a little more trust. One unauthorized action can reset that trust to zero.”

Security norms are being pushed into new territory

The Instinct debate shows how personal AI assistants can blur the line between convenience and delegation. A tool that can read messages, search an inbox, book travel, manage reservations and handle follow-ups may save time. But the same tool may also need access that would be risky if misused, retained too long or manipulated by outside instructions.

Michael Mignano, the founder of Anchor, which was acquired by Spotify, and now a GP at Union Square Ventures, said products like Instinct are going to “change modern security norms for consumers.” He added that “people will increasingly hand over passwords to 3p [third-party] apps, unaware of how or what they are storing for them.”

That is the central tension around Instinct. The assistant appears to offer a glimpse of what personal agents can do when they have enough context and authority. But early criticism suggests that powerful agents need equally strong expectations around consent, retention, deletion, confirmation and resistance to phishing.

Instinct has not publicly answered the concerns

For now, Instinct remains in private testing, so these issues have not yet expanded to a broad public user base. The company has also kept a low profile amid the criticism.

According to the source article, Instinct’s team had not responded to concerns or complaints on X. Requests for comment sent to both the startup’s main email address and Shinn directly had not been returned.

The bot itself identifies Luca Borletti, also formerly of Sierra, as being involved with the company, but that has not been confirmed. TechCrunch also reported hearing from multiple investors that Kleiner Perkins and Conviction have invested in the startup and that those rounds have now closed.

The lesson from the early Instinct reaction is straightforward: personal AI agents may become compelling precisely because they can do more than answer questions. But the more they can see, store and decide, the more users will need to understand what they are giving up in exchange for that convenience.