Anthropic launches AI Interviewer tool to collect user feedback on how people actually use AI

I used Anthropic's Interviewer tool to share my AI complaints, and enjoyed it - how you can too

Follow ZDNET: Add us as a preferred source on Google. ZDNET's key takeaways * Anthropic's new Interviewer chatbot tool collects user feedback. * The feedback is then given to human researchers for analysis. * The tool reveals insights into what people want from AI. As AI tools become increasingly pervasive, they're also starting to feel like copies of each other rather than being tailored to your actual needs. Anthropic has developed a tool to collect user feedback to learn what you really want out of AI products. The new Anthropic Interviewer tool, a chatbot experience launched Thursday, conducts real-time, adaptive interviews of 10 to 15 minutes with participants. Afterwards, a human researcher collaborates with Anthropic Interviewer's analysis of the transcripts to better understand how people integrate AI tools into their daily workflows. Also: Claude Code made an astonishing $1B in 6 months - and my own AI-coded iPhone app shows why "Millions of people now use AI every day. As a company developing AI systems, we want to know how and why they're doing so, and how it affects them," said Anthropic in the blog post. "We want to use people's feedback to develop better products -- but it's also because understanding people's interactions with AI is one of the great sociological questions of our time." Users can opt in now to participate in the next phase of the research. However, the pilot is only a week long, so you'll want to act quickly. How Anthropic Interviewer works (and how to use it) Before launching Interviewer, Anthropic tested it with a sample of 1,250 professionals. The company used the insights from that test to compile a report with early findings on the use of AI at work, but the ultimate goal was to test the capabilities of Anthropic Interviewer. Anthropic detailed how the tool works, breaking down Interviewer's operations into three stages: planning, interviewing, and analysis. With the first planning step, the tool creates an interview framework that poses consistent research questions across hundreds or thousands of interviews, while accommodating the nuances that come with different interviewers. In its test, Anthropic created a system prompt that it fed into Interviewer to develop its methodology. Then, Anthropic Interviewer crafted specific questions and conversation flow, which human researchers refined. Also: Your favorite AI tool barely scraped by this safety review - why that's a problem In its second step, the tool conducted interviews at scale by following a system prompt on best practices for interviews. Lastly, the researchers collaborated with Claude to analyze the transcripts, identify emerging themes, and produce a final report. Using those findings, the company released a public pilot of the tool. Anyone can opt to participate in the research, and participant insights will be analyzed anonymously as part of the company's societal impact research and published in a future report. Also: 92% of young professionals say AI boosts their confidence at work - how they use it I tried Interviewer myself and was impressed at the flow of the conversation and the depth of its questions. The tool acknowledged my answers and built on them with different questions. Interviewer also double-checked on several of answersto ensure it understood my aims correctly, and every time it did. Compared to any other survey I have ever taken, the process felt much more comprehensive and, as a result, enjoyable. To wrap up the process, Interviewer also included a broad, open-ended question where I could share any insights that weren't specifically cited, allowing me to make any additional points. The full experience took six minutes, which is about half of the estimated time that Anthropic suggested. Interviewer's findings Alongside the announcement, Anthropic published detailed AI research results. Overall, the report aligned with recent study findings from Google regarding the adoption of AI and productivity enhancements among working professionals, which have been leaning positively. Topline, the survey found that 86% of professionals believe AI saves them time, and 65% said they are satisfied with the role AI plays in their work. Also: How to learn ChatGPT in under an hour using my favorite guides and videos - for free Furthermore, 65% of participants described AI's primary role as augmentative, while only 35% described it as automative. However, anxieties about the technology and job security persist, with 55% expressing concerns about the impact of AI on their future and 25% expressing concerns about setting boundaries for AI use. Since the survey was geared toward creatives, scientists, and the general workforce, the report also delved deeper into the sentiments of working professionals toward AI across various sectors, including writers, visual artists, craftspeople, physicists, chemical engineers, astronomers, and others. You can view the full results here.

Anthropic will start using AI to interview its users... about their experience with AI.

The research pilot program will run for a week, and each AI interview will take 10 to 15 minutes, per Anthropic. Questions include what the user would most ideally like AI's help with and whether there are "ways that AI might be developed or deployed that would be contrary to your vision or what you value." It seems to be part of Anthropic's societal impacts team's push to do more social science research on how AI affects people. But, as the AI interviewer itself tells people who opt in, "AI asking about AI [is a] bit self-referential."

Anthropic launches Interviewer bot to quiz users on AI usage

Anthropic, the San Francisco-based artificial intelligence firm, released a new Interviewer tool on Thursday designed to collect user data on sentiment and preferences regarding its technology. The company has structured Interviewer as a chatbot that engages users in conversations about their usage and perspectives on AI, and Anthropic plans to publicly share the findings from these interactions. Interviewer is available for a week-long pilot within Anthropic's Claude chatbot. Users may encounter Interviewer as a pop-up window inviting participation. Prior to this pilot, Anthropic used the tool to survey 1,250 professionals regarding their views on work and artificial intelligence, the results of which the company has published in a separate blog post. Anthropic developed Interviewer utilizing its own language model, Claude. The tool automates drafting questions, conducting interviews, and summarizing responses. Interviewer operates in three primary phases: The recent test run of the AI tool involved a diverse participant group, including 1,000 individuals from general occupations, 125 from scientific fields such as chemistry, physics, engineering, and data science, and 125 from creative professions like writing, art, design, and music. Anthropic's shared results indicate a majority of participating professionals expressed positive views regarding AI's role in their work. For instance, 86% of general workforce respondents reported AI saves them time, and 65% indicated satisfaction with AI's involvement in their tasks. Many respondents characterized AI as a beneficial tool for routine or time-consuming work, allowing humans to concentrate on higher-level creative or oversight responsibilities. However, some concerns emerged. Creative professionals and those in scientific fields voiced apprehension regarding job identity, data security, and potential loss of control should AI become excessively integrated into workflows. Other participants expressed hesitation about relying on AI for critical tasks, such as scientific research design, citing issues of trust and accuracy. The surveyed data also revealed two distinct patterns of AI use: "augmentation," where humans and AI collaborate, and "automation," where AI manages tasks more independently. Approximately 65% of respondents described their AI use as augmentative, while 35% characterized it as automation. This suggests a user preference for AI as a collaborative tool rather than a replacement.

Anthropic's New Claude Tool That Interviews Users About Their AI Usage

When using Claude, users might see the tool appear in a pop-up window Anthropic announced the release of a new Interviewer tool on Thursday. But, unlike most artificial intelligence (AI) tools built by the company, this one is not primarily for the users. The San Francisco-based AI firm said that Interviewer is a specifically built tool that interviews users about their usage and perspective over the technology. Effectively, the company is using the chatbot to collect user data on user sentiment and preferences, and Anthropic has said that it will publicly share the findings. Anthropic Built an AI Interviewer In a post on X (formerly known as Twitter), the AI firm stated that interviewer is available in Claude for a week-long pilot. So, while using the chatbot, users might find Interviewer pop-up in a window, asking users to participate in an interview. Notably, this is not the first time Interviewer is being tested in real-world situations. Anthropic revealed that the tool has already been used to interview 1,250 professionals about their views on work and AI. The results of that survey has been shared in a separate blog post. Essentially, Interviewer is built on Anthropic's own language model, Claude, and is designed to automate the process of drafting questions, carrying out interviews and summarising responses. Anthropic says Interviewer works in three phases: planning, interviewing and analysis. In the planning phase, the AI generates a set of questions based on a research goal, then human researchers review and refine them if needed. During interviews, Claude interacts with participants in a conversational format, following a dynamic flow that adapts based on responses. After interviews conclude, a human-AI team analyses transcripts: Claude helps cluster responses and identify recurring themes, while researchers add context and judgement. Coming to the test run of the AI tool, the company revealed that participants were drawn from a wide range of occupations, including 1,000 from general jobs, 125 from science fields (like chemistry, physics, engineering, and data science), and 125 from creative professions (writers, artists, designers, musicians and more). Based on the results shared by Anthropic, majority of participating professionals expressed positive views about AI's role in their work. For instance, 86 percent of general-workforce respondents said AI saves them time, and 65 percent said they were satisfied with AI's role in their tasks. Many described AI as a helpful tool for routine or time-consuming work, leaving humans to focus on higher-level creative or oversight tasks. However, there were some concerns too. Some of the creative professionals and those belonging to scientific fields were noted to be worried about job identity, data security, and the possible loss of control if AI becomes too embedded in workflows. Others expressed hesitation about relying on AI for critical tasks like scientific research design, citing trust and accuracy issues. Interestingly, the surveyed data also highlighted two patterns of AI use. First is "augmentation," where humans and AI collaborate, and "automation," where AI handles tasks more independently. Among respondents, around 65 percent said their AI use was augmentative, while 35 percent described it as automation. This suggests many people prefer AI as a collaborator rather than a replacement.