Microsoft Copilot: The Next Wave of AI Integration Across Microsoft's Ecosystem

Microsoft Copilot: Everything you need to know about Microsoft's AI | TechCrunch

Copilot is Microsoft's take on productivity-boosting generative AI, and it continues to grow and expand with Microsoft's AI ambitions. Today, there are around a dozen Copilot-branded products powering various capabilities in Microsoft software and services, like summarizations in Microsoft Outlook and transcriptions in Microsoft Teams. That's in addition to Microsoft-owned GitHub's Copilot tool for generating code, and the Copilot that lives on Windows and the web, which serves as a general-purpose assistant. In this post, we explain the many Microsoft Copilots available and what they do, and the differences between the premium and free editions. Microsoft Copilot, previously known as Bing Chat, is built into Microsoft's search engine, Bing, as well as Windows 10, Windows 11, and the Microsoft Edge sidebar. (Newer PCs even have a dedicated keyboard key for launching Copilot.) There's also stand-alone Copilot apps for Android and iOS and an in-app Telegram room. Powered by fine-tuned versions of OpenAI's models (OpenAI and Microsoft have a close working relationship), Copilot can perform a range of tasks described in natural language, like writing poems and essays, as well as translating text into other languages and summarizing sources from around the web (albeit imperfectly). Copilot, like OpenAI's ChatGPT and Google's Gemini, can browse the web (in Copilot's case, via Bing) for up-to-date information. It sometimes gets things wrong, but for timely queries, access to search results can give Copilot an advantage over offline bots such as Anthropic's Claude. Copilot can create images by tapping Image Creator, Microsoft's image generator built on OpenAI's DALL-E 3 model. And it can generate songs via an integration with Suno, the AI music-generating platform. Typing something like "Create an image of a zebra" or "Generate a song with a jazz rhythm" in Copilot will pull in the relevant tool. On the subject of integrations, Copilot supports plug-ins for third-party apps and websites. There's plug-ins for Instacart (for meal planning and cooking-related questions), Kayak (for trip planning), OpenTable (for restaurant reservations) and Shopify, to name a few examples. More are being added on a regular basis. Copilot also drives Copilot Pages, an embeddable digital canvas where users can edit and share Copilot-originated content. Paying customers (more on that below) get access to BizChat, a business-focused hub that ties into Pages to pull data from the web (and work files) to help create things like project plans, meeting notes, proposals, and more. On Windows 11 (but not necessarily Windows 10), Copilot can control certain settings and functions, acting as a digital concierge of sorts. With Copilot, either by typing or using Windows 11's speech recognition functionality, users can perform actions on a PC like turning the battery saver on or off, showing device and system information, launching live captions, displaying the PC's IP address, and emptying the recycle bin. A toggle in the Copilot experience on Windows 11 switches between "Work" and "Web" mode, with the former bringing Copilot's Microsoft 365 capabilities in the Windows interface. Copilot Pro is Microsoft's premium Copilot product, priced at $20 per month. Copilot Pro customers get priority access to the most capable OpenAI models (e.g., o1) during peak times. And select features of Copilot can only be accessed with a Pro subscription, such as higher-resolution images from Image Creator. Copilot Pro also gives users access to generative AI functions across the Microsoft 365 suite of productivity apps: Word, Excel, PowerPoint, Outlook, and OneNote. In Word and OneNote, Copilot can write, edit, summarize, and generate text. Copilot in Excel and PowerPoint can turn natural language prompts into presentations and visualizations (optionally grounded in data from files and templates). And in Outlook, Copilot can help draft email responses with toggles for adjusting the length or tone. More features as part of Microsoft's Copilot Wave 2 update are coming down the pipeline. Copilot in PowerPoint will soon pull in company-approved images from a SharePoint library, while Outlook will get a "Prioritize my inbox" capability that summarizes each email (and gives insights like who you've been most responsive too). In late 2024, Outlook users will also gain the ability to "teach" Copilot topics, keywords, and people of interest so that those emails are always marked as high-priority. In Excel, Copilot can format data, create graphs, generate pivot tables and guide users through creating new formulas and macros. It can also make use of the programming language Python for advanced data analysis - in natural language, users can describe forecasting, risk analysis, and data visualization tasks and Copilot will translate the text to the necessary Python code to perform these tasks. A future version of Copilot in Word will let you quickly pull in data from outside Word, PowerPoint, and PDF documents as well as emails, encrypted docs, and meetings. Elsewhere, in OneDrive, Copilot will summarize, show metrics about, and compare the differences between files for you. Beyond the Microsoft 365 upgrades, Copilot Pro subscribers get landscape formatting options and 100 "boosts" per day in Image Creator (versus only 15 per day for free users) to speed up the image-generation process. Importantly, Copilot Pro does not come with Copilot in Teams, a Copilot feature in Microsoft Teams that provides real-time summaries and action items while handling tasks like identifying people for follow-ups. Copilot in Teams is exclusive to enterprise Copilot customers, meaning those with an enterprise-class (or equivalent) Microsoft 365 license. Separate and distinct from the consumer Copilot SKU is Microsoft 365 Copilot, a collection of generative AI add-ons to Microsoft 365 with an emphasis on business apps. Microsoft 365 Copilot is priced at $30 per user per month and available to customers with a Microsoft 365 E3, E5, Business Standard, or Business Premium license. It delivers many of the same capabilities across the Microsoft 365 family of apps as Copilot Pro, but with the addition of "enterprise-grade data protection" and the Semantic Index, a back-end system that creates a map of the data and content in an organization to allow Copilot to deliver more personalized responses. Microsoft recently launched Microsoft 365 Chat, for example, a tool that pulls info from content across Microsoft 365 apps (e.g., Word docs, PowerPoint presentations) to answer questions. There are many, many Copilots besides. Here's a partial list of them and their "skills": Note that some of Microsoft's Copilots, like Copilot in Business Central, are included in the base software licenses and don't require paying an additional fee. Others, like Copilot for Sales and Copilot for Service, cost an extra $20 per user per month or $50 per user per month without an active Microsoft 365 Copilot subscription. Copilot Studio is a dashboard that lets customers give Microsoft 365 Copilot access to data in their, or a third party's, customer relationship management software, enterprise resource management systems, and other databases and repositories using prebuilt connectors or connectors they build themselves. Through Copilot Studio, customers can build guardrails for Copilots and create and publish their own custom-tailored "copilots." Microsoft 365 Copilot subscribers can tap Copilot Studio to create their own custom copilot by describing it in natural language. Copilots can filter to specific datasets for particular teams or users, or connect to an automation, plug-in, or third-party service to kick off actions or a workflow. Copilot Studio is also where customers can craft what Microsoft calls Copilot agents. These AI bots -- which can be "@ mentioned" in Outlook and Teams -- leverage memory and knowledge of context to navigate different business workflows, learning from user feedback and asking for help when they encounter situations they don't know how to handle. Copilot agents range in capability from simple, prompt-and-response agents to more sophisticated bots -- think bots that can monitor email inboxes and automate data entry. Microsoft provides a number of pre-built ones, including the Visual creator agent that generates images, designs, and soon videos. Not to be confused with the many other Copilots in Microsoft's portfolio, GitHub Copilot is a set of tools for generating code and supporting programming work. GitHub Copilot can be installed as an extension for IDEs including Visual Studio Code, Visual Studio, Neovim, and JetBrains, or used in the cloud with GitHub Codespaces. The generative AI model underpinning GitHub Copilot has been trained on billions of lines of Python, JavaScript, TypeScript, Ruby, Go, and dozens of other programming languages -- many of them hosted and available publicly on GitHub. When you're writing code, GitHub Copilot suggests code as you type; you can cycle through suggestions and accept or reject them. GitHub Copilot can also translate code into natural language descriptions, and Copilot Extensions allow developers to extend Copilot with third-party skills. GitHub Copilot is available for free for students and for "verified" open source contributors and educators. For individuals, it's $10 per month. For business customers, it's $19 per month per user. And for enterprises, it's $39 per user per month. Individual, business and enterprise subscribers get Copilot Chat along with GitHub Copilot, a chatbot-like flow that's aware of the entire context of the code they're working on and can answer questions about that code. In addition to answering coding queries, Chat can help developers fix errors and bugs and address security issues through code analysis. The enterprise and business GitHub Copilot plans include license management, IP indemnity, organization-wide policy management, and added privacy features. Enterprise customers can customize for their codebases and knowledge bases and fine-tune the underlying models, as well as access Copilot through the Microsoft Copilot on the web and use Copilot Chat on GitHub.com. In April, GitHub launched Copilot Workspace, a sort of take on AI-powered software engineering. Workspace provides a dev environment that taps AI-powered agents to help brainstorm, plan, build, test and run code in natural language. Owing to the complex and fraught nature of today's generative AI tech, Microsoft's Copilots have their issues. The models occasionally make mistakes when summarizing or answering questions because of their tendency to hallucinate, including while summarizing meetings. The Wall Street Journal cited an instance where, for one early adopter using Copilot for Teams meetings, Copilot invented attendees and implied that calls were about subjects that were never actually discussed. As for GitHub Copilot, GitHub itself warns that it can produce insecure coding patterns, bugs, and references to outdated APIs, or idioms reflecting the imperfect code in its training data. The code Copilot suggests might not always compile or run -- or even make sense. Security and privacy concerns loom large over Copilot, as well. But perhaps the elephant in the room is the unresolved fair-use question. Like most generative AI models, the models powering Microsoft's Copilots were trained on public data, some of which is copyrighted or under a restrictive license. Microsoft -- among others -- argues that the fair-use doctrine shields it from copyright claims. But that hasn't stopped data owners from filing class action lawsuits against the company, GitHub, OpenAI, and many more over what owners allege are clear licensing and IP violations. Microsoft offers policies to protect certain customers from courtroom battles arising from fair use challenges -- at least in narrow circumstances. That doesn't resolve the ethical quandary of training models on data without permission, however, which may be more than some customers can swallow

Microsoft Copilot's Wave 2 is here. Everything you need to know

Microsoft has released several Copilot upgrades to compete with OpenAI's models. Since its launch, Microsoft Copilot has proved a worthy ChatGPT competitor, even occasionally lapping OpenAI's offering. However, as ChatGPT continued to advance, Copilot lagged -- until now. Microsoft has unveiled several upgrades to reclaim its throne. On Monday, Microsoft held a live-streamed event titled "Microsoft 365 Copilot: Wave 2," during which it unveiled all of the upgrades coming to its AI assistant, both for enterprise and personal consumers. The upgrades were impressive, including further integration with Microsoft 365 applications and collaborative ways to use AI as a team, group, or family. To see all of the announcements, keep reading below. If you sign into Copilot with your work or organization's account, you will likely notice a tab option to switch from Work to Web. The Work tab is what Microsoft refers to as BizChat, a workflow within Copilot that can pull answers from your work data in the Microsoft 365 applications. BizChat has now been upgraded with a new Pages feature. Also: The Linux file system structure explained With Pages, you can take the insights Copilot found for you -- which are rooted in your business data -- edit them, add them to a Page, and share them with your team to collaborate. Pages are much like any other editable document and can be accessed by a shareable and editable link. This feature emphasizes multiplayer AI, a trend seen in other AI companies' recent offerings, such as You.com's collaborative AI assistants and Salesforce's Agentforce, a suite of autonomous and collaborative AI agents. Pages will begin rolling out to Microsoft 365 Copilot customers today and will be available to free Microsoft Copilot users who are signed in with their Microsoft ENtra account in the coming weeks. One of Microsoft Copilot's biggest advantages is its ability to help users within the Microsoft 365 apps, which have become a steady cornerstone of many people's workflows. With Wave 2, Microsoft is using customer feedback to further its Copilot assistance in the applications and improve user accessibility. Also: Is your Windows license legal? Should you even care? For starters, Copilot in Excel is generally available for all users and provides support for formulas, data visualization, conditional formatting, and more. Furthermore, Microsoft announced Copilot in Excel with Python in public preview, which allows users to work with Python in Excel using only natural language. This means users can leverage Python in Excel to conduct advanced analysis, such as forecasting and risk analysis, without entering any code. Microsoft also made Narrative Builder in PowerPoint generally available. This feature builds you the first draft of your presentation in minutes -- just type in a topic, and the Builder will produce an outline. Copilot in Teams can now synthesize both the contents of an actual meeting and what was sent in the chat to create a summary of the entire meeting. For example, if a user asks Copoilot what they missed in the meeting, the response will also include content from the chat. This feature will be generally available next month. Copilot in Outlook couldn't be left behind: It now has a "Prioritize my inbox" feature that, as the name implies, analyzes your inbox and flags the most important emails while providing a summary of it and the reason it was prioritized. Microsoft shares that users will even "soon" be able to instruct Copilot on the topics or keywords it should look for to mark as high priority. These features are unavailable now, but Microsoft says they will be in public preview starting in late 2024. Since users often pull on external resources when writing in Microsoft Word, Microsoft is introducing a new feature later this month that allows users to quickly reference web and work data, including emails, PDFs, PowerPoints, and more. Copilot in Word also got a new "on-canvas start experience, " allowing users to collaborate with Copilot inline as they work on sections of their document. Both these Word features are now generally available. Also: Still have a Windows 10 PC? You have 5 options before support ends in 2025 Lastly, Microsoft unveiled Copilot in OneDrive, allowing users to quickly leverage the assistant to find files in their OneDrive repository, get summaries, and even compare files. Copilot in OneDrive is rolling out to users starting today and will be generally available this month. Microsoft introduced Copilot agents, AI assistants that can carry out specific tasks with as little or as much human intervention a user may need. For example, Microsoft shares that some agents can perform fully autonomously while others are more simple "prompt-and-response" agents. Users can build Copilot agents using the new agent builder, an experience powered by Copilot Studio that allows users to build an agent in BizChat or Sharepoint. These agents can then be summoned in the 365 applications like any other teammate, using the "@" to call on it. Copilot agents and the agent builder in BizChat are generally available starting today and will be rolled out to all customers in the coming weeks. However, Copilot agents and agent builder in SharePoint will enter public preview in early October, according to Microsoft. In May, OpenAI unveiled GPT-4o, its most advanced model with GPT-4-level intelligence and multimodal capabilities. It then infused ChatGPT with this model for both free and paid ChatGPT users to enjoy. Since then, there has been much speculation on whether Copilot would follow suit, upgrading its AI chatbot to the latest version. In the event blog post on Monday, Microsoft confirmed that Copilot uses GPT-4o, which, according to Microsoft, has dramatically improved performance with responses that are more than two times faster on average. Furthermore, Microsoft said it would continue to bring the latest models to Copilot, including OpenAI's o1, which was introduced last week and has the most advanced reasoning of OpenAI models thus far.