Meta AI and Martin Challenge ChatGPT: A Comparative Analysis of AI Assistants

Meta AI vs. ChatGPT: The jack of all trades takes on the master of one. We find out which is best

ChatGPT might've got all the attention when it launched in public preview back in November 2022, but these days, every company wants to get in on the action. That includes the parent company of Facebook and Instagram, which with its market cap of $1.5 trillion, has enough resources to make a mighty impressive chatbot. Is it a case of a jack of all trades and a master of none, or does Meta have what it takes to rival industry giants like ChatGPT? It should come as no surprise that Meta, a company founded on social media and content sharing, has built its artificial intelligence system to generate all manner of content, from text to images. We wanted to see whether the established social media giant could outpace a relatively new AI startup with a series of tasks set to catch them out and separate the wheat from the chaff. To put them to the test, we used six different types of prompts: a factual explanation, a mathematical equation, a broader request for advice, an open-ended request, an expression of creativity and the generation of a completely custom image. This should be a fairly easy one, because there should be scientific backing behind the answer that's either right or wrong. They both covered the basics well, writing a summary of the three key states of water - solid, liquid and gas. Although the overall outcome is a draw, it's not because both systems went above and beyond. Rather, they both missed out on some information. Meta included two further, lesser-referenced states: plasma and supercritical fluid. But ChatGPT shared information about the transition between states that enables water molecules to go from solid right the way through to gas. Meta AI 1 - ChatGPT 1 This would usually be one for the calculator, but could an intelligent computer replace the need to ever find the calculator app again? Sometimes AI struggles with these types of questions, because rather than referencing existing information, it has to run the calculation itself, creating more of a margin for error. The two chatbots stated that the answer is approximately 1.80, but only ChatGPT produced further expansion to reveal the full answer to 15 decimal places. In a scenario where you might be using AI to help with your homework, those decimal places might make all the difference. Meta AI 0 - ChatGPT 1 There's no right answer for this one, I'm afraid, because it depends entirely on what you need your laptop for. That's exactly what Meta AI and ChatGPT advised me. Again, the two systems draw because they both missed out on something that the other one included. Meta AI noted other factors like resolution, refresh rate and screen technology, whereas ChatGPT mentioned broader considerations like the impact on battery life and the ergonomics of the physical device. In essence, neither really painted the full picture, and yes, further prompts will have gotten me there, but this is a test of single-prompt output, not an entire conversation. Meta AI 1 - ChatGPT 1 With just a few hours to cram in a whole lot of sightseeing, the plan needs to be details and inclusive. The two systems broke down the day into time slots, with varied allocations depending on the type of attraction. ChatGPT added context, like why you might want to visit certain locations and what you might want to do there, but Meta took the points on this one because it was able to reference other elements of the daytrip. Meta AI included dinner and snacks, and also offered advice with regards to accommodation, transport and budget. Meta AI 1 - ChatGPT 0 The artificial intelligence would need to first identify the rules of a haiku, which stipulate how many syllables and lines should make up the poem, before captivating an audience of dog fans. There's not a lot you can do with a total of just 17 syllables, but both outputs were pretty unimaginative and referenced typical traits and themes, like loyalty and a dog's tail. Creating an image is notoriously challenging for AI, because it's a lot more involving than just stringing a sentence of existing words together. Particularly when it's an outlandish request like a giraffe driving a convertible. Meta AI took 16.04 seconds, which compared with other chatbots is pretty respectable. Four cartoon-style images were created, but neither was especially usable. They mostly illustrated a long neck coming from a car, but no visual cues that the giraffe was driving, such as 'arms' (front legs) on the steering wheel. ChatGPT took less time - 9.41 seconds - but produced just a single image. It was much more childish in its design, but it was instantly usable. This was a really close game, with half of the tests turning out to be inconclusive. The two systems performed strongly and produced solid results all round. ChatGPT might've come out victorious, but you wouldn't do badly to pick Meta AI to handle your prompts. There's no wonder Meta's Llama LLM and OpenAI's GPT LLM are often chosen by businesses.

I put Martin vs. ChatGPT head-to-head to find out which is best - this is the winner

ChatGPT is credited for making generative AI popular, but in the two years that it's been publicly available, it's faced growing competition from all manner of AI tools. It remains first and foremost a chatbot, with text and image generation capabilities, but new features have been added along the way such as support for internet searches, image generation and more. A new era of AI is emerging, though. Personalized AI and AI assistants might seem like the ultimate goal when it comes to this time-saving tech, but can Martin's AI keep up with OpenAI's more established product? ChatGPT is primarily a text and image generation tool, so it's this part of Martin that we wanted to put to the test, even though it's capable of integrating more deeply into everyday routines. Still, because Martin is sold as an AI personal assistant, we wanted to check whether its integrated chatbot offered a more personalized experience. The questions included a factual explanation, a mathematical equation, a broader request for advice, an open-ended request, an expression of creativity and freedom to express creativity. We wanted to understand the biological process and implications of sunburn, and both chatbots offered a full overview of how the damage occurs in a logical, step-by-step breakdown. They also offered further context regarding long-term effects and advice to prevent sunburn in the first place. On the whole, we were pleased with the well-rounded responses from both sources. Martin 1 - ChatGPT 1 Two really simple equations bundled into a single, third equation should produce a really straightforward answer, but getting there can be a riddle for anybody who wasn't paying attention in school. The two systems broke down the steps involved to reach the correct answer - 110. However, while ChatGPT was able to lay everything out clearly and in a range of formats, Martin struggled, making understanding how 110 was reached slightly less digestible. Martin 0 - ChatGPT 1 There's no definite answer here, because everybody's answer should be slightly different. Martin and ChatGPT both shared advice per lifestyle, for example sedentary and active days. Martin also included advice for different age groups, which ChatGPT omitted, however there's little evidence or data to suggest that these offer the correct advice. The hope was that they would give us broad advice, which they did, and then go on to explain how to calculate the correct number of calories more accurately. Martin failed to do this, but ChatGPT cited the Mifflin-St Jeor Equation. Martin 0 - ChatGPT 1 Two or three days should be enough to see everything Reykjavic has to offer for anybody who's short on time, but can AI pinpoint what we should see and when to make the best use of our time? Both systems counted Friday night as part of the weekend, but Martin assumed we had the whole day. They also highlighted some good places to eat, and more or less nailed the brief. Martin and ChatGPT responded well to the prompt, but OpenAI's chatbot went further and included advice on the weather, transportation and currency. Martin 0 - ChatGPT 1 If your toddler is struggling to get to sleep and you're out of books, you'll need a story whipped up in seconds - Martin and ChatGPT delivered on this front. They both generated an easily digestible and fairly captivating tale split into short paragraphs of no more than a sentence or two in length. Both centered around some sort of a mission and a positive outcome, however we found the vocabulary used in Martin's story to be marginally more challenging. This can either be a positive or a negative, depending on your parenting and educational preferences, but on the whole, ChatGPT's story was easier to follow. Realistically, you could imagine both being printed in a children's book or being turned into a short animation. Putting Martin's chatbot head-to-head against ChatGPT resulted in a clear verdict - ChatGPT performed well across all five tasks, while we felt Martin was lacking across three of them However, that's just the start of Martin's abilities, and if you're after a more contextual assistant then Martin can integrate with to-do lists, calendars, reminders and your email inbox to learn more about your preferences and habits. Martin also integrates across messaging, calling and email platforms, including Slack, so it can be easy to share information with the AI assistant across more settings. OpenAI recently added voice calls for ChatGPT in the US and Canada, or VoIP calls via WhatsApp, but this is currently limited.