Google’s ChatGPT rival Bard is almost equal – just slower

Earlier this month, Google announced the launch of Gemini, its most powerful AI model to date. It hopes this will attract more users than its biggest competitor, OpenAI’s ChatGPT.

ChatGPT and the new Bard with Gemini are similar products – Gemini Pro is very similar to the GPT-4 found in the subscription-based ChatGPT Plus. So we decided to test the two chatbots to see their accuracy, speed and overall ease of use.

Comparing Gemini and ChatGPT: the basics
ChatGPT Plus and Gemini Pro are very advanced chatbots based on extensive language models. They are the latest and greatest options from their respective companies and promise to respond to queries faster and better than their predecessors. Most importantly, both are trained based on the latest information, rather than just knowing what’s happening on the internet until 2021. They are also quite easy to use as standalone products, unlike something like X’s new Grok bot.

However, the two aren’t exactly the same: GPT-4-powered ChatGPT Plus costs $20 USD per month, while Bard is free. Another issue is that Bard with Gemini Pro doesn’t have multimodal features like ChatGPT Plus. The multimodal language model allows users to receive text prompts and respond in another medium, such as a photo or video; Gemini and Bard will eventually do this, but it will be a larger version of Gemini called Ultra, which Google has yet to release.

On the other hand, Bard also offers a way to check for other draft answers, a feature ChatGPT lacks.

One of the challenges of testing chatbots is that the answers can change dramatically when the same question is re-run many times. The main changes I encountered are outlined in the description. To be fair, I gave each bot the same initial prompt, starting with simple requests and moving on to more complex ones if necessary.

One general difference was that Bard tended to be slower than ChatGPT. It usually took five to six seconds to ‘think’ before it started typing, whereas ChatGPT took one to three seconds to start delivering results. (The total turnaround time for both depended on the nature of the information requested – more complex prompts tended to produce longer responses and took more time to fill in). This speed difference persisted over home and office Wi-Fi for the few days I played with both apps.

Both OpenAI and Google place some restrictions on the types of responses chatbots can give. Through a process called red teaming – where developers test content and security policies by repeatedly trying to break the rules – AI companies are building guardrails to make sure they don’t violate copyright protection or give racist and harmful answers. In general, I encountered Google’s restrictions more often than ChatGPT’s.

‘Give me a chocolate cake recipe.
I was given a chocolate cake recipe on both platforms. This was one of the prompts used by The Verge earlier this year in a comparison of Bing, ChatGPT and Bard, and since recipes are a popular search topic on the web, AI chatbots are no exception.

As a baker, I usually know a good cake recipe when I see one. But I double-checked with a reliable non-AI source for comparison: Claire Saffitz’s cookbook Dessert Person. Safitz’s version is certainly a bit fancier, but comparable to both Bird’s and ChatGPT’s.

However, there were some complications. Coffee is more common in chocolate cake recipes, so ChatGPT’s version using boiling water was questionable. Bird’s seems to have copied the recipe verbatim from Sally’s Baking Addiction blog. However, seemingly random changes have been made, such as doubling the eggs.

Twins and Conversation GPT’s cake (and by contrast Sally’s cake) is to bake a cake. The result. Both cakes were functional but tasted nothing like Claire Saffitz. Gemini’s cake was a bit gummy – a friend described it as ‘like rice cake’ – but it was the moistest of the three cakes. I didn’t like it at all but the editor thought it was quite good; the ChatGPT cake was rich, smooth and chocolatey – I would call it the perfect breakfast cake.