# Google's Gemini Breakthrough: Testing the AI Model That Beat GPT-4o

## 🌟 Introduction

In a significant development for the AI industry, Google's latest language model, currently known simply as Gemini Exp-1114, has surpassed OpenAI's leading model in the prestigious LMArena benchmarks. This breakthrough marks a potential shift in the competitive landscape of large language models (LLMs).

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731854151093/c5953ddb-d17b-4de9-b45c-277f1a240d24.png align="center")

## 📈 Performance Breakthrough

The most striking aspect of Gemini Exp-1114's emergence is its comprehensive dominance across multiple domains. According to the [LMArena](http://lmarena.ai) benchmarks, the model has achieved remarkable improvements:

🏆 **Category Improvements**

* 🧮 **Mathematics**: #3 ➡️ #1
    
* 🎯 **Hard Prompts**: #4 ➡️ #1
    
* ✍️ **Creative Writing**: #2 ➡️ #1
    
* 👁️ **Vision Tasks**: #2 ➡️ #1
    
* 💻 **Coding**: #5 ➡️ #3
    

## 🔍 Quick Observations

* 📏 Model operates with a 32K context window, notably smaller than the 2 million token context length of Gemini Pro-002
    
* ⏱️ Current testing indicates slower inference times compared to its competitors
    
* 🧪 Still in experimental phase, as evidenced by its temporary designation
    

## 🚀 Try It Yourself

For those interested in experiencing this breakthrough firsthand, Google has made Gemini Exp-1114 available for free testing through their [AI Studio platform](https://aistudio.google.com/prompts/new_chat).

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731850875298/26f90a26-68f4-4416-b88a-0e897a99c455.png align="center")

## 🧪 Testing the Beast

I put both Gemini Exp-1114 and Gemini Pro-002 through a series of real-world tasks to understand their capabilities and differences.

### 1\. 🎭 Creative Humor Generation

**🔮 Gemini Exp-1114**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851301286/65d77067-993d-474a-8309-23cfff47e1c1.png align="center")

**⚡ Gemini Pro-002**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851467568/454820e0-055a-4e00-a6d6-22c273c5244a.png align="center")

### 2\. 🖼️ Mango Detection

Enquired the model about the following image without being specific in the prompt:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851578487/fa730cec-a36d-4a7e-8b93-618d440d4853.jpeg align="center")

**🔮 Gemini Exp-1114**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851839306/edaedfb9-34cd-4933-8bde-8137a2e47bcf.png align="center")

**⚡ Gemini Pro-002** \[IMAGE\]

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851818443/6b247c05-a257-45e7-aed3-e8154e192c21.png align="center")

> Both the models were able to identify the partially cut mango but the count was wrong

### 3\. 🔤 Character Counting

**🔮 Gemini Exp-1114**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851967497/96f5771c-8aa1-4c93-9090-495bc5c9662b.png align="center")

**⚡ Gemini Pro-002**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731851991400/9a4a5f71-eb7a-4424-9ac3-6cac57a343fe.png align="center")

> The Gemini Exp-1114 Model does not give the right answer owing to “over-analysis” of a simple problem

### 4\. 🧮 Mathematical Reasoning

**🔮 Gemini Exp-1114**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731852149337/cf4c2f70-aa42-48db-a1df-b313fae2a5a4.png align="center")

**⚡ Gemini Pro-002**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731852164418/ac701f1e-a686-4bbf-a4e4-ba283078518d.png align="center")

> Both the models give the correct answer, the Gemini Exp-1114 additionally gives detailed steps to solve the problem

### 5\. 📝 Word-Limited Writing

**🔮 Gemini Exp-1114**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731852317646/724bfdb1-0aff-4712-ac2a-307963e87b5d.png align="center")

> Verification using the traditional [wordcounter.net](https://wordcounter.net)

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731852362906/291a9737-f19c-43aa-936c-23e7ecee1507.png align="center")

**⚡ Gemini Pro-002**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731852390836/1991bc38-809c-44e4-89d5-88643e3e6038.png align="center")

> Verification using the traditional [wordcounter.net](https://wordcounter.net)

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731852408269/03f03aa2-fee6-4790-8ddf-494eff81638e.png align="center")

> Both the models generate relevant content, the Gemini Exp-1114 follows the content limit instructions more strictly and hence generates a more accurate output

### 6\. 🎨 SVG Code Generation

**🔮 Gemini Exp-1114**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731853893945/8163f76d-3852-4414-81da-42fd90b38920.png align="center")

> Rendered using [svgviewer.dev](http://svgviewer.dev)

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731853854941/aae4ffb3-a299-450f-9368-d7765b80e2a5.png align="center")

**⚡ Gemini Pro-002**

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731853925531/85233a82-44cd-4fd1-8181-76155cbd5fa5.png align="center")

> Rendered using [svgviewer.dev](http://svgviewer.dev)

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1731853943796/cf83f2f4-dcaf-4689-a868-1371546f488a.png align="center")

> Since the prompt is minimal, the generated SVGs are minimal as well. Gemini Exp-1114 does a decent job at v0.0.1 prototype while the Gemini Pro-002 model completely messes it up

## 📋Closing Notes

Google is yet to launch Gemini Exp-1114 in its full capacity, but preliminary benchmarks and hands-on testing show promising results. With its ability to go head-to-head with Claude and GPT-4, Gemini represents a significant step forward in Google's AI capabilities, potentially reshaping the competitive landscape of large language models.