Databricks

Magic AI

Databricks

Databricks runs on a consumption-based pricing model. Companies pay for the compute and storage they actually use on the Databricks platform, measured in "Databricks Units" (DBUs).

The more data you process, the more you pay. This is brilliant because it means revenue grows automatically as customers' data volumes grow — which in the age of AI, they always do.

The platform runs on top of the major cloud providers — AWS, Azure, and Google Cloud. Databricks doesn't own servers.

They're a software layer that makes those clouds dramatically more useful for data work. They take a margin on top of the underlying cloud compute costs, essentially acting as a "toll booth" between companies and their data.

They also pioneered the "lakehouse" architecture — a mashup of data warehouses (structured, fast querying) and data lakes (cheap, handles any data format). Before Databricks, companies had to maintain both.

The lakehouse collapses them into one system. This isn't just clever marketing — it genuinely saves enterprises millions in duplicate infrastructure.

Magic AI

Magic is building an AI that works like a software engineer, not a chatbot. The goal is a model you can hand a whole code base and a task, and it writes, reviews, and debugs like a teammate.

Their edge is memory. In 2024 they showed off LTM-2-mini, a model with a 100 million token context window.

To put that in perspective, that is about 10 million lines of code the model can look at in one shot, by far the biggest of any model at the time. The plan is to sell this as an AI coworker to companies that write a lot of software.

As of the 2024 rounds, the product was still not for sale.

Databricks

Databricks started as a research project at UC Berkeley's AMPLab around 2009. Matei Zaharia, a PhD student, was frustrated with how slow Hadoop MapReduce was for iterative machine learning workloads.

His answer was Apache Spark — an open-source engine that could process data up to 100x faster than MapReduce by keeping data in memory instead of writing to disk after every step.

Spark took off fast in the open-source community. By 2013, it was the most active open-source project in big data.

Zaharia and six Berkeley colleagues — Ali Ghodsi, Andy Konwinski, Arsalan Tavakoli-Shiraji, Ion Stoica, Patrick Wendell, and Reynold Xin — decided to build a company around it. They incorporated Databricks in 2013 with the idea that Spark was powerful but brutally hard to set up and manage.

The company would offer a managed cloud platform that made Spark accessible to data teams who weren't distributed systems engineers.

Their first product was essentially "Spark as a service" — a collaborative notebook environment where data scientists and engineers could write Spark jobs without managing clusters. The bet was that enterprises had massive data problems but not enough PhDs to solve them.

They were right.

Magic AI

Eric Steinberger and Sebastian De Ro started Magic in 2022. Steinberger is an AI researcher who became convinced that the missing piece for a true AI software engineer was memory.

Chatbots forget what you told them a few thousand words ago. He wanted a model that could read your entire code base and remember all of it.

They raised a small seed in 2022, then a $23 million Series A in early 2023 led by Alphabet's growth fund CapitalG. From day one the pitch was not a chatbot.

It was an AI colleague that could handle real engineering work.

Databricks

Databricks grew by being genuinely useful before being profitable. They contributed massively to Apache Spark's open-source ecosystem, which meant thousands of companies were already using Spark when Databricks offered to manage it for them.

The open-source-to-enterprise pipeline is the most powerful go-to-market motion in software.

They also bet big on partnerships. The Microsoft partnership was transformational — Azure Databricks became a first-party service on Azure, meaning Microsoft's sales force was effectively selling Databricks to every enterprise customer.

That single deal probably added billions in annual recurring revenue.

Acquisitions were strategic and well-timed. MosaicML in 2023 for $1.3 billion gave them proprietary AI training capabilities right when every enterprise wanted to build custom AI models.

Tabular in 2024 brought the creators of Apache Iceberg, another critical open-source data format. They bought the talent and the technology simultaneously.

Magic AI

Magic's strategy is research first, product later, which is both unusual and expensive. Instead of shipping an app and iterating in public, they raised huge sums to chase a technical breakthrough in model memory.

The headline move was the 100 million token context claim, which got the whole AI world talking. They also cut a deal with Google Cloud in 2024 to build two supercomputers, called Magic-G4 and Magic-G5, using NVIDIA chips, to train bigger models.

Star backers like Eric Schmidt served as validation. The bet is that if the tech is far enough ahead, the product and the customers follow.

Databricks

The elephant in the room is Snowflake. Both companies want to be the single platform where enterprises do all their data work, and the overlap is growing fast.

Snowflake started in SQL analytics and is pushing into data engineering and ML. Databricks started in data engineering and ML and is pushing into SQL analytics.

The collision is inevitable and expensive — both are spending billions on sales and R&D.

There's also the cloud provider threat. AWS, Azure, and Google Cloud all have their own data analytics services and could theoretically squeeze Databricks by making their native tools better or cheaper.

Databricks runs ON these clouds, which means their biggest partners are also their biggest potential competitors. It's the classic platform risk problem.

So far, Databricks has stayed ahead by innovating faster than the cloud providers' internal teams, but it's a race that never ends.

Magic AI

Magic has raised close to half a billion dollars and, years in, still has no product the public can buy. That is the whole problem in one sentence.

A giant context window is a great demo, but rivals like Cursor, GitHub Copilot, and Cognition's Devin already have real users and revenue. Magic has to prove that more memory actually makes a smarter engineer, not just a bigger number on a slide.

And it has to do that while burning enormous sums on compute against competitors who are already shipping. The clock and the cash both keep running.

Databricks

Unity Catalog — a universal governance layer that lets companies manage permissions, lineage, and access control across all their data and AI assets in one place. Delta Lake — an open-source storage layer that brings reliability to data lakes with ACID transactions, schema enforcement, and time travel (yes, you can query your data as it existed at any point in the past).

Databricks SQL — a serverless SQL analytics product that competes directly with Snowflake on their home turf. Mosaic AI — their machine learning and generative AI platform, supercharged after acquiring MosaicML in 2023 for $1.3 billion.

Databricks Notebooks — collaborative workspaces where data teams write code, visualize results, and build pipelines together in real time.

Magic AI

The centerpiece is Magic's LTM, or Long-Term Memory, model family. In 2024 they unveiled LTM-2-mini, a model with a 100 million token context window, pitched as the key to an AI that can understand an entire code base at once.

The broader product is an AI software engineer, or coworker, that can write and change code across a whole project. The company also announced Magic-G4 and Magic-G5, two supercomputers built with Google Cloud and NVIDIA to train these models.

Much of it is still research rather than a shipping product.

Databricks

Andreessen Horowitz led multiple early rounds and has been the longest-standing institutional backer. Microsoft made a massive strategic investment alongside the Azure Databricks partnership.

T. Rowe Price, Tiger Global, and Franklin Templeton participated in later growth rounds.

NEA was an early investor. The $10 billion Series J in 2024 valued the company at $62 billion and was led by Thrive Capital, with later rounds multiplying that several times over with participation from Andreessen Horowitz, DST Global, GIC, Insight Partners, and WCM Investment Management.

Magic AI

Magic's backers are a murderers' row of AI insiders. It started with a $5 million seed round in 2022 and a $23 million Series A in early 2023 led by Alphabet's CapitalG.

Then in February 2024 came a $117 million round led by Nat Friedman and Daniel Gross, two of the most connected investors in AI. The blowout was August 2024.

Magic raised $320 million with checks from former Google CEO Eric Schmidt, CapitalG again, Atlassian, Jane Street, Sequoia, and Elad Gil. That pushed total funding to around $465 million, all for a company still building toward its first sellable product.