AI

Modular secures $100M to build tools to optimize and create AI models

Comment

Futuristic digital blockchain background. Abstract connections technology and digital network. 3d illustration of the Big data and communications technology.
Image Credits: v_alex / Getty Images

Modular, a startup creating a platform for developing and optimizing AI systems, has raised $100 million in a funding round led by General Catalyst with participation from GV (Google Ventures), SV Angel, Greylock and Factory.

Bringing Modular’s total raised to $130 million, the proceeds will be put toward product expansion, hardware support and the expansion of Modular’s programming language, Mojo, CEO Chris Lattner says.

“Because we operate in a deeply technical space that requires highly specialized expertise, we intend to use this funding to support the growth of our team,” Lattner said in an email interview with TechCrunch. “This funding will not be primarily spent on AI compute, but rather improving our core products and scaling to meet our incredible customer demand.”

Lattner, an ex-Googler, cofounded Palo Alto-based Modular in 2022 with Tim Davis, a former Google colleague in the tech giant’s Google Brain research division. Both Lattner and Davis felt that AI was being held back by an overly complicated and fragmented technical infrastructure, and founded Modular with a focus on removing the complexity of building and maintaining AI systems at large scale.

Modular provides an engine that tries to improve the inferencing performance of AI models on CPUs — and beginning later this year, GPUs — while delivering on cost savings. Compatible with existing cloud environments, machine learning frameworks like Google’s TensorFlow and Meta’s PyTorch and even other AI accelerator engines, Modular’s engine, currently in closed preview, lets developers import trained models and run them up to 7.5 times faster versus on their native frameworks, Lattner claims.

Modular’s other flagship product, Mojo, is a programming language that aims to combine the usability of Python with features like caching, adaptive compilation techniques and metaprogramming. Currently available in preview to “hundreds” of early adopters, Modular plans to release Mojo in general availability early next month.

“Our developer platform enables our customers, and the world’s developers, to defragment their AI technology stacks — pushing more innovations into production faster and realizing more value from their investment in AI,” Lattner said. “We’re attacking the complexity that slows AI development today by solving the fragmentation issues that plague the AI stack, starting with where AI software meets AI hardware.”

Ambitious much? Perhaps. But none of what roughly-70-employee Modular is proposing is out of the realm of possibility.

Deci, backed by Intel, is among the startups offering tech to make trained AI models more efficient — and performant. Another in that category is OctoML, which automatically optimizes, benchmarks and packages models for an array of different hardware.

In any case, to Lattner’s point, AI demand is fast approaching the limits of sustainability — making any tech to cut down on its compute requirements hugely desirable. The generative AI models in vogue today are 10 to 100 times bigger than older AI models, as a recent piece in The Wall Street Journal points out, and much of the public cloud infrastructure wasn’t built for running these systems — at least not at this scale.

It’s already had an impact. Microsoft is facing a shortage of the server hardware needed to run AI so severe that it might lead to service disruptions, the company warned in an earnings report. Meanwhile, the sky-high appetite for AI inferencing hardware — mainly GPUs — has driven GPU provider Nvidia’s market cap to $1 trillion. But Nvidia’s become a victim of its own success; the company’s best-performing AI chips are reportedly sold out until 2024.

For these reasons and others, more than half of AI decision makers in top companies report facing barriers to deploying the latest AI tools, according to a 2023 poll from S&P Global.

“The compute power needed for today’s AI programs is massive and unsustainable under the current model,” Lattner said. “We’re already seeing instances where there is not enough compute capacity to meet demand. Costs are skyrocketing and only the big, powerful tech companies have the resources to build these types of solutions. Modular solves this problem, and will allow for AI products and services to be powered in a way that is far more affordable, sustainable and accessible for any enterprise.”

Modular
Modular’s Mojo programming language, a “fast superset” of Python. Image Credits: Modular

That’s reasonable. But I’m less convinced that Modular can drive widespread adoption of its new programming language, Mojo, when Python is so entrenched in the machine learning community. According to one survey, as of 2020, 87% of data scientists used Python on a regular basis.

But Lattner argues that Mojo’s benefits will drive its growth.

“One thing that is commonly misunderstood about AI applications is that they are not just a high-performance accelerator problem,” he said. “AI today is an end-to-end data problem, which involves loading and transforming data, pre-processing, post-processing and networking. These auxiliary tasks are usually done in Python and C++, and only Modular’s approach with Mojo can bring all these components together to work in a single unified technology base without sacrificing performance and scalability.”

He might be right. The Modular community grew to more than 120,000 developers in the four months since Modular’s product keynote in early May, Lattner claims, and “leading tech companies” are already using the startup’s infrastructure, with 30,000 on the waitlist.

“The most important enemy of Modular is complexity: complexity in software layers that only work in special cases, software that’s tied to specific hardware and complexity driven by the low-level nature of high-performance accelerators,” he said. “The very thing that makes AI such a powerful and transformative technology is the reason it requires so much effort to reach scale, so much talent invested in building bespoke solutions and so much compute power to deliver consistent results. The Modular engine and Mojo together level the playing field, and this is just the start.”

And — at least from a funding standpoint — what an auspicious start it is.

More TechCrunch

Featured Article

What Snowflake isn’t saying about its customer data breaches

As another Snowflake customer confirms a data breach, the cloud data company says its position “remains unchanged.”

16 mins ago
What Snowflake isn’t saying about its customer data breaches

Investor demand has been so strong for Rippling’s shares that it is letting former employees particpate in its tender offer. With one exception.

Rippling bans former employees who work at competitors like Deel and Workday from its tender offer stock sale

It turns out the space industry has a lot of ideas on how to improve NASA’s $11 billion, 15-year plan to collect and return samples from Mars. Seven of these…

NASA puts $10M down on Mars sample return proposals from Blue Origin, SpaceX and others

Featured Article

In 2024, many Y Combinator startups only want tiny seed rounds — but there’s a catch

When Bowery Capital general partner Loren Straub started talking to a startup from the latest Y Combinator accelerator batch a few months ago, she thought it was strange that the company didn’t have a lead investor for the round it was raising. Even stranger, the founders didn’t seem to be…

7 hours ago
In 2024, many Y Combinator startups only want tiny seed rounds — but there’s a catch

The keynote will be focused on Apple’s software offerings and the developers that power them, including the latest versions of iOS, iPadOS, macOS, tvOS, visionOS and watchOS.

Watch Apple kick off WWDC 2024 right here

Welcome to Startups Weekly — Haje’s weekly recap of everything you can’t miss from the world of startups. Anna will be covering for him this week. Sign up here to…

Startups Weekly: Ups, downs, and silver linings

HSBC and BlackRock estimate that the Indian edtech giant Byju’s, once valued at $22 billion, is now worth nothing.

BlackRock has slashed the value of stake in Byju’s, once worth $22 billion, to zero

Apple is set to board the runaway locomotive that is generative AI at next week’s World Wide Developer Conference. Reports thus far have pointed to a partnership with OpenAI that…

Apple’s generative AI offering might not work with the standard iPhone 15

LinkedIn has confirmed it will no longer allow advertisers to target users based on data gleaned from their participation in LinkedIn Groups. The move comes more than three months after…

LinkedIn to limit targeted ads in EU after complaint over sensitive data use

Founders: Need plans this weekend? What better way to spend your time than applying to this year’s Startup Battlefield 200 at TechCrunch Disrupt. With Monday’s deadline looming, this is a…

Startup Battlefield 200 applications due Monday

The company is in the process of building a gigawatt-scale factory in Kentucky to produce its nickel-hydrogen batteries.

Novel battery manufacturer EnerVenue is raising $515M, per filing

Meta is quietly rolling out a new “Communities” feature on Messenger, the company confirmed to TechCrunch. The feature is designed to help organizations, schools and other private groups communicate in…

Meta quietly rolls out Communities on Messenger

Featured Article

Siri and Google Assistant look to generative AI for a new lease on life

Voice assistants in general are having an existential moment, and generative AI is poised to be the logical successor.

13 hours ago
Siri and Google Assistant look to generative AI for a new lease on life

Education software provider PowerSchool is being taken private by investment firm Bain Capital in a $5.6 billion deal.

Bain to take K-12 education software provider PowerSchool private in $5.6B deal

Shopify has acquired Threads.com, the Sequoia-backed Slack alternative, Threads said on its website. The companies didn’t disclose the terms of the deal but said that the Threads.com team will join…

Shopify acquires Threads (no, not that one)

Featured Article

Bangladeshi police agents accused of selling citizens’ personal information on Telegram

Two senior police officials in Bangladesh are accused of collecting and selling citizens’ personal information to criminals on Telegram.

24 hours ago
Bangladeshi police agents accused of selling citizens’ personal information on Telegram

Carta, a once-high-flying Silicon Valley startup that loudly backed away from one of its businesses earlier this year, is working on a secondary sale that would value the company at…

Carta’s valuation to be cut by $6.5 billion in upcoming secondary sale

Boeing’s Starliner spacecraft has successfully delivered two astronauts to the International Space Station, a key milestone in the aerospace giant’s quest to certify the capsule for regular crewed missions.  Starliner…

Boeing’s Starliner overcomes leaks and engine trouble to dock with ‘the big city in the sky’

Rivian needs to sell its new revamped vehicles at a profit in order to sustain itself long enough to get to the cheaper mass market R2 SUV on the road.

Rivian’s path to survival is now remarkably clear

Featured Article

What to expect from WWDC 2024: iOS 18, macOS 15 and so much AI

Apple is hoping to make WWDC 2024 memorable as it finally spells out its generative AI plans.

1 day ago
What to expect from WWDC 2024: iOS 18, macOS 15 and so much AI

As WWDC 2024 nears, all sorts of rumors and leaks have emerged about what iOS 18 and its AI-powered apps and features have in store.

What to expect from Apple’s AI-powered iOS 18 at WWDC 2024

Apple’s annual list of what it considers the best and most innovative software available on its platform is turning its attention to the little guy.

Apple’s Design Awards highlight indies and startups

Meta launched its Meta Verified program today along with other features, such as the ability to call large businesses and custom messages.

Meta rolls out Meta Verified for WhatsApp Business users in Brazil, India, Indonesia and Colombia

Last year, during the Q3 2023 earnings call, Mark Zuckerberg talked about leveraging AI to have business accounts respond to customers for purchase and support queries. Today, Meta announced AI-powered…

Meta adds AI-powered features to WhatsApp Business app

TikTok is testing streaks that are similar to Snapchat’s in order to boost engagement, including how long people stay on the app.

TikTok is testing Snapchat-like streaks

Welcome back to TechCrunch Mobility — your central hub for news and insights on the future of transportation. Sign up here for free — just click TechCrunch Mobility! Your usual…

Inside Fisker’s collapse and robotaxis come to more US cities

New York-based Revel has made a lot of pivots since initially launching in 2018 as a dockless e-moped sharing service. The BlackRock-backed startup briefly stepped into the e-bike subscription business.…

Revel to lay off 1,000 staff ride-hail drivers, saying they’d rather be contractors anyway

Google says apps offering AI features will have to prevent the generation of restricted content.

Google Play cracks down on AI apps after circulation of apps for making deepfake nudes

The British retailers association also takes aim at Amazon’s “Buy Box,” claiming that Amazon manipulated which retailers were selected for the coveted placement.

Amazon slammed with £1.1B data abuse lawsuit from UK retailers

Featured Article

Rivian overhauled the R1S and R1T to entice new buyers ahead of cheaper R2 launch

Rivian has changed 600 parts on its R1S SUV and R1T pickup truck in a bid to drive down manufacturing costs, while improving performance of its flagship vehicles.  The end goal, which will play out over the coming year, is an existential one. Rivian lost about $38,784 on every vehicle…

1 day ago
Rivian overhauled the R1S and R1T to entice new buyers ahead of cheaper R2 launch