Dollars to Data: Kelsey Hightower Talks LLMs in the Enterprise

The day after he moderated a panel about the technology path to adoption of Large Language Models (LLMs) in the enterprise, Kubernetes guru Kelsey Hightower spoke with Union.ai about the business case for LLMs and AI and their potential to reshape the workplace. 

Is your business prepared to implement these tools? What demands will they make on hiring? And who’s in a position to profit from their growth?

“I'm not an AI or ML  expert,” Hightower cautioned. “I don't work on models, I don't do research. I'm just a casual observer, thinking through the pros and the cons. If I were challenged to get an infrastructure team to enable a company to do this kind of stuff, this is the way I would think about it.”

The last mile

Before they can even get to that stage, Hightower said, most enterprises have to make significant changes to their tech stacks. “There's a spectrum in terms of enterprises’ talent and ability to execute,” he said. “Outside of the big tech companies, the majority of common enterprises are in the business of doing a couple of things and outsourcing everything else,” including installation and support of most IT systems.

“So given that this advanced technology isn't just a cookie-cutter, off-the-shelf, buy-it-from-anyone commodity, you don't have the talent to fill in the gaps,” he continued. “You can lie to yourself; you can go hire the fancy C-suite person, but that one person will not transform you into an AI-ready enabled organism. 

“I think the reality is, you can't buy your way into this yet. This ain't iPhone territory yet! This is for the people who are close: You have data scientists effectively doing business analysis. You’ve got your Oracle dialed. You’ve  got your data dialed in. You're not running around saying, ‘How do we put all that data in one place?’ 

“If you're beyond that — if you already are doing data science, you're probably answering 80, 85, 90% of the questions you want to ask anyway, — you're already doing pretty well. Now you're looking for that last mile: ‘Yo, we got all these unstructured data, we got all these documents. We know that we have knowledge gaps, and maybe this advanced tool can close knowledge gaps and help us see things we could never see before. Because we have some of these larger, general-purpose models that we're going to then fine-tune for our situation. And that fine-tuning is the skill gap. And the truth is, you probably can't buy it because most people can't even sell it. And I think this is the excitement: Whoever closes your gap, the belief is they're going to leapfrog everybody else.”

Staffing for the future

Data is the lifeblood of most modern businesses, and companies in fields like biotechnology and advanced logistics are leading the way for data-intensive workflows. As data demands grow, by how much will businesses need to grow their data-science teams? 

Hightower said he expects expertise to grow in a different direction based on past precedent. “When mainframes became a thing, and more people bought mainframes, more people were needed who knew how to load and unload punch cards,” he said. “The more people started using mainframes, the more punch-card people we needed. Then we stopped using punch cards. Now, did we go out and get a bunch of floppy-disk people? I don't think we did. I think it ended up being much easier to deal with data using disks than punch cards. So the people who were already doing the work just also did that part, too. 

“I think the way ML tools are shaping up, we may end up in a world where it's the exact same number of data scientists. Maybe the value of a data scientist goes up, maybe the average person’s ability to use ML goes up. So instead of having to dump a bunch of data into some database and learning SQL to get anything done, now I'm watching the same developers who are building normal features just integrate with something like OpenAI. Like, ‘Hey, we'll just use open AI for natural language processing, or we’ll use some other third-party service.’ 

“Does that mean we just got a whole bunch of new data scientists? I don't think so. I think we're going to scale in a much more quadratic fashion. It's not gonna be more data, more people. I think if these tools are working, people do more.”

Dollars vs. data

To make meaningful contributions to human endeavors, Hightower insists, an LLM must be tailored to the specific context the organization wants it to address. Handling that kind of development in-house might be a barrier to entry for most companies — but it can be a unique competitive advantage for those that can provide these solutions. 

“Lots of companies write programs using programming languages — Java, Python, you name it,” he said “But how many of them employ people working on compilers? Not a lot.

“Five years ago, an insurance company might say, ‘Man, what if we could process claims by flying a drone over a crash site, taking a bunch of photos, and then using proprietary models? And now they might say, ‘There’s already a crash-detection algorithm that you can download off the Internet. We may not need to develop our own unique crash analyzer. Let's just use the other one — because it's so good — and we'll tune it.’ “

Companies that can provide those tools stand to cash in, Hightower said. “In that brief period where no one knows who the winners are going to be, it's just going to be raining money! Venture capitalists are going to figure it out: ‘If I give a team $5 million that so they're going to be working on this … Worst case, that team can be bought for $30 million to $50 million by some corporation that just wanted to hire some great data scientists! It doesn't have to go all IPO [to show a profit]. And maybe there's some IP that comes out of it. So I do think there'll be a lot of funding. 

Startups may have an edge in innovation, Hightower said, large, established companies can access data sets beyond those upstarts’ scope. “If you're a startup, where are you getting the data from? You're getting it from State Farm? They’re just going give you their data for free and not take any equity?” Hightower laughed. “I think we're going to find out that while VCs may supply the dollars, the incumbents are going to supply the data.”