Pivotal CTO: Kubernetes means we’re all distributed systems programmers now

Source: devclass.com

Kubernetes is changing the way organisations develop their software and manage their systems. But can we expect it to start changing the nature of organisations themselves?

It’s certainly changed the nature of Pivotal – which launched a drive onto Kubernetes last month, not long after being lambasted by financial analysts for scaling back financial targets and being late to the Kubernetes party.

We sat down with Cornelia Davis, Pivotal’s CTO at DevOps Enterprise Summit in June. She was there to talk about Functional Programming – but not as it applies to software, rather as it applies to the systems space. And, in her view, Kubernetes is central to this.

To understand her position, it helps to understand how she got there.  “I’m a developer by background,” she says, and when she began learning Compsci, “it was all imperative – I learned Pascal and C and all those languages.”

Many years on, she argues, that “detour” is suddenly starting to look a lot more relevant. “We are starting to see a resurgence, or an emergence of the relevance of functional programming and in languages that are being used in industry…Now we’re seeing individuals start to programme in languages like Kotlin, Clojure, Scala”

“I believe it’s because we’re all distributed systems programmers now. When I started my degree 30 years ago, that was niche. Now, in the cloud, everything is distributed and it just turns out functional programming works far better in distributed systems.”

This is relevant more broadly, she says, because if you think about Conway’s law – the notion that organisations will produce designs which replicate their communications structures – “This gives us the opportunity, even the necessity to change our organisation structures. It actually permeates out to the people and process side.”

“So, if you’re thinking about what was the functional analogue in the systems space, it’s declarative expressions. With Kubernetes, we say here’s what I want my topology to look like…and that’s the way the maintainers of the applications deploy their systems which is very very different.”

“I spend a lot of time in the Kubernetes community,” she continues, “And there’s always this tension between purists and pragmatism….My whole thesis is if we change the programming model, it allows the humans to turn their cognitive load to solve problems they couldn’t solve before.”

So, if Davis has been getting deep into the Kubernetes world, why does Wall Street, amongst others, think Pivotal has missed the Kubernetes boat? As always, it depends on your perspective.

Davis recalls a conversation with a colleague, who asked “Did we really need to go to Kubernetes? Could we just put our head down and stick with our PAS product, because that was working?”

“And I said we had to go to Kubernetes…I think honestly the reason we got a bit of a kicking was because people are starting to understand the potential of Kubernetes…and their instinct is this is going to change things.

And Pivotal knows this, she says, “because we’ve been doing it for about six years…our platform has been container-based.”

But it hasn’t got that crucial K word…”And so I think we fell into a bit of a trap when we said ‘we’re already doing this, Kubernetes is an implementation detail’.”

The reality she accepts, is that Kubernetes has “This energy that is independent of Pivotal…we have a few 100s of customers, and Kubernetes has captured the mindshare of 100s, 1000s of organisations.”

“So we perhaps could be accused of coming to the Kubernetes party a little late, and it’s because we felt we were already there.”

“But the good news is we also are going to offload some of the stuff we used to have to build and maintain ourselves.”

At the same time, she said, a “big portion of the value” Pivotal’s customers derive from PAS is it’s turnkey nature. “In the Kubernetes world – which really comes from more of a Linux philosophy, components and compositions – creating a platform that is both turnkey and gives room for optionality, is our big challenge going forward.”

Which brings us back to Conway’s Law, which in the Kubernetes space, is a “completely unsolved problem” Davis says.

Pivotal’s platform, “allowed us in a lot of our customers to do something very interesting from an organisational perspective. It allowed us to establish something called a platform team, responsible for providing this platform.” The app team then can just go to the platform on a self-service basis.

“In the Kubernetes space, where those personas live and what they can do is not really being talked about…are you going to allow your app teams the access to actually change the compute infrastructure?”

“And this value of the platform team and the app team has probably been the most enabling thing in our six year history. We’re going to take all of that experience and knowledge and apply it now to this Kubernetes space.\”

So, Pivotal with its so far still alpha Kubernetes strategy will bring some of the traditional disciplines to the still somewhat unruly Kubernetes space. It’s an argument that companies have often used in the past, when they’ve been superceded in the tech arms race. Will it be different this time? Well, who’d have expected functional programming to come back into vogue.