Kubernetes

imho storage and networking concepts will always be something to modernize and make programmable.

Storage is hard. Maybe the hardest of all the infra problems. It's not surprising that it is lagging. The need for really dynamic cluster FSes has never been greater, but those things take a long time to stabilize.

@furrier in the virtualization wave it took us almost a decade to "fix" storage and networking (storage led networking). Networking has been leading storage in the container and #k8s maturation

k8s has done a great job defining storage APIs (PV, PVC, storageclasses). the storage implementations in cloud are well defined, but there is still a lot of options on-prem. choice usually comes down to minimizing storage ops burden on-premises. check out portworx for that.

@thockin to your point... in reflecting on this challenge within HPC-land, new parallel filesystems take many years to harden and prove seaworthy.

@lcalcote Precisely, and Containers just gave them 100x the problem they used to have.

@lcalcote that's a great point. What is the status of K8/container security?

(edited)

indeed. CSI is in it's very early days and doesn't allow to expose of the unique storage characteristics yet..

Ceph get/put object store simplifies however is that the storage platform for distributed fault tolerant systems?

two neighbors of #Kubernetes, storage and network can’t catch up fast enough. It’s like she has her lawn all green and mowed but two neighours aren’t even watering their grass. She is stuck in bad neighborhood. 😂

The storage side of the stack continues to proliferate projects. There's also Apache Hadoop Ozone, which is developing scalable distributed object store that is designed for containerized environments such as K8x in which storage and compute are decoupled.

(edited)

@Dee_Marketing K8s+container security has seen much advancement - more so than K8s+storage. Advancements in Docker and K8s themselves have certainly helped, while separate OSS and/or vendor tooling have operator's concerned well-covered, I believe.

The problem with storage is that we have, for 40 years, told developers to use fopen() and rely on POSIX semantics. That doesn't scale to Cloud Native. Objects stores are better but are NOT compatible with the old mindset and the billions of LOC that rely on POSIX semantic

@jameskobielus agreed. the decoupling of compute and storage makes it much easier to run legacy AI/ML on k8s

@thockin agreed. the dependence of apps on POSIX slows their migration to cloud/k8s

@tapbluedata I see this as the fundamental stumbling block for SO MANY apps - They are all-in on cloud native, but they want a multi-writer, strongly consistent, high-perf, low latency shared filesystem.

My sense is that there's a future in self-provisioned locally managed K8s clusters at the edges, capable of autonomous self-management.

we need all of the form factors. cloud-connected on-prem makes management easier where possible, but fully isolated on-prem k8s has many, many use-cases in the enterprise -- more than i appreciated before i joined Microsoft :)

freedom of choice is what we need in the industry, but certainly managed k8s services have made things so easy that going on-prem needs a strong validation. GKE-on prem and variants will definitely be super strong offering that will compete with PKS and Openshift.

@nigelpoulton if someone could give us a date on when on-prem IT will go away, it would help us all know when to short stocks.

@bgracely April 1st 2021

@sebgoa perfect. looking forward to the Google April Fool's Day video.

@bgracely yes - on-prem storage, especially for storing sensitive data, will not be going away anytime soon.

Service meshes are certainly afire at the moment. While there's much to be built into those various projects (e.g. Linkerd, Istio, etc.), this additional layer - a layer just after orchestration - is crucial to those running many services.

I would have to say #knativeproject because to me it goes in the direction of a better dev workflow. But projects that are CRD based like the AWS service operator https://github.com/a... are terrific.

Operator Framework, leveraging native Kubernetes concepts (e.g. CRDs) to make app-deployments simpler across any cloud. Tensorflow for AI/ML on Kubernetes. Knative will bring some consistency to Serverless on Kubernetes.

the Virtual Kubelet project is on track to unlock "Serverless Kubernetes" (Kubernetes without VMs). Microsoft is working with AWS, Ali Cloud, VMware, Hyper.sh and others on this project, which elimates VMs in favor of hyperscale pods-as-a-service backends.

@sebgoa I have to say that I love service meshes

@gabrtv thx I have a serverless k8 question coming up bc I see those 2 as great moves being made

Kubernetes as an API to build APIs (CRDs, webhooks, controllers) is very intriguing to me. It's the sort of thing that almost always fails if you set out to do it on purpose, but this was emergent.

Ok i will give a +1 to Virtual Kubelet as well :) I think there is a use case for it in edge computing

agree with @Thockin on that. Kubernetes architecture, but particularly its extensible API design, has proven to be one of its most important features.

Istio (service mesh), Kubeflow (for DevOps workflows on containerized ML apps), Virtual Kubelet (for serverless interfaces to Kubernetes apps), and KubeVirt (for building, modifying, and deploying microservices into containers and VMs).

@gabrtv agreed. Moreover that the shape "what" Kubernetes is isn't lost in its extensibility. Something that OpenStack lost along its way?

@thockin good highlight. I was just reviewing a new controller this morning that my team has written to integrate our Kubernetes deployment with our products.

Another hot new open source project is KubeDirector, for deploying and managing complex stateful applications on Kubernetes: https://twitter.com/...

keep an eye on KubeVirt, the ability to also manage VMs under Kubernetes

We (Google) tend to think of ops in 2 groups - cluster ops and app ops. They might be the same people but they are different hats. I think that is the split in those statements.

presumably the former refers to k8s infra, whereas the latter is around success with k8s to deliver distributed apps -- at least that's my read. thanks to managed k8s services like AKS/GKE/EKS, the former is a lot easier than the latter.

(1) is likely someone that manages the Kubernetes platform, and (2) is someone that's either a developer (doesn't know about the platform) or uses a managed Kubernetes service

One area that I see lots of interest in is the customer architecture for cloud as many are putting toes in the water on not being on one cloud. So I worry about what I call the hadoop problem - running it isn't the same as using it for something impactful

poor Hadoop

I mean that's what I'm seeing. Containers/Kubernetes used for experiments or new project.

It also hints at a challenge in the industry where people may adopt a tech without clear problems. You first need to have a clear business problem and then solve it with various tools. Running Kubernetes does not help, it is what you do with it that matters.

@sebgoa Mesosphere has a really interesting case study about Royal Caribbean using containers to deliver internet and apps on cruise ships.

for a list of good use cases go to https://kubernetes.i... and hit the CNCF youtube channel for talks form Kubecon, lots of production people there, definitely not just greenfield

lots of examples of customers/companies running Kubernetes/OpenShift across multiple cloud environments. it provides a nice abstraction from the differences between clouds.

Financial Times had an interesting talk on why and how they adopted Kubernetes, and the time consuming conversion from their in-house platform to get there.

Those statements sound like they're saying the same thing.

@sebgoa CNCF also posts a ton of case studies / end users that show cloud native adoption (outside of just Kubernetes) https://www.cncf.io/...

@jameskobielus I saw lots of people running things but not accomplishing things and the group parsed it well and end users have to understand running k8 to achieve an objective; running it for running sake isn't the goal

But getting it running at all is the prerequisite for getting K8s to do useful things.

@cra yes and I always point people to your youtube channel :) https://www.youtube....