Kubernetes

Kubernetes Influencer Chat
Getting smart influencers together for a sharing data and commentary on #Kubernetes. Question will be asked & answered by the community
   5 years ago
#KubernetesKubernetes AdoptionThis channel members can discuss about Kubernetes adoption at the MultiCloud (AWS, Azure, GCP), DevOps toolchain and its ecosystem.
John Furrier
What is Kubernetes used for and why?
Sebastien Goasguen
deploy and manage distributed apps made of containers. Orchestrates all things that need to fall in place for the app to work well, be resilient and scale
jameskobielus
Kubernetes’ core function is to orchestrate, automate, and scale the DevOps pipeline for cloud-native microservices.
jameskobielus
Kubernetes enables deployment of containerized microservices that are platform agnostic, dynamically managed, loosely coupled, distributed, isolated, efficient, and scalable.
John Furrier
What use cases specifically are customers using it for
jameskobielus
Kubernetes executes Linux containers on clustered hosts that may span public, private, hybrid, and multiclouds.
jameskobielus
Kubernetes groups the containers that make up an application into logical units for easy management and discovery.
Captain Dee 🧜‍♂️
@sebgoa why are Kubernetes so hot right now as compared to other container management systems?
Tom Phelan
k8s has historically been used to run stateless apps, but we are seeing more and more users wanting to run AI/ML/DL workloads on k8s
Brian Gracely
@Dee_Marketing largest OSS community, broadest support for range of applications, community not dominated by a single vendor, extensible technology
Tim Hockin
1) Manage applications with modern, dynamic, scalable techniques; 2) provide a platform for building and extending APIs to manage applications
Sebastien Goasguen
@Dee_Marketing Kubernetes is a clone of Google's Borg written from scratch in the open. People recognize where it comes from and that it is a real strength compared to other solutions. Releases are also very strong and stable.
jameskobielus
Customers are using Kubernetes to orchestrate containerized microservices across multiclouds.
David Floyer
@Dee_Marketing Kubernetes Reached 50% of adoptions first, with Swarm a long way back. The user & project momentum has gone with Kubernetes.
Donald C. Klein
@tapbluedata What has to happen to run more stateful AI/ML/DL workloads in containers?
ConsumingML
in the DS world. Deploying AI and Machine Learning models. An often overlooked use case is scaling Python and R w/o Spark. While not distributed, being able to Send R and Python jobs to other nodes makes processing and scaling easier for large DS teams not on Hadoop.
Tom Phelan
either the AI/ML/DL apps need to evolve to a micro services architecture or k8s needs to have improved support for data persistent on the "root" storage as well as management of multiple services running within a single container
Tim Hockin
@tapbluedata Local Volumes are in flight now. Not sure what the last point means?
Tom Phelan
PVs are great for non-root storage, but most unmodified AI/ML/DL apps want to store config data in the "/" (root file system) and PVs cannot currently be mounted as the root storage of a container on k8s
jameskobielus
@Grant_Case Right. K8s is transforming the AI, ML, and big data analytics market. In Hadoop, for example, what Hortonworks/IBM/Red Hat are doing under the Open Hybrid Architecture initiative.
Tim Hockin
Why is the directory "/" important as compared to "/data" ?
Lee Calcote
among other things, Kubernetes is used to extend the initial promise of portability that made Docker so valuable (among other qualities).
ConsumingML
curious if anyone sees interest in Spark on k8s or is grafting Spark on missing the point. There seems to be a lack of processing platforms for really large ML/AI problems beyond Spark and the only place you seem to go is Hadoop.
Tom Phelan
@Thockin most AI/ML/DL (Hadoop, Spark, etc) write to "/etc" for config files. PVs can be mounted on /etc, but you need to be careful when crafting the original image otherwise files that were originally placed in /etc will be hidden when the PV is mounted.
jameskobielus
@tapbluedata That's config data. Where do AI/ML apps want to store application data (for training, inferencing, etc.)? Are K8s Persistent Volumes OK for that latter use case?
Tom Phelan
@Thockin Ai/ML/DL apps can be modified to work well with PVs and storing their config data - it just takes some care when setting up the container image
Tom Phelan
@jameskobielus absolutely. PVs are ideal for storing app data
Tim Hockin
@tapbluedata The more rigid apps are, the harder they are to port to different environments. REQUIRING that you can write to /etc is a terrible idea. Some things need to be made more flexible in order to work well in more dynamic, managed environments.
Tim Hockin
That said, I would really like to find a way to have a merging FS abstraction in k8s
Tom Phelan
@Thockin I completely agree. The AI/ML/DL apps must evolve to a micro services architecture

(edited)

Tom Phelan
@thockin I agree wrt merging FS abstraction support in k8s - that would make running stateful legacy apps on k8s easier
Sarbjeet Johal
high level: new apps because it makes it easy to ship & operate em. But there are certain type of workloads which are better fit for #kubernetes
Stuart Miniman
Via @nigelpoulton "I'm keen for thoughts on what is being done to improve simplicity for more than just spinning up a cluster? And related.... is the job of making #K8s simple the job of the core project or the ecosystem"
Sebastien Goasguen
there are lots of tools for application management out there, 70+ and definitely the goal is to make onboarding devs much easier. Helm is one for sure, kustomize is another great little tool
Gabe Monroy
we have a lot left to go in simplifying k8s for devs and operators. i feel very strongly that this should be done outside of k8s core, which is not a developer tool.
Tom Phelan
agreed. k8s is a platform
Tim Hockin
This is a tough one. There's some inherent complexity in the problem domain
and I am always wary of anyone who says they can paper over it. That said, we
have historically tended to err on the side of "exposed ductwork" in
Kubernetes. I get a lot of feedback from peopl
Brian Gracely
Operator Framework will significantly help with Day1/Day2 deployments of applications on Kubernetes.
Tim Hockin
e who want MORE configurability
in kubernetes, which is antithetical to simplicity in many ways. I think we've
done an OK job building higher levels of abstraction (Deployment > Pod) but
there's more to do. It's not clear whether those higher layers are "part of"
k8s o
Stuart Miniman
@gabrtv well said, and addresses another of @nigelpoulton comment that he is worried about scope creep. Balance of what is in the core vs too many ancillary projects. I do consistently hear that #k8s is "hard" and that "victory" is often just getting it running.
Gabe Monroy
nailing simplicity requires a deep understanding of user personas. k8s is not designed for use by developers, but by operators. a lot of failed attempts at simplicity can be tied back to this misunderstanding.
Nigel Poulton
I think the *getting K8s* running is waaaaay simpler these days. One cloud platforms we can just ask for a cluster and get one. That's a real score! But...... then there's so much more complexity if doing it the right way - pod resource requests, HPAs... just to name some
Nigel Poulton
I still feel like running K8s properly for production is a large mountain to climb. And one where the rapid development of the core product makes it a fast-moving target.
Tim Hockin
@thockin r ecosystem, though. I lean towards ecosystem.
Nigel Poulton
@gabrtv Agree with that Gabe!
Tim Hockin
@nigelpoulton There are a lot of concepts, but many/most are orthogonal and additive. Making FULL use of k8s should not be the day-1 goal - find ways where it can help you achieve your goals.
Stuart Miniman
@nigelpoulton had to run, but asked: Is there a future on self-provisioned on-prem K8s clusters (will the tooling etc be developed going fwd) or is the more likely future on-prem managed by cloud. E.g. GKE On-prem?
Tim Hockin
K8s is in the "awkward adolescence" stage - I think it will get simpler and more robust as it matures, but it's still experiencing painful growth spurts right now, and still has a fair number of "blemishes"
John Furrier
What’s the hottest project in opensource that illustrates the promise and direction of Kubernetes? And Why?
Lee Calcote
Service meshes are certainly afire at the moment. While there's much to be built into those various projects (e.g. Linkerd, Istio, etc.), this additional layer - a layer just after orchestration - is crucial to those running many services.
Sebastien Goasguen
I would have to say #knativeproject because to me it goes in the direction of a better dev workflow. But projects that are CRD based like the AWS service operator https://github.com/a... are terrific.
Brian Gracely
Operator Framework, leveraging native Kubernetes concepts (e.g. CRDs) to make app-deployments simpler across any cloud. Tensorflow for AI/ML on Kubernetes. Knative will bring some consistency to Serverless on Kubernetes.
Gabe Monroy
the Virtual Kubelet project is on track to unlock "Serverless Kubernetes" (Kubernetes without VMs). Microsoft is working with AWS, Ali Cloud, VMware, Hyper.sh and others on this project, which elimates VMs in favor of hyperscale pods-as-a-service backends.
John Furrier
@sebgoa I have to say that I love service meshes
John Furrier
@gabrtv thx I have a serverless k8 question coming up bc I see those 2 as great moves being made
Tim Hockin
Kubernetes as an API to build APIs (CRDs, webhooks, controllers) is very intriguing to me. It's the sort of thing that almost always fails if you set out to do it on purpose, but this was emergent.
Sebastien Goasguen
Ok i will give a +1 to Virtual Kubelet as well :) I think there is a use case for it in edge computing
Gabe Monroy
agree with @Thockin on that. Kubernetes architecture, but particularly its extensible API design, has proven to be one of its most important features.
jameskobielus
Istio (service mesh), Kubeflow (for DevOps workflows on containerized ML apps), Virtual Kubelet (for serverless interfaces to Kubernetes apps), and KubeVirt (for building, modifying, and deploying microservices into containers and VMs).
Lee Calcote
@gabrtv agreed. Moreover that the shape "what" Kubernetes is isn't lost in its extensibility. Something that OpenStack lost along its way?
Lee Calcote
@thockin good highlight. I was just reviewing a new controller this morning that my team has written to integrate our Kubernetes deployment with our products.
BlueK8s
Another hot new open source project is KubeDirector, for deploying and managing complex stateful applications on Kubernetes: https://twitter.com/...
Brian Gracely
keep an eye on KubeVirt, the ability to also manage VMs under Kubernetes