EMCCloud

Openstack Operations
OpenStack Operations: Challenges and Solutions
Kenneth Hui @rubrikInc HQ
On the topic of skill gaps, what do u look 4 when u want 2 hire someone that u think can learn OpenStack ops?
Cheyenne Bryant
The ability to be agile and learn. Python and networking are big plusses.
Kenneth Hui @rubrikInc HQ
How important are DevOps experience?
Tim Bell
At CERN, we're generally looking for Linux/Windows admins with an open source community approach. We find people learn very quickly once they get involved in the community.
Scott Carlson
high quality troubleshooting skill, the ability to think, the ability to read/interpret other people's code.
Cheyenne Bryant
DevOps experience is important for the troubleshooting aspect, but in general it's more important to find someone with a willingness and ability to learn without falling behind.
Kenneth Hui @rubrikInc HQ
@noggin143 Can u elaborate on OSS community approach?
Sean Winn
I look for people with problem solving skills and critical thinking skills who can admit to not knowing. Not having the answer and admitting to it is hard.
Tim Bell
The willingness to share your knowledge and learn from the experiences of others
cloud foreign
all of our code is open source. We require that our engineers contribute code with that in mind. We accept pull requests from outside the org and curate our releases.
Danny Abukalam
There's still a massive gap with people who just aren't used to working in the open and capitalizing on the open source community resources available because they come from a closed source background.
Kenneth Hui @rubrikInc HQ
@craig_tracey Does this mean OpenStack operators have 2 be able 2 write code?
David Pollack
@Dannyabukalam What's best way to bridge gap?
Tim Bell
we've found the OpenStack summits help people to understand community. Sitting in an ops design summit session is a great way to start to understand the benefits of sharing.
John Furrier
DevOps is an engineering position not just coder; top exec & geeks I speak with say ability to engineer is key and coding is requirement in some cases or at least some basic coding capability
Scott Carlson
Yes, but not necessarily python code. scripts/html/bash/perl/python all counts because automation is key
Tim Bell
Lots of ways to help other than coding... help on the documentation is an easy one
Tim Bell
Helping out people on http:://ask.openstack.org
Kenneth Hui @rubrikInc HQ
@noggin143 Yes but I wonder if @craig_tracey is implying that 2 be a proficient OpenStack operator u have 2 be able 2 code?
Danny Abukalam
@DavidMPollack It's a different way of thinking. I don't think it's something that will happen overnight. It's not just individuals but also entire organisations that have this way of thinking.
Sean Winn
To become a better operator, I've been learning to code. I've operated Infra for 20+ years thinking I could avoid it. I can't.
cloud foreign
absolutely not. We have a spectrum of "coders" and "operators"...we do expect that our operators can work with configuration management.
Tim Bell
From our experience, python experience is not needed to run a cloud. Understanding logs and googling tracebacks is needed.
David Pollack
@dannyabukalam Vendor marketing and product management has its work cut out for them. No time to waste.
Sean Winn
@colinmcnamara has a great talk on contributing as an operator - https://www.youtube....
Contributing Back As an Operator
Contributing Back as an Operator features Colin McNamara of Nexus IS, Inc. OpenStack needs more operators contributing back. Learn about the challenges, and ...
Ashok Bhojwani
This should be a 3 fold approach. Prepare for the Cloud admins, Get the right Use Cases for the Business, Get the right set of developers who can code for 3rd Platform apps.
Azhagarasu A
Dear Experts, May be this is a very basic question but I would like to understand what would happen to instances running on a Nova compute node (KVM based) when it fails and disappears from cluster?
cloud foreign
those instances would be unavailable. Your ability to resurrect them depends wholly on your architecture. But cloud advocates cattle over pets.
Sean Winn
That depends on how you created that instance. Heat has ways to deal with this using Ceilometer and autoscaling. There's also PaaS.
Sean Winn
also application architecture has a tremendous impact on what effect the outage will have
Kenneth Hui @rubrikInc HQ
It's assumed that app owner is responsible 4 handling failures.
Scott Carlson
If you have a single compute node with local storage, they die. If you have shared storage, you can likely resurrect elsewhere.
Sean Winn
most of the hypervisors support some form of vm migration - live or not depends on other architectural choices like shared storage
Shamail Tahir
The loss of a VM != loss of service necessarily
Kenneth Hui @rubrikInc HQ
This brings up an interesting ops question - should OpenStack pivot 2 provide HA 4 traditional enterprise apps?
Azhagarasu A
@craig_tracey Agreed. But instead of using KVM as hypervisor. if i use vCenter (ESXi) and integrate OpenStack. would that make a high available solution?
Scott Carlson
@hui_kenneth If it doesn't, the people who want to run traditional enterprise apps CANNOT move.... you always have vmware with vmotion underneith though.
Shamail Tahir
@craig_tracey If you use vCenter then you could leverage vMotion, HA, etc. to make the instance resilient.
cloud foreign
again, that all boils down to your architecture. For me, relying on a hypervisor to provide "reliability" is not an option. The application itself needs to be fault tolerant and cloud aware.
Sean Winn
@hui_kenneth #OpenStack lets the architect/deployer choose what level of availability there is. I usually advocate application resiliency over redundant infra, but some policies require it.
Scott Carlson
@craig_tracey Many "COTS" software cannot be configured w/o much IT pain. Those are the apps that have the toughest time becoming "cloud ready" unless you stick them on VSphere with VMotion
cloud foreign
I can guarantee that the most heavily utilized web services on the planet are not migrating failed VM's.
Shamail Tahir
@craig_tracey @seanmwinn Agreed, but depending on the use-case the user may not always have access to modify app. This is the duality of OpenStack use cases.
Azhagarasu A
@relaxed137 Yes its a shared storage from dedicated storage nodes and also in my example its multiple compute nodes.
Kenneth Hui @rubrikInc HQ
@relaxed137 @craig_tracey What are ur thoughts on vSphere with OpenStack & with VIO, VMware's OpenStack distro?
cloud foreign
@ShamailXD 100% agree. But this is why folks need to stop considering lift and shift of legacy apps.
David Pollack
@relaxed137 Aren't there growing number of tools to help with the transition?
Scott Carlson
@hui_kenneth i'm not a fan of 'vendor' distro unless you are willing to go "all in" with that distro and make it meet your needs. sometimes you have to adjust your business process rather than adjusting your cloud. It depends on if you have devs and ops.
cloud foreign
@DavidMPollack absolutely! OpenStack even has an attempt or 10 at this sort of thing. My bets are are Cloud Foundry at the moment.
Shamail Tahir
@craig_tracey I see users making that decision but we need to, as a community, help them understand the decision.
Azhagarasu A
@ShamailXD But can i eliminate this by using ESXi hypervisor?
David Pollack
@ShamailXD Great as a community, but it sure does hurt when EMC or others step to plate from business driving perspective
Scott Carlson
@DavidMPollack sure there are and often the "Tech Fighting" is the hardest to overcome. There are still people who don't think workloads can be 'virtualized' and god forbid if we ever talk 'cloud'. remember its just servers, cpu, and ram
Shamail Tahir
You have to use vCenter along with ESXi to leverage the VMware drivers.
Azhagarasu A
@craig_tracey yes, but if someone wants to use the Instance just as a server and he/she plans to run a application of their choice would using ESXi or Citrix help to achieve 99 % availability of instances.
Azhagarasu A
@hui_kenneth Yes i think if we use ESXi or citrix the environment becomes redundant and highly available from the infrastructure perspective.
Azhagarasu A
@ShamailXD yes, vCenter provides good high availability features for a traditional DC. but if OpenStack is used on top of vCenter are we making OpenStack highly available because of the underlying infrastructure?
Shamail Tahir
There are different layers that you need HA. The API, the meta-data, the actual data plane, etc. In this case, you would be accounting for data plane HA and not necessarily the others.