ConfigMgmt

Definitely! We're seeing more teams move to immutable infrastructure and work their security into their development pipelines a la DevSecOps. As this happens, people are questioning whether they need full blown config management or whether bash and powershell are sufficient.

I think that the lack of automation in container-aware infrastructure patching, can lead to that, but at the same time it's easier than ever to do live patching in a container environment. Also integrating container security scanner tools will force updating the base images.

How do you do live patching of container-aware infrastructure? Would you ever run a config mgmt. agent in the container and depend on it to pull updates from package repos or is that an anti-pattern?

I agreed but there are still issues with many organizations trusting public docker images, which is a large concern. We do need more automation and security automation as, like you said, they are lacking. So patching is easy, but who's doing the looking?

live patching would mean identify the host, drain it (do not allow new containers to start on it) terminate gracecefully the containers it runs, then patch, reboot, test, add it back, all automated but the automation tool needs to "talk" to the container orchestration also

This is a task for configuration management tools, more for push-type (ansible) than others

Primarily my concern was directed to getting patches onto the container image itself, but this is also an interesting point.

DC/OS currently does support this automation, and in my experience is works pretty well. There are some caveats, for instance if the images need persisted storage, but for stateless containers it handles live failovers well.

To rely on the immutable infrastructure principle there would be no patching, you would simply start a new image with the new software versions on it and join it in the cluster, and drain then destroy the old instances.

and here the config management tools would be used only in the instance launch/configuration phase, if needed, or in the image build phase.

If you were building a brand new pipeline, would you select Puppet, Chef, Salt, etc.. for this? How about if you already had Puppet/Chef/Salt deployed and working? Would it be worth changing your pipeline?

mutable = servers change. A server might have ver. 1 of an app, then v2, then v3. Immutable servers don't change. An update means creating a new server image and launching a new VM, container, etc... Immutable requires a mature CI/CD and automation pipeline.

Traditionally, teams build servers and then deploy their apps to them. When the app changes, updates are sent to the server. With immutable infra., servers never change. They are destroyed and new servers are created. Update and rollbacks are standard inf. rollouts.

It's also a security layer as you can have immutability mean also readonly data in containers, an attacker has a much reduced access from within the container to deploy tools and attack further.

In general yes, but there are some common best practices that are easily broken. For instance DC/OS allows containers to have sudo access, which is required for some of their out of the box images.

We lean on the 12 factor app!. "An app’s config is everything that is likely to vary between deploys". I argue that config should be stored outside of image artifacts and pulled in at deployment time. Is a tool like Puppet or Chef the right solution to pull cfgs?

I think it's important to remember that Puppet is over 10 years old, there are still many companies just now migrating to some form of config management. There are also a lot of companies still using bare metal servers. Any form of management is better than none.

And it's mature with lots of modules to make it turn-key. It's definitely still relevant. Underlying infrastructure to enable containers need to be managed, too!

We would love to hear from you. Type in your thoughts, opinions, or questions.

Thanks Krista! We want to talk about config mgmt and its role in the changing world of cloud computing. Agent or agentless? Do we still need heavy config management tools or are lighter weight solutions now preferred?

@ghbaker Lighter weight solutions definitely seem to be more favored recently but there is always a use case for the 'old school' puppet/chef options. But in my experience there are rarely advantages to using an agent based management tool when you are using immutable infra.

I think lighter weight solutions like Ansible have become popular because they put the control in the hands of developers and enable them to quickly create new environments without needing a centralized config mgmt server. Containers support this view.

Some larger teams resist this due to the complexity of their infrastructure and their regulatory/compliance needs. They need tighter control of the infrastructure tier while also allowing development teams to dictate how their app is deployed and configured. Can we have both?

@ghbaker Yeah but there is the other issue. Generating ansible inventory lists can get cumbersome. They do have nice scripts for big vendors, but often times you want some more customization. And if you aren't using immutable infra, config drift is a common concern.