LocalData

service providers, microservices, hybrid cloud, and IoT applications to name a few

There's not enough CPU,mem,storage in a single system to solve just about any problem

Unpredictable yet fast infrastructure growth along with decreasing administrative time per unit of things to be managed.

Many applications are now being born "cloud-native." That brings with it an assumption of horizontal scale and other architectural properties.

humblebrag: Wikibon nailed the #TruePrivateCloud market analysis cloud is in high demand onprem

Also never know how fast an application might grow from the outset

dev ops

data growth ... ridiculous exponential data growth driven by digital transformation, cloud, IoT, etc

As a macro trend, data analytics / Machine Learning / AI / Deep learning / whatever you want to call it ... it's here and it demands massive scaling of data coupled with CPU capacity.

Data growth! http://inside.igneou...

There's also a whole set of scale-out technologies pioneered at FaceBook/Google/etc. that are now looking for for problems to solve + monetize in the mid-market and enterprise.

so I think partly of it as being driven by customer demand but also partly driven by tech searching for a problem in other markets

@steve_pao Agreed... Businesses are generating a ton of new data and are afraid to delete any of the old. All are looking to a world where they might get answers from big data one day.

Virtualization, I believe, actually stagnated innovation for about a decade. This has been needed for a while.

Data explosion demands new architectures. 6 Zetabytes today, growing to 90+ Zetabytes in 2025.

Another trend is the shift of IT capacity planning from OpEx to Capex. Business leaders demand agility and to pay only for what they use.

- #TruePrivateCloud is a departure from "private cloud" claims of legacy providers. https://cube365.net/...

Andrew: @andriven what problems are being solved by these new architectures

the need to independently scale resources will always be there, any time you purchase resources that aren't absolutely required you're missing an opportunity for cost optimization

people (mobile) and data (sensors) at the edge

One driver is the flexibility that comes with knowing you can grow as needed with little or no disruption. You don't have to size it correctly the first time.

Another driver is the amount of data being stored. Legacy architecture is not capable of storing this much data in a useful way. Users want to do more than pump and dump data into some repository.

@fdmts So business leaders want to take the risk out of decisions they have made?

A lot - I'd argue hyperconvergence is possible due to scale-out tech. So is containers/Docker/Kubernetes. Obviously what I do at my day job is made possible there too.

Clustering adds fault tolerance scale out does not necessarily add ft

I think that most scale out systems are "clustered" in one sense of the word. The distinction lies in scale out's ability to expand capacity in a balanced way without introducing bottlenecks.

cluster = quorum and more active/passive. Scale out provides all nodes to perform a function.

clustering often refers to a dual node system, though not always, in a sense, most traditional scale up storage architecture are actually 2 clustered controllers

Clustering is way to achieve scale-out.

begs the question "how do clusters scale out" :-)

historically clustering was about resiliency and removing single points of failure. Let's go back to Novell Cluster Services for instance. Scale-out is focused at very different thigns.

clustering usually means tightly coupled. all the resources are pooled together. scale-out many times just means dividing up the work

@csharney Clustering doesn't necessarily add fault tolerance. It's a big place.

Clustering may not add performance or capacity

some companies have sought to merge those though - aka NetApp with scaling out dual controller clusters for their Data Fabric. I'd argue that's not scale-out per se.

I don't view clustering as a way to increase horsepower just resiliency. Scale out is all about adding linear available resources.

@johna_white true that clusters require quorum, but most scale out systems do as well

It seems like some of us are using the word "clustering" to mean "High Availability Pairs." That's pretty old school, isn't it?

@johna_white definitely agree. clustering helps survive, not increase capacity

Clustering is a tech that can potentially be used for scaling out but it depends on the architecture. For example a master/slave architecture cannot scale out in the same way as a shared-nothing architecture.

@dinisco sure, a new age quorum typically exists across all nodes now... not just one place.

Clustering focus = keep my services online. Scale-out = grow forever.

Often, clustering can be used for scale-out, but also clustering might restrict scale-out to achieve redundancy. So, the terms are somewhat orthogonal.

@fdmts I think of a cluster as a logical collection of resources performing the same task...versus a set of discreet resources allocated to different things

@fdmts I'll agree that's how I think of it. In some cases cluster capabilities could be a subset of scale-out capabilities.

Didn't Isilon invent "scale out" ? :-)

pretty sure that was Al Gore

remember the vax cluster in the early 80's...that's old school

Yup. And they spent a decade explaining what the hell it meant too.

I know I always said scale-out to describe Isilon when at an EMC partner. People seemed to get it and loved the theory.

One example is Oracle RAC. High-availability without much scale.

@dvellante DecNet was a great net protocol stack killed SNA but then TCP took over #OpenWins

Isilon scales but also hits the wall at 144 nodes, so not limitless.

@andriven and @fdmts, agree with both, think one fundamental difference between cluster and scale out is that clusters can have passive resources, scale out generally has all active resources

@dvellante VMS (DEC's) OS is what microsoft copied with Windows NT (WNT) one letter after VMS

Spinnaker :-)

@andriven Something about terms like Kleenex and Post-it.

@dinisco Good distinction - in scale-out often all resources can be used and during failure scenarios the total resources available diminishes rather than inherent performance decrease.

love that story...Dave Cutler