LocalData

   7 years ago
#LocalDataTiering to Public CloudHow to think about and manage tiering from the private data center to public cloud.
   6 years ago
#localdataData ManagementData Management: Protection, Movement, Search & Discovery, Usage
John Furrier
Q7: What does it take to build scale out architectures?
Andrew Miller
Significant engineering + QA investment.
Jeff Hughes
Have to understand what the end-user is looking to "scale" There's trade-offs, nothing is free
Chris Dwan
Fundamentals, and a system architecture clear and clean enough that the whole team can understand it.
Patrick Rogers
Anticipating and removing all logical limitations and/or bottlenecks in your design
Patrick Rogers
Lots of cloud instances on which to test your scale-out design. Can't afford to strap together than much hardware in a data center.
Chris Dwan
@patrickrogersca I would build on this to say - an engineering culture that embraces finding the next bottleneck as a win.
Stephen Pao
Scale-out generally requires loosely-coupled, distributed architectures.
Andrew Miller
Often a custom file system, metadata/catalog work, locking mechanisms, some ACID characteristics, easy growth, masterless architecture, etc.
John Furrier
followup question what is the mix of hardware vs software? thoughs color commentary?
Andrew Miller
this is why I'm skeptical by default if I see an existing product or company start to claim scale-out. Really doing is major under the covers work.
jeff dinisco
patience and emphasis rock solid internode communication, write plans, etc. before going after checkbox features that customers want to talk about
I am John White
IMO, it should be all software for it to be successful. The hardware should be irrelevant.
John Furrier
tangent comment: #blockchain cyptocurrencies are showing both decentralized & distributed models to be very interesting.. thoughts
Andrew Miller
I'd call scale-out an interesting inflection point where products with older "design centers" will be overtaken overall by products that are built with scale out in their "design center".
Chris Dwan
I think that it starts with the software architecture - the algorithms - and that the architecture imposes constraints on the hardware. As Rickover said, "The devil is in the details, but so is salvation."
Andrew Miller
I'm not throwing shade on older products but overhauling an existing product to make it scale-out is HARD. Not impossible but seriously hard.
Stephen Pao
Agree. Blockchain potential goes way beyond cryptocurrency!
John Furrier
love @jhughes comment 98% software that makes HW vendors 2%ers :-)
Andrew Miller
Heavily, heavily software. You can't move fast enough with ASIC and/or hardware design to adapt.
Chris Dwan
@johna_white Disagree. Hardware is where the bytes live. I've seen too many designed that assume that a cow is a unit sphere filled with milk.
Chris Dwan
Oh God, Blockchain came up. Everybody DRINK.
Andrew Miller
@fdmts Here you go then. Scale out = An innovative, game changing, and leading solution that leverages actionable synergies across unique, optimized paradigm shifts to deliver bottom line results that you can double click on.
I am John White
@fdmts Yeah I think you are missing my point. I shouldn't have to buy specialty hardware to make a scale-out solution work. Commodity stuff all day.
Andrew Miller
Everyone now wins buzzword bingo. ;)
Kenneth Hui @rubrikInc HQ
At the compute and storage layer, it's mostly software. You can build some massive scale out systems using standard white boxes. For networking, there are benefits to using custom ASICS to minimize latencies.
Stephen Pao
@fdmts Drinking is in order. Cryptocurrency is a distributed transaction model, not a distributed infrastructure model. 😀
Patrick Rogers
Software has to assume there will always be hardware limitations. The challenge is living within the confines of the hardware units.
Chris Dwan
@johna_white Fair enough. I've seen good solutions tip over when the sales team agreed to try to run them on my legacy hardware.
Kenneth Hui @rubrikInc HQ
So if a vendor tries to sell you on custom gear for scale out storage, run the other way. :)
Chris Dwan
@steve_pao We could use the BitCoin chain for file level locking. 10 minute lags FTW.
John Furrier
Q2: What is the difference between "scale out" and clustering? Any examples would be great
Chris Harney
Clustering adds fault tolerance scale out does not necessarily add ft
Chris Dwan
I think that most scale out systems are "clustered" in one sense of the word. The distinction lies in scale out's ability to expand capacity in a balanced way without introducing bottlenecks.
I am John White
cluster = quorum and more active/passive. Scale out provides all nodes to perform a function.
jeff dinisco
clustering often refers to a dual node system, though not always, in a sense, most traditional scale up storage architecture are actually 2 clustered controllers
Patrick Rogers
Clustering is way to achieve scale-out.
John Furrier
begs the question "how do clusters scale out" :-)
Andrew Miller
historically clustering was about resiliency and removing single points of failure. Let's go back to Novell Cluster Services for instance. Scale-out is focused at very different thigns.
Jeff Hughes
clustering usually means tightly coupled. all the resources are pooled together. scale-out many times just means dividing up the work
Chris Dwan
@csharney Clustering doesn't necessarily add fault tolerance. It's a big place.
Chris Harney
Clustering may not add performance or capacity
Andrew Miller
some companies have sought to merge those though - aka NetApp with scaling out dual controller clusters for their Data Fabric. I'd argue that's not scale-out per se.
I am John White
I don't view clustering as a way to increase horsepower just resiliency. Scale out is all about adding linear available resources.
jeff dinisco
@johna_white true that clusters require quorum, but most scale out systems do as well
Chris Dwan
It seems like some of us are using the word "clustering" to mean "High Availability Pairs." That's pretty old school, isn't it?
Jeff Hughes
@johna_white definitely agree. clustering helps survive, not increase capacity
Kenneth Hui @rubrikInc HQ
Clustering is a tech that can potentially be used for scaling out but it depends on the architecture. For example a master/slave architecture cannot scale out in the same way as a shared-nothing architecture.
I am John White
@dinisco sure, a new age quorum typically exists across all nodes now... not just one place.
Andrew Miller
Clustering focus = keep my services online. Scale-out = grow forever.
Stephen Pao
Often, clustering can be used for scale-out, but also clustering might restrict scale-out to achieve redundancy. So, the terms are somewhat orthogonal.
Dave Vellante
@fdmts I think of a cluster as a logical collection of resources performing the same task...versus a set of discreet resources allocated to different things
Andrew Miller
@fdmts I'll agree that's how I think of it. In some cases cluster capabilities could be a subset of scale-out capabilities.
John Furrier
Didn't Isilon invent "scale out" ? :-)
I am John White
pretty sure that was Al Gore
Dave Vellante
remember the vax cluster in the early 80's...that's old school
Chris Dwan
Yup. And they spent a decade explaining what the hell it meant too.
Andrew Miller
I know I always said scale-out to describe Isilon when at an EMC partner. People seemed to get it and loved the theory.
Patrick Rogers
One example is Oracle RAC. High-availability without much scale.
John Furrier
@dvellante DecNet was a great net protocol stack killed SNA but then TCP took over #OpenWins
Patrick Rogers
Isilon scales but also hits the wall at 144 nodes, so not limitless.
jeff dinisco
@andriven and @fdmts, agree with both, think one fundamental difference between cluster and scale out is that clusters can have passive resources, scale out generally has all active resources
John Furrier
@dvellante VMS (DEC's) OS is what microsoft copied with Windows NT (WNT) one letter after VMS
Chris Dwan
@andriven Something about terms like Kleenex and Post-it.
Andrew Miller
@dinisco Good distinction - in scale-out often all resources can be used and during failure scenarios the total resources available diminishes rather than inherent performance decrease.
Dave Vellante
love that story...Dave Cutler