LocalData

Significant engineering + QA investment.

Have to understand what the end-user is looking to "scale" There's trade-offs, nothing is free

Fundamentals, and a system architecture clear and clean enough that the whole team can understand it.

Anticipating and removing all logical limitations and/or bottlenecks in your design

Lots of cloud instances on which to test your scale-out design. Can't afford to strap together than much hardware in a data center.

@patrickrogersca I would build on this to say - an engineering culture that embraces finding the next bottleneck as a win.

Scale-out generally requires loosely-coupled, distributed architectures.

Often a custom file system, metadata/catalog work, locking mechanisms, some ACID characteristics, easy growth, masterless architecture, etc.

followup question what is the mix of hardware vs software? thoughs color commentary?

@fdmts #agree

this is why I'm skeptical by default if I see an existing product or company start to claim scale-out. Really doing is major under the covers work.

98% software

patience and emphasis rock solid internode communication, write plans, etc. before going after checkbox features that customers want to talk about

IMO, it should be all software for it to be successful. The hardware should be irrelevant.

tangent comment: #blockchain cyptocurrencies are showing both decentralized & distributed models to be very interesting.. thoughts

I'd call scale-out an interesting inflection point where products with older "design centers" will be overtaken overall by products that are built with scale out in their "design center".

I think that it starts with the software architecture - the algorithms - and that the architecture imposes constraints on the hardware. As Rickover said, "The devil is in the details, but so is salvation."

I'm not throwing shade on older products but overhauling an existing product to make it scale-out is HARD. Not impossible but seriously hard.

Agree. Blockchain potential goes way beyond cryptocurrency!

love @jhughes comment 98% software that makes HW vendors 2%ers :-)

Heavily, heavily software. You can't move fast enough with ASIC and/or hardware design to adapt.

@johna_white Disagree. Hardware is where the bytes live. I've seen too many designed that assume that a cow is a unit sphere filled with milk.

Oh God, Blockchain came up. Everybody DRINK.

@fdmts if #bitcoin drink twice

@fdmts Here you go then. Scale out = An innovative, game changing, and leading solution that leverages actionable synergies across unique, optimized paradigm shifts to deliver bottom line results that you can double click on.

@fdmts Yeah I think you are missing my point. I shouldn't have to buy specialty hardware to make a scale-out solution work. Commodity stuff all day.

Everyone now wins buzzword bingo. ;)

At the compute and storage layer, it's mostly software. You can build some massive scale out systems using standard white boxes. For networking, there are benefits to using custom ASICS to minimize latencies.

@fdmts Drinking is in order. Cryptocurrency is a distributed transaction model, not a distributed infrastructure model. 😀

Software has to assume there will always be hardware limitations. The challenge is living within the confines of the hardware units.

@johna_white Fair enough. I've seen good solutions tip over when the sales team agreed to try to run them on my legacy hardware.

So if a vendor tries to sell you on custom gear for scale out storage, run the other way. :)

@steve_pao We could use the BitCoin chain for file level locking. 10 minute lags FTW.

Can we get a virtual group hug? :)

Right after your maintenance contract is due

Stood up a 400 node Isilon system. "It can be done"

when the back end interconnect has no port capacity

without being to product specific, there was a "scale-out" architecture (RAIN) that maxed at 16 nodes. Just a hard stop...

I'm a consultant, they don't call me to fix the scale out system when it's working right.

@jhughes #nope #hardpass

@andriven hahaha, always thought that architecture was a bit of joke

It's usually when the backplane gets saturated, so the system can't stay in sync anymore.

@jhughes Just to be clear, it didn't "scale", but it stood up

@jhughes And you backed away very slowly, both hands in clear view.

Network almost always defines the limits. There's a reason Google brags about their CLOS setup

We ran out of API calls per second on GCS - that was fun.

That moment when you realize that east-coast-2 is actually a finite resource.

@dinisco would usually see this with Infiniband before 10 GigE based architectures. Of course Infiniband could usually drive lower latency more predictably.

Any architecture that requires a locking mechanism or uses a one of more master nodes will have scaling limits.

@kenhuiny Excellent point - scalable locking mechanisms as well as masterless architecture are huge.

Something about split-brain.