bigdataNYC

How to Hadoop?
What's the better model for commercial Hadoop: Open Core or Open Source? Thought Leaders discuss.
   11 years ago
#BigDataNYCData-Driven EnterpriseWhat are the Requirements, Challenges, & Approach to transform into a Data-Driven Enterprise
Dave Vellante
Here's the elephant in the room - Olson's article implies the Hortonworks model is doomed...but if HW has deep pockets doesn't that doom the cloudera model?
Matt Asay
I don't think it's anywhere near that simple. @mikeolson was simply arguing that to get to scale you need more than merely an OSS support model. History suggests he's right
Dave Vellante yes but past is not prologue...what are your thoughts on canonical disrupting Red Hat
Jeff Kelly agree - if Hortonworks succeeds as a the winner in this market and becomes a true mega-vendor, it will be the first pure open source SW vendor to do so.
Jeff Kelly
IMO Hortonworks is playing the long game - seeding the market, then build significant revenue when mainstream moves to large-scale production deployments
Yves de Montcheuil
I seem to recall that at first Cloudera was also playing the purity card. Until they came up with Cloudera Manager.
Dave Vellante I think that's fair Yves - bit of a flip flop although why not - they have every right to pivot
Dr.Cos And that's another illustration on why "Open Core" is essentially harmful for open source development: it let you to pretend to be an open source rather than be an open source.
Jeff Kelly
not necessarily - this is a HUGE market opportunity and may well be room for two very successful players - a bit of a cop-out on my part but I think its true.
Yves de Montcheuil There is space, no question. But can the support-only model be sustainable and scalable for HW?
Jim Walker
literally just LOL'd... ok, seriously... doomed? our customers, partners, employees and investors think VERY differently
Dave Vellante this is the crux of the debate Jim...my personal opinion is that open source is a weapon in this game and it's the most powerful - shit it neutralized msft's monopoly
Dr.Cos Shall we try not to steer it toward HW vs. C argument? Cause I'd rather flame about vim vs emacs ;)
Jim Walker vi... of course. come on!
Jack Norris
It wasn't that long ago that articles were written about Cloudera being the open source champion and that models like MapR's were proprietary. The Elephant in the room is the acknowledgement that our model is right.
Jeff Kelly very true - Cloudera has done a 180 degree turn on messaging and seems to me to have the same biz model as MapR. big difference is that Olson says the platform must be open source.
Jack Norris When you don't have the required innovations at the platform level that would be an expected response. The key here is innovating at the platform level while embracing industry standards. This drives benefits without lock-in.
Yves de Montcheuil
My guess is that HW will pivot too. But that they will invent another model. After all, not 2 open source successes are exactly identical.
Jim Walker guess again mon frier.... ;)
Dave Vellante that's another big question Yves - will HW ultimately "IP up?" - so far they've been very pure on that topic
Dave Vellante see what I mean :-)
Jim Walker
there was one comment on post that called out central point to this argument, working within the ecosystem is how you multiply the model... for HW it is microsoft, teradata, SAP... stay tuned.
Yves de Montcheuil Interesting point. Seed the community but the revenue streams come from OEMs? Now, that would be scalable.
Jeff Kelly you nailed it Jim - the key for Hortonworks long term success is the channel and partnerships.
Jeff Kelly you nailed it Jim - the key for Hortownorks if it is to win the market is partnerships and the channel - made much easier with HW's open source model.
Dr.Cos OEM game is definitely very scale-able: given you have a coherent community around the project and its commitments to stability and backward compatibility. Fortunately, Hadoop has #ASFBigtop to keep it in-check
Dave Vellante
I come back to cononical - shuttleworth is essentially funding a petri dish for this argument
John Furrier
open source will always win
Ercan Yilmaz
What is the definition of commercial success? 10x exit or building the next ORCL, MSFT? Market and time will tell which is a better model unless they pivot.
Dave Vellante
great question Ercan - to me "commercial success" is the realization of a worthy commercial cause - adding value for customers in a sustained manner
Dave Vellante even if that entity gets absorbed -- if it continues to add value it's successful imo
Yves de Montcheuil
10x? tsss... be ambitious :-)
Jim Walker
success is really defined as making the market function with hadoop as the core component, embraced by the broader ecosystem... not commercial terms so much.
tostaypuft
Do you think the pre-Oracle MySQL community feels that way Dave?
Dr.Cos As well, as pre-Oracle Hudson, JDK, Solaris and some others.
Dr.Cos Another evidence of how 'open core' dumb-down innovation and openness of software
Jeff Frick
Some like to build companies, some like to build toward liquidity event. Horses for Courses
Jeff Kelly
success for hadoop is enabling the data economy leading to wave of new biz applications with data at the heart
Jeff Frick Yes, new class of aps, new way of identifying & solving problems
Jeff Kelly
Again specific to Hadoop, which model - pure open source or open core + proprietary overlay - innovates faster?
John Furrier
http://twitter.com/WadeMapR/status/388382620158943232
WadeMapR
RT @mapr: The post @jacknorris is referring to by MapR CEO John Schroeder is here: http://t.co/Vz89WNV8WA #bigdatanyc via http://t.co/4x0xm…
a minute ago
Jim Walker
look at the rate of innovation in the Apache Hive community over the past few months. more importantly, look at the pure amount of brainpower applied. huge.
Dr.Cos Is it because Hive initially has set the expectation so low? ;)
Tim Crawford
Today, Hadoop is not enough of a business model to sustain itself. Needs ecosystem + commercial.
Dr.Cos Well, I would expect to see any enterprise business being in need of ecosystem and commercial. No one develop software for free; most of OSS contributions are paid by commercial.
Jim Walker amen... amen! and it is getting it
Dr.Cos
Open Source is the best, IMO. It has more flexibility and well openness, which leads to better collaboration, faster advancements and adaptability of the technology.
Yves de Montcheuil
It's the richness of the ecosystem that innovates faster. Having several vendors with different philosophies allows them to differentiate.
Jeff Kelly
does the open source community ever slow innovation - particularly if it isn't well organized or if there are major disagreements about direction?
Yves de Montcheuil One could argue that Singer vs Impala is such an example...
Dr.Cos "Open source" biology has the same issue, right? And 95% of the species that ever existed are gone by now. Perhaps, they weren't well organized in the first place. I wonder if anyone has done such stats for software development?
Ercan Yilmaz
Faster innovation does not mean faster adoption. Commercial success depends on adoption.
Matt Asay
I'm not sure this is the right question. Cloudera can contribute heavily to the core OSS (which it does) and still spend heavily on developing additional, proprietary functionality. The two aren't mutually exclusive. Indeed, what we've seen
Matt Asay
in open source is that it is the proprietary companies that tend to contribute much more open source than pure-play open source companies. Why? Because they can afford to do so. Look at Facebook or IBM....
John Furrier
Pure open source always wins services on top is the way to add value.. going proprietary off open is risky
Jim Walker
IS hadoop fundamentally different than past OSS markets. how much does history apply?
Yves de Montcheuil
IMHO it's the first major open source project co-led by several similar-weight heavyweights
Dave Vellante
there are differences: 1/ maturity of OSS; 2/ World is now built on OSS; 3/ whale's got into the ecosystem much earlier; 4/ more deliberate funding
Jack Norris
I think the difference largely depends on the timeframe. Unix operating systems had been around 30 years and were well understood with Linux was created. Customers today benefit from innovation plus open source.
Jim Walker
the world is a different place. post 2000 college grads now making buying decisions. they EXPECT open source.
Jim Walker
i am so happy to be part of this debate because the emergence of open source makes the developer king again...
Dave Vellante
I think history is very hard to de-code here - so much in flux and potential for unknown phenomenon
Jim Walker
i think the tech involved (hadoop) however represents a fundamental shift in the way we think about data and that alone makes this movement different.
Dave Vellante
the other big difference is the data-centricity of the current climate - co's like Socrata are emerging around open data initiatives - fascinating
Jim Walker
open source moves up and down as well... look at open stack and open compute... it IS fascinating.. i like to say AWESOME
Yves de Montcheuil
You guys surely follow the #GartnerSYM tweetstream. Notice how much more is about data/information than it was last year?
Dave Vellante
Ok so what are the big takeaways of this crowdchat?
Jeff Kelly
i think we can all agree that open source is here to stay and dramatically impacting the enterprise SW market.
Dave Vellante
1/ OS model is still highly unpredictable; 2/ definition of success is still fuzzy; 3/ Open source is running the world 4/ lack of historical dominance fm OSS co's may not be future indicator
Jeff Kelly
hadoop, or any technology serving as a data platform, must be open to enable innovation on top of the stack
Jeff Kelly
What about the commercial Hadoop market specifically? Is there anything inherently unique about Hadoop that makes it different than previous open source technologies?
Dr.Cos
commercial Hadoop has two ways of growing, IMO:
Dr.Cos platform play (e.g. horizontal) and application play (e.g. vertical); The former quickly becomes a commodity; where the latter is rapidly growing. My concern, that latter might give a push for Open Core model, eventually.
Jeff Kelly which do you think is the better model - platform or application/vertical?
Joe Stein
Yes, Hadoop is not just a technical solution but gets into real business needs. The business didn't care if you ran SCO or Windows or Slackware whatever reduced risk and cost justified, it is a game changer, Big Data IS a profit center!!!
Dr.Cos Business certainly cares about ROI and capital costs, as well as true costs of ownership. Closed source means that oftentimes you're at mercy of your vendor: MS, Oracle, etc.
Jack Norris
The broad use cases require innovations combined with Apache Hadoop to provide a more stable and robust platform
Dr.Cos
Re: Singer vs Impala are you consider them "open source" or "open core"?
Jim Walker
first of all, stinger is not a product
Jim Walker
both are open source, both are NOT open community
Dr.Cos Tightly controlled influx of commits by a small group of company's employees doesn't strike me as a poster child of "open source"
Jeff Kelly good point - from what I've seen and heard Impala is essentially a Cloudera project with few if any outside contributors - but if its a good piece of technology, why wouldn;t Cloudera Enterprise customers use it?
Yves de Montcheuil
If the license is OSI compliant, it is open source
Dr.Cos
Totally agree - very relevant and very illustrative!
Jack Norris
I also thought that John Schroeder's post about business models that are "Built to Last" provided a perspective on driving innovation to spur customer success.
Dave Vellante
do you have a link?
Jack Norris Here's the link to the John's blog http://bit.ly/1bUzlKa
Yves de Montcheuil
@Dr.Cos says open core is harmful. Why? My thinking is that being candid/honest about what you get is the key.
Jeff Kelly
say what you will, but MapR and Hortonworks haven't wavered in their messaging or approaches.
Dave Vellante
Question: is open source a business model or a weapon against traditional approaches? another arrow in the quiver so to speak
Jeff Kelly
both - we've seen a handful of open source companies develop solid businesses while disrupting traditional old school vendors
Jim Walker
i dont think of it as a weapon, more of a vehicle
Dave Vellante weapon in the same way amazon crushes the comp. If funded, pure open source ultimately wins (aka my weapon analogy Jim)
Yves de Montcheuil
Neither. it's an approach for building software and fostering adoption. You can separately build a business model around open source software.