sparksummit

Crowd Doc

What are the advantages/disadvantages of deploying Spark with Hadoop YARN ? Which one is the first class citizen as a resource manager for Spark ? YARN or Mesos ? cc/ @spark_summit

Dr.Cos YARN suffers from higher scheduling latency compared to Mesos. Which is fine for most of the Hadoop applications, but is critical for Spark and alike.

John Furrier

What are the advantages/disadvantages of deploying Spark with Hadoop YARN? What are the advantages/disadvantages of deploying Spark with Mesos? Which has better cluster mgt?

Dr.Cos

On-Mesos deployment is very beneficial as it provides much faster scheduling than YARN. Also, with Mesos you can make Hadoop and Spark clusters coexist on the same infra.

Crowd Doc0

What are the advantages/disadvantages of deploying Spark with Hadoop YARN ? Which one is the first class citizen as a resource manager for Spark ? YARN or Mesos ? cc/ @spark_summit

Dr.Cos12

It is new generation of data analytic platform. MR is batch and slow. New applications need interactive system to churn models quickly. That's why spark is gaining popularity so quickly. Commercial companies are backing it up: first @WANdisco, now others

8 Votes Vote

Dean of Big Data

What is Spark's relationship with YARN?

Will Davis Support for running Spark on YARN was added to Spark in version 0.6.0, and improved in 0.7.0 & 0.8.0 - http://bit.ly/1bSvsTE

theCUBE There's no relation between Spark and YARN. The latter is a Hadoop resource scheduler. Spark supports YARN and can work as a YARN application though. But the same way it works with AMPlab Mesos https://www.crowdchat.net/post/4903

Crowd Captain

is the use case of Spark only limited to social data or only Graph DBs and Machine Learning environments?

Stephanie McReynolds We're using Spark across a wide variety of use cases. Social analysis is there but more so, supply chain ditribution, localized market demand, and a host of others. Anytime exploratory analysis is key. @CrowdCaptain

Dean of Big Data4

I'm an old school DW and BI guy, so what are the ramifications to SQL on Hadoop from Spark?

Jeff Kelly

good questions - where does Spark fit with other efforts to make Hadoop more SQL-like (i.e. Impala, Stinger, HAWQ, etc.)?

Alex Gorbachev Shark is Spark's component that does R/T SQL but there is more - streaming, graph, ML base.

Crowd Captain

I think that it's a not as impressive as the graph and ML performance of in memory to bypass the network and storage bottlenecks.. Others have been doing SQL on Hadoop DW since two #hadoopsummit ago

John Furrier6

Why is Spark so important and successful gain traction with developers in #bigdata

Stephanie McReynolds

For #bigdata analytics to have business impact, performance at the sped of thought is key. Spark delivers on fast, iterative analysis. @furrier

3 Votes Vote

Jeff Frick "Performance at the speed of thought" good one.

Jeff Kelly

I see Spark as part of the larger evolution of #Hadoop from batch to multi-applications, in this case real-time analytics

theCUBE

developers want a unified platform to abstract away complexities emerging for innovation ontop of MapReduce bc as @slangenfeld pointed out multipass, ad hoc queries, and interactivity are now table stakes speed and ease of programming key

Dr.Cos0

Morning guys. So, let's talk about #sparksummit?

John Furrier

what do you expect this trend to turn into for value to developers in #bigdata

John Furrier0

welcome to the open crowdchat to do some #pregaming of the Spark Summit. Just ask a question or post a thread.. Reply to the each post in threaded manner. You can vote for the best content as well