EMCDataLake

> Problem or Opportunity or Both?

the numbers are staggering - 80% of data is unstructured and doubling every two years

Data from IDC - the question for enterprises becomes how do you manage all this growth

unstructured data volume #bigdata and increasingly speed / #fastdata requires new tools to process

@logicalblock can you share some customer examples of companies getting new value or insight out of that data? is it typically machine-generated or human-generated?

@logicalblock > What public unstructured data set are enterprises integrating the most?

@logicalblock > From exhaust that pollutes to exhaust with potential insight.

batch processing log data is what first drove #bigdata and #datalake requirements - this was refined via ELT process for #datawarehouse usecases

@bgracely Mix of both - machine generated data is what is causing the volume. Examples abound - analytics to predict how to stock warehouses, genome sequencing, etc

@logicalblock our data suggests that unstructured is even higher - over 90% of the data created is unstructured and growing much faster than structured

@dvellante this is an emerging field with data trends changing real time #bigdata

@dvellante would love to see the data - if anything it bolsters the DataLake 2.0 story

i am hoping for 100% someday! :)

#datalake has been widely adopted term in #hadoop to refer to uncurated data that needs refinement, enrichment

very real and very strategic! customer demand mass consolidation of unstructured data AND the ability to drive insight over this information...we have delivered and expanded this strategy with the data lake 2.0 announcements today!

this is where John makes his case for "Data Ocean"

Especially exciting is the broad spectrum from cloud to edge devices.

To me the Data Lake is about data warehouse market

au contriari mon frere data warehouse is a great 1999 conversation, but the technology has moved quickly and is being dominated by unstructured data...audio, video, images, files, objects, social, etc... data lakes are required now to store this

@sgrocott so is data lake more reliable than a data ocean :-)

with all the new machine readable data emerging, the #datalake is going beyond #datawarehouse use cases

depends, is it running on EMC Isilon?

@ggilbert41 @furrier this Sam guy is good

@ggilbert41 new use cases are lower latency capturing and processing data nearer origination

Wikibon practitioners tell us that the traditional EDW is still a key part of their #bigdata strategy...does data "ocean" imply a superset of the data lake that includes EDW or is EDW part of the lake?

@dvellante EDW is a use case of a data lake, but it doesn't entirely define the space.

A Data Lake is part efficiency, part value. The efficiency of coalescing unstructured data, and the value gained by reasoning over it through multi-protocol access.

Hey Phil! ok...so it's in the scope of DL

@dvellante Right. We have a lot of customers with EDW-focused Data Lake workloads, more focused on historical data. Our tiering capabilities (perf, capacity, archive and cloud tiers) enables multiple uses cases in a single Data Lake.

I see your edge, core and cloud message. Why is this something customers need today?

#datalake started as adjunct to #datawarehouse. now it's becoming key part of near real-time #fastdata analytic #datapipeline

starts with getting value from data and then keeping data management costs down #SimpleIsSmart

a petabyte #Isilon cluster was a rarity, soon this will become the average

@logicalblock > Demands for storage will never decrease

@logicalblock > Sounds easier in theory than practice, especially if you're just "capturing" not really sure what the value it.

@logicalblock > Begs Question, what are some qualifying questions to ask before turning your data exhaust from simply flowing onto the floor, and into the data lake?

@JeffFrick completely fair - the first step starts by having a set of use-cases that you know you want to look at and then expanding from there

imo customers are spending 75-80% of their time messing w/ data quality issues and the like

Hadoop complexity has hurt ROI so they're demanding more from suppliers of data lakes #ROI

so glad to be here! Exciting day today!

Welcome to the EMC AMA on the big news today about Data Lake 2.0 - I'm tweeting from Palo Alto CA. Where are you posting from?

> Hello Phil, Sam and Ashish!

thanks john! looking forward to it

I will start with a series of question but this is for anyone to come in and ask questions hence AMA

Hi all, this is my 10th Crowdchat this year.. For once not as a host. Looking forward to the chat.

I'm sailing the Data Ocean

@suzyspaatz I've been admiring your chats lately and tell us what we can do better

welcome crowd captain!

@sgrocott > Who doesn't love Captn Stubing.

EMC Release is here http://www.emc.com/a...

thanks for hosting John...

@sgrocott and sam

@Schmarzo > Where is the Dean?

btw: I like the hashtag selection here great job whoever thought of it - it will have legs