BigData

Systems of Intelligence
Conversation with Wikibon #bigdata research analyst on Systems of Intelligence & it's impact

Which statement best reflects the future of big data?

   5 years ago
#bigdata@theCUBE Talks Big DataConversation with George Gilbert Wikibon Analyst about bigdata Hadoop & other cool news & trends
John Furrier
Q2: Where are we on the customer journey to Systems of Intelligence? Give some examples or share data you might have.
George Gilbert
so, first a definition...
John Furrier
How are systems of intelligence different from Systems of Record? @bobpicciano uses systems of record & systems of engagement all the time
George Gilbert
#SystemsofEngagement are modern mobile UI with cloud delivery. #SystemsofIntelligence have predictive analytics to anticipate and influence consumer interactions for better outcomes on both sides
Kirk Borne
#SystemsofIntelligence reflect a concept that I've taught for years "Intelligent Archives of the Future"
Dave Vellante
Article in the WSJ about Apple & Google's race to predict what you want...#Data_Arms_War
Kirk Borne
@ggilbert41 ...apply #MachineLearning to customer behaviors (breadcrumbs) to learn interests, intents, and preferences #CEM
Rodrigo Gazzaneo
@ggilbert41 would you consider #IoT platforms to be #SystemsOfIntelligence as well?
John Furrier
. @vGazza I personally say 100% YES
John Furrier
. @vGazza #iot has to be intelligent at many levels so this will be a fun area to watch
George Gilbert
@vGazza great question - they are future versions of SysofIntel - where instead of anticipating consumer interaction, the application automatically responds to machine data
George Gilbert
@vGazza for example - a smart grid system might auto tune energy consumption of appliances in an individual home
Kirk Borne
#IoT platforms need to be #SystemsOfIntelligence in order to be useful, successful,.. But that isn't automagic
Jen(Cohen)Cheplick
As with many emerging technologies, there are some organizations that are on the "bleeding edge" if you will. But it takes more than technology to enable it -- you need a shift in skills and in organizational mindset to make the transition
Rodrigo Gazzaneo
@ggilbert41 I see #IoT as the #SysOfIntel for unintelligent elements. Intelligence at system level.
Dave Vellante
@vGazza that's an interesting perspective...you're basically saying #IoT produces "raw material" for SoI?
Jen(Cohen)Cheplick
We see online gaming organizations collecting user data real-time to dynamically enhance the experience, for example. Those organizations with a tangible benefit will be the earliest adopters
Crowd Captain
@jscheplick love the online gaming bc the use of big data is huge
Kirk Borne
@jscheplick You are right on the money -> culture beats strategy (or is that "eats strategy"? :)
Rodrigo Gazzaneo
@dvellante raw material is already there with sensors and element monitoring, I think
Kirk Borne
@jscheplick #MachineLearning in Online Gaming is a forerunner to #Gamification of #Analytics in many different orgs
Rodrigo Gazzaneo
@dvellante correlation at scale, #MachineLearning and insights can help build #SysOfIntel out of this raw material
John Furrier
Q4: Besides big and semi-structured data, just how different are Data Lakes and Data Warehouses?
George Gilbert
#datalake is repository for unrefined, uncurated data that data scientists and biz analysts can explore. repeatable analytics can go to DW or a production #Hadoop cluster
Rodrigo Gazzaneo
it's about the schema. Data Lakes are flexible, Data Warehouses are rigid.
Jen(Cohen)Cheplick
Data lakes should be easier to adapt to changing business & infrastructure needs vs. EDW
George Gilbert
@vGazza also correct. the flexible schema is part of making the #datalake a self-service environment - you add the schema as you explore
Kirk Borne
Diverse multi-source heterogeneous data sets are the norm in Data Lakes, but not in Data Warehousing.
Rodrigo Gazzaneo
Data Lakes support schemas on demand, so you can improve the models continuously
Crowd Captain
Data warehouses and data lakes are slow and sound so old..what's new & different with Systems of Intelligence models ?
Kirk Borne
Data Warehouse = schema on write.. Data Lake = schema on read.
Rodrigo Gazzaneo
@CrowdCaptain Data Lakes support flexible ingestion and insight layers, so they can be fast also
Kirk Borne
Data Lake allows easy updates (data "columns"). DW requires new schema and index builds when adding new
Jen(Cohen)Cheplick
Data lakes retain all (more) data vs. DW
John Furrier
ok it's all about the #dataocean bc oceans have currents and are always highly dynamic so the real time intelligence algos and tech are in the #dataoceans
Jasdeep Singh
DW are archived hierarchially... Data Lakes are mostly object-based
George Gilbert
@vGazza flexible ingestion enables self-service, but repeatability invites structure and therefore performance
Jen(Cohen)Cheplick
We are talking a lot about technical differences, but what is the difference for business value?
Kirk Borne
@CrowdCaptain #SystemsOfIntelligence should derive their value from its apps (smart Machine Learning) not its data model (DW or Data Lake)
David Floyer
data lakes are just a cheaper and bigger version of the failed data warehouses model
George Gilbert
@KirkDBorne it's all about flexibility vs. performance trade-off
Rodrigo Gazzaneo
a Data Lake can be a source of data for a Data Warehouse once you know what to ask
George Gilbert
@KirkDBorne Kirk, i couldn't have said it better. Data Lakes are training wheels. #MachineLearning is what drives Systems of Intelligence
Rodrigo Gazzaneo
@ggilbert41 systematic queries can be optimised and run on Data Warehouses for performance
David Floyer
data streams, data rivers power things along. Data lakes fester.
George Gilbert
@vGazza that's why DataLakes coexist with DataWarehouses - exploration vs. production performance
David Wild
Completely new methods needed for Data Lakes. #Machine Learning and stats miss many of the possibilities
Jeff Frick
@CrowdCaptain > Getting out front in the decision process.
John Furrier
Question 1: Systems of Record (SoR). What are they?
George Gilbert
We've been building these for 50 years - they automate business processes and report on historical performance.
Dave Vellante
So we're talking payroll? transaction systems? Warehouses?
George Chow
@dvellante I think of them collectively as ERP
George Gilbert
original was airline reservations, then ERP/CRM...
Jen(Cohen)Cheplick
ERP, CRM, databases
John Furrier
How sophisticated are their analytics?
David Floyer
Transaction systems & systems of record are the heart of enterprise processes
George Gilbert
pretty basic - it's just about historical performance reporting - equivalent to steering a ship by looking backwards at its wake
Kirk Borne
@jscheplick Those 3 (ERP, CRM, databases) are best SoR examples, and yet they trick some firms into thinking they already have NextGen Analytics (Systems of Intelligence)
John Furrier
@KirkDBorne love that comment - the classic head fake
Jasdeep Singh
With all due respect, in my opinion sophistication lies in the way they run, reports and UI have become user friendly #bigdata
John Furrier
. @KirkDBorne many are getting fooled by recycled technologies passed as tranformational
Dave Vellante
@KirkDBorne @jscheplick - good point - we're not talking about just semantics are we?
John Furrier
@Jase_mtlngs915 I agree on the user friendly but being modern is another story..horizontally scalable? real time? low latency? dealing with semantic intelligence..etc
George Gilbert
Jasdeep - BI/reporting is more friendly, but it doesn't inform the consumer in real-time at the point of interaction - especially by anticipating their actions and then influencing them - kind of like ads work
Kirk Borne
Yes, there are many "new" features: semantics, machine learning, personalization, behavior analytics, real-time,...
Timo Elliott
isn't it usually SoR vs "system of engagement". Is "system of intelligence" different?
David Floyer
Transaction systems & systems of record are the heart of enterprise processes
Dave Vellante
@timoelliott it's a concept being put forth (first) by Geoff Moore and @ggilbert41 is re-defining Systems of Intell uses machine learning to anticipate, respond and affect outcomes
Kirk Borne
@timoelliott Yes, SoEngagement includes those #CEM #CX pieces, but SoIntelligence is predictive, prescriptive, cognitive
John Furrier
I love the systems of engagement bc that data is active and with context then it can be intelligence #SystemofIntelligence
George Gilbert
with context and with advanced analytics - like machine learning that turns data into predictive model - like Amazon recommendations
Kevin Petrie
@ggilbert41 Great point - recommendations, #NLP, other #machinelearning apps are becoming must-have feature for many enterprise offerings. We expect Amazon to make recommendations, customer service to email an auto-response, Facebook to tag our images
Dave Vellante
@KevinPetrieTech yes this idea of a personal assistant that anticipates my data needs #scary
George Gilbert
@KevinPetrieTech couldn't have said it better - but coming up with #machinelearning recommendations is going to involve big data for greatest accuracy and fast/streaming data for currency
Kirk Borne
@dvellante Check out this personal assistant concept (from 2004) ScheduleNanny: http://arxiv.org/abs...