BigData

so, first a definition...

How are systems of intelligence different from Systems of Record? @bobpicciano uses systems of record & systems of engagement all the time

#SystemsofEngagement are modern mobile UI with cloud delivery. #SystemsofIntelligence have predictive analytics to anticipate and influence consumer interactions for better outcomes on both sides

#SystemsofIntelligence reflect a concept that I've taught for years "Intelligent Archives of the Future"

Article in the WSJ about Apple & Google's race to predict what you want...#Data_Arms_War

@ggilbert41 ...apply #MachineLearning to customer behaviors (breadcrumbs) to learn interests, intents, and preferences #CEM

@ggilbert41 would you consider #IoT platforms to be #SystemsOfIntelligence as well?

. @vGazza I personally say 100% YES

. @vGazza #iot has to be intelligent at many levels so this will be a fun area to watch

@vGazza great question - they are future versions of SysofIntel - where instead of anticipating consumer interaction, the application automatically responds to machine data

@vGazza for example - a smart grid system might auto tune energy consumption of appliances in an individual home

#IoT platforms need to be #SystemsOfIntelligence in order to be useful, successful,.. But that isn't automagic

As with many emerging technologies, there are some organizations that are on the "bleeding edge" if you will. But it takes more than technology to enable it -- you need a shift in skills and in organizational mindset to make the transition

@ggilbert41 I see #IoT as the #SysOfIntel for unintelligent elements. Intelligence at system level.

@vGazza that's an interesting perspective...you're basically saying #IoT produces "raw material" for SoI?

We see online gaming organizations collecting user data real-time to dynamically enhance the experience, for example. Those organizations with a tangible benefit will be the earliest adopters

@jscheplick love the online gaming bc the use of big data is huge

@jscheplick You are right on the money -> culture beats strategy (or is that "eats strategy"? :)

@jscheplick great example!!

@dvellante raw material is already there with sensors and element monitoring, I think

@jscheplick #MachineLearning in Online Gaming is a forerunner to #Gamification of #Analytics in many different orgs

@dvellante correlation at scale, #MachineLearning and insights can help build #SysOfIntel out of this raw material

#datalake is repository for unrefined, uncurated data that data scientists and biz analysts can explore. repeatable analytics can go to DW or a production #Hadoop cluster

it's about the schema. Data Lakes are flexible, Data Warehouses are rigid.

Data lakes should be easier to adapt to changing business & infrastructure needs vs. EDW

@vGazza also correct. the flexible schema is part of making the #datalake a self-service environment - you add the schema as you explore

Diverse multi-source heterogeneous data sets are the norm in Data Lakes, but not in Data Warehousing.

Data Lakes support schemas on demand, so you can improve the models continuously

Data warehouses and data lakes are slow and sound so old..what's new & different with Systems of Intelligence models ?

Data Warehouse = schema on write.. Data Lake = schema on read.

@CrowdCaptain Data Lakes support flexible ingestion and insight layers, so they can be fast also

Data Lake allows easy updates (data "columns"). DW requires new schema and index builds when adding new

Data lakes retain all (more) data vs. DW

ok it's all about the #dataocean bc oceans have currents and are always highly dynamic so the real time intelligence algos and tech are in the #dataoceans

DW are archived hierarchially... Data Lakes are mostly object-based

@vGazza flexible ingestion enables self-service, but repeatability invites structure and therefore performance

@bigdataryan care to chime in?

We are talking a lot about technical differences, but what is the difference for business value?

@CrowdCaptain #SystemsOfIntelligence should derive their value from its apps (smart Machine Learning) not its data model (DW or Data Lake)

data lakes are just a cheaper and bigger version of the failed data warehouses model

@KirkDBorne it's all about flexibility vs. performance trade-off

a Data Lake can be a source of data for a Data Warehouse once you know what to ask

@KirkDBorne Kirk, i couldn't have said it better. Data Lakes are training wheels. #MachineLearning is what drives Systems of Intelligence

@ggilbert41 systematic queries can be optimised and run on Data Warehouses for performance

data streams, data rivers power things along. Data lakes fester.

@vGazza that's why DataLakes coexist with DataWarehouses - exploration vs. production performance

Completely new methods needed for Data Lakes. #Machine Learning and stats miss many of the possibilities

@CrowdCaptain > Getting out front in the decision process.

Databases

We've been building these for 50 years - they automate business processes and report on historical performance.

So we're talking payroll? transaction systems? Warehouses?

@dvellante I think of them collectively as ERP

original was airline reservations, then ERP/CRM...

ERP, CRM, databases

How sophisticated are their analytics?

Transaction systems & systems of record are the heart of enterprise processes

pretty basic - it's just about historical performance reporting - equivalent to steering a ship by looking backwards at its wake

@jscheplick Those 3 (ERP, CRM, databases) are best SoR examples, and yet they trick some firms into thinking they already have NextGen Analytics (Systems of Intelligence)

@KirkDBorne love that comment - the classic head fake

With all due respect, in my opinion sophistication lies in the way they run, reports and UI have become user friendly #bigdata

. @KirkDBorne many are getting fooled by recycled technologies passed as tranformational

@KirkDBorne @jscheplick - good point - we're not talking about just semantics are we?

@Jase_mtlngs915 I agree on the user friendly but being modern is another story..horizontally scalable? real time? low latency? dealing with semantic intelligence..etc

Jasdeep - BI/reporting is more friendly, but it doesn't inform the consumer in real-time at the point of interaction - especially by anticipating their actions and then influencing them - kind of like ads work

Yes, there are many "new" features: semantics, machine learning, personalization, behavior analytics, real-time,...

isn't it usually SoR vs "system of engagement". Is "system of intelligence" different?

Transaction systems & systems of record are the heart of enterprise processes

@timoelliott it's a concept being put forth (first) by Geoff Moore and @ggilbert41 is re-defining Systems of Intell uses machine learning to anticipate, respond and affect outcomes

@timoelliott Yes, SoEngagement includes those #CEM #CX pieces, but SoIntelligence is predictive, prescriptive, cognitive

I love the systems of engagement bc that data is active and with context then it can be intelligence #SystemofIntelligence

with context and with advanced analytics - like machine learning that turns data into predictive model - like Amazon recommendations

@ggilbert41 Great point - recommendations, #NLP, other #machinelearning apps are becoming must-have feature for many enterprise offerings. We expect Amazon to make recommendations, customer service to email an auto-response, Facebook to tag our images

@KevinPetrieTech yes this idea of a personal assistant that anticipates my data needs #scary

@KevinPetrieTech couldn't have said it better - but coming up with #machinelearning recommendations is going to involve big data for greatest accuracy and fast/streaming data for currency

@dvellante Check out this personal assistant concept (from 2004) ScheduleNanny: http://arxiv.org/abs...