@dorkninja What does it means to "commoditize pattern data"? Does this have something to do with building and training ML models that can be reused in new IT infra mgt apps and possibly even resold in online marketplaces?
@dorkninja Right. And balance the investment in IT tooling/infra against investment in IT management human resources, including developers of IT mgt machine learning assets.
Agreed, assuming you can actually make use of that data lake in am meaningful way. So often people get huge cess pools of unrefined data that isn't useful, doesn't have appropriate ML features, and they can garner no insight from. #SadMLFails
+1. Hurts my data loving soul, but it's a song all too common. Companies know they "need" big data, but have no idea how to implement/refine/use/benefit. So they end up with a non useful solution, are mired in it for years, and fall behind.
Precisely. Suddenly the message is "We tried that big data thing. It's bogus. Why should we try it again?" when new, innovative, powerful approaches surface. Same issue "cloud" and many new techs have faced, frankly.
@colin_walker some vendors build in the data/ML smarts for a particular domain so their application collects and organizes only the data they need in the data lake
@jameskobielus i think incident response is more horizontal. i was thinking about intrusion detection or management of an application's full stack such as SAP, etc
@colin_walker We're the emergence lineage, provenance, governance and security capabilities for data lakes, but what is still laking is a clear vision of the desired outcome.
@NeilRaden Oof, that hits home hard. 100% agree. People don't know what they want to know, yet. They're looking for not only the answers, but the questions they're supposed to be asking. Makes it tough to set up appropriate ILM, practices, etc.
@colin_walker There are some emerging autodiscovery and pattern sniffing techniques that can help here I think. Akin to pharma saving all clinical trial and vector data forever, in case a future superbug appears.
@NeilRaden problem I believe is confusion from vendors about what outcome is best for the data you have ingested - specific use cases should be designed by industry to enable governance to work and be compliant with regulatory rules GDPR etc
@dorkninja Agreed. Lots of options coming that use ML to ask the questions of the data without humans having to know what to ask. Definitely the way forward, IMO. #OneMansOpinion. Part of why we use this technique in our tech. #DontKnowWhatYouDontKnow
ILM based on regulatory compliance and industry methodology can be tuned to be both flexible and encompassing - problem with Big Data technologies is that they primarily did not start out that way so difficult to enact ILM correctly
@mcauth What's the role of machine learning in unifying the treatment of different data sources while also enabling differentiated task-specific IT management insights?
@jameskobielus Increasingly large. Putting data sources aside for a moment, the growth alone warrants machine assist. Human analyst error rates are high vs. machines for a huge swath of ops intel.
@mcauth Exactly. Forget sources or silos. The sheer growth and volume of inbound to be processed means ML is either currently or soon 100% required, full stop. Otherwise what's your option? Call center sized buildings full of data analysts? #Pass
@colin_walker Industrial-grade feature discovery on that infrastructure data can call out the predictors from unstructured data. Perhaps cluster analysis algorithms.