datafriction

Absolutely right. And the governance challenge increases over time if not thought of early.

Hence the perpetual, almost zombie-like resilience of HSM, ILM, linear media, etc... :)

@dorkninja Just say it! TAPE!

@plburris Boo! Hiss! ;)

@plburris I like tape, and dammit I'm not ashamed to admit it! :)

@plburris Lol you said what I was thinking. Well played, sir

@colin_walker Hey, I could have mentioned punch cards.

@plburris It's too early in the morning for that kind of cruelty. brb, fetching a shredder.

@plburris you're dating yourself

@mcauth Data retention is a key issue, especially for the scads of infastructure data too low-level (or not relevant to compliance) enough to merit long-term retention. There's just so much log data that has marginal value for retaining.

@cselland probably should have put a ;) on that one

Look, the proper place to USE that data is still a data warehouse, and they have overcome many of their drawbacks. I say, cloud DW, not data lake if you think it will be used.

@NeilRaden but surely Clolud DW, Data Lake, EDW are all part of the Enterprise Data Fabric - should we not worry where the data is held as long as it can be used across the Enterprise?

Tough question to answer absent context... Who are the users? What are their specific needs?

big issue is not the data, its which answers u r looking for

In other words, how much friction is there for people trying to find the data and analytics: Do they need to know a query language? Can they easily search? How long do they have to wait?

@yaronhaviv I'd argue that you are looking for answers to questions you haven't thought of yet

I'll parse this along the boundaries of data sources: Machine data (logs, snmp, etc.) Agent data, synthetic data, and wire data. Most are quite easy to get at, it's harder to wrangle it into value.

No common reference model for what is the data, making it difficult to know what to find. Machine learning can really help there.

Varies *wildly* with tooling and instrumentation. For some I've spoken with it's hopeless in their current deployment. For others it's seconds to visibility and insight. It's all about surfacing insights rapidly, automatically.

@dorkninja Great point! But as IT resources become more core to business behavior, a more diverse user community will demand simple ways to find, collaborate on, and act on answers.

start w a goal (e.g. oper efficiency, security, downtime ..) define the data u need from there

@yaronhaviv Agree. There is a "half-life" to data value and it differs based on the analytics use case. With incident response and downtime, the half-life of data value is shorter, but for long-term capacity planning the half-life is longer.

All service -- and adoption -- is local.

Too many disregard the value of data, presuming that it's disposable. The change in perspective happens in pockets.

@plburris Data. Is not. Disposable. Literally impossible to replace. Preach. #SaveTheData #NotARedShirt

@colin_walker AMEN.

@dorkninja Hilariously data is the thing that allows you to replace everything else. Or fix it. Or learn from it. Or ... #AllTheThings. Data is the truth/life/love. #BringMeData #AndBacon

@RalphFinos you have to know how to use and what you are doing. data itself is kind of useless unless you know what you are doing and can use the right tools

@plburris Agree Peter, regulatory compliance rules (GDPR et al) will start to rain this problem in I think. It will be interesting to see if it has an affect on the use of data in the Data Lake as this is the usual place the data is just thrown

This is a must have for organizations

@colin_walker And once it gets out, it can't be chased down.

@plburris Right?! You have to be able to grab it *in* the moment. #YouveOnlyGotOneShot after all.

And once DATA gets out, it can't be chased down.

@RalphFinos Agree. But there are also cases where data that's useless today, and tomorrow, and the next day - becomes invaluable 10 years from now. See: The CDC.

@dorkninja This is interesting, and 100% true. Data's value changes over time depending on the use case. Too few take this into consideration, let's hope that changes :)

@dorkninja I'd argue that's not data that's useless today or tomorrow. It's just a single data point that is useful in a trend, rather than on its own. Also something we can catalog, capture and covet.

@dorkninja But yes, often *appears* useless at first, and for a while, and gets wrongly chucked. Definitely an issue.

@colin_walker Yup - people forget the temporal element and assume all data is ephemeral or disposable.

@mcauth Yes, everyone knows data is valuable, but no one knows how to value it. That's why you don't see it on the balance sheet

@NeilRaden From an ops perspective - as an ex-Ops person myself, a very simple value model I use is (velocity / friction ) * the number of users that can put the data into action. Not academically rigorous but folks find it useful.

@mcauth - There should be some conformity, or canonical models, in industry verticals. There may be different valuation models across departments. Unless there is a market for data, all data valuation models will be subjective

a deeper dive on the data value equation from @mcauth https://www.extrahop...

@NeilRaden +1 This is a fantastic point and is absolutely true.

@dorkninja Data often indicates a state, status, or condition of the infrastructure at a point in time, or it can represent a trend over time. For infrastructure, all s essential for historical analysis, real-time monitoring, and preventive.

most applications today are vomiting data - state, status, condition in a vacuum. not actionable - need AI to synthesize and deliver accelerated insight

@Jshoc Largely true, especially given sheer volume. But it's also possible to - from a practice level - extract the stuff that you know matters and present it proactively, pre-AI.