DataOps

A powerful combination of #Data #Engineering, Data #Flow mgmt and #integration, Data #Quality, Data Security, Data Privacy, and Governance

converting data capital to org high performing capital

# devops is dataops, no feedback without data

#DevOps is about feature velocity whereas #DataOps is analytic velocity. Need to talk about devops in order to understand dataops better :)

too many terms, data is data and analyzing data drives decisions. Just like dev, data must add value, velocity should be in line with feedback

in my opinion DataOps, like DevOps, should be about the behavior of creating teams that collaborate to ensure what is being delivered is timely, uses common practice etc (but should be data focused)

DataOps is a #teamsport

1) Data Gathering, 2) Cloud Adoption 3) Analytics Velocity
#DataOps

Alignment to business goals and purpose of the data being gathered. 2) Treat data as a value stream and reduce friction from capture to insights 3) Seamless integration - with existing systems, cloud, disparate data sources

agree Dr. G - before solutioning we need to understand what data we are addressing the determine the method to treat.

Know your Why of Data using data, ensure alignment, target low hanging for experiment and bitter fruits to bring focus of organisation

I have found that the ecosystem of DataOps focused tools is still maturing and does still have some important gaps, do you agree?

(edited)

as I already called out. to start your journey you don't need much tool. Having data modelling any would work.

starting with the right tool too often means not asking the right data question

Agree with you. I would say we should first evaluate our data pipeline and see what gaps we currently have and look for a tool that can easily integrate into the pipeline, has a quick time to implement and a low learning curve and that will increase data value

Its hard to find a single window to study data. its very isolated

But as you progress, DataOps teams will want to automate, validate data consistency, etc. Consider for instance the context of automating the production readiness of your data.

snowflake

Focusing on right culture, treating data as your second most valuable asset (first is people), Automation, data pipelines, observability, Value stream management are some of the areas from #DevOps that can help #DataOps

agree with all you said. I'd like to add ownership as key element of culture and go boldly forward from observability to telemetry, structured logs approach, etc

collaboration and mindset are key, its a culture shift before a technology one

in my experience, having a transition plan is key, messaging, supporting failure without finger pointing, and having top management support

Lean agile mindset, feedback loops, collaboration and automation can be take to start with

teamup for data, dataops is not project a mindset shift

probably decreased value generation compared to initial devops

dont newd advanced tools until initial gains stop, same as weightlifting #westsidepl

@TinyCyber implicitly then, Value Stream Management is a good partner for DataOps

value stream management should be thw underpinning of any strategic change. If no value in action, then change. Meaximize the work not done #agile

@TinyCyber agree, but who is ensuring this is occuring? there needs to be governance that supports the organizational goals and direction. Enterprise data would ensure the underpinning and DataOps would provide the practice to support delivery (in my opinion)

Disagree. Strategic alignment should be everyones mission. Collaborative ownership of the end product, external governance can decrease value and create resentment. Different than compliance

not a who ensures governance but what are the teams goals for data? Data analytics should be clear. Maybe a data person to offer various options?

@TinyCyber Like cybersecurity, it is everyone's mission but not everyone can be accountable. Having guidance and governance to ensure regulatory and industry best practice is essential. This needs to be overseen to review for consistency in application

ha ha

We’ve established flow and now feedback, perhaps an experiment. If we are chatting than the chat has started

point taken. I am seeing a trend, DevOps to DataOps, and now seeing MLOps. thoughts?

Too many word for similar processes. Key is simplicity, afterall attachment is the root of sufferings

Whichever ops, goal is reducing silos and accelerating value to the customer

I attended last week, concerned with meshing methodologies rather than creating one that customly fits

Data results are always unique, data frameworks not so much, measure what matters

I believe this is where enterprise data structure will help.

transparency! Clone your own data copy to do unusual things, then share results with others

good question and I feel Value Stream Management or even a basic cost/benefit analysis can provide the answer here.

within the Banking & Finance Industry we go by the #governance #compliance rules. for e.g. we need to save transactions for X years then. Easy to segregate :)

@WSzepesi keep all the data for the current process and store old atuff in varying archive layers

@pareeksiddharth those transactions are different than dev data, can use dev for compliance but frequently includes policy and other elements

start with your business question

Depends on the question, early gains need leas data for growth, the closer to potential, the more difficult to close the gap

I believe we need to understand what 'data' you are referring.

Any data associated with the value stream strategically

I do believe that is needed going forward from lean to enterprise #leandata