eweekchat

DataOps and Data Management
JOIN US: Discuss trends in DataOps and Data Management.
   5 years ago
#eweekchatData Storage and ProtectionJOIN US: Discuss trends in Data Storage, Protection and Privacy.
   4 years ago
#eweekchatNew Tech to Expect for 2022JOIN US: Discuss trends in new technology in the year head.
James Maguire
Q10. To wrap up, any predictions for the future of DataOps?
Radhika Krishnan
A10. As #ITandOT converge, the demand for #DataOps will end up serving more interconnected applications, devices, algorithms and #MachineLearning models that drive #digital experiences. #IoT
Bas Kamphuis
A10: We will continue to see a rapid pace of innovation as modern services come together / converge. And I believe it will continue to increase its importance, because digital transformations largest challenge is all about data(ops).
Cognite
A10: DataOps will evolve from a practice and methodology conversation into a SaaS platform discussion in 2022 already - thus significantly expanding both the market and value delivery of #DataOps to enterprises
James Maguire
@BasDutch1 I like this: "digital transformations largest challenge is all about data(ops)."
Bas Kamphuis
A10: despite all the innovations in everything hyperscale-cloud providers do and everything startups are launching to add to the mix of capabilities, one of the hardest hurdles remains data: trying to get data out of a system and knowing what that data means.
Bruce Kornfeld
A10. ML/AI integration for automatic tuning/performance improvements and creating more business value with less human intervention.
Chris Ehrlich
10A: In terms of the future of DataOps, it will become a standard enterprise discipline that does for data what DevOps did for software. #DataOps #DataManagement
Sam Lakkundi
A10. Ever-increasing sources of valuable data will intensify the need for smart, automated data analysis. Automation via #AI #ML will make this diverse and dynamic data be economically manageable. (1/4)
Radhika Krishnan
@BasDutch1 #DigitalTransformation is definitely a catalyst for implementing #DataOps as part of an enterprise wide change. We often see our customers building a #data operating model to serve their new #digital customer experience. #CX
Sam Lakkundi
@samlakkundi A10. Organizations will have to turn to both open source and commercial components that can address the complexity of the modern data supply chain, and they must be integrated into end-to-end solutions. (2/4)
Cognite
A10: In 3 years time, enterprise CDOs and digital transformation executives in industry alike will wonder how they ever managed to support their business without an Enterprise DataOps Platform as part of their enterprise architecture
Sam Lakkundi
@samlakkundi A10. Organizations must consider crowdsourcing for data discoverability, maintenance, and quality improvement. The people required to make data unification effective aren't data engineers, but SMEs who enable a new level of productivity in data delivery. (3/4)
Bas Kamphuis
@RKs2cents 100% agree - and I think it is more and more understood that the Digital transformation is not merely about launching new services but also overcoming the technical debt that keeps a team anchored in yesterdays capabilities.
Bas Kamphuis
@RKs2cents and hence #DataOps is the ability to overcome and increasing the value of the data that is harnessed in those legacy applications whilst creating the services that will make us successful in the future.
Sam Lakkundi
@samlakkundi A10. On a human element other IT roles and responsibilities will shrink, DataOps will grow rapidly. Professionals with skills, vision & talent will be in high demand, & it's safe to predict salaries for DataOps professionals will rise rapidly to match demand (4/4)
Cognite
@BasDutch1 and operationalising and scaling the new services that are deployed/piloted somewhere. Large disconnect there today...
James Maguire
Q9.What do you see as a core best practice for DataOps?
Bruce Kornfeld
A9. To create auditable paths of how the data has been thinned and serviced so to insure the integrity and decision process and to track data from the point of entry to its end of life.
Radhika Krishnan
A9. Automating large volumes of repetitive tasks such as #DataClassification and #DataProtection, can improve productivity by using business policies applied to volumes of data. Automation to curate #Metadata and analysis helps to augment human effort and drive #DataOps.
Bas Kamphuis
A9: . Think about this formula for a second: (data * Model) ^ People. In other words: by making data insightful through a model (context rich and business driven preferably) many more people will be empowered to take smarter actions.
Cognite
A9: As core best practice for DataOps, focus on understanding what DataOps really can do for your business, not just you individually. For those in industry, our free book offers a solid guide to this for all those working in industrials. Download at https://content.cognite.com/the-definitive-guide-t...
https://content.cognite.com/the-definitive-guide-to-industrial-dataops
The Definitive Guide to Industrial DataOps
The Definitive Guide to Industrial DataOps
The guide, written by Cognite, explains the emerging discipline of Industrial DataOps for business and IT professionals - a first for the industry.
Chris Ehrlich
A9: In terms of a core best practice for DataOps, it is dedicated staffing allocation to the discipline. #DataOps #DataManagement
Sam Lakkundi
A9. Assess your environment by starting small. Create a data operations department, align with the organization. Educate your team, make it cross-functional, build for reuse & automation, & use data development tools.
James Maguire
Q8. What industries do you see benefitting the most from DataOps?
Bas Kamphuis
A8 I am biased (appologies), but industries with complex supply chains and/or manufacturing operations at a global scale will reap the highest rewards.
Cognite
A8: As Cognite is only focused on the industrials verticals with our Industrial DataOps Platform, we cannot say much about other industries - but the benefit rating for industrials struggling with operationalising and scaling their digital transformation programs from PoCs is BIG
James Maguire
@BasDutch1 Care to name a couple?
Bas Kamphuis
@BasDutch1 A8 Industries like Oil and Gas, Consumer packaged Good or Life Science will benefit the most because the ability to react faster to a supply chain disruption is key in increasing competitiveness and achieving results

(edited)

Bruce Kornfeld
A8. Industries that are at the edge (retail, manufacturing, smart cities, healthcare) have unique data needs in that too much "noise" is being gathered. I think these would benefit the most.
Sam Lakkundi
A8. Every company, no matter the vertical, will greatly benefit from DataOps, however, I think the customer sectors will see a larger boost. Harnessing insights directly from big data can provide significant competitive advantage.
Radhika Krishnan
A8. We are seeing demand increase across all industries with the most exciting #DataOps innovations in #Banking, #Technology, #Consulting, #Government, #Telecommunications, #Insurance, #Healthcare.
Sam Lakkundi
A8. Generally we have seen that companies in banking, retail and healthcare are more likely to have larger data plays.
Chris Ehrlich
8A: Banking, finance, health care, transportation, travel, manufacturing, energy, retail and technology. Those industries where the competitive imperative to harness the full operational and revenue value of data is inherent now. #DataOps #DataManagement
James Maguire
Q7. So many vendors claim to do DataOps – and they approach it differently. Is the concept losing clarity?
Sam Lakkundi
A7. It’s not losing clarity. DataOps platforms are primarily used by analytics and data teams within an organization; they’re cross-functional and can be used across verticals. IT operation teams can reduce storage infrastructure and increase staff productivity.
Radhika Krishnan
A7. Far from diluting the value of #DataOps, we are seeing the more widespread DataOps becomes, the more powerful the organization becomes at managing and governing #data well. #DataGovernance
Cognite
A7: Looking at the transformative potential of an Enterprise DataOps Platform, there are in fact surprisingly few vendors really focusing on building a unified, seamless UX DataOps Platform (not just relabelling some former product portfolio as DataOps)
Bruce Kornfeld
A7. I think it's lost some of the focus. Many vendors have created lock-in approaches to their integrated technologies that require manual processes for a market that needs a simple yet company agnostic work-flow. Still a wide-open market for innovation.
Cognite
A7: Also, it feels very natural to expect differentiated DataOps Platforms to emerge, as different verticals have very different data types and use case portfolios a DataOps Platform needs to elegantly support, not just support with a set of workarounds...
Chris Ehrlich
7A: In terms of the clarity of DevOps, as a segment ripe for entrants, it will be further grayed. Buyers, however, will recognize the need to clarify problems and spec solutions, tech- or talent-related, and in doing so, define the segment. #DataOps #DataManagement

(edited)

James Maguire
@brucekornfeld "company agnostic work-flow" is an elusive quality.
Sam Lakkundi
@samlakkundi At @BMCSoftware, it's clear for us given our numerous customer engagements in this space.
Bas Kamphuis
A7: probably true, probably a lot of vendors do focus here because it's such an interesting problem to solve in the bigger context of the industry challenges. We had Magnitude decide to focus and it's all about the ERP data for us.
James Maguire
Q6. Can a company “buy” DataOps or is it simply a process to implement? Will it adopt a SaaS model?
Radhika Krishnan
A6.1 ) #DataOps isn’t a noun, it’s a verb i.e. it’s something that is performed day-in, day-out as the new norm. A company can always procure new tools and technologies and then buy in or even hire in new expertise to run it. #DataCulture #DataCitizens
Sam Lakkundi
A6. Not quite. There are a lot of vendors that play in the space, but that means you’ll will swim in a sea of tools and APIs. To get the most benefit, organizations will need robust orchestration and automation to bring the pieces together to best serve their needs.
Bruce Kornfeld
A6. Today is a process and a philosophy. In the future, products will emerge that end users can actually deploy to solve these difficult data problems in moving and using data for maximum value. I don't see this as a SaaS model - at least for data at the edge.
James Maguire
@samlakkundi But "robust orchestration and automation" is a buy-able solution, right?
Radhika Krishnan
A6.2) Being successful means #DataOps practice is understood, adopted and performed by a critical mass of producers and consumers across the enterprise.
Sam Lakkundi
Spot on. However, not without heavy customization.

(edited)

Bas Kamphuis
A6 within context yes on both. This is what we do at Magnitude. Specific to business apps, we are taking a lot of the proprietary IP / knowledge hurdles and enabling the customer's workforce and putting it into software code
Cognite
DataOps is shaping up into a SaaS play for sure, incl. verticalized versions addressing different markets. For DataOps to become transformative at scale, it needs to become a true product experience with UX that makes the previously impossible easy.
Chris Ehrlich
6A: DataOps is a process before it’s a tool. More ultra-niche tools, particularly around automation, will be developed, but DataOps ought to be fundamentally process-driven to reach goals. #DataOps #DataManagement
Bas Kamphuis
A6 In '21 of course the automation, or engine if you will, is provisioned as a Service in a broader analytics architecture.
Bas Kamphuis
A6 but that is a key point today - Dataops is not an objective, its a core element in the ability to transform to a 'data driven organization'. Doing dataops right will help secure success, but its only one element