eweekchat

DataOps and Data Management
JOIN US: Discuss trends in DataOps and Data Management.
   4 years ago
#eweekchatData Storage and ProtectionJOIN US: Discuss trends in Data Storage, Protection and Privacy.
   4 years ago
#eweekchatNew Tech to Expect for 2022JOIN US: Discuss trends in new technology in the year head.
James Maguire
Q8. What industries do you see benefitting the most from DataOps?
Bas Kamphuis
A8 I am biased (appologies), but industries with complex supply chains and/or manufacturing operations at a global scale will reap the highest rewards.
Cognite
A8: As Cognite is only focused on the industrials verticals with our Industrial DataOps Platform, we cannot say much about other industries - but the benefit rating for industrials struggling with operationalising and scaling their digital transformation programs from PoCs is BIG
James Maguire
@BasDutch1 Care to name a couple?
Bas Kamphuis
@BasDutch1 A8 Industries like Oil and Gas, Consumer packaged Good or Life Science will benefit the most because the ability to react faster to a supply chain disruption is key in increasing competitiveness and achieving results

(edited)

Bruce Kornfeld
A8. Industries that are at the edge (retail, manufacturing, smart cities, healthcare) have unique data needs in that too much "noise" is being gathered. I think these would benefit the most.
Sam Lakkundi
A8. Every company, no matter the vertical, will greatly benefit from DataOps, however, I think the customer sectors will see a larger boost. Harnessing insights directly from big data can provide significant competitive advantage.
Radhika Krishnan
A8. We are seeing demand increase across all industries with the most exciting #DataOps innovations in #Banking, #Technology, #Consulting, #Government, #Telecommunications, #Insurance, #Healthcare.
Sam Lakkundi
A8. Generally we have seen that companies in banking, retail and healthcare are more likely to have larger data plays.
Chris Ehrlich
8A: Banking, finance, health care, transportation, travel, manufacturing, energy, retail and technology. Those industries where the competitive imperative to harness the full operational and revenue value of data is inherent now. #DataOps #DataManagement
James Maguire
Q7. So many vendors claim to do DataOps – and they approach it differently. Is the concept losing clarity?
Sam Lakkundi
A7. It’s not losing clarity. DataOps platforms are primarily used by analytics and data teams within an organization; they’re cross-functional and can be used across verticals. IT operation teams can reduce storage infrastructure and increase staff productivity.
Radhika Krishnan
A7. Far from diluting the value of #DataOps, we are seeing the more widespread DataOps becomes, the more powerful the organization becomes at managing and governing #data well. #DataGovernance
Cognite
A7: Looking at the transformative potential of an Enterprise DataOps Platform, there are in fact surprisingly few vendors really focusing on building a unified, seamless UX DataOps Platform (not just relabelling some former product portfolio as DataOps)
Bruce Kornfeld
A7. I think it's lost some of the focus. Many vendors have created lock-in approaches to their integrated technologies that require manual processes for a market that needs a simple yet company agnostic work-flow. Still a wide-open market for innovation.
Cognite
A7: Also, it feels very natural to expect differentiated DataOps Platforms to emerge, as different verticals have very different data types and use case portfolios a DataOps Platform needs to elegantly support, not just support with a set of workarounds...
Chris Ehrlich
7A: In terms of the clarity of DevOps, as a segment ripe for entrants, it will be further grayed. Buyers, however, will recognize the need to clarify problems and spec solutions, tech- or talent-related, and in doing so, define the segment. #DataOps #DataManagement

(edited)

James Maguire
@brucekornfeld "company agnostic work-flow" is an elusive quality.
Sam Lakkundi
@samlakkundi At @BMCSoftware, it's clear for us given our numerous customer engagements in this space.
Bas Kamphuis
A7: probably true, probably a lot of vendors do focus here because it's such an interesting problem to solve in the bigger context of the industry challenges. We had Magnitude decide to focus and it's all about the ERP data for us.
James Maguire
Q6. Can a company “buy” DataOps or is it simply a process to implement? Will it adopt a SaaS model?
Radhika Krishnan
A6.1 ) #DataOps isn’t a noun, it’s a verb i.e. it’s something that is performed day-in, day-out as the new norm. A company can always procure new tools and technologies and then buy in or even hire in new expertise to run it. #DataCulture #DataCitizens
Sam Lakkundi
A6. Not quite. There are a lot of vendors that play in the space, but that means you’ll will swim in a sea of tools and APIs. To get the most benefit, organizations will need robust orchestration and automation to bring the pieces together to best serve their needs.
Bruce Kornfeld
A6. Today is a process and a philosophy. In the future, products will emerge that end users can actually deploy to solve these difficult data problems in moving and using data for maximum value. I don't see this as a SaaS model - at least for data at the edge.
James Maguire
@samlakkundi But "robust orchestration and automation" is a buy-able solution, right?
Radhika Krishnan
A6.2) Being successful means #DataOps practice is understood, adopted and performed by a critical mass of producers and consumers across the enterprise.
Sam Lakkundi
Spot on. However, not without heavy customization.

(edited)

Bas Kamphuis
A6 within context yes on both. This is what we do at Magnitude. Specific to business apps, we are taking a lot of the proprietary IP / knowledge hurdles and enabling the customer's workforce and putting it into software code
Cognite
DataOps is shaping up into a SaaS play for sure, incl. verticalized versions addressing different markets. For DataOps to become transformative at scale, it needs to become a true product experience with UX that makes the previously impossible easy.
Chris Ehrlich
6A: DataOps is a process before it’s a tool. More ultra-niche tools, particularly around automation, will be developed, but DataOps ought to be fundamentally process-driven to reach goals. #DataOps #DataManagement
Bas Kamphuis
A6 In '21 of course the automation, or engine if you will, is provisioned as a Service in a broader analytics architecture.
Bas Kamphuis
A6 but that is a key point today - Dataops is not an objective, its a core element in the ability to transform to a 'data driven organization'. Doing dataops right will help secure success, but its only one element
James Maguire
Q5. Apart from the challenges listed above, is DataOps’s greatest challenge human or technological?
Radhika Krishnan
A5. It’s both and necessarily so. Challenge is how we effectively #collaborate in a large organization, how we create processes to add end-to-end visibility, what tech do we use for scalable #DataManagement. #UX #AI
Bas Kamphuis
A5: It’s a combination of both: technology hasn't quite advanced to do it all, and human capital is not infinite in its capacity. And so, between those two, the challenges are probably about equal. The answer is automation.
Sam Lakkundi
A5. Human element is DataOps’ greatest challenge. We need a different approach to DataOps where self-service and low-code/no-code data pipelines are a fundamental principle. Only practical solution is democratization of data integration enabled through automation
Chris Ehrlich
A5: In terms of DataOps’ greatest challenge being human or technical, it is human-related. DataOps must compete as another sub-discipline for internal budgets and tech talent and a field for pros to specialize in over other more mature data fields. #DataOps
Bruce Kornfeld
A5. Technological. It's just not easy to create work-flows for ingesting, processing, moving, and storing data for all customers' use cases - but the ones that solve this will have true market disruption.
Cognite
A5: The greatest challenge with all new technology remains the product maturity until that is overcome. Human, organisational, cultural, etc. topics are just more trendy to discuss. For example, all talk on the "cultural" issues with smartphones ended after iPhone @JamesMaguire
James Maguire
@RKs2cents I agree! Might have been a trick question :)
Sam Lakkundi
@samlakkundi A5. At our BMC Innovation Labs, we believe while technology challenges can be difficult to overcome, some of the human barriers around their DataOps initiatives are even more pressing. #BMCInnovationLabs
Cognite
@brucekornfeld Truth spoken. And not necessary to solve universally either, within specific verticals is a great place to start - and is in general the mega trend across enterprise software
Radhika Krishnan
@samlakkundi We agree with democratization point along with enablers of #DataGovernance at scale
James Maguire
Q4. What’s DataOps’s greatest challenge: Cohesion between the teams? Process efficiency? Diversity of technologies? Or...?
Radhika Krishnan
A4. The reality is 87% of #DataScience projects never make it into production and the majority of #DataLakes have become unmanaged and ungoverned. See the report: https://htchivantara.is/3DtKLXc. #DataOps provides better practices to bring back value to data.
https://htchivantara.is/3DtKLXc
451 Research: DataOps Unlocks the Value of Data
451 Research: DataOps Unlocks the Value of Data
A recent survey from 451 Research revealed that 100% of respondents are currently planning or actively pursuing initiatives to deliver more agile and automated data management to manage vast new data flows. The method for unlocking value from the vas...
Chris Ehrlich
A4: DataOps’ greatest challenge is leadership buy-in for an organizational commitment to specialized DataOps functions — and not pushing the discipline as an add-on function on data professionals with other core priorities. #DataOps #DataManagement
Sam Lakkundi
A4. Effective communication among key stakeholders, yet poor coordination can make building, deploying, and maintaining data pipelines more difficult than needed. Many #DataOps teams don't understand how data pipeline works.
Cognite
A4: Technology immaturity combined with very real change management is always a challenging combo... as much as many love talking about culture being the greatest challenge to all new things, genuine fit-for-purpose technology platform (SaaS) availability is alike @JamesMaguire
Radhika Krishnan
@Chris_Ehrlich Totally agree on the leadership buy-in. That's crucial to sustain the transformation required.
Bruce Kornfeld
A4. Generic vs. Specific. It's hard to build a data solution that works across all industries - it'll be too generic. But, building a solution for a particular industry is so narrow that it won't get widespread traction. I wonder if this is slowing innovation???
Cognite
A4: There's just something magical that happens when the right user is given access to the right product that actually makes their life meaningfully easier - it takes care of a lot by itself, incl. change management and plenty of executive leadership support need @JamesMaguire
Sam Lakkundi
@Chris_Ehrlich Couldn't agree more on leadership buy-in.
Cognite
@brucekornfeld There will be vertical specialization for sure within DataOps Platforms, simply as the needs in different markets are so very different, as is the data alike
Bas Kamphuis
A4: I think there are 3 major challenges: 1: Data silos, 2: proprietary technology stacks; and 3: the state of automation.
Bas Kamphuis
A4: Lastly the lack of intelligence within BI is a scaling challenge: there is not a whole lot of self-learning algorithms that are helping you interpret data at the BI layer of the architectural stack.