eWeekChat

A5: #DataOps is an intelligent approach that offers agility and automation of data management. Through collaboration focused on the outcome, we bring cross-functional data and IT teams together to deliver the right data to the right people at the right time with the right quality

A5. Evaluate #opensource and open algorithms. Find less complex tools. Allocate budget, people, and time. Start with proven use cases (#UBA, #AIOps). Plan to fail, plan to learn!

A5 : Data analytics stack must be designed in a flexible way to accommodate this data growth. This includes managed data platforms, integrated DevOps, and scalable and managed compute resources.

@vamsipaladugus1 A5:Re: talent issues, build training and certifications & encourage employee initiative, idea ownership, and innovation that has them thinking outside the box.

.@AndiMann Failing fast is key to learn from any data project

A5: Introducing DataOps is an enabler to successfully adopting enterprise-wide data analytics. Orgs that realize the benefit of analytics often employa multi-horizon approach, focusing on using DataOps for a small set of high value use cases before scaling across the company.

(edited)

A5: Invest in data governance and data management platforms for data analytics. #eWEEKchat

A5: We often emphasize the tools aspect of #DataOps, but critical to remember the org aspect is just as critical. Intentional alignment and accountability of how individuals can engage in the #DataCloud makes a HUGE difference. It won't happen 'by accident'

@madhoop Exactly! I advise my clients and teams - fail fast, fail small, fail cheap, fail forward. Make space for #ContinuousLearning!

A5 (1/2): 1. Work with technology that puts flexibility and openness at the forefront. 2. Avoid lock in with proprietary vendor storage and compute — Your data services need to work for you as required on any cloud, not trap you.

Prioritize enablement and training, invest in data governance, select tools that reduce siloed data sets

@KatySalamati I wish I had more upvotes for this. Data governance is its own reward, not just in compliance or privacy, but the agility to reliably use the right data at the right time!

.@jeffhollan Organizational culture is front and center to any data problem. #DataOps is fundamentally a cross collaboration between #Data and #Ops teams.

A5: Know the relationships between data management, data science, and data analytics and commit to investing in those functions as a competitive advantage

A5 (2/2): 3. interoperability is essential. Regardless of what system you use, your tooling should enable streamlined and secure end-to-end workflows. 4. Avoid proprietary formats — If you're data is in an open usable format it gives your business more agility to stay ahead

@AndiMann, couldn't have said it better!

@madhoop Right on regarding #DataOps — Automation and smart tooling will in many ways define the winners and losers this year

.@namessanti - #2023 will be the year of #DataOps

@madhoop You heard it here first!

A4: Not knowing their own data and not having data governance in place are two of the major challenges that companies face with data analytics. #eWEEKchat

A4:As we move into 2023, data sets have become massive and continue to grow rapidly, making the scalability of data analytics platforms an ongoing challenge.

@vamsipaladugus1 A4:Hiring and retaining the right talent can also be a major challenge. In AI/Ml space , the market is competitive, it can be hard to retain talent

(edited)

A4. Skills obvs, but also reliability, predictability, verifiability, actionability, and ultimately, trust. Research shows over and over that leaders do not trust ‘black box’ analytics!

A4: (1/3) Pipeline complexity – increasing data volumes, multiple types & sources & growing use of real-time data are driving pipeline complexity. This is further magnified by the lack of coordination/collaboration across teams involved in data pipelines.

A4. (cont) Too many engines promise the world but never show their working, so insights are unverifiable, unpredictable, and do not drive action.

A4: (2/3) Governance & Transparency – traceability, observability of data requires new processes & skills to deliver at scale & at the pace that the business require. Use of AI analytics requires greater end-to-end transparency of data, beyond those provided by traditional tools.

A4: More and more people are coming into the "data analytics" tent in an organization, so finding ways to scale the tools for that level of collaboration. DataOps is increasingly critical here

A4. (cont) No wonder leaders typically say their biggest roadblock to using #data #analytics is #trust!

A4: (3/3) Benefit realization – new technologies (such as #edgecomputing) are being implemented without clear approaches to define, measure and realize value from the data. Many #analytics projects are still failing to deliver expected outcomes.

A4: Having fragmented data sets and not being able to easily correlate data between multiple tools

(edited)

A4: (1/2) Secure collection and movement of data in a timely or real time way. Too often we see businesses working of analysis that took long enough for data to be outdated and not yield the best results

A4 (1/2): #DataGovernance remains a significant challenge for companies. It’s been estimated that businesses with low #data quality data are losing around $3.1 trillion in the U.S. alone – or 20% of their revenue.

A4 (2/2): Being able to know your #data, understand its meaning and apply policies to automate #DataGovernance is critical to getting it in the hands of the business to drive data ANALYTICS.

@BMCSoftware Spot on there too - these are all additional elements in building trust. Great add!

A4: (2/2) Automation is a key factor here that cuts both human and resource cost drastically.

@namessanti Santiago, definitely see that too. Privacy, confidentially, regulations, and more often lead to an inability to use data in any real and meaningful way.

@vamsipaladugus1 this is spot on. How can you increasingly do more with less.

.@BMCSoftware Simplifying pipeline complexity is key. But also more real time processing of data and moving away from #Batch

A4: Never first having a handle on data management, particularly with growing AI and IoT data, to realize organization-wide analytics

@jeffhollan DataOps is a key to any successful data analytics project. otherwise it is garbage in garbage out!

@AndiMann Ditto. Timeliness of insights is the most critical variable to decision making — with blockers at the data ingest point, the analytic technology, and the regulators etc.

A3: Cost, in terms of the time and right resources that are needed at any level. #eWEEKchat

A3:Cost is a barrier because data is scattered, siloed, and consumed in multiple ways. Solutions include well-governed, centralized data lakes that provide visibility and access, evaluating deployment criteria and monitoring characteristics.

A3 (1/2): With #data becoming more distributed, diverse and dynamic, cost is always a challenge in data analytics, at any level. The biggest cost comes from data management … to onboard, cleanse, curate and use it for analytics. This has led to the practice of #DataOps.

A3 (2/2): A largely unrealized cost element that not enough people are talking about is the cost of running #DataAnalytics in the public cloud, leading to functions such as #FinOps.

A3: There are services—our Seagate Lyve Cloud for example—that eliminate multicloud friction as well.

A3: (1/3) There are multiple potential cost dimensions associated with #dataanalytics: 1) Access to suitably qualified and experienced talent increases the cost of implementation of new #analytics solutions and impairs realization of expected value.

A3: This brings me back to my previous prediction — companies need to run compute workloads where it's most economical. Major public clouds are amazing for some things, but cost-prohibitive for others. This is where a portable and interoperable platform is key.

A3:(2/3) 2)Delivery & operation of modern data pipelines is hugely complex & requires significant effort & ongoing support. 3)There are considerable costs associated w/collection & processing of non-valuable data. Often there is an assumption that more data = more value

@madhoop well said. that is why it is important to have efficient data analytics platforms that are fast, reliable and can scale! #sasviya #SASsoftware

A3. For sure, there area lot of great free #opensource tools and algorithms, but the most useful technologies are also complex and difficult. So the tech is $$$

A3: (3/3) which is often sadly not the case.
4) Finally, the impact and cost to the business from poorly performing analytics cannot be understated. Poor data analytics and subsequent bad decisions can have significant cost impact upon all aspects of a business.

.@BMCSoftware - Agreed on experienced talent part. Esp. with the recession looming and hiring freeze, I wonder what it will do to the talent pool of analytics professions.

A3: Yes, staffing cost is a major challenge — getting buy-in to build analytics teams that can get past the associated data management and become targeted department resources

A3: On the other side, training is also a cost-issue. Many platforms require you to work with notebook only env. which requires hard skills and time. The more we can adopt standardized open software and harden it for enterprise the more skilled professionals will be available

@madhoop Aha! Great add Madhoop - I was evacuating cloud-based 'Computer Vision' tools last year, and the compute was $$$, plus major cost of ingress/egess for HD 4K video!

.@AndiMann AI hardware has a lot of cost tied to it for sure. People like #ChatGPT. Do they know how expensive of an infrastructure it requires to run optimally?

(edited)

@madhoop Hopefully as data analytics technology continues to mature, one of the resulting outcomes will be to reduce the skills threshold necessary to support effective analytics

(edited)

@madhoop Oh, so much this too! Simply the cost of compute power for some of the available tools will wreck most #innovation budgets!

.@AndiMann Perhaps #Cloud can help commoditize the running costs of hardware intensive #AI workloads