eWeekChat

   a year ago
#eWeekChatEnterprise Tech in 2023JOIN US: Discuss the future of enterprise tech.
   a year ago
#eWeekChatMulticloud ChallengesExperts discuss multicloud computing
James Maguire
Q5. What 1-2 strategies do you recommend to overcome these challenges with data analytics?
Madhup Mishra
A5: #DataOps is an intelligent approach that offers agility and automation of data management. Through collaboration focused on the outcome, we bring cross-functional data and IT teams together to deliver the right data to the right people at the right time with the right quality
Andi Mann
A5. Evaluate #opensource and open algorithms. Find less complex tools. Allocate budget, people, and time. Start with proven use cases (#UBA, #AIOps). Plan to fail, plan to learn!
Vamsi Paladugu
A5 : Data analytics stack must be designed in a flexible way to accommodate this data growth. This includes managed data platforms, integrated DevOps, and scalable and managed compute resources.
Vamsi Paladugu
@vamsipaladugus1 A5:Re: talent issues, build training and certifications & encourage employee initiative, idea ownership, and innovation that has them thinking outside the box.
Madhup Mishra
.@AndiMann Failing fast is key to learn from any data project
BMC Software
A5: Introducing DataOps is an enabler to successfully adopting enterprise-wide data analytics. Orgs that realize the benefit of analytics often employa multi-horizon approach, focusing on using DataOps for a small set of high value use cases before scaling across the company.

(edited)

katy Salamati
A5: Invest in data governance and data management platforms for data analytics. #eWEEKchat
Jeff Hollan
A5: We often emphasize the tools aspect of #DataOps, but critical to remember the org aspect is just as critical. Intentional alignment and accountability of how individuals can engage in the #DataCloud makes a HUGE difference. It won't happen 'by accident'
Andi Mann
@madhoop Exactly! I advise my clients and teams - fail fast, fail small, fail cheap, fail forward. Make space for #ContinuousLearning!
Santiago Giraldo
A5 (1/2): 1. Work with technology that puts flexibility and openness at the forefront. 2. Avoid lock in with proprietary vendor storage and compute — Your data services need to work for you as required on any cloud, not trap you.
Jemiah Sius
Prioritize enablement and training, invest in data governance, select tools that reduce siloed data sets
Andi Mann
@KatySalamati I wish I had more upvotes for this. Data governance is its own reward, not just in compliance or privacy, but the agility to reliably use the right data at the right time!
Madhup Mishra
.@jeffhollan Organizational culture is front and center to any data problem. #DataOps is fundamentally a cross collaboration between #Data and #Ops teams.
Chris Ehrlich
A5: Know the relationships between data management, data science, and data analytics and commit to investing in those functions as a competitive advantage
Santiago Giraldo
A5 (2/2): 3. interoperability is essential. Regardless of what system you use, your tooling should enable streamlined and secure end-to-end workflows. 4. Avoid proprietary formats — If you're data is in an open usable format it gives your business more agility to stay ahead
katy Salamati
@AndiMann, couldn't have said it better!
Santiago Giraldo
@madhoop Right on regarding #DataOps — Automation and smart tooling will in many ways define the winners and losers this year
Madhup Mishra
.@namessanti - #2023 will be the year of #DataOps
James Maguire
@madhoop You heard it here first!
James Maguire
Q4. Apart from cost, what are the major challenges that companies face with data analytics?
katy Salamati
A4: Not knowing their own data and not having data governance in place are two of the major challenges that companies face with data analytics. #eWEEKchat
Vamsi Paladugu
A4:As we move into 2023, data sets have become massive and continue to grow rapidly, making the scalability of data analytics platforms an ongoing challenge.
Vamsi Paladugu
@vamsipaladugus1 A4:Hiring and retaining the right talent can also be a major challenge. In AI/Ml space , the market is competitive, it can be hard to retain talent

(edited)

Andi Mann
A4. Skills obvs, but also reliability, predictability, verifiability, actionability, and ultimately, trust. Research shows over and over that leaders do not trust ‘black box’ analytics!
BMC Software
A4: (1/3) Pipeline complexity – increasing data volumes, multiple types & sources & growing use of real-time data are driving pipeline complexity. This is further magnified by the lack of coordination/collaboration across teams involved in data pipelines.
Andi Mann
A4. (cont) Too many engines promise the world but never show their working, so insights are unverifiable, unpredictable, and do not drive action.
BMC Software
A4: (2/3) Governance & Transparency – traceability, observability of data requires new processes & skills to deliver at scale & at the pace that the business require. Use of AI analytics requires greater end-to-end transparency of data, beyond those provided by traditional tools.
Jeff Hollan
A4: More and more people are coming into the "data analytics" tent in an organization, so finding ways to scale the tools for that level of collaboration. DataOps is increasingly critical here
Andi Mann
A4. (cont) No wonder leaders typically say their biggest roadblock to using #data #analytics is #trust!
BMC Software
A4: (3/3) Benefit realization – new technologies (such as #edgecomputing) are being implemented without clear approaches to define, measure and realize value from the data. Many #analytics projects are still failing to deliver expected outcomes.
Jemiah Sius
A4: Having fragmented data sets and not being able to easily correlate data between multiple tools

(edited)

Santiago Giraldo
A4: (1/2) Secure collection and movement of data in a timely or real time way. Too often we see businesses working of analysis that took long enough for data to be outdated and not yield the best results
Madhup Mishra
A4 (1/2): #DataGovernance remains a significant challenge for companies. It’s been estimated that businesses with low #data quality data are losing around $3.1 trillion in the U.S. alone – or 20% of their revenue.
Madhup Mishra
A4 (2/2): Being able to know your #data, understand its meaning and apply policies to automate #DataGovernance is critical to getting it in the hands of the business to drive data ANALYTICS.
Andi Mann
@BMCSoftware Spot on there too - these are all additional elements in building trust. Great add!
Santiago Giraldo
A4: (2/2) Automation is a key factor here that cuts both human and resource cost drastically.
Andi Mann
@namessanti Santiago, definitely see that too. Privacy, confidentially, regulations, and more often lead to an inability to use data in any real and meaningful way.
Jeff Hollan
@vamsipaladugus1 this is spot on. How can you increasingly do more with less.
Madhup Mishra
.@BMCSoftware Simplifying pipeline complexity is key. But also more real time processing of data and moving away from #Batch
Chris Ehrlich
A4: Never first having a handle on data management, particularly with growing AI and IoT data, to realize organization-wide analytics
katy Salamati
@jeffhollan DataOps is a key to any successful data analytics project. otherwise it is garbage in garbage out!
Santiago Giraldo
@AndiMann Ditto. Timeliness of insights is the most critical variable to decision making — with blockers at the data ingest point, the analytic technology, and the regulators etc.
James Maguire
Q3. Is cost a major challenge in data analytics at any level? Cost of application, or training? How to remedy this?
katy Salamati
A3: Cost, in terms of the time and right resources that are needed at any level. #eWEEKchat
Vamsi Paladugu
A3:Cost is a barrier because data is scattered, siloed, and consumed in multiple ways. Solutions include well-governed, centralized data lakes that provide visibility and access, evaluating deployment criteria and monitoring characteristics.
Madhup Mishra
A3 (1/2): With #data becoming more distributed, diverse and dynamic, cost is always a challenge in data analytics, at any level. The biggest cost comes from data management … to onboard, cleanse, curate and use it for analytics. This has led to the practice of #DataOps.
Madhup Mishra
A3 (2/2): A largely unrealized cost element that not enough people are talking about is the cost of running #DataAnalytics in the public cloud, leading to functions such as #FinOps.
Vamsi Paladugu
A3: There are services—our Seagate Lyve Cloud for example—that eliminate multicloud friction as well.
BMC Software
A3: (1/3) There are multiple potential cost dimensions associated with #dataanalytics: 1) Access to suitably qualified and experienced talent increases the cost of implementation of new #analytics solutions and impairs realization of expected value.
Santiago Giraldo
A3: This brings me back to my previous prediction — companies need to run compute workloads where it's most economical. Major public clouds are amazing for some things, but cost-prohibitive for others. This is where a portable and interoperable platform is key.
BMC Software
A3:(2/3) 2)Delivery & operation of modern data pipelines is hugely complex & requires significant effort & ongoing support. 3)There are considerable costs associated w/collection & processing of non-valuable data. Often there is an assumption that more data = more value
katy Salamati
@madhoop well said. that is why it is important to have efficient data analytics platforms that are fast, reliable and can scale! #sasviya #SASsoftware
Andi Mann
A3. For sure, there area lot of great free #opensource tools and algorithms, but the most useful technologies are also complex and difficult. So the tech is $$$
BMC Software
A3: (3/3) which is often sadly not the case.
4) Finally, the impact and cost to the business from poorly performing analytics cannot be understated. Poor data analytics and subsequent bad decisions can have significant cost impact upon all aspects of a business.
Madhup Mishra
.@BMCSoftware - Agreed on experienced talent part. Esp. with the recession looming and hiring freeze, I wonder what it will do to the talent pool of analytics professions.
Chris Ehrlich
A3: Yes, staffing cost is a major challenge — getting buy-in to build analytics teams that can get past the associated data management and become targeted department resources
Santiago Giraldo
A3: On the other side, training is also a cost-issue. Many platforms require you to work with notebook only env. which requires hard skills and time. The more we can adopt standardized open software and harden it for enterprise the more skilled professionals will be available
Andi Mann
@madhoop Aha! Great add Madhoop - I was evacuating cloud-based 'Computer Vision' tools last year, and the compute was $$$, plus major cost of ingress/egess for HD 4K video!
Madhup Mishra
.@AndiMann AI hardware has a lot of cost tied to it for sure. People like #ChatGPT. Do they know how expensive of an infrastructure it requires to run optimally?

(edited)

BMC Software
@madhoop Hopefully as data analytics technology continues to mature, one of the resulting outcomes will be to reduce the skills threshold necessary to support effective analytics

(edited)

Andi Mann
@madhoop Oh, so much this too! Simply the cost of compute power for some of the available tools will wreck most #innovation budgets!
Madhup Mishra
.@AndiMann Perhaps #Cloud can help commoditize the running costs of hardware intensive #AI workloads