clearskyedge

Storage at Intelligent Edge
Discuss the trend under which data storage is moving to the cloud's intelligent edge.
Peter Burris
What are the chief apps and benefits of moving storage to the intelligent edge? http://www.via-cc.at...

jameskobielus
Moving storage to the intelligent edge speeds data access, boosts data avaialbility, and enables more reliable disaster recovery.
Keith Townsend - Light will overcome darkness
Analytic applications in general. So AI/ML/IoT are but buzzwords today. Collaboration is a 2nd beneficiary.
Bert Latamore
Avoid latency issues.
Peter Burris
Apps that monitor and triage SCADA/OT performance.
Courtney Pallotta
- data-intensive applications: ML, AI are perfect examples, traditional DB apps...all benefit from the intelligent edge
Keith Townsend - Light will overcome darkness
While we don't always consider it so, Public Cloud is also an edge location in an Hybrid-IT environment.
Bert Latamore
Also edge locations may not have strong network connections that can handle transmission of large data volumes.
jameskobielus
@CTOAdvisor Data applications in general: intelligent edge storage can also facilitate fast in-memory OLTP at the edge.
Dewin Chandra
analytics, machine learning, or any time-critical apps that handle patient care where milliseconds can have big consquences
jameskobielus
@CTOAdvisor Really? Can you explain how you define public cloud as an edge location?
Courtney Pallotta
@BertLatamore True. It's important to be in the right edge location. Highly connected colos with lots of network options.
Darren Miller
@CTOAdvisor I agree with you Keith but utilizing both the public cloud and local DC as intelligent edge options pose data mobility issues in most cases. Then an intelligent mobility solution is in order
Marc Keating
@BertLatamore Intelligent edge technology works best when you have excellent network connectivity and a diversity of carriers.
Peter Burris
Bandwidth providers are still charging enormous rates for edge support. Sometimes, even if data can be moved, it is too expensive to move.
Keith Townsend - Light will overcome darkness
@jameskobielus If you still have a enterprise DC strategy but use Public Cloud for purpose built applications such as R&D, it's an edge location. It's massive in compute/storage capacity but it's not your central data repository.
jameskobielus
Good one. Yes, each colocated storage facility is its own "edge" for the purpose of persisting data there for speeding access to that data by downstream users/apps.
Darren Miller
easier to move, expensive to retrieve!
Bert Latamore
@MarcJKeating Sometimes the edge locations do not have that connectivity. A drilling platform in the middle of the Caribbean, for instance.
Marc Keating
Bandwidth charges vary greatly based on location and the availability of carriers. For example, if you are in a well connected co-lo facility or a highly connected metro area, you'll have a far greater choice and price point for connectivity
jameskobielus
@CTOAdvisor Oh, yes. The edge storage can refer to any purpose-built node, cluster, or zone for data persisting/access to a replicated segment of the larger central data repository.
Marc Keating
@BertLatamore no doubt. In the case of an off-shore drilling platform, you're likely limited to satellite connectivity, which can be cost prohibitive, unless the value of moving the data in near real time was worth it to the business
Marc Keating
It also depends on the purpose of moving or accessing data is. If you're simply trying to back up the data on the oil rig, for example, and you have a 24 hour RPO, you could send tapes on the daily supply ship.
Marc Keating
On the other hand, if your business sees enough value in sharing that data within minutes, you could certainly obtain a high enough satellite link, but the relative cost of doing that would be much higher than the tape transport method
Bert Latamore
@MarcJKeating Microwave would be another choice, again fairly expensive. Better to analyze the data on site & send the results of the analysis rather than the raw data in many cases.
Peter Burris
How should you manage primary storage alongside backup and recovery at the intelligent edge?
http://www.via-cc.at...

Keith Townsend - Light will overcome darkness
I'd like not to worry about this at all. It either integrates into my existing backup platform or is part of the package for a intelligent edge solution. I don't want a new solution for backup because the data is physically located at the edge.
Courtney Pallotta
You can SETTLE for the status quo. Primary, backup and DR are one set of challenges and can be solved together - not in silos. #clearskyedge
Darren Miller
The edge should be used for it's purpose.. high speed data access while backups can be pushed out to the cloud for long term.
John Furrier
this is a great question many are learning what is best practice I haven't seen a consensus of what's a good solution
jameskobielus
A well-architected storage architecture should seamlessly replicate data from primary storage to backup/recovery edge nodes/clusters. That's are core definition of intelligent edge storage.
Peter Burris
@CTOAdvisor Edge RPO/RTO: That's policy-driven, right? Many B/R solutions don't support edge-oriented policies well.
Courtney Pallotta
You CAN'T :) SETTLE for the status quo. Primary, backup and DR are one set of challenges and can be solved together - not in silos. #clearskyedge
Darren Miller
Unless I'm using my BUs during the day for purposes other than traditional backup, I don't want them on my edge. I want them available when I need them and my edge to service my app compute.
Keith Townsend - Light will overcome darkness
Like all things it depends. What are you trying to protect against in backups vs. snapshots? If you have to restore from backup then do you enact your BC plan vs. recovering a corrupted file locally?
Marc Keating
@jameskobielus replicating data from primary storage to backup/recovery is certainly one way of doing that, but it involves making, paying for, and moving copies of data
jameskobielus
@DarrenMilr So, essentially, you're defining edge storage's primary use case as fast data access. Right?
Keith Townsend - Light will overcome darkness
Synthetic (do they even use that term anymore) backups from the edge to the core combined with local snapshots may meet most recovery scenarios. How often do you need to recovery TB's from backup? That's never a good situation.
Peter Burris
What does it mean to push storage to the intelligent edge? http://www.via-cc.at...

Keith Townsend - Light will overcome darkness
the ability to push/pull data as close to the consumption point as possible.
Peter Burris
Follow the data. Increasing, we're moving services to the "moment of execution."
jameskobielus
Pushing storage to the intelligent edge involves persisting data close to users/apps to speed query, analysis, transactions. etc.
Keith Townsend - Light will overcome darkness
We are no longer able to say just move compute to the data. How do you move cloud-based AI/ML to the edge?
Courtney Pallotta
The intelligent edge makes all data available quickly, as quickly as on prem - keep your data close, but it doesn't HAVE TO live in your data center.#clearskyedge
jameskobielus
Pushing storage to the intelligent edge is all about delivering flash-grade performance consistently through smart data movement/placement near access.
Keith Townsend - Light will overcome darkness
@jameskobielus Why does it have to be flash-grade? Does every edge use case require flash-level performance?
jameskobielus
@CTOAdvisor Yes. Perhaps also it's the abiity to selectively cache the most-accessed latency-sensitive data to edge devices and gateways on the fly.
Peter Burris
One more thing about intelligent edge: Encourage use of AI, ML and other technologies that can triage actions or provide automation.
Peter Burris
Move data to models at the edge.
jameskobielus
Yes, and perhaps also dynamically move selected data to mobile devices to ensure fast access with discontinous connectivity.
ClearSky Data
Distributed architectures help bring data to the edge, where it can be analyzed and interacted with in real time, as if it were local. #clearskyedge
Natasha Bishop
So all this sounds great...how do we actually make it happen? Are organizations already doing this successfully?
Marc Keating
Using flash selectively where necessary can speed up the workloads that require that level of performance, while keeping the costs reasonable
jameskobielus
@CTOAdvisor Yes, of course not every use case requires flash-grade data access latencies. Intelligent storage infrastructure matches user latency requirements with data movement/persistence.
jameskobielus
Right. Moving data to the edge should also involve moving "intelligence"--in other words, machine learning algorithms etc all--to execute on edge nodes. This supports the new generation of data-driven AI apps at the edge.
Keith Townsend - Light will overcome darkness
Physical storage is just one part of the intelligent edge. Data services at the edge is even more critical. It's the great normalizer of hybrid-IT.