NetApp AI Data Engine Structures Unstructured Data and Eliminates 20x Bloat

Original source: NetApp
This article is an editorial summary and interpretation of that content. The ideas belong to the original authors; the selection and writing are by Streamed.News.

This video from NetApp covered a lot of ground. 6 segments stood out as worth your time. Everything below links directly to the timestamp in the original video.

Discover how NetApp's AI Data Engine streamlines the complex process of preparing unstructured data for AI, while simultaneously solving the critical problem of data bloat caused by vectorization.

NetApp AI Data Engine Structures Unstructured Data and Eliminates 20x Bloat

The NetApp AI Data Engine offers out-of-the-box vectorization and embedding capabilities to help data scientists prepare unstructured enterprise data for AI applications. This process converts raw, unstructured data into numerical vectors, known as embeddings, which are essential for training AI models. A common challenge with vectorization is the significant data bloat it causes, often increasing storage requirements by 10 to 20 times. NetApp's AI Data Engine addresses this by incorporating built-in technology specifically designed to eliminate this storage overhead, delivering highly efficient, AI-ready data without the massive capacity increase.

"Not only do customers get AI ready data, they get a highly efficient form of AI ready data."

▶ Watch this segment — 21:23

NVIDIA and NetApp Announce AFX and AI Data Engine, Reinventing Storage for AI Era

Jensen Huang, CEO of NVIDIA, announced a "complete reinvention of data processing and storage" for the AI era through a joint initiative with NetApp, introducing AFX and the AI Data Engine. Huang highlighted the long-standing partnership, noting that NVIDIA has used NetApp's multi-cloud and hybrid-cloud solutions since the DJX1 supercomputer, which was the world's first AI supercomputer. The collaboration leverages NetApp's capability to manage diverse data types, particularly unstructured data—which constitutes over 90% of NVIDIA's own data. Huang emphasized that the new offerings will coherently manage data across cloud and hybrid environments, positioning the joint technologies to help clients extract knowledge from vast datasets and capitalize on the "biggest industrial revolution the world's ever seen."

"AFX and the AI data engine is a complete reinvention of data processing as we know it... today we're reinventing storage for the very first time."

▶ Watch this segment — 8:25

NetApp AFX Boosts ONTAP Performance Fourfold, Adds 99% Accurate Ransomware Detection

NetApp's AFX enhances the ONTAP data management platform, making it more adaptable for demanding AI workloads. The new integration delivers four times the performance for the same storage capacity, enabling terabyte-scale throughput for data-intensive operations. Beyond performance gains, AFX significantly strengthens data security with real-time ransomware detection boasting 99% accuracy. These advancements establish AFX as a critical component within the NetApp data platform, addressing both the speed and security requirements of modern AI deployments.

"We're seeing it in four times the performance for the same amount of capacity up to a terabyte-scale throughput and of course real-time ransomware detection with 99% accuracy."

▶ Watch this segment — 17:01

NetApp AI Data Engine Secures LLM Deployments, Prevents Data Leakage and Compliance Breaches

NetApp's AI Data Engine integrates robust security features to mitigate privacy and compliance risks associated with AI applications and large language models (LLMs). During the creation of AI-ready data, the engine scans customer data estates to identify, extract, and categorize sensitive information. By building "guardrails" around this sensitive data, the AI Data Engine prevents data leakage and ensures compliance with critical regulations such as GDPR and HIPAA. This allows organizations to confidently deploy AI agents and LLMs, safeguarding against potential legal fines and reputational damage from privacy breaches.

"Essentially we try to close all the privacy and compliance risks so that customers can deploy their AI agents and LLMs with confidence."

▶ Watch this segment — 22:59

NetApp Unveils AFX: Ultra-Scalable, Secure Storage for AI Enterprises

NetApp has introduced AFX, a new high-performance, ultra-scalable disaggregated storage system engineered specifically for AI-powered enterprises. AFX features a massive global namespace, providing extensive data visibility and delivering the performance benefits of a parallel file system without the associated complexity. The system offers granular linear scalability for both compute and capacity, simplifying optimization, and is powered by NetApp's enterprise-grade ONTAP intelligent data management. This integration ensures seamless data mobility across on-premises and cloud environments, further bolstered by integrated real-time ransomware detection, positioning AFX as a highly secure storage solution.

"AFX delivers a massive global name space, unmatched data visibility and parallel file system performance without all the complexity."

▶ Watch this segment — 4:11

NetApp AFX Enables Adaptive Scaling for AI Production, Simplifies Parallel File System Benefits

NetApp AFX, featuring a disaggregated architecture, is designed to support the transition of AI projects from development to production by providing adaptable and cost-effective scaling of either performance or capacity. This approach helps break down data silos often encountered in complex AI environments. Unlike traditional parallel file systems, which can be difficult to manage and often carry security risks due to client software requirements, AFX delivers comparable scalability and performance with the inherent availability, data security, and robustness of NetApp's ONTAP enterprise product. This eliminates the management complexities and security concerns of older solutions.

"AFX is not a parallel file system. It delivers the scalability and performance of a parallel file system, but with all of the security, availability, and robustness of an enterprise product."

▶ Watch this segment — 15:28

Also mentioned in this video

• Generative AI has kicked off an AI adoption race, disrupting industries and… (0:56)
• Entering a new age of enterprise AI requires unlocking data's full value… (1:35)
• Enterprise teams need efficiency, reliability, protection, integration, and… (2:27)
• To accelerate AI outcomes, data infrastructure must turbocharge the data… (5:03)
• NetApp AI Data Engine allows data teams to quickly discover and curate datasets… (6:10)
• Is trust in data usage and infrastructure readiness, which NetApp addresses… (6:47)
• The excitement for NetApp's new AI data infrastructure among partners and… (11:01)
• Their CG films are entirely digital data, requiring enterprise-grade solutions… (11:24)
• Enterprise AI adoption is driven by competition, requiring companies to move AI… (13:16)
• Companies are moving slowly with AI adoption due to the high cost of mistakes… (14:07)
• NetApp AFX and AI Data Engine change the game for enterprise AI by providing… (14:31)
• NetApp has built a metadata engine on top of ONTAP, allowing it to create a… (17:38)
• Brad Adams of Worldwide Technology (WWT) explains that while enterprises… (18:02)
• Customers seeking an AI advantage need reassurance about their investments, and… (19:13)
• Data discoverability, understanding, and access to well-governed, high-quality… (20:34)
• The AI ecosystem is critical for enterprise AI, comprising three elements (24:12)
• NetApp's unique long-standing integration strategy with key public clouds and… (25:29)

Summarised from NetApp · 30:10. All credit belongs to the original creators. Streamed.News summarises publicly available video content.