Using AI in storage management

Jason Nadeau from Pure Storage (left) and Man Law from EMC (right)

Using AI in storage management Data volumes are exploding in the digital era, driven by cloud computing, mobility and increasingly by machines such as sensors, internet of things (IoT) and connected devices.

In the Data Age 2025 study, IDC forecasts that by 2025 the global datasphere will grow to 163 zetabytes. That is ten times the 16.1 zetabytes of data generated in 2016. The global datasphere is a measure of all new data that is captured, created and replicated around the globe.

The explosive growth rate of data has brought new challenges—the increased demand for high-capacity storage, faster access to data and the ability to manage storage.

Continuous demand for all-flash

With the continuous improvements in capacity and response times, flash storage is growing rapidly. In the same study by IDC, by 2025 flash is estimated to account for over 40% of storage capacity shipments.

The demand for all flash array systems remains robust not only in data centers but also in companies across different verticals to support their digital initiatives.

“All-flash and scale-out storage solutions—capable of delivering both the high performance and rich data services needed for today’s demanding applications—are critical elements for enterprises that want to achieve IT transformation,” said Mark Peters, practice director & senior analyst at Enterprise Strategy Group.

“Data is the lifeblood of the digital generation,” said Michael Alp, regional VP at Pure Storage Asia Pacific and Japan. “Modern digital businesses require an all-flash data platform that enables businesses to extract new insights from data and to do so in real time.”

Apart from the need to store more data and process data faster, enterprises are also concerned about storage management, particularly storage failure and capacity prediction. Storage failures can have a big impact on productivity. Artificial intelligence (AI) can help manage data, storage and predict storage failures.

Failure & capacity prediction

“Storage management and support has evolved to be machine learning and AI-based,” said Jason Nadeau (pictured, left), senior director of products and solutions marketing at Pure Storage. “Our AI engine Pure1 Meta enables effortless management, analytics and support.”

As part of the company’s data platform, Meta is touted to be a real time global sensor network that has thousands of flash arrays deployed globally. It collects and analyzes over one trillion array telemetry data points per day, accumulating to a data lake of more than seven petabytes of data. It delivers predictive intelligence to manage and analyze its arrays, and alert customers if analomies are found.

According to the company, Meta works like antivirus scanning. It scans the logs from all incoming arrays against a library of fingerprints (early symptons of a known issue). If there is a match, the arrays may be susceptible to the issue. Pure Storage’s support will be automatically notified for remediation before the issue impacts customer environments.

Apart from incident analytics, Meta can also predict workloads and capacity such as predicting if or when an array will run out of capacity. Customers can view the capacity, performance of all their arrays, alerts and support cases through a dashboard.

Besides Meta, Pure Storage’s data platform also includes all-flash array hardware products and software-defined data services.

Apart from Pure Storage, EMC also uses analytics to manage the health, performance and availability of its storage systems.

“Digital transformation and new emerging technologies like IoT have led to the rising demand for flash storage in Hong Kong,” said Man Law (pictured, right), head of systems engineering for Hong Kong and Macau at EMC. “We provide CloudIQ that utilizes AI and cloud to support storage management.”

As a cloud-based SaaS analytics platform, CloudIQ helps storage administrators to monitor and be aware of any potential vulnerabilities with their storage systems. It is available for EMC’s Unity and SC series storage systems for free.

Law said data from EMC’s storage systems globally like configuration, performance and capacity are sent to CloudIQ. It uses near real time intelligence, proactive monitoring and predictive analytics to provide customers with a health score of their storage arrays based on the collected data.

Recommended remediation information is provided to help customers react and resolve issues before they occur. For instance, CloudIQ provides information about whether data compression performance of a storage system may be affected with a non-compliant operating environment, or predict when a storage pool will run out of space.

“Customers can know in detail what is happening in their storage systems via a web-based user console,” said Law.