HPC and machine learning convergence is changing how organizations think about data-intensive computing. High-performance computing (HPC), traditionally associated with modeling and simulation, and big data (BD), focused on ingesting and analyzing information from varied sources, are increasingly used alongside machine learning (ML). For industrial automation teams, the central idea is not simply to process more data. It is to connect data collection, analytics and computational models so that each can inform the others.
The convergence is driven by growing data volumes and more demanding applications. Data generation is no longer the only constraint: managing, curating and analyzing data can become the bottleneck. A connected computing environment could help organizations bring together sensor or instrument data, large-scale analytics and compute-intensive simulation. Realizing that potential, however, requires changes across infrastructure, software and operations.
Why combine HPC, big data and machine learning?
HPC systems have long supported large simulations, while big data systems have been designed to handle information from diverse sources. These approaches are now overlapping. ML can analyze data, help build predictive models and, in some cases, guide simulations or experiments. In turn, simulations can produce data that ML methods use to identify patterns or inform subsequent computational work.
This creates a more iterative workflow: data can inform a model, a model can help direct computation, and the resulting output can feed further analysis. The workshop report describes research teams combining experimental data with ML to steer simulations and experiments. It also notes that ML and deep learning workloads can require HPC-scale resources, while complex simulations may benefit from ML embedded in their workflows.
For industrial organizations, the relevance is the broader computing pattern rather than any one specific implementation. Data may be generated across distributed systems, analyzed at different scales and used to support predictive modeling or operational decisions. The source identifies predictive maintenance and supply chain operations as examples of applications where data and equipment condition can inform diagnostics and maintenance planning.
HPC and machine learning convergence depends on a connected architecture
The emerging vision is not a choice between edge, cloud and HPC. It is a combination of these environments, each contributing different capabilities. Computing near the data source can support intelligent data collection or triage. Cloud environments can support data ingestion and large-scale analytics. HPC can handle computationally intensive analysis, optimization and simulation.
Moving information between these layers is not automatic. A useful end-to-end workflow must account for where data is produced, how it is prepared and retained, and what computing resources are needed for each task. In distributed or real-time applications, processing closer to the data can matter; larger analyses may require more extensive compute resources. The report calls this “smart computing at the edge” and highlights the need for systems that can work across edge, cloud and HPC environments.
That connected architecture also has to accommodate different computing workloads. Simulation, data analytics and ML may share methods such as dense linear algebra, but they can place different demands on memory, storage, latency, bandwidth and networking. A system that accelerates one part of a workflow may still be constrained by data movement or another part of the infrastructure.
Infrastructure challenges: heterogeneity, memory and data movement
Future systems are expected to include a wider mix of processors, including GPUs and field-programmable gate arrays. Specialized hardware can support particular workloads, but heterogeneous systems also increase the need for software and tools that help teams understand performance and manage different devices. The report emphasizes that compute, memory, input and output, latency and bandwidth must be considered together.
Several infrastructure challenges stand out:
- Interconnects: Efficient communication within and between nodes is important for large-scale applications. Off-node operations can affect performance.
- Memory and storage: Both simulation and analytics can be memory-bound, while file systems designed for traditional HPC workloads may not suit newer ML applications.
- Power efficiency: The report identifies a need for further work on power efficiency in both processors and memory.
- Measurement and benchmarks: Common tools and end-to-end benchmarks are needed to compare workloads and identify bottlenecks across simulation, big data and ML.
These issues are closely linked. For example, a workflow that moves large datasets between storage and accelerators may depend as much on data access and network performance as on processor capability. Assessing a converged system therefore requires looking beyond peak compute capacity.
Software and operational readiness matter
HPC and big data environments have developed distinct system designs and software ecosystems. HPC has often relied on tightly coupled systems, while big data platforms have commonly used large collections of more loosely coupled nodes. These differences have shaped the tools and practices available to users. Bringing workloads together calls for software that can bridge environments, manage resources and make hardware easier to use.
The report identifies several areas for development: programming models and compilers, operating and runtime systems, common libraries, and system management tools. Large jobs also need support for lower launch latency, real-time monitoring and handling runtime failures. For users who are new to large-scale computing, accessible interfaces, training and technical support are important parts of the operating environment.
Data readiness is equally important. ML depends on datasets that are curated and tagged appropriately; simply having a large volume of data does not make it ready for analysis. Data management, access, retention and reproducibility also require attention. Publishing the data and software that support a result can help others evaluate and reproduce the work.
Build the workflow, not just the compute system
HPC and machine learning convergence is an ecosystem challenge as well as a hardware challenge. The practical implication for technical decision-makers is to evaluate the full workflow: data capture and preparation, movement and storage, analytics, simulation, deployment across computing environments, and the skills needed to operate the system.
Progress also depends on collaboration among domain specialists, data scientists and software engineers. The report describes this multidisciplinary approach as necessary because domain experts may not have experience with the integrated technologies, while specialist teams can help connect application needs to software and infrastructure choices.
For industrial automation, the direction is toward more coordinated use of edge, cloud and high-performance computing—not a single platform that replaces all others. Organizations considering data-intensive analytics or simulation should account for data quality, interoperability, performance bottlenecks, user support and reproducibility alongside compute capacity. Addressing these elements together can make it more practical to connect operational data with analysis and predictive models as computing needs evolve.