From Raw Data to Real Insights: Synergizing Data Engineering and AI for HPC

Date: May 11-12, 2026 - 01.00 pm - 05.00 pm Online Workshop
Hosted by CSG Data Engineering & AI

All about our speakers and talks!

Talk: From Queries to Algorithms to Insights in One Optimizer

Data system architectures are converging: storage in lakehouses, execution in embedded engines, querying in converged systems. Yet the optimizer still draws a hard line between queries it can plan and algorithms it cannot. Nowhere is this gap more visible than in graph analytics, where pipelines routinely chain queries with algorithms like PageRank and shortest paths across multiple engines, with serialization dominating data-access time. I argue that the optimizer is the next frontier in this convergence, and show how, through our work on AvantGraph at TU/e, cost-based optimization can be extended from pattern matching through recursive queries to graph algorithms. I introduce the concept of an optimizer-visible fragment, connecting this to the broader trend of architectural convergence and its implications for HPC and AI workloads.

Talk: Efficient Use of LLMs for Data Preparation

Data preparation accounts for 80% of a data scientist’s time and remains one of the least enjoyable yet essential tasks in the workflow. While LLMs offer new opportunities for automating structured data preparation, challenges persist in efficiency, scalability, and adaptability. In this talk, we explore the efficient use of LLMs for data preparation, including generating transformation code for data wrangling tasks and fine-tuning small LLMs for entity matching. We highlight how LLMs can be leveraged not just for reasoning but as scalable, cost-effective automation tools in data-cleaning pipelines. Finally, we discuss future research opportunities in the area, paving the way for more adaptable and interpretable AI-driven data science workflows.

Talk: Agentic data systems and Multimodal Analytics

Over the past few years, retrieval-augmented generation (RAG) and LLM-based agents have rapidly evolved from simple document retrievers to complex architectures capable of orchestrating end-to-end data pipelines. This progress has opened the door to querying data in ways that blur the line between search, question answering, and database-like operations. In this talk, we cover some of the recent systems that incorporate optimization layers, dynamic retrieval, or reasoning strategies to query unstructured data. Yet, when moving from text-only corpora to multimodal datasets—where information may be stored as images, tables, or structured metadata—the challenges multiply: join-like reasoning across modalities, scaling over high-cardinality queries, and balancing precision with recall under context limits. To address these challenges, we present our vision for an analytical system over unstructured data. We highlight the open challenges for the design of such system and also in the often overlooked problem of benchmarking, presenting some of the most recent efforts on evaluating such systems on real-world pipelines data.

Panel discussion: Will AI eat HPC software? Potentials, Pitfalls, and the Future of Scientific Computing

As HPC systems continue to scale, so does the complexity of managing data, software, and workflows. At the same time, advances in AI, particularly in LLMs, are reshaping how software is written, optimized and maintained. This panel explores the question: Will AI fundamentally change, or even replace, how HPC software is developed and used?

Talk: SemPipes - Optimizable Semantic Data Operators for Tabular Machine Learning Pipelines

Machine learning on tabular data relies on complex data preparation pipelines that are costly to design and optimize. We introduce SemPipes, a declarative programming model that integrates LLM-powered semantic data operators into tabular ML pipelines. Semantic operators express data transformations in natural language, while execution is handled by a runtime system that synthesizes task-specific implementations based on data characteristics and pipeline context. Using LLM-based code synthesis guided by evolutionary search, SemPipes enables automatic optimization of data operations. Experiments across diverse tabular ML tasks show that SemPipes improves end-to-end predictive performance for both expert-designed and agent-generated pipelines, while reducing pipeline complexity.

Talk: Under the Hood of HPC: Analyzing Cluster Processes Through Data

HPC clusters power much of modern computational science, but what actually happens inside and how users interact with them at scale often stays hidden. In this talk, I'll show how data from schedulers like SLURM can be turned into structured event logs that make these inner workings visible. We'll look at how to extract such data in practice, why a single perspective (just jobs, or just nodes) rarely tells the full story, and how object-centric process mining helps tie the different pieces together. Along the way, we look at findings observed in real-life cluster data: From everyday job patterns to what the data looked like during an unplanned shutdown. Finally, we turn to some thoughts on where all of this could go, for operators making sense of their clusters, and for users getting more out of them.

Talk: AI-Assisted Data Analysis without Hallucinations: Giving LLMs the Right Tools

Large Language Models (LLMs) like ChatGPT can suggest analysis strategies and write fluent explanations. But when asked to compute a p-value or count events in a dataset, they confidently make up numbers. This makes naive "just ask the AI" approaches dangerous for any analysis that requires correctness. In this talk, I present an alternative: Strict tool chaining. Instead of generating code or computing results itself, the AI interacts with existing tools, just like we would. It decides what to compute, then delegates to verified tools (statistics or data science libraries, a process mining engine, visualization backends) that do the actual work. In a live demo, I show how a prototype AI copilot with pre-built, trusted tool calls produces correct, verifiable results where an unconstrained LLM would hallucinate. The talk closes with a discussion of when this works, where it breaks, and how the pattern generalizes beyond process mining.