Geothermal Well Array AI Decision Support with LLMs

Geothermal Well Array AI Decision Support with LLMs

Introduction

Geothermal energy is having a moment—but the hard part isn’t just finding heat underground. It’s planning what to build: where to drill, how to arrange multiple wells into well arrays, and how to optimize systems like closed-loop coaxial wells so they’re both effective and fault-tolerant. That’s exactly where new research from the paper comes in, showing how large language models (LLMs) can support geothermal decision-making and even help with parts of geothermal modeling workflows.

In this new study (from February 2025 workshop proceedings), the authors focus on a practical problem with LLMs: reliability. LLMs can sound convincing while being wrong—often called hallucinations—and geothermal decisions can’t afford sloppy answers. So the research asks: can today’s LLMs act like expert assistants for geothermal data, recommendations, and modeling tasks, and how dependable are they?

To make that test possible, the authors use a clever workaround: they build new geothermal benchmark questions quickly using Google’s NotebookLM, and then evaluate multiple models—general-purpose LLMs like ChatGPT o1, Gemini Advanced, Claude 3.5 Sonnet, Grok 2, plus the geothermal-focused assistant AskGDR. They also run a case study on whether LLMs can help generate parallelized numerical code for a 3D coaxial-well heat equation model—something that matters a lot for scaling digital twins.

Why This Matters

This research is significant right now because geothermal projects are moving from “cool idea” to “real infrastructure,” and the complexity gap is widening fast. The moment you design a well array (multiple wells working together) or a closed-loop coaxial system (pipe-in-pipe heat exchange), you’re juggling geometry, heat transfer, operational tradeoffs, costs, and risk. The people doing that work need decision support that’s fast, organized, and traceable—not just “smart-sounding.”

A scenario where this can be applied today: imagine a design team evaluating a proposed 10×10 array (100 wells) of closed-loop coaxial wells. Before committing to drilling-scale decisions, teams typically rely on domain experts and repeated numerical runs. In this study, LLMs are explored as helpers for brainstorming and structuring tradeoffs—like well diameters vs. spacing—and for accelerating parts of the numerical workflow via auto-parallelization. Even if an LLM can’t be trusted as the sole decision-maker, it can still be useful as a co-pilot that speeds up iteration.

Finally, it builds on a growing line of AI work in geoscience: specialized models (like K2) and retrieval-backed assistants (like AskGDR) aim to reduce the mismatch between general language skills and domain correctness. But this paper adds a missing ingredient: benchmark freshness. As LLMs evolve quickly, older benchmarks can become outdated, so the authors focus on generating new quantitative evaluation sets rapidly—using NotebookLM to create unpublished-style question sets with citations.

Main Content Sections

1) What “Decision Support” Looks Like in Geothermal (Not Just Q&A)

Most people hear “LLM in science” and think: “Great, it answers questions.” But geothermal decision support is more like building a reliable workflow around uncertain inputs—data, assumptions, modeling choices, and operational constraints.

The paper frames two big use cases:

1. LLMs for geothermal data analysis and recommendations

   • Here, the challenge is not only generating text—it’s generating accurate interpretations from technical data.
   • Since LLMs can be wrong while remaining confident, the research emphasizes the need for measurable reliability.

2. LLMs for improving geothermal modeling

   • Instead of only answering questions, LLMs can help with the modeling side: code generation, parameter exploration, and scaling simulations.
   • The paper’s key modeling example is integration with digital twins / digital multiplets, where you want multiple model variants working together (e.g., different geometries, parallel computations, and scenario comparisons).

Think of this like moving from a calculator to a workshop tool:
- A calculator (like “just answer questions”) gives numbers.
- A workshop toolchain (like “digital twin workflow”) helps you build and iterate on the system, faster and more safely.

And crucially, the paper’s evaluation strategy is designed to test whether the models behave more like the calculator (reliable answers) or like a noisy brainstorming partner (creative but inconsistent).

2) Building New Geothermal Benchmarks Fast with NotebookLM

A core obstacle in LLM evaluation is that benchmarks get stale. When new models (or new versions) arrive, you need fresh tests, ideally with questions that aren’t copied from somewhere public. The paper tackles this by generating unpublished quantitative geothermal benchmarks quickly.

How the benchmark generation works

The authors upload 20 academic articles covering:
- general geothermal energy,
- geothermal closed-loop systems (including coaxial wells),
- geothermal well arrays.

Then they use NotebookLM to synthesize the material and generate a set of 90 multiple-choice questions, divided into microquizzes by category:
- general geothermal energy,
- well arrays,
- closed-loop systems,
- coaxial wells.

Each microquiz contains between 15 and 30 questions.

Why multiple-choice questions?

The paper explicitly chooses multiple-choice rather than free-form answers because subjective human grading can vary. Multiple-choice allows a more objective accuracy measurement (though it still has design challenges—like whether a model says “I don’t know”).

Reliability measurement and the AskGDR twist

One really practical detail: the geothermal assistant AskGDR often responds with “I do not know” instead of guessing. So the authors calculate accuracy two ways:
- Strict-choice accuracy: only count questions where AskGDR selects one of the provided options.
- Honesty-penalized accuracy: treat “I do not know” as wrong.

This matters because it highlights a real reliability philosophy difference:
- An assistant that refuses uncertain answers can be more trustworthy,
- but a naive scoring rule might make it look worse.

The paper’s overall point is that benchmark design has to reflect how you want the system to behave, especially in high-stakes domains like energy infrastructure. (This benchmark methodology is central—see the full details in the paper.)

3) Which LLMs Were Most Reliable? The Multiple-Choice Results

With the benchmark in place, the authors evaluate:
- ChatGPT o1,
- Gemini Advanced,
- Claude 3.5 Sonnet,
- Grok 2,
- AskGDR.

The headline reliability result

Across categories, the models that emerge as most reliable are Gemini, Claude, and Grok. The paper defines “most reliable” using a minimum-accuracy view:
- the highest minimum accuracy observed is 85%.

In other words: they didn’t just average well—they avoided large dips across question sets.

Where dispersion matters

The paper notes that while mean accuracy stays relatively consistent across models (except AskGDR depending on how you score its “I don’t know” responses), there are significant differences in:
- minimum accuracy (worst-case performance),
- maximum accuracy (best-case performance).

That’s a subtle but important takeaway for decision support. Teams usually need consistent competence—especially when questions get more specific (like coaxial design details or array geometry nuances).

Quick comparison of the evaluated models (as framed by the paper)

The “citations even when incorrect” point

One interesting observation: AskGDR frequently includes relevant source citations, even when the answer is wrong. That’s not perfect, but it provides a pathway for reliability improvements:
- If citations can be verified automatically (or reviewed efficiently),
- you can build a trust layer on top of the assistant’s reasoning.

4) LLMs as Code Co-Pilots: Auto-Parallelizing a 3D Coaxial-Well Model

Benchmarks are one thing. The paper goes further and tests LLMs in a more “engineering-real” setting: modifying code for parallel execution.

The modeling testbed

The authors use an in-house closed-loop coaxial well simulation as a target. The physics model is based on a diffusion-convection form of the heat equation, solved in 3D using a finite difference method on a uniform mesh. Velocities are treated as fixed, and convection is not modeled in rock formation or well subdomains.

The goal here isn’t to claim the physics is fully production-ready—it’s to test whether the LLM can:
- correctly identify where to apply threading,
- generate parallel, compilable, executable code.

What the LLMs were asked to do

The prompt (as described) was essentially:
- “Parallelize the Julia code below using CPU multi-threading.”

Then the authors compare the generated parallelization choices.

Results: correct threading placement vs. over-parallelization

The paper highlights two LLM behaviors:

• ChatGPT o1 and Claude 3.5 Sonnet:

  • correctly identify the right function/line to apply @threads,
  • produce parallel code that compiles and runs without errors.

• Gemini Advanced:

  • parallelizes all loops indiscriminately,
  • creating too many spatial partitions,
  • which increases parallel overhead and leads to inefficiency.

Here’s the analogy: parallel computing is like running a warehouse with too many managers.
- If you delegate the right tasks to the right teams, throughput improves.
- If you split everything into tiny jobs, the coordination overhead kills performance.

So the lesson isn’t “use threads everywhere.”
It’s “parallelize with structure.”

Why this matters for digital twins and multiplets

In digital twins, you typically run many scenarios:
- parameter sweeps,
- geometry variants,
- what-if operational changes.

If LLMs can reliably help generate efficient parallel code, the iteration cycle shortens—making digital multiplets more feasible in practice.

5) LLM-Assisted End-to-End Design: Where It Helps—and Where It Can Mislead

After the benchmarks and code experiment, the paper explores unstructured interviews with LLMs for geothermal array design and cost modeling. The setup includes a conceptual 10×10 array (100 wells) and assumes closed-loop coaxial wells.

What LLMs contributed in brainstorming

ChatGPT and Gemini were used for end-to-end brainstorming around well and array design. They suggested cost-saving strategies such as:
- modular assembly,
- just-in-time delivery,
- networked redundancy.

They also produced parameter tradeoff discussions—for example, ChatGPT o1 discussing tradeoffs tied to different diameters and array spacing.

The reliability limitation: missing assumptions and weak citation discipline

The paper emphasizes that LLM outputs can be hard to validate because they often:
- don’t clearly state underlying assumptions,
- don’t consistently cite the sources of numerical inputs,
- produce “new LCOE” style claims without showing where the data came from.

And this is a big deal in energy economics:
- If you can’t trace the basis of an LCOE calculation or the assumed parameters,
- the recommendation can’t be audited.

The authors argue that for LLM-assisted design to become a decision-support foundation—not just brainstorming—validation mechanisms must be built in.

Key Takeaways

• LLMs can be useful for geothermal decision support, but reliability varies by model and question type.
• The study created a fresh geothermal benchmark (90 multiple-choice questions) using NotebookLM fed with 20 academic papers, aiming to reduce issues like benchmark staleness and test contamination.
• In the benchmark, Gemini, Claude, and Grok showed the best reliability, with a highest minimum accuracy of 85%.
• AskGDR often responds “I do not know” and may include citations even when answers are incorrect—useful for traceability, but scoring depends on how you treat refusal.
• In a code case study, ChatGPT o1 and Claude 3.5 Sonnet correctly placed @threads to produce parallel, compilable code, while Gemini Advanced over-parallelized by threading too many loops, causing inefficiency.
• For end-to-end design brainstorming (e.g., a conceptual 10×10 array of 100 closed-loop coaxial wells), LLMs can propose cost-saving strategies—but they may omit assumptions and sources, so validation is essential before acting on outputs.

If you want, I can also turn this into a more “builder-friendly” checklist (e.g., how to design LLM benchmarks for geothermal, and what reliability controls to add for decision support).