Open Problems & Future Directions

67.1 Overview & Motivation

The preceding chapters of this survey have documented a remarkable acceleration in LLM-powered evolutionary systems between 2024 and 2026. From FunSearch's demonstration that large language models can serve as mutation operators for program synthesis, through AlphaEvolve's industrial-scale deployments, to the proliferation of open-source platforms such as OpenEvolve, GEPA, and LLM4AD, the field has moved from proof-of-concept to production use in under two years. Yet this velocity has outpaced the field's theoretical, methodological, and safety infrastructure. Many systems work well in practice without anyone being able to explain precisely why they work, under what conditions they will fail, or how to compare them fairly.

This chapter synthesizes the open problems that have surfaced throughout the survey — not as a wish list, but as a structured analysis of the gaps that most urgently constrain scientific progress and responsible deployment. We organize these into seven domains: theoretical foundations, scalability, safety, benchmarking, methodology, societal implications, and a concrete research roadmap. Each section identifies the specific gap, explains why it matters, and suggests tractable research directions.

67.2 Theoretical Foundations

The most significant intellectual deficit in LLM-powered evolution is the near-complete absence of formal theory. Classical evolutionary computation has developed convergence proofs, schema theorems, no-free-lunch results, and runtime analysis over decades. LLM-based systems inherit almost none of this machinery, because the mutation operator — the language model itself — violates the assumptions underpinning most existing theory.

67.2.1 The Mutation Distribution Problem

In classical evolutionary algorithms, mutation operators are typically characterized by a well-defined probability distribution over the search space. For bitstring GAs, single-bit-flip mutation is uniform over Hamming neighbors. For Gaussian mutation in evolution strategies, the perturbation distribution is parameterized by a covariance matrix. These distributions enable formal analysis of exploration–exploitation tradeoffs, drift behavior, and convergence rates.

When an LLM serves as the mutation operator, the induced distribution over program variants is:

where $x$ is the parent program, $x'$ is the mutated offspring, $c$ is the prompt context (including task description, fitness feedback, and other candidates), and $\theta$ represents the frozen model parameters. This distribution is:

• Implicit — it cannot be written in closed form or efficiently sampled from without autoregressive generation.
• Context-dependent — the same parent $x$ produces different mutation distributions under different prompts $c$, making the operator non-stationary even with fixed $\theta$.
• High-dimensional — the support is the space of all syntactically valid programs, which has no natural metric topology.
• Opaque — the internal representations driving $p_{\text{LLM}}$ are not interpretable, so we cannot characterize locality, bias, or coverage analytically.

Open Problem 1. Develop a formal characterization of the mutation distribution induced by LLM-based operators that is sufficient to prove convergence or runtime bounds for at least one non-trivial problem class.

One promising direction is to treat $p_{\text{mut}}$ as a kernel in a Markov chain over program space and analyze mixing times empirically. Another is to study the effective neighborhood of an LLM mutation — the set of programs reachable with probability above some threshold $\epsilon$ — and relate its structure to fitness-landscape properties.

67.2.2 Fitness Landscape Theory for Program Spaces

Classical fitness landscape analysis relies on a distance metric (typically Hamming or Euclidean) to define concepts like ruggedness, neutrality, and basins of attraction. For programs, no single metric captures semantic similarity. Edit distance on source code is syntactic and poorly correlated with behavioral difference. Execution-trace similarity is expensive to compute and task-dependent.

The autocorrelation function $\rho(f, d)$, which measures how fitness correlation decays with distance $d$ in the search space, is a standard tool for landscape analysis. Here $f$ is the fitness function, $x$ and $x'$ are solutions at distance $k$ under metric $d$, and $\text{Cov}$ and $\text{Var}$ denote covariance and variance over the solution space. For LLM-powered evolution, computing $\rho$ requires both a meaningful distance metric and the ability to sample uniformly at fixed distances — neither of which is straightforward in program space.

Open Problem 2. Define a distance metric over program space that (a) is computationally tractable, (b) correlates with behavioral similarity, and (c) enables meaningful fitness landscape analysis for LLM-guided search.

Embedding-based distances (using code-embedding models) are a candidate, but their relationship to fitness-landscape structure has not been studied systematically. The interaction between the LLM's implicit code representation and the fitness landscape's structure is a fundamental open question: does the LLM's training distribution create an implicit smoothing of the landscape, and if so, can we characterize when this smoothing helps versus hurts?

67.2.3 Convergence and Runtime Analysis

No LLM-powered evolutionary system surveyed in this book has a formal convergence guarantee. The closest analogy in classical EC is the $(1+1)$-EA on simple functions like OneMax, where expected runtime is $\Theta(n \log n)$. Extending such results to LLM-based mutation requires bounding the probability that the LLM produces an improving solution, which in turn requires understanding the relationship between prompt context and output quality.

A tractable starting point might be to analyze a simplified model: a $(1+1)$-LLM-EA on a restricted problem class (e.g., optimizing a single numerical parameter in a program template), where the LLM's mutation distribution can be approximated by a parametric family. Even this restricted case would represent significant theoretical progress.

Open Problem 3. Prove a non-trivial runtime bound for any LLM-powered evolutionary algorithm on any well-defined problem class, even under simplifying assumptions about the LLM's behavior.

67.3 Scalability Challenges

LLM-powered evolutionary systems face scalability constraints along three axes: computational cost, population management, and multi-objective scaling. These are not merely engineering problems — they have deep algorithmic implications.

67.3.1 Cost and Budget Efficiency

The dominant cost in LLM-powered evolution is inference: each mutation requires one or more LLM calls, and each evaluation may require sandbox execution. For a population of size $N$ over $G$ generations with $M$ LLM calls per mutation, the total inference cost scales as:

where $\bar{c}_{\text{call}}$ is the average cost per LLM inference call (a function of input/output token counts and per-token pricing), and $\bar{c}_{\text{eval}}$ is the average cost per candidate evaluation (sandbox execution time, compute resources). For frontier models, $\bar{c}_{\text{call}}$ can range from $0.01 to $0.50+ per call depending on context length and model tier. A run with $G = 100$, $N = 50$, $M = 2$ requires 10,000 LLM calls — potentially $100–$5,000 in API costs alone.

This cost structure creates a fundamental tension: the evolutionary paradigm's strength is population-based exploration, but LLM inference costs penalize large populations far more than classical mutation operators do. Systems surveyed in this book have adopted various mitigation strategies — hierarchical model selection (using cheaper models for initial mutations and expensive models for refinement), evaluation cascades (cheap static checks before expensive execution), and caching (deduplicating semantically equivalent candidates). However, none of these strategies has been analyzed for optimality.

Open Problem 4. Develop principled budget-allocation strategies for LLM-powered evolution that optimize the tradeoff between population diversity, mutation quality, and evaluation thoroughness under a fixed monetary or compute budget.

# Pseudocode — illustrative budget allocation framework
# No public implementation available

def adaptive_budget_allocation(
    total_budget: float,
    generation: int,
    population_fitness: list[float],
    model_costs: dict[str, float],  # model_id -> cost_per_call
    model_quality: dict[str, float],  # model_id -> estimated mutation quality
) -> dict[str, int]:
    """
    Allocate LLM calls across models for the next generation.
    
    Key insight: early generations benefit from cheap, diverse mutations;
    later generations benefit from expensive, high-quality refinements.
    This is analogous to cooling schedules in simulated annealing,
    but the 'temperature' here controls model tier, not acceptance.
    """
    fitness_variance = variance(population_fitness)
    stagnation_signal = fitness_variance < STAGNATION_THRESHOLD
    
    # Compute value-per-dollar for each model
    # quality_estimate could come from a bandit over recent success rates
    value_per_dollar = {
        m: model_quality[m] / model_costs[m]
        for m in model_costs
    }
    
    if stagnation_signal:
        # Shift budget toward expensive models for deeper reasoning
        allocation = allocate_to_top_k_models(
            value_per_dollar, total_budget, k=2, weight_expensive=True
        )
    else:
        # Spread budget across cheap models for diversity
        allocation = allocate_proportional_to_value(
            value_per_dollar, total_budget
        )
    
    return allocation  # {model_id: num_calls}

67.3.2 Population Scale and Distributed Evolution

Island models — where subpopulations evolve semi-independently with periodic migration — are the primary mechanism for scaling LLM-powered evolution across multiple compute nodes. Systems like OpenEvolve and GEPA implement island topologies, but several fundamental questions remain open:

• Migration policy: When should candidates migrate between islands, and which candidates should migrate? Classical island-model theory provides results for fixed topologies and simple fitness functions, but the interaction between migration and LLM-based mutation is unstudied.
• Heterogeneous islands: If different islands use different LLM providers or prompt strategies, how should migration account for the distributional shift in mutation operators?
• Fault tolerance: LLM API calls fail stochastically (rate limits, timeouts, model deprecation). How should the evolutionary process adapt to partial island failures without losing population diversity?

Open Problem 5. Develop migration and topology-adaptation strategies for island-model LLM evolution that provably maintain population diversity under heterogeneous mutation operators and stochastic failures.

67.3.3 Multi-Objective and Many-Objective Scaling

Real-world program optimization is inherently multi-objective: correctness, runtime performance, memory usage, code readability, robustness, and maintainability all matter. Systems like GEPA support Pareto-based multi-objective optimization, but the interaction between many objectives and LLM-based mutation creates challenges that go beyond classical MOEA theory.

The LLM's prompt must communicate multiple objectives and their relative priorities. As the number of objectives $k$ grows, the prompt becomes longer and more complex, potentially degrading the LLM's ability to produce coherent mutations. Moreover, the Pareto front in $k$-dimensional objective space grows exponentially, making archive management and selection pressure increasingly difficult.

where $|PF_k|$ is the expected number of non-dominated points when $k$ objectives are evaluated over $N$ candidates. As $k$ grows, almost every candidate becomes non-dominated, collapsing selection pressure. This is a known problem in classical MOEAs, but LLM-powered systems add the dimension that the mutation operator itself must be steered toward under-explored regions of objective space via prompt engineering — a coupling between representation and search that has no classical analogue.

67.4 Safety & Alignment Concerns

LLM-powered evolutionary systems generate and execute code autonomously. This creates safety challenges that combine the risks of autonomous code generation with the open-ended nature of evolutionary search.

67.4.1 Sandbox Adequacy

Every system surveyed in this book employs some form of execution sandboxing — restricted subprocesses, containers, or resource-limited environments. However, as noted in our analysis of individual systems, the security guarantees vary widely and are often overstated. Process-level isolation (e.g., Python's subprocess with resource limits) does not constitute a security sandbox; it prevents accidental resource exhaustion but not deliberate escape.

The threat model for LLM-evolved code is distinct from both traditional software testing and adversarial ML. The generated code is not written by a human attacker, but it is also not constrained by human assumptions about "reasonable" behavior. Evolutionary pressure selects for fitness, and if the fitness function has exploitable gaps, evolution will find them — this is precisely the specification gaming problem studied in the AI safety literature, but applied to program synthesis.

Open Problem 6. Define a formal threat model for LLM-evolved code execution and develop sandbox specifications that provide provable containment guarantees against specification gaming, resource exhaustion, and information exfiltration.

67.4.2 Fitness Function Integrity

Goodhart's Law — "when a measure becomes a target, it ceases to be a good measure" — is an existential risk for autonomous code evolution. If the fitness function is even slightly misaligned with the true objective, evolutionary search will exploit the misalignment. Unlike human programmers, who self-correct when they notice their code is "gaming" a metric, evolutionary systems have no such meta-awareness.

Concrete examples of fitness-function exploitation observed in the broader program-synthesis literature include:

• Programs that detect test-case patterns and hardcode outputs rather than computing them.
• Sorting algorithms that modify the comparison function rather than the array.
• Optimization heuristics that exploit floating-point edge cases to achieve artificially high scores.

The systems surveyed in this book use various mitigations — held-out test sets, multi-stage evaluation cascades, static analysis checks — but none provides formal guarantees against specification gaming. This is particularly concerning as these systems are applied to increasingly safety-critical domains.

67.4.3 Recursive Self-Improvement and Containment

A more speculative but potentially consequential concern arises when LLM-powered evolution is applied to improving the evolutionary system itself — its prompts, selection strategies, or evaluation criteria. Several systems already implement prompt co-evolution, where the prompts used to guide mutation are themselves subject to evolutionary optimization. This creates a feedback loop:

where $\theta_{t+1}^{\text{prompt}}$ is the updated prompt population at generation $t+1$, $c_i$ are candidate prompts, $\text{Mutate}(x, c_i, \theta^{\text{LLM}})$ is the offspring produced by mutating parent $x$ using prompt $c_i$ with LLM parameters $\theta^{\text{LLM}}$, and $\text{Select}$ retains prompts that led to fitness-improving mutations. If extended to evolving evaluation criteria or system configuration, this becomes a form of recursive self-improvement — a topic of significant concern in the AI safety literature.

Open Problem 7. Develop formal containment frameworks for self-modifying evolutionary systems that bound the rate and scope of recursive improvement while preserving the system's ability to optimize effectively.

67.5 Benchmark Gaps

The benchmarking landscape for LLM-powered evolution is fragmented, inconsistent, and often methodologically unsound. This section catalogs the most pressing gaps.

67.5.1 The Reproducibility Crisis

Reproducing results from LLM-powered evolutionary systems is fundamentally harder than reproducing classical algorithm benchmarks, for reasons that go beyond the usual concerns about random seeds and hardware variation:

Reproducibility Factor	Classical EA	LLM-Powered EA
Mutation operator	Deterministic given seed	Stochastic, model-version-dependent
Model availability	N/A	Models deprecated, APIs changed, weights not released
Cost to replicate	CPU-minutes	$10–$10,000+ in API costs
Evaluation determinism	Usually deterministic	May depend on LLM-based judges or stochastic execution
Prompt sensitivity	N/A	Minor prompt changes can significantly alter results
Version pinning	Algorithm is the code	Model behind API may change without notice

The core issue is that the LLM is a black-box, externally hosted, non-deterministic, non-versioned component. When a system reports results using "GPT-4" or "Claude 3.5 Sonnet," the exact model weights behind that API endpoint may differ from month to month. This makes longitudinal comparison effectively impossible unless the research community adopts explicit version-pinning protocols.

Open Problem 8. Establish community standards for LLM-EA reproducibility, including minimum reporting requirements for model versions, API timestamps, prompt templates, evaluation protocols, random seeds, trial counts, and cost breakdowns.

67.5.2 Fairness in Cross-System Comparison

As highlighted repeatedly throughout this survey (particularly in our analyses of LLM4AD's benchmark platform and cross-system comparisons), comparing LLM-powered evolutionary systems is methodologically treacherous. Systems differ in:

• Budget type: Some report LLM calls, others report evaluations, generations, wall-clock time, or dollar cost. These are not interconvertible without detailed per-run metadata.
• Model tier: A system using GPT-4 at $30/million tokens operates in a fundamentally different regime than one using a local 7B model.
• Evaluation fidelity: Some systems execute candidates in full sandboxes; others use proxy evaluations or LLM-based scoring. Higher fidelity costs more but produces more reliable fitness signals.
• Seed programs: The starting program dramatically affects convergence. Systems that begin with a well-optimized seed have an unfair advantage over those starting from scratch.

No existing benchmark suite controls for all of these variables simultaneously. The field urgently needs what the machine-learning community has developed for supervised learning (e.g., standardized train/test splits, fixed compute budgets, leaderboards with controlled evaluation) but adapted for the unique challenges of evolutionary program synthesis.

67.5.3 Domain Coverage

The benchmark tasks used across the surveyed systems cluster heavily in a few domains: combinatorial optimization (bin packing, TSP, vehicle routing), mathematical optimization (circle packing, function discovery), and algorithm design (sorting, scheduling heuristics). Significant application domains remain under-explored:

Domain	Current Coverage	Gap Description
Scientific computing	Sparse	PDE solvers, numerical methods, simulation kernels
Systems programming	Minimal	Memory allocators, schedulers, network protocols
Data processing	Low	ETL pipelines, query optimization, data cleaning
ML pipelines	Emerging	Feature engineering, architecture search, loss functions
Security	Very low	Fuzzing strategies, vulnerability detection, patch generation
Robotics control	Minimal	Motion planning, control policies, sensor fusion

Open Problem 9. Develop standardized benchmark suites for LLM-powered evolution that cover diverse domains, control for budget and model tier, include multiple difficulty levels, and provide reference implementations with known-optimal or best-known solutions.

67.6 Methodological Open Questions

67.6.1 Prompt Engineering as Algorithm Design

The prompt template in an LLM-powered evolutionary system is not merely an interface detail — it is a core algorithmic component that determines the mutation distribution, the effective neighborhood structure, and the balance between exploration and exploitation. Yet prompt design remains largely empirical, guided by intuition and trial-and-error rather than principled methodology.

# Pseudocode — no public implementation available
# Illustrating the prompt-as-algorithm-component concept

class MutationPromptSpace:
    """
    A prompt template defines an implicit mutation operator.
    Different prompts induce different search behaviors,
    analogous to different crossover/mutation operators in classical EAs.
    
    Open question: Can we formally characterize the mapping
    from prompt features to search behavior?
    """
    
    # Each dimension represents a prompt design choice
    # that affects the mutation distribution
    dimensions = {
        "context_window": [
            "parent_only",       # Show only the parent program
            "parent_and_best",   # Show parent + current best
            "parent_and_diverse",# Show parent + diverse archive sample
            "full_history",      # Show recent evolution trajectory
        ],
        "instruction_style": [
            "direct",            # "Improve this function"
            "analytical",        # "Analyze weaknesses, then improve"
            "comparative",       # "Compare with best, then improve"
            "creative",          # "Find an unconventional approach"
        ],
        "constraint_encoding": [
            "implicit",          # Constraints embedded in examples
            "explicit_rules",    # Constraints stated as rules
            "test_cases",        # Constraints as input/output pairs
            "formal_spec",       # Formal specification language
        ],
        "feedback_granularity": [
            "score_only",        # Just the fitness value
            "rank_in_population",# Relative ranking
            "detailed_metrics",  # Multiple sub-scores
            "execution_trace",   # Full execution trace
        ],
    }
    # Total design space: 4^4 = 256 prompt configurations
    # Each maps to a different implicit mutation operator

The key insight is that prompt design choices are algorithm design choices in disguise. Choosing to include the top-$k$ candidates in the prompt context is analogous to choosing a selection pressure; choosing between "improve" and "find a completely different approach" is analogous to tuning mutation step size. Making this analogy precise — and developing theory for how prompt features map to search dynamics — is a major open problem.

Open Problem 10. Develop a formal framework for understanding prompt templates as parameterized mutation operators, with predictable effects on exploration–exploitation balance, semantic locality, and convergence behavior.

67.6.2 Credit Assignment in Co-Evolutionary Systems

Modern LLM-powered evolutionary systems involve multiple co-evolving components: the candidate population, the prompt population, the model selection policy (via bandits), and sometimes the evaluation criteria themselves. When a successful mutation occurs, attributing credit to the right component is a combinatorial problem:

where $\Delta f$ is the fitness improvement, and each $\delta$ term represents the contribution of a different system component to the improvement. Here, $\delta_{\text{model}}$ captures the LLM's contribution, $\delta_{\text{prompt}}$ captures the prompt template's contribution, $\delta_{\text{parent}}$ captures the quality of the parent selected for mutation, $\delta_{\text{context}}$ captures the effect of the surrounding population context provided in the prompt, and $\epsilon$ is irreducible stochastic noise from LLM sampling. This decomposition is not identifiable from observational data alone without controlled experiments — yet the bandit algorithms used for model selection and prompt adaptation implicitly assume that credit can be assigned to individual components.

Open Problem 11. Develop credit-assignment methods for LLM-powered co-evolutionary systems that correctly handle the confounding between model, prompt, parent, and context contributions.

67.6.3 Knowledge Transfer and Continual Learning

Current LLM-powered evolutionary systems treat each run as independent. Knowledge gained during one evolutionary run — which mutation strategies worked, which code patterns were productive, which regions of the search space are barren — is discarded when the run ends. Some systems maintain "learning logs" or "skills databases," but these are heuristic and local.

The opportunity is to develop methods for transferring evolutionary knowledge across runs, tasks, and even domains. This connects to the meta-learning literature but adds the unique dimension of evolving code: can patterns discovered in evolving sorting algorithms transfer to evolving scheduling heuristics? Can a skill library trained on combinatorial optimization accelerate scientific computing tasks?

Open Problem 12. Develop principled methods for cross-task and cross-domain knowledge transfer in LLM-powered evolution, with theoretical or empirical guarantees on when transfer helps versus hurts.

67.7 Broader Implications

67.7.1 Toward Self-Improving AI Systems

LLM-powered evolutionary systems represent one of the most concrete instantiations of AI systems that improve AI systems. AlphaEvolve's reported application to improving components of Google's own infrastructure — including hardware design verification and compiler optimization — demonstrates that the feedback loop from AI-generated code to AI system performance is already closing.

This raises questions that extend beyond computer science into philosophy and governance:

• Pace of improvement: If LLM-powered evolution can improve the LLMs themselves (through better training data preprocessing, architecture tweaks, or optimization heuristics), what bounds — if any — exist on the rate of recursive improvement?
• Predictability: Classical software has the property that its behavior can (in principle) be understood by reading its source code. Evolved programs may be correct and efficient but opaque — their logic may resist human comprehension. How do we maintain oversight of systems built from evolved components?
• Concentration of capability: The cost structure of LLM-powered evolution favors organizations with access to frontier models, large compute budgets, and proprietary evaluation infrastructure. This creates a risk of capability concentration that open-source platforms partially mitigate but do not eliminate.

67.7.2 The Role of Human Oversight

A recurring theme across the systems surveyed in this book is the tension between automation and oversight. Fully autonomous evolution is more efficient but harder to control; human-in-the-loop evolution is safer but slower and limited by human attention. The optimal balance depends on the domain's risk profile, the quality of the fitness function, and the maturity of the sandbox infrastructure.

We propose a classification of oversight levels for LLM-powered evolutionary systems, drawing on the autonomy levels defined in the broader AI governance literature:

Level	Name	Human Role	Appropriate When
L0	Manual	Human writes mutations; system only evaluates	Safety-critical, novel domains
L1	Suggested	System proposes mutations; human approves each	High-stakes with well-defined specs
L2	Supervised	System evolves autonomously; human reviews periodically	Well-understood domains, good fitness functions
L3	Autonomous	System evolves and deploys; human monitors dashboards	Low-risk, mature evaluation infrastructure
L4	Self-governing	System evolves its own evaluation criteria and scope	Research exploration only; no production use

Most current systems operate at L2–L3. Moving to L4 — which some systems' architectures already support in principle through prompt co-evolution and adaptive evaluation — requires safety infrastructure that does not yet exist.

67.8 Research Roadmap

Based on the analysis in this chapter, we propose a three-horizon research roadmap for the field.

67.8.1 Horizon 1: Empirical Foundations (2026–2027)

The most impactful near-term work is empirical, not theoretical. The field needs shared infrastructure before it can support rigorous science:

1. Benchmark standardization. A community-maintained benchmark suite with fixed evaluation budgets (measured in both LLM calls and wall-clock seconds), reference seed programs, and canonical fitness functions. This should cover at least four domains: combinatorial optimization, numerical optimization, algorithm design, and code repair.
2. Reproducibility reporting standards. A minimum reporting template for papers: model identifiers with version/date, prompt templates (full text), number of trials with seeds, cost breakdown, and evaluation protocol. Journals and conferences should require this as supplementary material.
3. Empirical mutation characterization. Systematic studies of what different LLMs produce when used as mutation operators: diversity of outputs, syntactic validity rates, semantic novelty, and sensitivity to prompt variation. This empirical groundwork is a prerequisite for theoretical analysis.

67.8.2 Horizon 2: Theoretical Development (2027–2029)

With empirical foundations in place, the field can pursue formal theory:

1. Simplified models with provable properties. Start with restricted settings — fixed prompt, single model, simple fitness landscape — and prove basic results about convergence, runtime, or approximation quality. Gradually relax assumptions.
2. Program-space geometry. Develop distance metrics and topological tools for analyzing the structure of program fitness landscapes under LLM-guided search.
3. Safety formalization. Formal specification of sandbox requirements, containment properties, and specification-gaming detection methods, building on work in formal verification and AI safety.

67.8.3 Horizon 3: Integration and Governance (2029+)

Longer-term work must address the systemic implications of self-improving code-generation systems:

1. Verified evolution. Combining LLM-powered evolution with formal verification to produce evolved programs that are provably correct — not just empirically tested.
2. Governance frameworks. Policy and technical standards for autonomous code evolution in production systems, including audit trails, rollback mechanisms, and human-override protocols.
3. Recursive improvement theory. Formal analysis of when and how self-improving systems can be safely deployed, drawing on work in AI alignment, formal methods, and control theory.

67.9 A Classification of Open Problems

To aid researchers in identifying where their expertise can contribute, we classify the open problems identified in this chapter according to their origin:

Problem	From EC	From LLM	Novel to Intersection	Section
Mutation distribution characterization		●	●	§67.2.1
Program-space fitness landscapes	●		●	§67.2.2
Convergence / runtime bounds	●		●	§67.2.3
Budget allocation strategies		●	●	§67.3.1
Island-model adaptation	●		●	§67.3.2
Many-objective + LLM steering	●		●	§67.3.3
Sandbox threat model		●	●	§67.4.1
Specification gaming in evolution	●		●	§67.4.2
Recursive self-improvement containment		●	●	§67.4.3
Reproducibility standards		●	●	§67.5.1
Budget-fair comparison	●		●	§67.5.2
Prompt-as-operator theory			●	§67.6.1
Co-evolutionary credit assignment	●		●	§67.6.2
Cross-task knowledge transfer	●	●	●	§67.6.3

A striking observation: every problem listed is marked as "novel to intersection." While several inherit structure from classical EC or LLM research, the specific form they take in LLM-powered evolution is sufficiently different that existing solutions do not directly transfer. This underscores the claim that LLM-powered evolution is not merely an application of existing techniques but a genuinely new research area requiring its own theoretical and methodological infrastructure.

67.10 Concluding Reflections

LLM-powered evolutionary systems occupy an unusual position in the landscape of AI research. They are simultaneously one of the oldest ideas in computing — programs that write programs, guided by selection — and one of the newest, enabled by the surprising capabilities of large language models as code generators. The field's rapid growth between 2024 and 2026 has produced impressive demonstrations, from mathematical discoveries to infrastructure optimization, but it has also accumulated significant intellectual debt.

The most pressing needs are not glamorous: standardized benchmarks, reproducibility protocols, and careful empirical characterization of basic mechanisms. These foundational investments will determine whether LLM-powered evolution matures into a rigorous discipline with predictable, reliable methods, or remains a collection of impressive but poorly understood demonstrations. The theoretical challenges — characterizing LLM mutation distributions, proving convergence bounds, formalizing prompt-operator mappings — are deep and may take years to resolve, but even partial progress would transform the field's ability to design systems principally rather than empirically.

The safety questions, though more speculative, are not premature. Systems that autonomously generate and execute code, that co-evolve their own prompts and evaluation criteria, and that are being applied to improve AI infrastructure itself, deserve careful containment analysis before they become ubiquitous — not after. The autonomy-level framework proposed in Section 67.7.2 offers one starting point for matching oversight to risk.

For researchers entering this field, the opportunity is substantial: virtually every foundational question is open, the tools for investigation are accessible (many systems are open-source, and LLM APIs are widely available), and the practical demand for principled methods is growing as industry adoption accelerates. The problems outlined in this chapter are not obstacles to the field's progress — they are the field's research agenda for the next decade.