Career Profile

Hi, I recently graduated with a PhD in Machine Learning, focusing on the use of evolutionary methods for the optimization of neural networks as intelligent agents. I specialize on learning methods that combine evolution, Reinforcement Learning and Quality-Diversity, and in my current postdoc at Imperial College London I focus on their application to adaptive and intelligent robotics. With a background in engineering combined with experience in consulting and teaching, I have built a wide range of skills to be comfortable at all stages of development: from research to implementation, from technical depth to high-level communication, from individual work to team collaboration.

Work Experiences

Research Associate

Aug 2024-Present
Supervised by Antoine Cully

Postdoc at Imperial College London focusing on adaptive and intelligent robotics, while continuing research on evolutionary methods and Quality-Diversity.

PhD in Machine Learning

2021-2024

Research on Evolution Strategies for policy search, covering deep RL, Quality-Diversity, Genetic Programming. Teaching: 150h in Python and Evolutionary Computation. Co-supervision of an intern leading to a CEC paper.

Visiting PhD Student

Mar-Jul 2023
Supervised by Antoine Cully

Visiting student at Antoine Cully’s Adaptative & Intelligent Robotics Lab (AIRL) at Imperial College Londo, focusing Evolutionary RL and mixes of ES with Quality-Diversity. This collaboration lead to 2 published articles and a postdoc.

Research Intern

May-Nov 2020
Supervised by Dennis G. Wilson

Evolution of neural networks with genetic algorithms for to video games. Implementing NEAT, HyperNEAT in Julia. Ranked first in the GECCO 2020 competition on evolving a DOTA 2 bot.

Cybersecurity Consultant

Feb-Aug 2019

Internship subject: uses of AI for cybersecurity intrusion detection & response, implementation of a PoC. Other missions: EBIOS risk analysis, impacts on cybersecurity of emerging technologies (i.e. quantum computing).

CEO's Right Hand

Jul-Dec 2018

Leading high-stake international projects with long-term implications in an AI-focused startup.

Education

PhD in Machine Learning

2021-2024
ISAE-SUPAERO

PhD in evolutionary machine learning, focusing on the optimization of neural networks with Evolution Strategies, their interactions with Deep Reinforcement Learning and Quality-Diversity.

MSc in Engineering

2016-2020
ISAE-SUPAERO

Masters in general engineering, applied to aerospace problems. Specialized in Data Science (major) and Robotics (minor). GPA: 4.18. Research projects:

  • Deep Learning to solve NP-hard problems
  • Deep Reinforcement Learning for human-machine cooperation.

MSc in Operations Research

2019-2020
ISAE-SUPAERO

Additional MSc coupled with the Data Science specialization with classes on:

  • Optimization,
  • Advanced combinatorial optimization
  • Stochastic and evolutionary methods

Publications

PhD thesis
Leveraging Structures in Evolutionary Neural Policy Search
Defended 2024-04-22
While training an artificial agent for complex tasks like driving a car, mastering a video game, or controlling plasma in a nuclear fusion reactor, innovations can lead to intelligent behavior. In such scenarios, a promising approach is to mimick the natural world's evolutionary process, which has honed the problem-solving capabilities of animal brains. Evolutionary Neural Policy Search (ENPS) draws inspiration from this concept. It creates a diverse population of ``brains'' represented by neural networks, allowing the system to ``evolve'' by selectively combining and mutating successful individuals. This thesis delves into the core components of ENPS and their intricate interplay. By analyzing the structures of ENPS, the goal is to design novel policy search methods that enhance these components, ultimately leading to the development of more efficient and effective learning algorithms for complex tasks.

Peer reviewed:

Quality with Just Enough Diversity in Evolutionary Policy Search
GECCO 2024 - Best Paper Award (Paper) (Code) (Video)
Paul Templier, Luca Grillotti, Emmanuel Rachelson, Dennis G. Wilson, Antoine Cully
Quality with Just Enough Diversity (JEDi) uses behavior information from Quality-Diversity to improve the search capabilities of ES, by learning and focusing on interesting behaviors.
Genetic Drift Regularization: on preventing Actor Injection from breaking Evolution Strategies
IEEE CEC 2024 (Paper) (Video)
Paul Templier, Emmanuel Rachelson, Antoine Cully, Dennis G. Wilson
By studying the injection of an RL actor into an ES population, we show they drift apart genetically, which can lead to the ES breaking. We introduce GDR, a simple regularization in the actor training loss, to fix it.
Searching Search Spaces: Meta-evolving a Geometric Encoding for Neural Networks
IEEE CEC 2024 (Paper) (Code)
Tarek Kunze, Paul Templier, Dennis G. Wilson
The Geometric Encoding for Neural Network Evolution (GENE) relies on pseudo-distance functions to encode neural networks as smaller genomes. Using Genetic Programming as a meta-evolution loop, we learned a new encoding based on GENE. The discovered encoding makes sparse networks emerge naturally.
LUCIE: An Evaluation and Selection Method for Stochastic Problems
Erwan Lecarpentier, Paul Templier, Emmanuel Rachelson, Dennis G. Wilson
To tackle the impact of uncertain evaluations in genetic algorithms, we introduce LUCIE, a resampling scheme based on a bandit approach. LUCIE is able to better select elite individuals, making the GA more robust to noise.
A Geometric Encoding for Neural Network Evolution
Paul Templier, Emmanuel Rachelson, Dennis G. Wilson
Directly optimizing the weights of a neural network with evolution can get expensive, especially with methods like XNES or CMAES. We introduce GENE, a new method to encode neural networks as genomes which keeps the performance of direct encoding while reducing the size of the genome by an order of magnitude.
Evolving a Dota 2 bot: Illuminating search in CGP and NEAT
Competition at GECCO 2020 (Paper) (Code)
Paul Templier, Lucas Hervier, Dennis G. Wilson

Blog articles:

  • Detecting security incidents with Machine Learning (FR)
  • Hugo Moret, Paul Templier
    RiskInsight blog (Wavestone)
  • Security of instant messaging applications (FR)
  • Wajih Jmaiel, Paul Templier
    RiskInsight blog (Wavestone)

    Grants & Awards

    Grants:
    Visiting PhD Scholarship
    10k€, 2023

    Grant to visit the AIRL lab at Imperial College London for 4 months. Combines grants awarded by ISAE-SUPAERO and the EDMITT doctoral school.

    PhD Scholarship
    55k€, 2021-2024

    3-year PhD scholarship from Région Occitanie, covering tuition fees and a monthly stipend.

    PhD Scholarship
    37k€, 2021-2024

    3-year PhD scholarship from ISAE-SUPAERO, to complete the Région Occitanie funding.


    Awards:
    GECCO 2024: Best Paper Award
    2024 (Link)

    Best Paper Award at GECCO 2024 for the article “Quality with Just Enough Diversity in Evolutionary Policy Search”, in the combined tracks of Complex Systems and Learning for Evolutionary Computation.

    GECCO 2020 Competition: Evolving a Dota 2 bot
    First place, 2020 (Link)

    First place at the GECCO 2020 competition on evolving a Dota 2 bot for 1v1 matches, with Lucas Hervier and Dennis Wilson.

    Talks

    Recorded talks:
    Quality with Just Enough Diversity in Evolutionary Policy Search
    August 2024
    GECCO 2024 (recorded) (Video)

    Presenting our JEDi paper at GECCO 2024. The video was pre-recorded, and the live talk was presented by Dennis Wilson.

    Genetic Drift Regularization: on preventing Actor Injection from breaking Evolution Strategies
    2 July 2024
    CEC 2024 (Yokoama, Japan) (Video)

    Presenting our GDR paper at CEC 2024. The video was pre-recorded.

    Ma Thèse en 180 secondes - Regional Finals (FR)
    25 March 2022
    Théâtre Sorano (460 people) (Video)

    Regional finals of the MT180 competition, where PhD students have 3 minutes to explain their research topic to a broad public.

    A Geometric Encoding for Neural Network Evolution
    21 July 2021
    GECCO 2022 (online) (Video)

    Presenting our GENE paper at GECCO 2021. The video was pre-recorded.


    Personal research presentations:
    Quality with Just Enough Diversity in Evolutionary Policy Search
    8 July 2024
    Tsukuba University, Japan

    Presenting my research on JEDi to Prof. Claus Aranha’s team at Tsukuba University.

    Evolution Strategies for Policy Search
    10 March 2023
    Imperial College London, UK

    Introduction presentation of my research to the AIRL lab at Imperial College London.

    Mid-PhD defense
    29 June 2022
    ISAE-SUPAERO, France

    Presentation to an external jury composed of Prof. Nikolaus Hansen and Prof. Olivier Sigaud.

    Research life contributions

    I believe one's contribution to the scientific community also lies in the support and organization of research activities. Here are some of the ways I try to contribute:

    Peer Review - Reviewer for GECCO (2023-2024-2025), IEEE TEVC (2024), ALIFE (2025).
    MSc student supervision - Co-supervised Tarek Kunze's MSc thesis on the meta-evolution of neural network encodings (2024), leading to a CEC 2024 paper. (Link)
    Hiring interviews (AIRL) - Participated in the hiring process for 4 postdoc positions in my Imperial College lab (2024). In charge of technical interviews for PhD student applications (2025).
    ICARL seminars - Part of the Imperial College Autonomous Reasoning & Learning seminars organization team (2025). (Link)
    Elected mandates - Elected mandate as Students Representative on the ISAE-SUPAERO board (2017-2020), and as CPGE Students Representative on the board of Lycée J-B Say (2015-2016).

    Teaching

    Evolutionary Computation

    2021-2022-2024
    Class managed by Dennis G. Wilson
    ISAE-SUPAERO (40h total)

    Elective module on evolutionary computation for students in their 1st year of MEng program. Teaching:

    • Evolution Strategies
    • Evolution of neural networks
    • Genetic representation and operator design
    • Quality-diversity approaches, evolution of behavior, coevolution
    • Final project supervision: policy search for soft robots with ES

    Python - Algorithm and Computing

    2021-2022-2023
    Class managed by Jérôme LACAN
    ISAE-SUPAERO (100h total)

    Teaching python to students in the FISA program and MSc in Aerospace Engineering:

    • Basics of Python and algorithms
    • 3D representation of planet movements
    • Introduction to embedded systems with Micro:Bit

    Bash & Python

    2022-2023
    Class managed by Dennis G. Wilson
    ISAE-SUPAERO (20h)

    Introduction to Bash, Git and Python for students in the last year of the MEng. program.

    OSS Contributions

    QDax with ES - Quality-Diversity with Evolution Strategies: contributing to the QDax library with Evosax integration
    Kheperax - High-performance JAX-powered simulator for robotic navigation in 2D mazes

    Projects

    Here are some of the recent projects I worked on, either during my PhD or my Masters degree, or as personal side projects.

    BERL - Benchmarking Evolutionary Reinforcement Learning: a python framework to test and evaluate Evolution Strategies for RL tasks, with MPI parallelism
    GENE - A Geometric Encoding for Neural Network Evolution
    NeuroEvolution.jl - A Julia implementation of NEAT-based neuroevolution algorithms (NEAT, CPPN, HyperNEAT)
    Multidimensional GP for multiclass classification - Jupyter notebook implementing, presenting and explaining a research paper for Data Science specialization.
    Genepy - Artificial life simulation in a 2D environment, with a custom implementation of NEAT for the brains.
    Groinkbot - Multi-platform chatbot framework, based on a modular architecture and with a high-level interface.
    Compute - Python tool to easily configure and run experiments on remote hosts with pre-defined configurations through SSH
    Solvers - Bruteforce solvers for puzzle games like Minesweeper or Scrabble.