profile picture

Paul Templier

PhD, Engineer

Languages

  • French (Native)
  • English (Certificate of Proficiency - C2)
  • Spanish (Independant - B2)

Coding

  • Python (Proficient, teaching experience)
  • Jax, pytorch, sklearn, pandas, MPI, Ray (Computing tools)
  • Julia (Professional experience)
  • C / Java (Academic experience)
  • JS / HTML / CSS (Side projects)

Interests

  • Evolutionary computation
  • Machine Learning
  • Reinforcement Learning
  • Evolution Strategies
  • Open-ended evolution
  • Auto-ML
  • Robotics

Hobbies

  • Cooking
  • Game development

Career Profile

Hi, I recently graduated with a PhD in Machine Learning, specializing in the use of evolutionary methods for the optimization of neural networks as intelligent agents. With a background in engineering combined with experience in consulting and teaching, I have built a wide range of skills to be comfortable at all stages of development: from research to implementation, from technical depth to high-level communication, from individual work to team collaboration.

Work Experiences

Research Associate

Aug 2024-Present
Supervised by Antoine Cully
Imperial College London, UK

Postdoc at Imperial College London focusing on adaptive and intelligent robotics, while continuing research on evolutionary methods and Quality-Diversity.

PhD in Machine Learning

2021-2024
Supervised by Emmanuel Rachelson, Dennis G. Wilson
ISAE-SUPAERO, Toulouse, France

Research on Evolution Strategies for policy search, covering deep RL, Quality-Diversity, Genetic Programming. Teaching: 150h in Python and Evolutionary Computation. Co-supervision of an intern leading to a CEC paper.

Visiting PhD Student

Mar-Jul 2023
Supervised by Antoine Cully
Imperial College London, UK

Visiting student at Antoine Cully’s Adaptative & Intelligent Robotics Lab (AIRL) at Imperial College London: studying Evolutionary RL and mixes of ES with Quality-Diversity.

Research Intern

May-Nov 2020
Supervised by Dennis G. Wilson
ISAE-SUPAERO, Toulouse, France

Evolution of neural networks with genetic algorithms for to video games. Implementing NEAT, HyperNEAT in Julia. Ranked first in the GECCO 2020 competition on evolving a DOTA 2 bot.

Member of the Board

2017-2020
ISAE-SUPAERO, Toulouse, France

Elected mandate as Students Representative on the ISAE-SUPAERO board.

Cybersecurity Consultant

Feb-Aug 2020
Wavestone, Paris, France

Internship subject: uses of AI for cybersecurity intrusion detection & response, implementation of a PoC. Other missions: EBIOS risk analysis, impacts on cybersecurity of emerging technologies (i.e. quantum computing).

CEO's Right Hand

Jul-Dec 2019
Pricemoov, Paris, France

Leading high-stake international projects with long-term implications in an AI-focused startup.

Education

PhD in Machine Learning

2021-2024
ISAE-SUPAERO

PhD in evolutionary machine learning, focusing on the optimization of neural networks with Evolution Strategies, their interactions with Deep Reinforcement Learning and Quality-Diversity.

MSc in Engineering

2016-2020
ISAE-SUPAERO

Masters in general engineering, applied to aerospace problems. Specialized in Data Science (major) and Robotics (minor). Research projects:

  • Deep Learning to solve NP-hard problems
  • Deep Reinforcement Learning for human-machine cooperation.

MSc in Operations Research

2019-2020
ISAE-SUPAERO

Additional MSc coupled with the Data Science specialization with classes on:

  • Optimization,
  • Advanced combinatorial optimization
  • Stochastic and evolutionary methods

Publications

PhD thesis
Leveraging Structures in Evolutionary Neural Policy Search
Defended 2024-04-22
(PDF) (Thèses.fr)
While training an artificial agent for complex tasks like driving a car, mastering a video game, or controlling plasma in a nuclear fusion reactor, innovations can lead to intelligent behavior. In such scenarios, a promising approach is to mimick the natural world's evolutionary process, which has honed the problem-solving capabilities of animal brains. Evolutionary Neural Policy Search (ENPS) draws inspiration from this concept. It creates a diverse population of ``brains'' represented by neural networks, allowing the system to ``evolve'' by selectively combining and mutating successful individuals. This thesis delves into the core components of ENPS and their intricate interplay. By analyzing the structures of ENPS, the goal is to design novel policy search methods that enhance these components, ultimately leading to the development of more efficient and effective learning algorithms for complex tasks.

Peer reviewed:

Quality with Just Enough Diversity in Evolutionary Policy Search
GECCO 2024 - Best Paper Award (Paper) (Code) (Video)
Paul Templier, Luca Grillotti, Emmanuel Rachelson, Dennis G. Wilson, Antoine Cully
Quality with Just Enough Diversity (JEDi) uses behavior information from Quality-Diversity to improve the search capabilities of ES, by learning and focusing on interesting behaviors.
Genetic Drift Regularization: on preventing Actor Injection from breaking Evolution Strategies
IEEE CEC 2024 (Paper) (Video)
Paul Templier, Emmanuel Rachelson, Antoine Cully, Dennis G. Wilson
By studying the injection of an RL actor into an ES population, we show they drift apart genetically, which can lead to the ES breaking. We introduce GDR, a simple regularization in the actor training loss, to fix it.
Searching Search Spaces: Meta-evolving a Geometric Encoding for Neural Networks
IEEE CEC 2024 (Paper) (Code)
Tarek Kunze, Paul Templier, Dennis G. Wilson
The Geometric Encoding for Neural Network Evolution (GENE) relies on pseudo-distance functions to encode neural networks as smaller genomes. Using Genetic Programming as a meta-evolution loop, we learned a new encoding based on GENE. The discovered encoding makes sparse networks emerge naturally.
LUCIE: An Evaluation and Selection Method for Stochastic Problems
GECCO 2022 (Paper) (Code)
Erwan Lecarpentier, Paul Templier, Emmanuel Rachelson, Dennis G. Wilson
To tackle the impact of uncertain evaluations in genetic algorithms, we introduce LUCIE, a resampling scheme based on a bandit approach. LUCIE is able to better select elite individuals, making the GA more robust to noise.
A Geometric Encoding for Neural Network Evolution
GECCO 2021 (Paper) (Code) (Video)
Paul Templier, Emmanuel Rachelson, Dennis G. Wilson
Directly optimizing the weights of a neural network with evolution can get expensive, especially with methods like XNES or CMAES. We introduce GENE, a new method to encode neural networks as genomes which keeps the performance of direct encoding while reducing the size of the genome by an order of magnitude.
Evolving a Dota 2 bot: Illuminating search in CGP and NEAT
Competition at GECCO 2020 (Paper) (Code)
Paul Templier, Lucas Hervier, Dennis G. Wilson

Blog articles:

  • Detecting security incidents with Machine Learning (FR)
    • Hugo Moret, Paul Templier
    RiskInsight blog (Wavestone)
  • Security of instant messaging applications (FR)
    • Wajih Jmaiel, Paul Templier
    RiskInsight blog (Wavestone)

    Teaching

    Evolutionary Computation

    2021-2022-2024
    Class managed by Dennis G. Wilson
    ISAE-SUPAERO (40h total)

    Elective module on evolutionary computation for students in their 1st year of MEng program. Teaching:

    • Evolution Strategies
    • Evolution of neural networks
    • Genetic representation and operator design
    • Quality-diversity approaches, evolution of behavior, coevolution
    • Final project supervision: policy search for soft robots with ES

    Python - Algorithm and Computing

    2021-2022-2023
    Class managed by Jérôme LACAN
    ISAE-SUPAERO (100h total)

    Teaching python to students in the FISA program and MSc in Aerospace Engineering:

    • Basics of Python and algorithms
    • 3D representation of planet movements
    • Introduction to embedded systems with Micro:Bit

    Bash & Python

    2022-2023
    Class managed by Dennis G. Wilson
    ISAE-SUPAERO (20h)

    Introduction to Bash, Git and Python for students in the last year of the MEng. program.

    OSS Contributions

    QDax with ES - Quality-Diversity with Evolution Strategies: contributing to the QDax library with Evosax integration
    Kheperax - High-performance JAX-powered simulator for robotic navigation in 2D mazes

    Projects

    Here are some of the recent projects I worked on, either during my PhD or my Masters degree, or as personal side projects.

    BERL - Benchmarking Evolutionary Reinforcement Learning: a python framework to test and evaluate Evolution Strategies for RL tasks, with MPI parallelism
    GENE - A Geometric Encoding for Neural Network Evolution
    NeuroEvolution.jl - A Julia implementation of NEAT-based neuroevolution algorithms (NEAT, CPPN, HyperNEAT)
    Multidimensional GP for multiclass classification - Jupyter notebook implementing, presenting and explaining a research paper for Data Science specialization.
    Genepy - Artificial life simulation in a 2D environment, with a custom implementation of NEAT for the brains.
    Groinkbot - Multi-platform chatbot framework, based on a modular architecture and with a high-level interface.
    Compute - Python tool to easily configure and run experiments on remote hosts with pre-defined configurations through SSH
    Solvers - Bruteforce solvers for puzzle games like Minesweeper or Scrabble.

    Talks

    Quality with Just Enough Diversity in Evolutionary Policy Search

    21 July 2021
    GECCO 2024 (recorded) - Video

    Presenting our JEDi paper at GECCO 2024. The video was pre-recorded, and the live talk was presented by Dennis Wilson.

    Ma Thèse en 180 secondes - Regional Finals (FR)

    25 March 2022
    Théâtre Sorano (460 people) - Video

    Regional finals of the MT180 competition, where PhD students have 3 minutes to explain their research topic to a broad public.

    A Geometric Encoding for Neural Network Evolution

    21 July 2021
    GECCO 2022 (online) - Video

    Presenting our GENE paper at GECCO 2021. The video was pre-recorded.