NEAT Neural Network for DenoJS

This project is a unique implementation of a neural network based on the NEAT (NeuroEvolution of Augmenting Topologies) algorithm, written in DenoJS using TypeScript.

Feature Highlights

Extendable Observations: The observations can be extended over time as the indexing is done via UUIDs, not numbers. This prevents the need to restart the evolution process as new observations are added, providing flexibility and scalability.
Distributed Training: Training and evolution can be run on multiple independent nodes. The best-of-breed creatures can later be combined on a centralized controller node. This feature allows for distributed computing and potentially faster training times, enhancing the efficiency of the learning process.
Life Long Learning: Unlike many pre-trained neural networks, this project is designed for continuous learning, making it adaptable and potentially more effective in changing environments. This feature ensures the model remains relevant and accurate over time.
Efficient Model Utilization: Once trained, the current best model can be utilized efficiently by calling the activate function. This allows for quick and easy deployment of the trained model.
Unique Squash Functions: The neural network supports unique squash functions such as IF, MAX, MIN, and HYPOT. These functions provide more options for the activation function, which can lead to different network behaviours, offering a wider range of potential solutions. More about Activation Functions.
Neuron Pruning: Neurons whose activations don’t vary during training are removed, and the biases in the associated neurons are adjusted. This feature optimizes the network by reducing redundancy and computational load. More about Pruning (Neural Networks).
CRISPR: Allows injection of genes into a population of creatures during evolution. This feature can introduce new traits and potentially improve the performance of the population. More about CRISPR.
Grafting: If parents aren’t “genetically compatible”, then the “grafting” algorithm from one parent to another parent onto the child will be used. This allows for species from islands to interbreed.
Memetic Evolution: The algorithm can now record and utilize the biases and weights of the fittest creatures to fine-tune future generations. This process, inspired by the concept of memes, allows the system to “remember” and build upon successful traits, enhancing the evolutionary process. Learn more about Memetic Algorithms.
Error-Guided Structural Evolution: Dynamically identifies and creates new synapses by analyzing neuron activations and errors. This targeted structural adaptation improves performance by explicitly reducing neuron-level errors, blending evolutionary topology adjustments with error-driven learning.
Visualization

Usage

This project is designed to be used in a DenoJS environment. Please refer to the DenoJS documentation for setup and usage instructions.

Contributions

Contributions are welcome. Please submit a pull request or open an issue to discuss potential changes/additions.

License

This project is licensed under the terms of the Apache License 2.0. For the full license text, please see LICENSE.