graph LR
Data_Processing_Encoding["Data Processing & Encoding"]
SNN_Core_Primitives["SNN Core & Primitives"]
SNN_Optimization_Acceleration["SNN Optimization & Acceleration"]
Pre_built_SNN_Models["Pre-built SNN Models"]
Training_Experimentation["Training & Experimentation"]
Model_Conversion_Deployment["Model Conversion & Deployment"]
Data_Processing_Encoding -- "provides input data to" --> SNN_Core_Primitives
SNN_Core_Primitives -- "forms the basis of" --> Pre_built_SNN_Models
Pre_built_SNN_Models -- "are trained/evaluated by" --> Training_Experimentation
Pre_built_SNN_Models -- "are prepared for deployment by" --> Model_Conversion_Deployment
Model_Conversion_Deployment -- "generates/modifies" --> Pre_built_SNN_Models
SNN_Core_Primitives -- "utilizes" --> SNN_Optimization_Acceleration
SNN_Optimization_Acceleration -- "optimizes operations of" --> SNN_Core_Primitives
Training_Experimentation -- "applies optimization techniques from" --> SNN_Optimization_Acceleration
click Data_Processing_Encoding href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/spikingjelly/Data_Processing_Encoding.md" "Details"
click SNN_Core_Primitives href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/spikingjelly/SNN_Core_Primitives.md" "Details"
click SNN_Optimization_Acceleration href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/spikingjelly/SNN_Optimization_Acceleration.md" "Details"
click Training_Experimentation href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/spikingjelly/Training_Experimentation.md" "Details"
click Model_Conversion_Deployment href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/spikingjelly/Model_Conversion_Deployment.md" "Details"
The spikingjelly architecture is designed to provide a comprehensive ecosystem for Spiking Neural Networks, extending PyTorch's capabilities. It follows a modular and layered approach, starting with data ingestion and encoding, progressing through core SNN primitive definitions, and incorporating advanced optimization techniques for efficient computation. The framework offers a rich set of pre-built SNN models, which are then subjected to a robust training and experimentation pipeline. A crucial aspect is the ability to convert and deploy these SNN models onto various platforms, including specialized neuromorphic hardware, ensuring practical applicability and performance. This structure facilitates a clear flow from raw data to deployable SNN solutions, emphasizing the unique requirements of neuromorphic computing within a familiar deep learning paradigm.
Data Processing & Encoding [Expand]
Manages the loading, preprocessing, and transformation of diverse neuromorphic datasets (e.g., DVS, N-MNIST, SHD) into formats suitable for SNN processing. This component handles the initial ingestion and preparation of event-based or rate-coded data.
Related Classes/Methods:
SNN Core & Primitives [Expand]
Implements the fundamental building blocks of Spiking Neural Networks. This includes various spiking neuron models (e.g., Leaky Integrate-and-Fire), SNN-adapted neural network layers (convolutional, linear, recurrent), and essential utility functions for SNN operation (e.g., backend selection, step mode management). It also incorporates CuPy-accelerated implementations for high-performance computation.
Related Classes/Methods:
spikingjelly.activation_based.neuronspikingjelly.activation_based.layerspikingjelly.activation_based.functionalspikingjelly.activation_based.neuron_cupy
SNN Optimization & Acceleration [Expand]
Provides critical mechanisms for enabling efficient training and execution of SNNs. This includes various differentiable approximations (surrogate gradients) for the non-differentiable spike function, which are crucial for backpropagation-based training. It also encompasses automated generation of highly optimized CUDA kernels for accelerated GPU execution.
Related Classes/Methods:
Contains a collection of pre-implemented and specialized Spiking Neural Network architectures. This includes adaptations of standard deep learning models (e.g., ResNet, VGG) for SNNs, as well as novel SNN designs, serving as a model zoo for common use cases and benchmarks.
Related Classes/Methods:
spikingjelly.activation_based.model.sew_resnetspikingjelly.activation_based.model.spiking_resnetspikingjelly.activation_based.model.spiking_vgg
Training & Experimentation [Expand]
Manages the end-to-end lifecycle of SNN model development, from training to evaluation. This component includes the core training and evaluation loops, handling data loading, optimization, loss calculation, and performance metric tracking. It also provides examples and implementations for applying SNNs in reinforcement learning scenarios.
Related Classes/Methods:
spikingjelly.activation_based.model.train_classifyspikingjelly.activation_based.examples.DQN_statespikingjelly.activation_based.examples.PPO
Model Conversion & Deployment [Expand]
Provides tools and methodologies for preparing SNN models for efficient deployment on various hardware. This includes techniques for converting pre-trained Artificial Neural Networks (ANNs) into SNNs, implementing quantization methods to reduce model precision, and facilitating interoperability with specialized neuromorphic hardware platforms (e.g., Intel's Lava, Lynxi).
Related Classes/Methods: