This repository provides a pipeline to forward model Dark Energy Survey Year 3 (DES Y3)-like weak lensing and galaxy clustering mocks from cosmological simulations:
- Input: Full-sky probe maps (weak lensing signal, intrinsic alignments, and dark matter density) from the CosmoGridV1 simulation suite [Kacprzak & Fluri et al. 2022] projected using
cosmogridv11 - Output: Self-consistent DES Y3-like weak lensing (convergence) and galaxy clustering (linear bias) maps with realistic survey properties [Thomsen et al. 2025]
- Deep Learning Integration: The data storage and loading are designed to work in conjunction with the training pipeline for mutual information maximizing graph convolutional neural networks in
y3-deep-lss
Requires Python >= 3.8, TensorFlow >= 2.0, and TensorFlow-Probability.
Step 1: Install this package
On HPC clusters with pre-installed TensorFlow (recommended):
pip install -e .On systems without TensorFlow:
pip install -e .[tf]Use the first option when TensorFlow is available via system modules (e.g., module load tensorflow) to preserve optimized GPU/MPI configurations.
msfm/apps- Production scripts for parallel mock generation usingesub-epipefor submissionmsfm/utils- Helper functionsmsfm/fiducial_pipeline.pyandmsfm/grid_pipeline.py- Data generators for neural network training
Configuration files for cosmological and astrophysical paremeter priors, fixed survey properties, forward-modeling choices, and other analysis settings.
CosmoGridV1 properties, survey masks, and catalog ellipticities used in the shape noise generation. The top level is exactly the set of inputs the current production config reads; data/cache/ holds regenerable catalog-derived maps and data/deprecated/ frozen superseded ones. data/README.md documents where every file comes from and which notebook produces it. Note: desy3_noise_512.h5 exceeds the repo's file size limit and must be generated from the source galaxy catalog via notebooks/noise_file.ipynb.
Notebooks that generate the contents of the data directory, one per product — see the table in data/README.md. This is the production set; the analysis and comparison notebooks that consume those products live in dev/notebooks instead.
Submission commands for distributed HPC execution via esub-epipe.
- Informative map-level neural summary statistics:
y3-deep-lss - Simulation-based inference:
multiprobe-simulation-inference