Synthetic Telescope Pointing Data Generator

Dependencies: Python, Pandas, Matplotlib, Astropy

• Clone the repository for the model: https://github.com/jzariski/Portfolio.git. This project uses the code in the folder Portfolio/telescope-api-main/api.

Training

syntheticPoint.py

The script simulates data-driven telescope telemetry modeled after the telescope in Haleakala, Hawaii (ogg.clma.2m0a). It is designed to create a baseline dataset for training the machine learning model presented at SPIE 2026 researching data-driven acquisition offset correction for telescope networks to validate its accuracy against synthetic Tpoint-based calculations. The generator executes according to the following steps:

1. Coordinate Frame Transformation: The script picks a dynamic observation timestamp and a random local sidereal time (LST), then translates randomly generated raw star coordinates from the static celestial ICRS frame into the local apparent Celestial Intermediate Reference System (CIRS) using astropy.

2. Mount Frame Projection: It calculates local hour angle (Roll) and declination (Pitch) parameters to project the target star's position onto an equatorial telescope mount structure.

3. Horizon Filtering: An automated horizon check filters out invalid observations. Any targets falling outside a realistic tracking window (Hour Angles beyond $\pm 5$ hours) are dropped to mimic actual observing conditions.

4. Mechanical Error Injection: The remaining valid coordinates are processed through traditional TPoint mathematical pointing formulations to inject mechanical imperfections—such as tube flexure, non-perpendicularity, and encoder index errors.

5. Offset and Actual Coordinate Derivation: The script computes the resulting angular offsets (Offset H, Offset D) and outputs the true physical coordinates where the telescope mechanics would actually land (Actual RA, Actual DEC).

6. Fixed-Width Export: The final output is written to a space-separated text file (data.h5)

Because the training data is generated outside of the repository in which the model was built, the .h5 file must be shifted into Portfolio/telescope-api-main/api/data for the training data to be accessed. The data is then fed into booster.py to train the model, which outputs model_ra.json and model_dec.json, which gives the relative weight of each feature when the model was learning.

Final Feature Schema

The model is trained on 9 input features, in this exact order:

1. lst_hours — local sidereal time (hours)
2. obs_ra_deg — demanded RA (degrees)
3. obs_dec_deg — demanded Dec (degrees)
4. sin_roll — sin(roll), roll = LST - RA (hour angle)
5. cos_roll — cos(roll)
6. sin_pitch — sin(pitch), pitch = Dec
7. cos_pitch — cos(pitch)
8. previous_acq_error_ra — previous row's (obs_RA - solv_RA), 0 for first row
9. previous_acq_error_dec — previous row's (obs_Dec - solv_Dec), 0 for first row

year/month/day/hour/minute/second are intentionally excluded — lst_hours already encodes sky rotation, and the individual date/time fields were found to be redundant proxies for it (0.0 feature importance in early testing) while risking poor generalization to real telescope data observed on different nights. tan(pitch) was tested as an additional feature and found to add no value (also 0.0 importance, since it's derivable from sin_pitch/cos_pitch already present). 1/cos(pitch) was tested and picked up non-trivial importance but did not improve validation accuracy, so it was not kept in the final schema.

Any script that calls the trained model (validateExternal.py, predict.py, Sobol.py if used) must construct its feature vector in this exact order or predictions will be silently wrong — XGBoost does not validate feature semantics, only shape.

Testing

validateExternal.py generates 1000 rows of fresh synthetic testing data and loads the trained model directly (via xgboost.XGBRegressor().load_model()) from Portfolio/telescope-api-main/api/models/ — no Docker or Flask API is involved in this validation step. The script outputs a .csv file containing all features used for testing, the model's predictions, the residual values for RA and Dec, and the individual TPoint terms used to calculate the true offsets for roll and pitch.

Analysis

plotting.py generates 4 categories of plots (Plots 4 and 5 are not relevant).

• PLOT 1: Residuals vs Roll and Pitch
• PLOT 2a: Delta-H TPoint terms vs Roll and Pitch
• PLOT 2b: Delta-D TPoint terms vs Roll and Pitch
• PLOT 3a: Residual RA and Dec vs Delta-H TPoint terms
• PLOT 3b: Residual RA and Dec vs Delta-D TPoint terms
• PLOT 6: Residual RA and Dec vs Trig Features

Note: weight.txt was manually created and is not generated by any of the scripts. It extracts information from the model.json files using extractWeights.py which is run in terminal to give text-based output.

Results Summary (eval set, arcsec)

Version	Features	Hyperparams	RA median	RA 90th	Dec median	Dec 90th
v1	11 (raw time fields)	default	5.57	25.79	1.74	5.69
v1.2	11 (raw time fields), split restored	default	~5.6	~25.8	~1.7	~5.7
v2	9 (sin&cos roll/pitch, no time)	default	0.63	1.99	0.22	0.76
v3	11 (+tan_pitch, +sec_pitch)	default	0.67	2.29	0.23	0.80
v4	10 (+sec_pitch only)	default	0.63	2.14	0.24	0.82
v5	9 (v3 features)	coord.py + MSE	0.51	1.71	0.23	0.62
v6 (final)	9 (v3 features)	coord.py + MAE	0.33	1.55	0.15	0.45

v6 is the recommended final configuration — best or tied-best on every metric, and the only hyperparameter variant that maintains tight residual clustering (visible in Plot 1 / Plot 3a-3b) rather than the more diffuse mid-range scatter seen under MSE (v6).

Version History

synthetic-pointer-data v.1: First draft. Built a basic validation pipeline that used standard values for Tpoint coefficients and used the same features for model training/testing as provided by the author of the model. Removed eval/testing split in booster.py, trained model using ALL data points. Generated 3 categories of plots. (This full-dataset-no-split approach was later reverted — see Change 1 below.)

synthetic-pointer-data v.2: Restored validation split, implemented feature engineering (removed all time-related features and included sine and cosine values of roll/pitch). Generated 6 categories of plots. This is the trig-only 9-feature baseline referenced throughout this README.

synthetic-pointer-data v.3: added tan(pitch) and 1/cos(pitch) as features. Result: no improvement over v.2 — tan(pitch) received 0.0 feature importance (fully redundant with sin(pitch)/cos(pitch) already present), and while 1/cos(pitch) was used by the model, overall eval metrics were flat-to-slightly-worse than v.2.

synthetic-pointer-data v.4: trained with just 1/cos(pitch) as the additional feature, to isolate it from tan(pitch). Result: still no meaningful improvement over v.2 (differences within run-to-run noise). Both v.3 and v.4 were ruled out; v.2's 9-feature set was kept going forward.

synthetic-pointer-data v.5: added XGBoost hyperparameters from a related model (deeper trees, more estimators, slower learning rate, subsample/colsample regularization, reg_lambda, hist tree method, larger max_bin) together with reg:squarederror (MSE) as the objective. Result: meaningful improvement in median/90th-percentile error over v.2, but residual plots showed increased scatter in the mid-range of errors — points were less tightly clustered near zero than in v.2, even though large outliers were reduced.

synthetic-pointer-data v.6: kept all v.5 hyperparameters the same, only changed the loss function from MSE back to MAE (reg:absoluteerror) so that the results would be more precise. This isolated the loss function as the cause of the v.5 scatter: reverting to MAE alone, on top of the other v.5 hyperparameters, gave the best results across every metric with tight residual clustering. This is the current final/recommended configuration (v.6 files are not in a folder).

Final XGBoost Configuration

The final, recommended xgb_params (used in booster_v7_coordparams_MAE.py, the active api/booster.py):

xgb_params = { 'booster': 'gbtree', 'objective': 'reg:absoluteerror', 'eval_metric': 'mae', 'n_estimators': 5000, 'learning_rate': 0.02, 'max_depth': 5, 'min_child_weight': 2, 'subsample': 0.95, 'colsample_bytree': 0.95, 'reg_lambda': 2.0, 'reg_alpha': 0.0, 'gamma': 0.0, 'tree_method': 'hist', 'max_bin': 1024, 'random_state': 7, 'n_jobs': -1, 'early_stopping_rounds': 150, }

These values are adapted from the model builder's coord.py experiment, with one deliberate deviation: the objective is kept as reg:absoluteerror (MAE) rather than coord.py's reg:squarederror (MSE). MSE was tested (see v6 in Version History) and gave a slightly better median error but visibly more mid-range residual scatter; MAE with the same capacity/regularization settings gave the best combination of accuracy and tight residual clustering.

Changes to booster.py

Change 1 — Train on full dataset (later reverted) Initially modified main() to train on all 20,000 points with no train/eval/test split, removing early_stopping_rounds. This was per the manager's first instruction before the meeting with the model builder.

Change 2 — Restored split + early stopping (per model builder's request) Reverted back to a chronological split, but simplified to train/eval only (90/10), no held-out test set — since validateExternal.py serves as the true external test. Restored early_stopping_rounds=100. Feature engineering overhaul (the trig features) In make_features_and_labels: Dropped: year, month, day, hour, minute, second (redundant proxies for lst_hours, confirmed by both independent analysis and the model builder's own coord.py design) Added: sin(roll_rad), cos(roll_rad), sin(pitch_rad), cos(pitch_rad) — computed by reconstructing roll_rad/pitch_rad from lst_hours and obs_ra_deg/obs_dec_deg Kept: lst_hours, obs_ra_deg, obs_dec_deg, previous_acq_error_ra, previous_acq_error_dec Net: went from 11 input features down to 9

Change 4 — Updated FEATURE_NAMES tuple to match the new 9-feature schema (documentation only, not functionally consumed elsewhere in the file)

Change 5 — Cleaned up train_and_evaluate to take only (X_train, y_train, X_eval, y_eval) instead of also test sets, updated print statements to say "eval set" for clarity, removed dead/commented-out code from main()

Change 6 — Tested tan(pitch) and 1/cos(pitch) as additional features (v.3), then 1/cos(pitch) alone (v.4). Both ruled out — see Version History above. Feature set reverted to Change 3's 9 features.

Change 7 — Replaced xgb_params with the full hyperparameter set listed above under "Final XGBoost hyperparameters" (v.5), then reverted only the objective from reg:squarederror to reg:absoluteerror (v.6, final).

Versioned copies of booster.py for each stage above are included in this repo under the folder booster-py-versions (e.g. booster_v1_full_dataset.py ... booster-v6-xgboos.py) so each experiment can be reproduced exactly. The active booster.py is the final v.6 configuration. Make sure to copy this over to Portfolio/telescope-api-main/api/booster.py before running the pipeline.

Pipeline Summary

1. Generate training data using syntheticPoint.py.
2. Move the file generated (data.h5) into Portfolio/telescope-api-main/api/data.
3. In another terminal, run booster.py. This will output model_ra.json and model_dec.json in Portfolio/telescope-api-main/api/models.
4. Run validateExternal.py to test the model.
5. Run plotting.py to generate plots.