Doby’s ESA Pyxel Logs🌙

☕🛰️ Pyxel Contribution: Prototype of Annotated Metadata for Model Constraints

Summary: Introduces a prototype implementation of typing.Annotated metadata to Pyxel model functions, using annotated-types (e.g. Gt(), Interval(), Unit()). This lays the groundwork for future automatic parameter validation and JSON Schema integration.

Merge Request: https://gitlab.com/esa/pyxel/-/merge_requests/1133

Date: 2025-11-26

✨ Overview

This contribution adds optional Python metadata using typing.Annotated to two Pyxel models:

  • pyxel.models.photon_collection.usaf_illumination
  • pyxel.models.charge_generation.dark_current_rule07

These annotations describe value constraints (e.g. Gt(0.0)), type expectations, and potential future units. The goal is to build a foundation for a full validation system spanning:

  • Model function signatures
  • YAML model configuration
  • JSON Schema generation

This work is part of issue #984.


🔧 Technical Details

Two model functions were updated to use the new metadata system:

  • USAF Illumination — annotated multiplier, time_scale, and position constraints
  • Rule07 Dark Current — annotated cutoff_wavelength and numerical parameters

Examples of new annotations:

from typing import Annotated
from annotated_types import Gt, Interval, Unit

multiplier: Annotated[float, Gt(0.0)] = 1.0
cutoff_wavelength: Annotated[float, Interval(ge=1.7, le=15.0)] = 2.5
position: tuple[
    Annotated[int, Gt(0)], 
    Annotated[int, Gt(0)]
]

This preserves full backward compatibility — nothing in runtime behavior changes.


🛠 JSON Schema Behavior

After applying the annotations, the JSON Schema generator (tox -e json_schema) fails with:

ModuleNotFoundError: No module named 'annotated_types'

This is expected and confirms that:

  • Pyxel detects the new Annotated metadata
  • The JSON Schema pipeline must be extended to support it
  • The prototype integrates cleanly into the import and reflection system

An error screenshot was added to the issue for maintainers.


📁 Branch & File Changes

Branch: feature/annotated-types-metadata-prototype

Modified files:

  • pyxel/models/photon_collection/usaf_illumination.py
  • pyxel/models/charge_generation/dark_current_rule07.py

Only type-level metadata was added. Function logic remains untouched.


🚀 Future Work

  • Extend annotations to all Pyxel models
  • Expose these constraints to the YAML metadata schema
  • Teach JSON Schema generator to interpret Annotated
  • Add unit-aware validation (e.g., µm, seconds, ADU)
  • Improve error reporting when invalid values are passed to models

🌱 Lessons Learned

This contribution became a powerful learning experience, helping me understand both Pyxel’s architecture and the broader workflow of scientific software development.

  • 1. Metadata can shape future features.
    Adding typing.Annotated doesn’t change runtime execution, but it enables future validation, documentation, and safer interfaces.
  • 2. Pyxel is intentionally modular.
    Model functions, YAML metadata, and JSON Schema run independently but communicate through well-defined layers — understanding these layers helped me implement a clean prototype.
  • 3. Tox and schema generation are central to Pyxel’s safety.
    Running tox -e json_schema demonstrated how Pyxel ensures its configuration stays valid.
  • 4. Open-source contribution is more than code.
    Branch naming, screenshots, merge request descriptions, and communication matter just as much as the implementation itself.
  • 5. Small prototypes guide big systems.
    The goal wasn’t to convert every model — just to demonstrate feasibility and provide a foundation for future work.
  • 6. Confidence grows through doing.
    From annotating the models to running tox to opening the MR, the process strengthened my confidence in contributing to real scientific tooling.

🐉 Challenges

Every contribution brings its own set of challenges. These were the main ones I encountered during this work:

  • 1. Understanding the reflection chain.
    Pyxel uses runtime reflection to discover models, populate metadata, and generate JSON Schema. Learning how and where this happens took time but was deeply rewarding.
  • 2. Finding the right boundaries for the prototype.
    It was important not to “over-annotate” or change behavior — the challenge was to keep the prototype small, focused, and correct.
  • 3. Managing imports inside the tox environment.
    annotated_types is not installed in the JSON schema tox environment, which intentionally triggered failures. Understanding why this happened and confirming it as expected behavior was a key challenge.
  • 4. Working across multiple layers of the codebase.
    Editing models, running tox, reacting to import failures, and preparing the MR required navigating several parts of Pyxel’s ecosystem.
  • 5. Learning GitLab workflow in a real scientific project.
    Forks, branches, pushes, issues, and merge requests became a full workflow that I had to coordinate correctly.

Overcoming these challenges made the contribution stronger and helped me build a deeper understanding of Pyxel’s design philosophy.

🧑‍🚀 Authored by: Doby Baxter
🌌 Contribution for: ESA Pyxel Project

⚙️ Config Lab