{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3f89a205",
   "metadata": {},
   "source": [
    "## Structure of utilized Pytrees  \n",
    "\n",
    "All methods and algorithms utilize three major pytrees `descent_state`, `measurement_info` and `descent_info`. All of these are instances of `pulsedjax.utilities.MyNamespace`. This custom type is used in order to dynamically define the structure of pytree instances via the `MyNamespace.expand()` method, which provides flexibility during developement. Unfortunately this flexibility easily leads to somewhat messy and opaque datastructures. Here an attempt is made to provide a rough overview.  \n",
    "\n",
    "Generally `measurement_info` and `descent_info` are treated as static/constant, while `descent_state` contains all variables which may be changed by an algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "854c571b",
   "metadata": {},
   "source": [
    "### Descent State  \n",
    "\n",
    "As stated above `descent_state` contains all (possibly) dynamic variables of an algorithm. Thus its content depends on the algorithm in question. It always contains the current guesses `descent_state.population`. The content and structure of which is again dependent on the algorithm.  \n",
    "In all algorithms the pulse(-parameters) are always described/stored in the frequency domain inside `population`. For algorithms which include randomization `descent_state` also holds a PRNG-key.  \n",
    "\n",
    "For classical algorithms `descent_state` may additionally contain information from the previous iteration(s) which are utilized by various nonlinear optimization approaches. In contrast in general algorithms it may contain dynamic variables/containers from optimistix or evosax.  \n",
    "The substructure of many variables in `descent_state` is subdivided into `.pulse` and `.gate`, since in the case of doubleblind retrievals twice the information needs to be stored. \n",
    "\n",
    "Below is an example of the `descent_state` structure used by `DifferentialEvolution`. \n",
    " \n",
    "```\n",
    "descent_state\n",
    "├── population\n",
    "│   ├── pulse\n",
    "│   │   ├── amp\n",
    "│   │   └── phase\n",
    "│   └── gate\n",
    "│       ├── amp\n",
    "│       └── phase\n",
    "├── best_individual\n",
    "│   ├── pulse\n",
    "│   │   ├── amp\n",
    "│   │   └── phase\n",
    "│   └── gate\n",
    "│       ├── amp\n",
    "│       └── phase\n",
    "├── mu \n",
    "└── key\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a52f7bbe",
   "metadata": {},
   "source": [
    "### Measurement Info  \n",
    "\n",
    "The `measurement_info` object contains all information related to the measurement method as well es derived parameters used for the retrieval. Thus it almost never depends on the algorithm used. If it varies it mainly does so with the measurement method.  \n",
    "Examples of its contents are `time`, `frequency`, `measured_trace`, `nonlinear_method`, `spectral_amplitude` (pulse spectrum if provided) and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "747a1f60",
   "metadata": {},
   "source": [
    "### Descent Info  \n",
    "\n",
    "The structure and content of `descent_info` is purely algorithm dependent. It holds all parameters that are/may be used by the algorithm. For a `DifferentialEvolution` this may for example include `crossover_strategy`, `mutation_strategy` `crossover_rate`, `mutation_rate` among other things.  \n",
    "\n",
    "Below is an example of the structure of `descent_info` for the `PtychographicIterativeEngine`.\n",
    "\n",
    "\n",
    "```\n",
    "descent_info\n",
    "├── measured_spectrum_is_provided\n",
    "│\t├── pulse\n",
    "│\t└── gate\n",
    "├── eta_spectral_amplitude\n",
    "├── optimize_calibration_curve\n",
    "│\t├── _local\n",
    "│\t└── _global\n",
    "├── population_size \n",
    "├── gamma\n",
    "│\t├── _local \n",
    "│\t└── _global\n",
    "├── alpha \n",
    "├── pie_method\n",
    "├── conjugate_gradients\n",
    "│\t├── _local\n",
    "│\t└── _global\n",
    "├── linesearch_params\n",
    "│\t├── linesearch\n",
    "│\t├── c1\n",
    "│\t├── c2\n",
    "│\t├── max_steps\n",
    "│\t└── delta_gamma\n",
    "├── newton\n",
    "│\t├── _local\n",
    "│\t├── _global\n",
    "│\t├── linalg_solver\n",
    "│\t├── lambda_lm\n",
    "│\t└── lbfgs_memory\n",
    "├── s_prime_params\n",
    "│\t├── _local\n",
    "│\t├── _global\n",
    "│\t├── number_of_iterations \n",
    "│\t├── r_gradient\n",
    "│\t├── r_newton\n",
    "│\t└── weights\n",
    "├── xi \n",
    "└── adaptive_scaling\n",
    "\t├── _local\n",
    "\t│\t├── order\n",
    "\t│\t└── factor\n",
    "\t└── _global\n",
    "\t\t├── order\n",
    "\t\t└── factor\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87ac1944",
   "metadata": {},
   "source": [
    "### Final Result  \n",
    "\n",
    "All algorithms perform a post processing step in which information from the final `descent_state` is extracted, analyzed and used to create the `final_result` object which is returned by `algorithm.run()`. The structure of this object is show below.\n",
    "\n",
    "```\n",
    "final_result\n",
    "├── time\n",
    "├── frequency\n",
    "├── pulses\n",
    "│   ├── pulse_t\n",
    "│   ├── pulse_f\n",
    "│   ├── gate_t\n",
    "│   ├── gate_f\n",
    "│   └── idx\n",
    "├── idx_best_individual\n",
    "├── x_arr\n",
    "├── frequency_exp\n",
    "├── trace\n",
    "├── measured_trace\n",
    "├── mu\n",
    "│   ├── _local\n",
    "│   └── _global\n",
    "└── error_arr\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}