Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 25 additions & 2 deletions Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ All notable Changes to the Julia package `AlgorithmsInterface.jl` are documented
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.1] unreleased
## [0.2.0] unreleased

### Added

Expand All @@ -16,12 +16,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Convenience two-argument `get_reason(algorithm, state)`, and likewise for `is_active`, `indicates_convergence` and `get_active_stopping_criteria`, extracting the criterion and its state the way `is_finished` already did.
- `StopReasonAction`, a `LoggingAction` that reports `get_reason` at the `:Stop` context.
- Defaults for `get_reason` (`nothing`) and for the type-domain `indicates_convergence` (`false`), so a criterion that implements neither no longer hits a `MethodError` from the derived convergence reporting.
- Exports for `DefaultStoppingCriterionState`, `StopAfterTimePeriodState` and `GroupStoppingCriterionState`, which a downstream criterion is expected to reuse.
- `StopAfterTimePeriodData`, exported, holding the clock `StopAfter` carries as the `data` of its state, which a downstream criterion working on time measurements is expected to reuse.
- Defaults for `initialize_state` and `initialize_state!` returning and resetting a `State`, so that an algorithm only has to provide a `Problem`, an `Algorithm` and a `step!`.
- Defaults for `initialize_state(problem, algorithm, stopping_criterion)` and its mutating variant, returning and resetting a `StoppingCriterionState`, with a `stopping_state_data` keyword to seed its `data`.
Every criterion honours that keyword rather than hardcoding its data: `nothing`, the default, leaves the criterion to decide, which on a reset means keeping the data the state already carries.
`StopAfter` takes the `StopAfterTimePeriodData` it is handed and otherwise makes a fresh one, and `StopWhenAll` and `StopWhenAny` read it as the states of the criteria they combine, in the order of their `criteria`.

### Changed

- `State` is no longer an abstract type but the concrete state every algorithm runs with, storing the `iterate`, the `stopping_criterion_state` and the `iteration` next to a single `data` field for anything else an algorithm has to carry from one step to the next.
The `data` field is opaque to this package and none of its contents are exposed as properties, so an algorithm reaches them through `state.data`, which is what makes one state type enough for all of them.
An algorithm that used to define a state of its own now defines nothing at all, or, if it wants to decide where to start from rather than being handed an `iterate`, only an `initialize_state` returning a `State`.
- `increment!` takes the problem and the algorithm as well, as `increment!(problem, algorithm, state)`, so that per-iteration bookkeeping beyond the iteration counter remains overloadable now that it can no longer dispatch on a state of its own.
- `initialize_state` and `initialize_state!` take their arguments positionally, as `(problem, algorithm, iterate, state_data, stopping_state_data)` and `(problem, algorithm, state, iterate, state_data, stopping_state_data)`, rather than through `iterate`, `state_data` and `stopping_state_data` keywords.
Everything beyond the `iterate` is optional, and `initialize_state!` defaults every argument to what the `state` already holds, so an in-place reset only takes what actually changes.
`solve` and `solve!` forward their trailing positional arguments there, so `solve(problem, algorithm; iterate = x)` becomes `solve(problem, algorithm, x)`.
An algorithm that decides where it starts still implements `initialize_state(problem, algorithm; kwargs...)`, which is what `solve(problem, algorithm)` reaches, and which no longer has to declare an `iterate` keyword to do so.
- `indicates_convergence` without a state moved to the type domain: a new criterion implements `indicates_convergence(::Type{YourCriterion})`, and `indicates_convergence(criterion)` forwards to it.
`StopWhenAll` and `StopWhenAny` combine their children in the type domain as well, so a criterion that only implements the variant taking an instance is no longer accounted for in a group.
- `StoppingCriterionState` is no longer an abstract type but the concrete state every criterion runs with, storing the `at_iteration` at which the criterion indicated to stop next to a single `data` field for whatever else it has to remember from one iteration to the next.
Both states put their `data` last, after the iteration they record: `StoppingCriterionState(at_iteration, data)` and `State(iterate, stopping_criterion_state, iteration, data)`, each with a shorter form omitting that iteration, as `StoppingCriterionState(data = nothing)` and `State(iterate, stopping_criterion_state, data = nothing)`.
A criterion therefore dispatches `is_finished!`, `get_reason` and the rest on itself rather than on a state of its own, and defines `initialize_state` only to fill that `data` field.
`StopAfter` keeps its clock there as a `StopAfterTimePeriodData`, and `StopWhenAll` and `StopWhenAny` the states of the criteria they combine, as a tuple in the order of their `criteria`.
- `StopAfterIteration` no longer carries its own `initialize_state` and `initialize_state!`, which the new criterion-level defaults now cover.

### Removed

- The abstract `State`: subtyping it is no longer how an algorithm provides a state, since the concrete `State` serves every algorithm.
- `AlgorithmsInterface.Test.DummyState`, which the concrete `State` replaces.
- The abstract `StoppingCriterionState`, along with `DefaultStoppingCriterionState`, `StopAfterTimePeriodState` and `GroupStoppingCriterionState`, all of which the concrete `StoppingCriterionState` replaces.

### Fixed

Expand Down
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "AlgorithmsInterface"
uuid = "d1e3940c-cd12-4505-8585-b0a4b322527d"
authors = ["Ronny Bergmann <git@ronnybergmann.net>", "Lukas Devos <ldevos98@gmail.com>"]
version = "0.1.1"
version = "0.2.0"

[deps]
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
Expand Down
117 changes: 77 additions & 40 deletions docs/src/interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,16 @@ Bundling these together loosely without forcing one giant monolithic type makes
* test algorithms in isolation by constructing minimal `Problem`/`Algorithm` pairs, and
* extend behavior (add new stopping criteria, new logging events) without rewriting core loops.

The interface in this package formalizes those roles with three abstract types:
The interface in this package formalizes those roles with three types:
* [`Problem`](@ref): immutable, algorithm‑agnostic input data.
* [`Algorithm`](@ref): immutable configuration and parameters deciding how to iterate.
* [`State`](@ref): mutable data that evolves (current iterate, caches, counters, diagnostics).

The first two are abstract, and every algorithm defines its own pair.
The third is a concrete type: since the data it carries is opaque to this package, one state serves them all.

It provides a framework for decomposing iterative methods into small, composable parts:
concrete `Problem`/`Algorithm`/`State` types have to implement a minimal set of core functionality,
concrete `Problem`/`Algorithm` types have to implement a minimal set of core functionality,
and this package helps to stitch everything together and provide additional helper functionality such as stopping criteria and logging functionality.

## [Concrete example: Heron's method](@id sec_heron)
Expand All @@ -48,6 +51,8 @@ They are illustrative; various performance and generality questions will be left

### Algorithm types

The [`Problem`](@ref) holds the immutable input and the [`Algorithm`](@ref) the immutable configuration:

```@example Heron
using AlgorithmsInterface

Expand All @@ -58,50 +63,21 @@ end
struct HeronAlgorithm <: Algorithm
stopping_criterion # will be plugged in later (any StoppingCriterion)
end

mutable struct HeronState <: State
iterate::Float64 # current iterate
iteration::Int # current iteration count
stopping_criterion_state # will be plugged in later (any StoppingCriterionState)
end
```

### Initialization

In order to start implementing the core parts of our algorithm, we start at the very beginning.
There are two main entry points provided by the interface:

- [`initialize_state`](@ref) constructs an entirely new state for the algorithm
- [`initialize_state!`](@ref) (in-place) reset of an existing state.
That leaves the third piece, the [`State`](@ref).
It holds the `iterate`, the `iteration` counter and the `stopping_criterion_state`, next to a `data` field for whatever else an algorithm might carry from one step to the next.
Since that field is opaque to this package, the same state serves every algorithm, so there is nothing to define here.

An example implementation might look like:

```@example Heron
function AlgorithmsInterface.initialize_state(problem::SqrtProblem, algorithm::HeronAlgorithm; kwargs...)
x0 = rand() # random initial guess
stopping_criterion_state = initialize_state(problem, algorithm, algorithm.stopping_criterion)
return HeronState(x0, 0, stopping_criterion_state)
end

function AlgorithmsInterface.initialize_state!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::HeronState; kwargs...)
# reset the state for the algorithm
state.iterate = rand()
state.iteration = 0

# reset the state for the stopping criterion
state = AlgorithmsInterface.initialize_state!(
problem, algorithm, algorithm.stopping_criterion, state.stopping_criterion_state
)
return state
end
```
The same goes for initialization: [`initialize_state`](@ref) and [`initialize_state!`](@ref), which respectively construct a fresh state and reset an existing one, both have a default.
An algorithm implements them only to [decide where to start from](@ref sec_custom_init), covered at the end of this section.

### Iteration steps

Algorithms define a mutable step via [`step!`](@ref). For Heron's method:

```@example Heron
function AlgorithmsInterface.step!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::HeronState)
function AlgorithmsInterface.step!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::State)
S = problem.S
x = state.iterate
state.iterate = 0.5 * (x + S / x)
Expand All @@ -112,33 +88,94 @@ end
Note that we are only focussing on the actual algorithm, and *not* incrementing the iteration counter.
These kinds of bookkeeping should be handled by the [`AlgorithmsInterface.increment!`](@ref) function, which will by default already increment the iteration counter.
The following generic functionality is therefore enough for our purposes, and does *not* need to be defined.
Nevertheless, if additional bookkeeping would be desired, this can be achieved by overloading that function:
Nevertheless, if additional bookkeeping would be desired, this can be achieved by overloading that function for your [`Algorithm`](@ref):

```julia
function AlgorithmsInterface.increment!(state::State)
function AlgorithmsInterface.increment!(problem::Problem, algorithm::Algorithm, state::State)
state.iteration += 1
return state
end
```

Since [`step!`](@ref) dispatches on the [`Algorithm`](@ref), sharing one state type across algorithms costs nothing.

### Running the algorithm

That is the whole algorithm: a [`Problem`](@ref), an [`Algorithm`](@ref) and a [`step!`](@ref).
With these definitions in place you can already run (assuming you also choose a stopping criterion – added in the next section):

```@example Heron
function heron_sqrt(x; maxiter = 10)
prob = SqrtProblem(x)
alg = HeronAlgorithm(StopAfterIteration(maxiter))
return solve(prob, alg) # allocates & runs
return solve(prob, alg, 1.0) # allocates & runs
end

println("Approximate sqrt: ", heron_sqrt(16.0))
```

The third argument is where the default initialization gets its starting point from, and it is required: there is no sensible guess this package could make on an algorithm's behalf.

Note that [`solve`](@ref) will default to returning `state.iterate`.
If desired, this can be customized by altering [`finalize_state!`](@ref).
We will refine this example with better halting logic and logging shortly.

### Carrying extra data

Anything an algorithm has to carry from one step to the next beyond those three properties goes into the `data` field of the [`State`](@ref).
It is passed along and reset alongside the rest of the state, but nothing is ever read from it, so an algorithm picks whatever container suits it.
Prefer a `NamedTuple` or a small struct of your own, which keep access to the contents type stable:

```@example Heron
struct TrackingHeronAlgorithm <: Algorithm
stopping_criterion
end

function AlgorithmsInterface.step!(problem::SqrtProblem, ::TrackingHeronAlgorithm, state::State)
S = problem.S
x = state.iterate
state.iterate = 0.5 * (x + S / x)
push!(state.data.residuals, abs(state.iterate^2 - S))
return state
end

problem = SqrtProblem(16.0)
tracking = TrackingHeronAlgorithm(StopAfterIteration(5))
state = initialize_state(problem, tracking, 1.0, (; residuals = Float64[]))
solve!(problem, tracking, state)
state.data.residuals
```

Note that the `NamedTuple` itself is immutable while the vector inside it is not, which is all this needs: the state keeps handing the same container back, and the algorithm mutates its contents.
Here the initial iterate and the data are passed positionally, as `initialize_state(problem, algorithm, iterate, state_data)`; the [`StoppingCriterionState`](@ref) can be given its own data as a further argument.

### [Deciding where to start](@id sec_custom_init)

The one thing the defaults cannot know is where an algorithm should start, which is why that argument has no default.
An algorithm that does know implements the variant without it, supplying that starting point and leaving the rest of the work to the generic methods:

```@example Heron
function AlgorithmsInterface.initialize_state(
problem::SqrtProblem, algorithm::HeronAlgorithm; kwargs...
)
return initialize_state(problem, algorithm, rand(); kwargs...)
end

function AlgorithmsInterface.initialize_state!(
problem::SqrtProblem, algorithm::HeronAlgorithm, state::State; kwargs...
)
return initialize_state!(problem, algorithm, state, rand(); kwargs...)
end
```

Note how both hand their guess to the generic method, which is where the [`State`](@ref) is allocated and reset.
Neither shadows that method: a caller who does pass an iterate reaches it directly and gets to say where the algorithm starts after all.
This `HeronAlgorithm` now falls back to an initial guess of its own rather than insisting on being handed one, so it runs with nothing but a problem and a criterion:

```@example Heron
println("Approximate sqrt: ", solve(SqrtProblem(16.0), HeronAlgorithm(StopAfterIteration(10))))
```

## Reference: Core interface types & functions

Below are the automatic API docs for the core interface pieces. Read them after grasping the example above – the intent should now be clearer.
Expand Down
33 changes: 8 additions & 25 deletions docs/src/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,26 +46,7 @@ struct HeronAlgorithm <: Algorithm
stopping_criterion
end

mutable struct HeronState <: State
iterate::Float64
iteration::Int
stopping_criterion_state
end

function AlgorithmsInterface.initialize_state(problem::SqrtProblem, algorithm::HeronAlgorithm; kwargs...)
x0 = rand()
stopping_criterion_state = initialize_state(problem, algorithm, algorithm.stopping_criterion)
return HeronState(x0, 0, stopping_criterion_state)
end

function AlgorithmsInterface.initialize_state!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::HeronState; kwargs...)
state.iterate = rand()
state.iteration = 0
initialize_state!(problem, algorithm, algorithm.stopping_criterion, state.stopping_criterion_state)
return state
end

function AlgorithmsInterface.step!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::HeronState)
function AlgorithmsInterface.step!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::State)
S = problem.S
x = state.iterate
state.iterate = 0.5 * (x + S / x)
Expand All @@ -75,11 +56,13 @@ end
function heron_sqrt(x; stopping_criterion = StopAfterIteration(10))
prob = SqrtProblem(x)
alg = HeronAlgorithm(stopping_criterion)
return solve(prob, alg) # allocates & runs
return solve(prob, alg, 1.0) # allocates & runs
end
nothing # hide
```

Note that this leaves the initialization to its defaults, as the [interface section](@ref sec_interface) does, so there is no `initialize_state` to be seen here and the starting iterate is handed to [`solve`](@ref).

It is already interesting to note that there are no further modifications necessary to start leveraging the logging system.

### Basic iteration printing
Expand Down Expand Up @@ -211,7 +194,7 @@ function AlgorithmsInterface.handle_message!(
action::CaptureHistory,
problem::SqrtProblem,
algorithm::HeronAlgorithm,
state::HeronState;
state::State;
kwargs...
)
push!(action.iterates, state.iterate)
Expand Down Expand Up @@ -287,7 +270,7 @@ end
StatsCollector() = StatsCollector(0, 0.0, 0.0)

function AlgorithmsInterface.handle_message!(
action::StatsCollector, problem::SqrtProblem, algorithm::HeronAlgorithm, state::HeronState;
action::StatsCollector, problem::SqrtProblem, algorithm::HeronAlgorithm, state::State;
kwargs...
)
action.count += 1
Expand Down Expand Up @@ -334,7 +317,7 @@ function solve!(problem::Problem, algorithm::Algorithm, state::State; kwargs...)
while !is_finished!(problem, algorithm, state)
emit_message(problem, algorithm, state, :PreStep)

increment!(state)
increment!(problem, algorithm, state)
step!(problem, algorithm, state)

emit_message(problem, algorithm, state, :PostStep)
Expand Down Expand Up @@ -415,7 +398,7 @@ Here we will illustrate this by a slight adaptation of our algorithm, which coul
To emit a custom logging event from within your algorithm, call [`emit_message`](@ref):

```@example Heron
function AlgorithmsInterface.step!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::HeronState)
function AlgorithmsInterface.step!(problem::SqrtProblem, algorithm::HeronAlgorithm, state::State)
# Suppose we check for numerical issues
if !isfinite(state.iterate) || mod(state.iteration, 10) == 0
emit_message(problem, algorithm, state, :Restart)
Expand Down
Loading
Loading