Optimization

A constant instance of OptRunning initialized with false.

A set of optimization methods that are disabled and not used with the BlackBoxOptim package.

A column in the progress bar representing the best optimization result.

• job

• segments

• measure

• best

This struct represents a column in the progress bar that displays the best result of the optimization job. It contains a ProgressJob, a vector of Segment objects, a Measure object, and a reference to the best result. The constructor creates a Segment with a string representation of the best result and sets the width of the measure to 15.

A named tuple representing the context and space in the optimization process.

A mutable structure representing the running state of an optimization process.

• value

This structure contains a single field value which is an atomic boolean. It is used to indicate whether the optimization process is currently running or not.

A structure representing an optimization session.

• s

• ctx

• params

• attrs

• results

• best

• lock

• s_clones

• ctx_clones

This structure stores all the evaluated parameters combinations during an optimization session. It contains fields for the strategy, context, parameters, attributes, results, best result, lock, and clones of the strategy and context for each thread. The constructor for OptSession also takes an offset and number of threads as optional parameters, with default values of 0 and the number of available threads, respectively.

A column in the progress bar representing parameters.

• job

• segments

• measure

• params

This struct represents a column in the progress bar that displays the parameters of the optimization job. It contains a ProgressJob, a vector of Segment objects, a Measure object, and a reference to the parameters. The constructor creates a Segment with a string representation of the parameters and sets the width of the measure to 15.

Optimize parameters using the BlackBoxOptim package.

bboptimize(
    s::Strategies.SimStrategy;
    seed,
    splits,
    resume,
    save_freq,
    zi,
    kwargs...
)

• splits: how many times to run the backtest for each step
• seed: random seed
• kwargs: The arguments to pass to the underlying BBO function. See the docs for the BlackBoxOptim package. Here are some most common parameters:
  • MaxTime: max evaluation time for the optimization
  • MaxFuncEvals: max number of function (backtest) evaluations
  • TraceMode: (:silent, :compact, :verbose) controls the logging
  • MaxSteps, MaxStepsWithoutProgress

From within your strategy, define four call! functions:

• call!(::Strategy, ::OptSetup): for the period of time to evaluate and the parameters space for the optimization.
• call!(::Strategy, params, ::OptRun): called before running the backtest, should apply the parameters to the strategy.

Applies parameters to strategy before backtest

call!(_::Strategies.Strategy, params, _::Executors.OptRun)

Returns Optimizations.ContextSpace for backtesting

call!(_::Strategies.Strategy, _::Executors.OptSetup)

The ctx field (Executors.Context) specifies the backtest time period, while space is either an already built BlackBoxOptim.SearchSpace subtype or a tuple (Symbol, args...) for a pre-defined BBO package search space.

Multi-threaded optimization function.

_multi_opt_func(
    splits,
    backtest_func,
    median_func,
    obj_type
) -> Optimization.var"#36#38"

The function takes four arguments: splits, backtest_func, median_func, and obj_type. splits is the number of splits for the optimization process, backtest_func is the backtest function, median_func is the function to calculate the median, and obj_type is the type of the objective. The function returns a function that performs a multi-threaded optimization for a given set of parameters.

Single-threaded optimization function.

_single_opt_func(
    splits,
    backtest_func,
    median_func,
    args...
) -> Optimization.var"#42#44"

The function takes four arguments: splits, backtest_func, median_func, and obj_type. splits is the number of splits for the optimization process, backtest_func is the backtest function, median_func is the function to calculate the median, and obj_type is the type of the objective. The function returns a function that performs a single-threaded optimization for a given set of parameters.

Returns the dimension of the search space.

_spacedims(params) -> Any

This function takes the parameters as input, which should include lower and upper bounds arrays as the second and third elements. It asserts that the lengths of these arrays are equal and returns their common length, which represents the dimension of the search space.

Converts the provided parameters into a string representation.

_tostring(prefix, params) -> String

The function takes a prefix and a set of parameters as input. It joins the prefix and the parameters into a single string, with each parameter converted to a compact number representation. The resulting string is then truncated to fit the display size.

Aggregates the results of an optimization session.

agg(sess::OptSession; reduce_func, agg_func) -> Any

The function takes an optimization session sess and optional functions reduce_func and agg_func. It groups the results by the session parameters, applies the reduce_func to each group, and then applies the agg_func to the reduced results.

Returns a set of optimization methods supported by BlackBoxOptim.

bbomethods() -> Set{Symbol}
bbomethods(multi) -> Set

This function filters the methods based on the multi parameter and excludes the methods listed in disabled_methods. If multi is true, it returns multi-objective methods, otherwise it returns single-objective methods.

Extracts the context, parameters, and search space from a given strategy.

ctxfromstrat(s)

This function takes a strategy as input and returns the context, parameters, and search space associated with that strategy. The search space can be a SearchSpace instance, a function, or a tuple where the first element is the BBO space type and the rest are arguments for the space constructor.

Calculates the small and big steps for the optimization context.

ctxsteps(
    ctx,
    splits
) -> NamedTuple{(:small_step, :big_step), <:Tuple{Any, Any}}

The function takes two arguments: ctx and splits. ctx is the optimization context and splits is the number of splits for the optimization process. The function returns a named tuple with small_step and big_step which represent the step size for the optimization process.

Defines the backtest function for an optimization session.

define_backtest_func(
    sess,
    small_step,
    big_step
) -> Optimization.var"#opt_backtest_func#33"

The function takes three arguments: sess, small_step, and big_step. sess is the optimization session, small_step is the small step size for the optimization process, and big_step is the big step size for the optimization process. The function returns a function that performs a backtest for a given set of parameters and a given iteration number.

Defines the median function for multi-objective mode.

define_median_func(
    ismulti
) -> Union{Optimization.var"#46#48", Optimization.var"#47#49"}

The function takes a boolean argument ismulti which indicates if the optimization is multi-objective. If ismulti is true, the function returns a function that calculates the median over all the repeated iterations. Otherwise, it returns a function that calculates the median of a given array.

Defines the optimization function for a given strategy.

define_opt_func(
    s::Strategies.Strategy;
    backtest_func,
    ismulti,
    splits,
    obj_type,
    isthreaded
)

The function takes several arguments: s, backtest_func, ismulti, splits, obj_type, and isthreaded. s is the strategy, backtest_func is the backtest function, ismulti indicates if the optimization is multi-objective, splits is the number of splits for the optimization process, obj_type is the type of the objective, and isthreaded indicates if the optimization is threaded. The function returns the appropriate optimization function based on these parameters.

Clears optimization sessions of a strategy.

delete_sessions!(s_name::String; keep_by, zi)

The function accepts a strategy name s_name and an optional keep_by dictionary. If keep_by is provided, sessions matching these attributes (ctx, params, or attrs) are not deleted. It checks each session, and deletes it if it doesn't match keep_by or if keep_by is empty.

Loads the BayesianOptimization extension.

The function checks if the BayesianOptimization package is installed in the current environment. If not, it prompts the user to add it to the main environment.

Filters the optimization results based on certain criteria.

filter_results(
    ::Strategies.Strategy,
    sess;
    cut,
    min_results
) -> Any

The function takes a strategy and a session as input, along with optional parameters for cut and minimum results. It filters the results based on the cut value and the minimum number of results.

Determines the fitness scheme for a given strategy and number of objectives.

fitness_scheme(
    s::Strategies.Strategy,
    n_obj
) -> BlackBoxOptim.ParetoFitnessScheme

This function takes a strategy and a number of objectives as input. It checks if the strategy has a custom weights function defined in its attributes. If it does, this function is used as the aggregator in the ParetoFitnessScheme. If not, a default ParetoFitnessScheme is returned.

Generates a grid from the provided parameters.

gridfromparams(params) -> Any

The function takes a set of parameters as input. It generates a grid by taking the product of the parameters and reshaping it to the length of the parameters.

Generates a grid from the optimization results.

gridfromresults(sess::OptSession, results; kwargs...) -> Any

The function takes an optimization session and results as input. It generates a grid by extracting the parameters from each row of the results.

Initializes a progress bar for grid optimization.

gridpbar!(sess, first_params) -> Base.RefValue

This function sets up a progress bar for the grid optimization process. It creates a ParamsColumn and a BestColumn and adds them to the default columns. The function returns a reference to the current parameters.

Backtests the strategy across combination of parameters.

gridsearch(
    s::Strategies.SimStrategy;
    seed,
    splits,
    save_freq,
    resume,
    logging,
    random_search,
    zi,
    grid_itr,
    offset
)

• seed: random seed set before each backtest run.
• splits: the number segments into which the context is split.
• save_freq: how frequently (Period) to save results, when nothing (default) saving is skipped.
• logging: enabled logging
• random_search: shuffle parameters combinations before iterations

One parameter combination runs splits times, where each run uses a period that is a segment of the full period of the given Context given. (The Context comes from the strategy call!(s, params, OptRun())

Checks if the optimization process is currently running.

isrunning() -> Bool

This function returns the value field of the RUNNING instance, indicating whether the optimization process is currently running.

Tests if if the strategy is thread safe by looking up the THREADSAFE global.

Loads an optimization session from storage.

load_session(name; ...) -> Any
load_session(name, startstop; ...) -> Any
load_session(name, startstop, params_k; ...) -> Any
load_session(
    name,
    startstop,
    params_k,
    code;
    zi,
    as_z,
    results_only,
    s
) -> Any

This function loads an optimization session from the provided zarr instance zi based on the given parameters. The parameters include the strategy name, start and stop date of the backtesting context, the first letter of every parameter, and a hash of the parameters and attributes truncated to 4 characters. The function returns the loaded session, either as a zarr array if as_z is true, or as an OptSession object otherwise. If results_only is true, only the results DataFrame of the session is returned.

Generates the path for the log file of a given strategy.

log_path(s) -> Tuple{Any, Any}
log_path(s, name) -> Tuple{Any, Any}

The function takes a strategy s and an optional name (defaulting to the current timestamp). It constructs a directory path based on the strategy's path, and ensures this directory exists. Then, it returns the full path to the log file within this directory, along with the directory path itself.

Returns the paths to all log files for a given strategy.

logs(s) -> Any

The function takes a strategy s as an argument. It retrieves the directory path for the strategy's log files and returns the full paths to all log files within this directory.

Clears all log files for a given strategy.

logs_clear(s)

The function takes a strategy s as an argument. It retrieves the directory path for the strategy's log files and removes all files within this directory.

Extracts the lower and upper bounds from a parameters dictionary.

lowerupper(params) -> Tuple{Vector{Any}, Vector{Any}}

The function takes a parameters dictionary params as an argument. It returns two arrays, lower and upper, containing the first and last values of each parameter range in the dictionary, respectively.

Calculates the metrics for a given strategy.

metrics_func(s; initial_cash)

The function takes a strategy s and an initial cash amount as arguments. It calculates the objective score, the current total cash, the profit and loss ratio, and the number of trades. The function returns these metrics as a named tuple.

Returns the number of objectives and their type.

objectives(s)

The function takes a strategy s as an argument. It returns a tuple containing the type of the objective and the number of objectives.

Removes results that don't have all the repeated evaluation.

optsession(s::Strategies.Strategy; seed, splits, offset)

The function groups the results by session parameters and removes those groups that don't have a complete set of evaluations, as defined by the splits attribute of the session.

Returns the zarrays storing all the optimization session over the specified zarrinstance.

optsessions(
    s_name::String;
    zi
) -> Union{Nothing, Dict{String, Zarr.ZArray}}

The function takes a strategy s as an argument. It retrieves the directory path for the strategy's log files and returns the full paths to all log files within this directory.

Prints the content of a specific log file for a given strategy.

print_log(s)
print_log(s, idx)

The function takes a strategy s and an optional index idx (defaulting to the last log file). It retrieves the directory path for the strategy's log files, selects the log file at the specified index, and prints its content.

A progressive search performs multiple grid searches with only 1 repetition per parameters combination.

progsearch(s; sess, rounds, cut, kwargs...)

After each search is completed, the results are filtered according to custom rules. The parameters from the results that match the filtering will be backtested again with a different offset which modifies the backtesting period. rounds: how many iterations (of grid searches) to perform sess: If a Ref{<:OptSession} is provided, search will resume from the session previous results halve: At each iteration

Additional kwargs are forwarded to the grid search.

Remove results that don't have all the repeated evalutaion.

Fetches the named tuple of a single parameters combination.

result_params(
    sess::OptSession
) -> Union{Nothing, NamedTuple}
result_params(
    sess::OptSession,
    idx
) -> Union{Nothing, NamedTuple}

The function takes an optimization session sess and an optional index idx (defaulting to the last row of the results). It returns the parameters of the optimization session at the specified index as a named tuple.

Resumes the optimization session from saved state.

resume!(sess; zi) -> Bool

The function attempts to load a saved session and resumes it. If the saved session does not match the current session in terms of strategy, context, parameters, or attributes, an error is thrown. If the session is successfully resumed, the results from the saved session are appended to the current session's results.

Generates a regular expression for matching optimization session keys.

rgx_key(startstop, params_k, code) -> Regex

The function takes three arguments: startstop, params_k, and code. These represent the start and stop date of the backtesting context, the first letter of every parameter, and a hash of the parameters and attributes truncated to 4 characters, respectively. The function returns a Regex object that matches the string representation of an optimization session key.

Sets the running state of the optimization process to true.

running!() -> Bool

This function changes the value field of the RUNNING instance to true, indicating that the optimization process is currently running.

Save the optimization session over the provided zarr instance

save_session(
    sess::OptSession;
    from,
    to,
    zi
) -> Union{Nothing, Dict}

sess is the OptSession to be saved. The from parameter specifies the starting index for saving optimization results progressively, while to specifies the ending index. The function uses the provided zarr instance zi for storage. The function first ensures that the zgroup for the strategy exists. Then, it writes various session attributes to zarr if we're starting from the beginning (from == 0). Finally, it saves the result data for the specified range (from to to).

Generates a unique key for an optimization session.

session_key(
    sess::OptSession
) -> Tuple{Union{Base.AnnotatedString{String}, String}, NamedTuple{(:s_part, :ctx_part, :params_part, :config_part), <:Tuple{Any, String, Union{Base.AnnotatedString{String}, String}, String}}}

This function generates a unique key for an optimization session by combining various parts of the session's properties. The key is a combination of the session's strategy name, context range, parameters, and a hash of the parameters and attributes.

Override attributes in a strategy with values from a given parameters dictionary.

overrides!(s::AbstractStrategy, params::Dict, pidx::Dict) -> AbstractStrategy

Override attributes in s with values from the params dictionary using the parameter index pidx. This is useful for updating strategy attributes during an optimization run.

Backtests by sliding over the backtesting period, by the smallest timeframe (the strategy timeframe).

slidesearch(s::Strategies.Strategy; multiplier)

Until a full range of timeframes is reached between the strategy timeframe and backtesting context timeframe.

• multiplier: the steps count (total stepps will be multiplier * context_timeframe / s.timeframe )

Sets the running state of the optimization process to false.

stopcall!() -> Bool

This function changes the value field of the RUNNING instance to false, indicating that the optimization process is not currently running.

Get the Opt group from the provided zarr instance.

Returns the zarr group for a given strategy.

zgroup_strategy(
    zi,
    s_name::String
) -> NamedTuple{(:s_group, :opt_group), <:Tuple{Zarr.ZGroup, Union{Zarr.ZArray, Zarr.ZGroup}}}

This function checks if a zarr group exists for the given strategy name in the optimization group of the zarr instance. If it exists, the function returns the group; otherwise, it creates a new zarr group for the strategy.