Fetch module

Fetch.MAX_ORDERSConstant

Defines an array representing possible numbers of orders to fetch.

Fetch.OB_CACHEConstant

Initializes a safe TTL cache for storing order book data with the default eviction TTL.

Fetch.OB_EVICTION_TTLConstant

Defines the eviction time-to-live (TTL) for an order book as 5 minutes.

Fetch.OB_FUNCTIONSConstant

Initializes a LittleDict for storing order book functions keyed by order book level and exchange ID.

Fetch.OB_TTLConstant

Defines the time-to-live (TTL) for an order book as 5 seconds (after which it is stale).

Fetch.OHLCVTupleTypesConstant

Defines the tuple type for OHLCV data, where each element represents a specific metric (Open, High, Low, Close, Volume).

Fetch.SINCE_MIN_PERIODConstant

Used to slide the since param forward when retrying fetching (in case the requested timestamp is too old).

Base.convertMethod

This is the fastest (afaik) way to convert ccxt lists to dataframe friendly format.

convert(
    _::Type{Tuple{Vector{Dates.DateTime}, Vararg{Vector{Float64}, 5}}},
    py::Py
) -> Tuple{Vector{Dates.DateTime}, Vararg{Vector{Float64}}}

This function converts the provided Python object to a tuple format suitable for dataframes, specifically tailored for OHLCV data.

Data.propagate_ohlcv!Method

Propagates OHLCV data to all timeframes in a data structure.

propagate_ohlcv!(
    data::SortedDict,
    pair::AbstractString,
    exc::Exchange
) -> Union{Nothing, SortedDict{TimeFrames.TimeFrame, DataFrames.DataFrame}}

The propagate_ohlcv! function propagates OHLCV data for a given pair from an exchange exc to all timeframes in the data SortedDict data structure.

Data.to_ohlcvMethod

Converts a Python object to a DataFrame with OHLCV columns

to_ohlcv(py::Py)
Fetch.__ensure_datesMethod

Ensures dates are within valid range for the exchange and timeframe.

__ensure_dates(exc, tf, from, to) -> Tuple{Any, Any}

The __ensure_dates function checks if the dates from and to are within the valid range for the given exchange exc and timeframe tf. If the dates are not within the valid range, the function adjusts them accordingly.

Fetch.__from_date_funcMethod

Determines the starting date for fetching data.

__from_date_func(
    update,
    timeframe,
    from,
    to,
    zi,
    exc_name,
    reset
) -> Union{Returns{<:Tuple{Nothing, Any}}, Fetch.var"#19#20"}

The __from_date_func function determines the starting date from for fetching data based on various parameters. If update is true, it will fetch data from the latest date available. If reset is true, it will fetch data from the earliest date possible. The function also considers the timeframe, to date, timezone zi, and exchange name exc_name in its calculations.

Fetch.__get_ohlcvMethod

Fetches OHLCV data for a specified exchange within a date range.

__get_ohlcv(
    exc,
    name,
    timeframe,
    from_date,
    to;
    out,
    cleanup,
    ohlcv_kind
) -> Tuple{Any, Any}

This function fetches OHLCV data for a given exchange exc, with the specified name and timeframe, within the date range specified by from_date and to. The fetched data is appended to the out data structure. The cleanup parameter determines if any post-processing should be done on the data before returning. The ohlcv_kind parameter determines the type of OHLCV data to fetch.

Fetch.__get_sinceMethod

Determines the 'since' parameter for fetching data from an exchange.

__get_since(
    exc,
    fetch_func,
    pair,
    limit,
    from,
    out,
    is_df,
    converter
) -> Any

This function calculates the 'since' parameter based on the specified 'from' timestamp, or finds the appropriate 'since' value if 'from' is 0.0.

Fetch.__handle_errorMethod

Handles errors during fetch operations.

__handle_error(
    e,
    fetch_func,
    pair,
    since,
    df,
    sleep_t,
    limit,
    converter,
    retry
) -> Any

This function takes an error e occurred during data fetching, and decides whether to retry the fetch_func based on the retry flag. If retry is true, it calls the fetch_func again with the same parameters.

Fetch.__handle_fetchMethod

Handles fetch operations for specified exchange and pair.

__handle_fetch(
    fetch_func,
    pair,
    since,
    limit,
    sleep_t,
    df,
    converter,
    retry,
    usetimeframe
) -> Tuple{Bool, Any}

This function calls the fetch_func for a given pair, starting from the since timestamp with a maximum limit of limit data points. It employs a delay sleep_t between fetches. The function also applies a given converter to the fetched data. If the retry flag is true, the function will try to fetch data again in case of an empty response. The usetimeframe flag indicates whether to use timeframe for fetching.

Fetch.__ordered_timeframesMethod

Returns an ordered list of timeframes for a given exchange

__ordered_timeframes(exc::Exchange) -> Tuple{Any, Any}

This function collects the timeframes from the exchange, converts them into periods, and sorts them in descending order. It then returns these sorted timeframes and periods.

Fetch.__pairdata!Method

Processes OHLCV data for a pair.

__pairdata!(
    zi,
    data,
    ohlcv,
    name,
    timeframe,
    z,
    exc_name,
    reset
) -> Data.PairData

The __pairdata! function processes the OHLCV data ohlcv for a pair name over a timeframe. It takes into account the timezone zi, data data, timezone offset z, exchange name exc_name, and a reset flag. If reset is true, it will reset the data for the pair before processing.

Fetch._cleanup_funding_historyMethod

Cleans up fetched funding history data.

_cleanup_funding_history(
    df,
    name,
    half_tf,
    f_tf
) -> Union{DataFrames.DataFrame, DataFrames.GroupedDataFrame, DataFrames.SubDataFrame}

The _cleanup_funding_history function takes a DataFrame df of fetched funding history data for a name and performs cleanup operations on it. The half_tf and f_tf parameters are used in the cleanup process.

Fetch._fetch_loopMethod

Iteratively fetches data over a specified date range.

_fetch_loop(
    fetch_func::Function,
    exc::Exchange,
    pair;
    from,
    to,
    sleep_t,
    out,
    converter,
    limit
)

This function calls the fetch_func function repeatedly until it has fetched data for the entire date range specified by from and to. Note: The total data points fetched may not match the expected number based on the date range.

Fetch._fetch_ohlcv_from_toMethod

Ensure a to date is set, before fetching.

_fetch_ohlcv_from_to(
    exc::Exchange,
    pair,
    timeframe;
    from,
    to,
    params,
    sleep_t,
    cleanup,
    out,
    ohlcv_kind
) -> Any

This function verifies that a 'to' date is set before attempting to fetch OHLCV data.

Fetch._fetch_ohlcv_with_delayMethod

Fetches OHLCV data with delay for a given exchange and arguments.

_fetch_ohlcv_with_delay(
    exc::Exchange,
    args...;
    ohlcv_kind,
    kwargs...
) -> Any

This function fetches OHLCV data for a specified exchange exc and additional args. The type of OHLCV data to fetch is determined by ohlcv_kind. It applies a delay between fetches as specified in kwargs.

Fetch._fetch_with_delayMethod

Wraps fetching function with error handling and backoff delay.

_fetch_with_delay(
    fetch_func::Function,
    pair;
    since,
    df,
    sleep_t,
    limit,
    converter,
    retry,
    usetimeframe
) -> Any

This function wraps a fetching function fetch_func with error handling and a backoff delay sleep_t. The fetch_func takes three parameters: pair, since, and limit, and returns a PyList. The converter function is used to tabulate the data such that the first column is the timestamp. The function will retry fetching in case of an error if retry is set to true.

Fetch._levelnameMethod

Returns the name of an order book level.

_levelname(level) -> String

The _levelname function takes an order book level and returns its name.

Fetch._orderbookMethod

Generates an order book of depth N.

_orderbook(
    N
) -> @NamedTuple{busy::Ref{Bool}, timestamp::Ref{Dates.DateTime}, asks::Vector{Tuple{Float64, Float64}}, bids::Vector{Tuple{Float64, Float64}}}

The _orderbook function generates an order book of depth N. The order book contains N levels of bid and ask prices along with their corresponding quantities.

Fetch._since_timestampMethod

Determines the start time for fetching data

_since_timestamp(
    actual::Dates.DateTime,
    p::Dates.Period
) -> Int64

This function calculates the timestamp from which to start fetching data. It ensures that the start time is not more than 20 years in the past or less than the given period.

Fetch._update_orderbook!Method

Updates an order book in place with new data.

_update_orderbook!(exc, ob, sym, lvl, limit; init)

The _update_orderbook! function takes an exchange exc, an order book ob, a symbol sym, an order book level lvl, and a limit limit, and updates the order book in place with new data. If init is set, the function will initialize the order book before updating it.

Fetch.extract_futures_dataMethod

Extracts futures data from a Python object.

extract_futures_data(data::Py) -> DataFrames.DataFrame

The extract_futures_data function takes futures data data from a Python object and extracts it into a format suitable for further processing or analysis.

Fetch.fetch_candlesMethod

Fetches candlestick data for a list of pairs from an exchange.

fetch_candles(
    exc::Exchange,
    timeframe::AbstractString,
    pairs::Union{AbstractString, AbstractSet, AbstractVector, Tuple{Vararg{T}} where T};
    from,
    to,
    ohlcv_kind
) -> Any

The fetch_candles function fetches candlestick data from a given exchange exc for a list of pairs over a specified timeframe. The from and to parameters define the date range for the fetched data. If from is not provided, it defaults to an empty string, which implies fetching data from the earliest available date. The type of candlestick data to fetch is determined by the ohlcv_kind parameter.

Fetch.fetch_limitMethod

Defines the fetch limit for an exchange.

fetch_limit(
    exc::Exchange,
    limit::Union{Nothing, Int64}
) -> Union{Nothing, Int64}

This function fetches the limit for an exchange. If no limit is specified, it retrieves the default limit for the exchange.

Fetch.fetch_ohlcvMethod

Fetches OHLCV data from an exchange for a list of pairs.

fetch_ohlcv(
    exc::Exchange,
    timeframe::String,
    pairs::Union{AbstractSet{T}, AbstractVector{T}, Tuple{Vararg{T}}} where T;
    zi,
    from,
    to,
    update,
    reset,
    progress,
    ohlcv_kind
) -> Dict{String, Data.PairData}

This function fetches OHLCV data from a given exchange exc for a list of pairs over a specified timeframe. The from and to parameters can represent dates or, if from is a negative number, the function fetches the last N=from candles. If update is true, the function checks for cached data and only fetches missing candles. If reset is true, the function removes cached data before fetching. The progress parameter determines whether a progress bar is shown. The type of OHLCV data to fetch is defined by the ohlcv_kind parameter.

Fetch.fetch_ohlcvMethod

Prompts user for confirmation before fetching OHLCV data.

fetch_ohlcv(::Val{:ask}, args...; kwargs...) -> Any

This function prompts the user for confirmation before fetching OHLCV data for the specified arguments args and keyword arguments kwargs. If the user inputs 'Y', 'y', or simply presses Enter, it proceeds with the fetch_ohlcv function. If any other input is given, the function returns nothing.

Fetch.fetch_ohlcvMethod

Fetches OHLCV data for multiple exchanges on the same timeframe.

fetch_ohlcv(
    excs::Vector{Exchange},
    timeframe;
    account,
    sandbox,
    parallel,
    wait_task,
    kwargs...
) -> Union{Nothing, Task}

This function fetches OHLCV data for multiple exchanges over the same timeframe. It accepts:

  • A vector of exchange instances excs.
  • The desired timeframe timeframe.

The function can run in parallel if parallel is set to true. If wait_task is set to true, the function will wait for all tasks to complete before returning.

You can provide additional parameters using kwargs.

Fetch.find_sinceMethod

Returns the oldest possible timestamp for a pair.

find_since(exc::Exchange, pair) -> Any

This function iterates over the timeframes and periods of the exchange to find the oldest available timestamp for a given pair. If no data is found in any timeframe, it defaults to 1 day in the past.

Fetch.funding_dataMethod

Retrieves all or a subset of funding data for a symbol from an exchange.

funding_data(exc::Exchange, sym::AbstractString) -> Any

The funding_data function retrieves all funding data returned by an exchange exc for a symbol sym.

Fetch.funding_historyMethod

Fetches funding rate history from an exchange for a list of Derivative pairs.

funding_history(
    exc::Exchange,
    assets::Vector;
    from,
    to,
    params,
    sleep_t,
    limit,
    cleanup
) -> Dict

The funding_history function fetches funding rate history from a given exchange exc for a list of assets. The from and to parameters define the date range for which to fetch the funding rate history. Additional parameters can be specified through the params dictionary. The function will wait for sleep_t seconds between each request to the exchange. The limit parameter can be used to limit the amount of data fetched. If cleanup is set to true, the function will perform a cleanup on the fetched data before returning it.

Fetch.funding_rateMethod

Retrieves the funding rate for a symbol from an exchange.

funding_rate(exc::Exchange, s::AbstractString) -> Any

The funding_rate function retrieves the funding rate for a symbol s from an exchange exc.

Fetch.ohlcv_func_bykindMethod

Returns the appropriate OHLCV fetching function based on the specified kind.

ohlcv_func_bykind(exc, kind) -> Any

The ohlcv_func_bykind function determines and returns the appropriate OHLCV fetching function for the given exchange exc and kind.

Fetch.orderbookMethod

Fetches an order book from an exchange for a symbol.

orderbook(exc, sym; limit, level) -> Any

The orderbook function fetches an order book from an exchange exc for a symbol sym. The limit parameter can be used to limit the depth of the order book. The level parameter specifies the level of the order book to fetch.

Fetch.parse_funding_rowMethod

Parses a row of funding data from a Python object.

parse_funding_row(r::Py) -> Tuple{Int64, String, Float64}

The parse_funding_row function takes a row of funding data r from a Python object and parses it into a format suitable for further processing or analysis.

Fetch.update_ohlcv!Method

Updates the tail of an OHLCV DataFrame with the most recent candles.

update_ohlcv!(
    df::DataFrames.DataFrame,
    pair,
    exc,
    tf;
    ohlcv_kind,
    from
) -> DataFrames.DataFrame

The update_ohlcv! function updates the tail of an OHLCV DataFrame df with the most recent candles for a given pair from an exchange exc over a timeframe tf. The type of OHLCV data to update is determined by the ohlcv_kind parameter.