src.training.dataset

🦉 AthenaAI is an AI-powered ensemble machine-learning trading bot for PocketOption, built on BinaryOptionsToolsV2 → Get it on GitLab
💬 Need help? Join the Chipa Discord · ⚡ Go no-code with ChipaEditor

Source: src/training/dataset.py. Dataset loading and training-sample generation, extracted from the bot so training can run standalone (via train.py) without a PocketOption connection.

load_candles(path: str) -> list[Candle]

Returns a chronological list[Candle] (order as in the file). Opens with encoding="utf-8-sig" (BOM-tolerant). Two formats are auto-detected from the first line:

1. HistData semicolon format — first line contains ; but none of time/open/high/date (case-insensitive): headerless rows YYYYMMDD HHMMSS;open;high;low;close[;volume]. A time string of only 8+ chars is parsed as a date-only YYYYMMDD. Rows with < 5 fields or parse errors are skipped silently.
2. Headered CSV — comma- or semicolon-delimited csv.DictReader. Time column: time, timestamp, date, or Date; parsed as YYYY-MM-DD HH:MM:SS when it contains both - and :, otherwise as a numeric epoch float. OHLC columns accept lower- or capitalized names (open/Open …). Volume falls back through tick_volume → volume → Volume → real_volume → 0.

Timestamps are produced with naive datetime.strptime(...).timestamp(), i.e. interpreted in the local timezone of the machine. Side effects: log lines Loading dataset from … , format detection, and Parsed N candles from file. Failure: FileNotFoundError propagates; malformed rows never raise.

See the data formats guide for examples of both formats.

build_samples(candles, feature_engine, window=20, lookback=200, label_horizon=5) -> tuple[np.ndarray, np.ndarray]

Slides over indices [max(window, 26), len(candles) − label_horizon). For each index i: compute features on candles[i−lookback : i+1] (skip if None), np.nan_to_num them, and label 1 if close[i+horizon] > close[i] (CALL wins), 0 if lower; flat closes are skipped.

Returns (X, y) — X an (n, 57) float array (np.vstack), y int64. If no samples were produced, returns (np.empty((0, 0)), np.empty((0,))).

Parallelism internals

• Worker count: min(os.cpu_count(), 16). Runs serially when workers ≤ 1 or there are fewer than 20,000 candidate indices.
• Otherwise the index range is split into workers contiguous chunks. Each task ships a self-contained candle slice (its range plus lookback context before and label_horizon after) to a ProcessPoolExecutor, so worker processes never share memory. Progress is logged as chunks finish (… N samples generated (k/n chunks done)), and results are reassembled in chronological chunk order regardless of completion order — output is deterministic.
• Because it uses process workers, calling build_samples on a large dataset from your own script requires the standard if __name__ == "__main__": guard on Windows.

_build_range(...) / _build_range_task(args) (internal)

_build_range(candles, start, end, feature_engine, window, lookback, label_horizon) is the serial core returning (list[np.ndarray], list[int]); _build_range_task is the picklable worker entry point that unpacks a task tuple and returns (order, X, y).

Usage example

from src.core.feature_engine import FeatureEngine
from src.training.dataset import load_candles, build_samples

if __name__ == "__main__":
    candles = load_candles("EURUSD_M1.csv")
    X, y = build_samples(candles, FeatureEngine(), window=20,
                         lookback=200, label_horizon=5)   # 5 x 60s = 300s expiry
    print(X.shape, y.mean())   # fraction of CALL-win labels

See also

• trainer.md — the pipeline that calls these functions
• Data formats guide

💬 Unsure how this interacts with the rest of the configuration? Ask in the Chipa Discord — or prototype strategies no-code with ChipaEditor.