# 08 - Historify

### Overview

Historify is OpenAlgo's historical market data manager built on DuckDB. It downloads OHLCV data from brokers and stores it locally for backtesting and analysis.

### Architecture

<figure><img src="https://17901342-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FmBwEhITzgv0O0fEGIIRN%2Fuploads%2FRyM7hL4fQL6dxj5kHVLF%2Fimage.png?alt=media&#x26;token=790c3f27-438d-4f8b-af91-aa71f7126b5d" alt=""><figcaption></figcaption></figure>

### Database (DuckDB)

**Location:** `db/historify.duckdb`

| Table             | Purpose                                                          |
| ----------------- | ---------------------------------------------------------------- |
| `market_data`     | OHLCV candles (symbol, exchange, interval, timestamp, OHLCV, oi) |
| `watchlist`       | Symbols to track                                                 |
| `download_jobs`   | Bulk download job tracking                                       |
| `job_items`       | Individual symbol status within jobs                             |
| `symbol_metadata` | Enriched symbol info (expiry, strike, lotsize)                   |

### Intervals

| Storage (Downloaded) | Computed (Aggregated from 1m) |
| -------------------- | ----------------------------- |
| `1m`, `D`            | `5m`, `15m`, `30m`, `1h`      |

Only 1-minute and Daily data are stored. Other timeframes are computed on-the-fly from 1-minute data.

### Key Features

* **Watchlist**: Track symbols for batch downloads
* **Bulk Download Jobs**: Download entire option chains with progress tracking
* **Pause/Resume/Cancel**: Job control with Socket.IO progress updates
* **Incremental Download**: Only fetch data after last available timestamp
* **CSV/Parquet Import/Export**: Data portability
* **FNO Discovery**: Find underlyings, expiries, and option chains

### Key Files

| File                               | Purpose                           |
| ---------------------------------- | --------------------------------- |
| `database/historify_db.py`         | DuckDB schema and queries         |
| `services/historify_service.py`    | Business logic and job processing |
| `blueprints/historify.py`          | Web UI routes                     |
| `frontend/src/pages/Historify.tsx` | React UI                          |

### Supported Exchanges

`NSE`, `BSE`, `NFO`, `BFO`, `CDS`, `MCX`