World markets: method

Code: pipeline/tipsheet/compute/world.py (build) and publish/world.py (bundles). Tests: pipeline/tests/test_world.py. Data bank adapters: src/databank/connectors/kite_global.py and ecb_exr.py in the Data bank.

The licence rule

Kite Connect data is licensed to the account holder for personal use (PLAN.md, decision 2), and index levels belong to their providers (S&P, Dow Jones, Nasdaq, FTSE Russell, Deutsche Börse, Euronext, Nikkei, Hang Seng Indexes, SSE, ASX). So this section publishes only derived statistics:

• percentage returns over fixed horizons, in local currency and in rupees;
• calendar-year returns;
• rolling correlations and betas of weekly returns;
• drawdown from peak, in percent.

No index level, price, quote or rebased growth curve is published for any index here. A growth-of-100 curve is a level series in disguise, so there is none. Every bundle repeats the rule in meta.licence_rule, and the licence gate (publish/licence_gate.py) refuses any world/ column whose name suggests a level, close, price or rebased value. The Data bank contract for the Kite dataset carries licence.redistribution: internal_only.

What is measured

For Nifty 50 and 11 overseas indices (S&P 500, Dow Jones Industrial Average, Nasdaq Composite, Nasdaq-100, FTSE 100, DAX, CAC 40, Nikkei 225, Hang Seng, Shanghai Composite, S&P/ASX 200):

• Return grid (world/returns_grid): 1 week, 1 month, 3 months, year to date, 1 year, and 3 and 5 years annualised, in local currency and in rupees. Plus the current drawdown, the worst drawdown since 2004 and its date, and the date of the last peak. Each row says whether its latest data is official or a provider_quote (see the break below), and whether it is stale.
• Calendar years (world/calendar_years): every year from 2005, local and rupee terms. The current year runs to ytd_through.
• Correlation and beta (world/correlation_weekly, world/beta_weekly): Nifty’s correlation with each market over the last 52 weeks of Friday-ending weekly returns in local currency (at least 40 weeks with both trading), and Nifty’s beta to each market (covariance over the market’s variance: the % Nifty moved per 1% move in that market). Weekly from 2005.
• Drawdown (world/drawdown_weekly): % below the highest close since 2004-01-01, weekly. A common start keeps the markets comparable; Nasdaq and Nifty histories reach back to 1990 but their peaks are searched from 2004 like everyone else’s.
• FX and commodities (world/fx_commodities, world/fx_commodities_calendar_years): % moves over the same horizons for the dollar, euro, pound and yen in rupees, the ICE dollar index, the Fed broad dollar index, gold in rupees and in dollars, and Brent in dollars and rupees.
• Stat bundles (world/stats/*): Nifty’s 1-year correlation with the S&P 500, and its median correlation across the 11 markets.
• Global proxies (world/proxies_grid, world/proxies_calendar_years, world/proxies_correlation_weekly): 17 more markets and asset classes measured through Indian international funds’ daily NAVs. See the section below.

Sources

Where an official public series exists it replaces Kite: the S&P 500 uses FRED from 2016-05-31 and Kite before (Kite equals the official close on all 2,125 overlapping days before the break), and both Nasdaq indices use Nasdaq’s own history throughout. The other eight overseas indices are Kite only.

Nifty 50 is the price index, not the total-return index, so it compares like for like with the overseas price indices. The DAX is the exception among those: it is a total-return index by construction, so its returns include dividends.

Rupee terms. Rupee return = (1 + local return) × (1 + change in rupees per unit of the local currency) − 1. Rupees per unit come from ECB reference rates: (INR per euro) ÷ (currency per euro), taking the rate on or before each close, within 4 days. ECB-derived USD/INR agrees with CCIL’s: median gap 0.04% over 5,136 days, 99th percentile 0.56%. The ECB fixing is at 14:15 CET, so for Asian and US closes the FX rate is a few hours off the close. That matters only for 1-week figures and only at the margin.

Kite: access, storage and cleaning

Access. The Data bank adapter reads the Kite session from the Aftermarket Report’s .env (pointed to by KITE_ENV_FILE in the Data bank .env), so both projects share one set of tokens. The daily access token expires around 06:00 IST. Before each fetch the adapter checks it and, if expired, renews it from the stored long-lived refresh token and writes the new token back. This runs unattended. Verified on 2026-10-01: the stored token had expired and was renewed without a login. If the refresh token is ever revoked, renewal fails, only this dataset fails in the 06:00 Data bank run, the warehouse keeps the previous snapshot, and this section keeps publishing from it with stale flags set. Fix: run amr login once in the Aftermarket Report.

Storage. Data bank dataset kite_global_index_history: raw JSON artifacts, contract, audit, a daily update policy (10-day look-back, ending yesterday because today’s candle is still forming), and the 2004 backfill. The authorization header is redacted from stored metadata and is not part of any fingerprint.

Cleaning (counts from the 2026-10-01 build, 59,000 rows):

1. Weekend rows dropped (918). None of these markets trades on a weekend. Most are US30’s forward-filled weekends from 2004-12; others are stray quotes, such as JAPAN225 Sundays in 2017 and a HANGSENG Sunday print 26% above the Friday close in 2008.
2. Rows on exchange holidays dropped, using each exchange’s own session calendar (exchange_calendars: NYSE, LSE, Xetra, Euronext Paris, Tokyo, HKEX, Shanghai, ASX). Before the break this removes only real closures (Hong Kong typhoon days, Whit Monday on Xetra). After it, it removes holiday quotes, 1 to 42 per instrument.
3. Forward-filled bars dropped (open = high = low = close = the previous close).
4. Bars dated on or after the day they were fetched (IST) dropped: the candle is still forming.

Two source bars whose high-low range excluded the close (UK100 2011-08-05, USCOMPOSITE 2023-08-04) keep their close; the Data bank nulls their range and flags range_nulled.

The series break

Kite’s GLOBAL series changed character on 2024-11-07, the day US100 and US10YRYIELD first appear. The date is found from the data: the first run of five days on which Kite’s US500 departs from the official S&P 500 by more than 0.01%. KITE_AUDIT.md put the change at 2025-03; the data shows it starts in November 2024.

Since then the values are provider quotes, at times CFD prices on a fixed tick grid, not official closes. Measured where an official close is public:

Before the break, US500 matches the official close on 100% of 2,125 days and USCOMPOSITE on 97.4% of 5,246 days.

The three US series above use the official closes, so this error does not reach them. For the eight Kite-only markets (Dow, FTSE 100, DAX, CAC 40, Nikkei, Hang Seng, Shanghai, ASX 200), every current figure carries an error of about this size. Their rows are marked basis: provider_quote. The share of closes on a 0.25 tick grid shows where the quotes are clearly CFD-like: US500 59% of the time since the break against 9.5% before, Hang Seng 41% against 4%, Nasdaq-100 always, and the DAX in some recent months. The other markets look like official closes on this test, but the Nikkei check below shows they can still drift.

Checks

Calendar-year returns against the providers’ published figures (price returns; the DAX is total return):

2023 matches exactly everywhere. In 2024 the year-end close falls after the break: the Dow is 0.07 pp off, the Nikkei 0.17 pp and Shanghai 0.66 pp, because Kite’s last 2024 bar is a provider quote (Shanghai 3,371.56 against the official 3,351.76; Nikkei 39,951.90 against 39,894.54). The others match. These comparisons run on every build and are stored in world/returns_grid under meta.checks.

Other checks: ECB-derived USD/INR against CCIL (above); the pipeline-wide licence gate; strict JSON (every bundle is serialised with NaN and infinity forbidden).

Caveats

• Post-break figures for the eight Kite-only markets carry the provider-quote error above. Treat 1-week moves under about half a percentage point as noise.
• Weekly correlations: US markets close after India each day. Weekly returns absorb most of that lag, but part of a US week’s move lands in Nifty’s next week, which pulls measured correlation down a little.
• Rupee-term returns use a 14:15 CET exchange rate, not a rate at each market’s close.
• Calendar years start in 2005 because Kite history starts in 2004. Drawdowns are measured from 2004, so a market still below a pre-2004 peak (the Nasdaq after 2000, the Nikkei after 1989) is shown against its post-2004 high.
• Gold in rupees is the domestic price including import duty; gold in dollars is the LBMA price.
• The Shanghai and Hang Seng figures are in yuan and Hong Kong dollars; the Hong Kong dollar is pegged to the US dollar.

Global proxies: Indian international funds’ NAVs

Added 2026-10-01. Code: build_proxies and its helpers in compute/world.py, _publish_proxies in publish/world.py.

Why fund NAVs, not ETF quotes

Kite has no overseas ETFs. The instrument dump of 2026-10-01 (109,396 instruments) has 12 rows on the GLOBAL exchange, all index CFDs already used above (US500 to US10YRYIELD), plus GIFT NIFTY on NSEIX. There are no US-, Hong Kong- or Europe-listed ETFs and no overseas bonds or commodities ETFs. Kite does carry the six Indian-listed international ETFs on NSE (MON100, MONQ50, MAFANG, MASPTOP50, MAHKTECH, HNGSNGBEES). Their exchange quotes are not market proxies, because the industry-wide overseas investment limit has pushed them far above NAV. From the ETF desk’s data (bhavcopy close against AMFI NAV), the median premiums in 2026 are 17.5% for MON100, 19.8% for MAFANG, 19.6% for MASPTOP50, 20.0% for MAHKTECH and 13.3% for HNGSNGBEES, with peaks of 26% to 57%. So basis: etf_quote is reserved and unused, and every proxy is basis: fund_nav.

AMFI NAVs are public. AMFI publishes every scheme’s daily NAV openly, so returns computed from them can be published. Only percentage returns, calendar-year returns, drawdown in percent and correlations are published; no NAV, index level or rebased curve. The indices used to check the proxies (FRED, Nasdaq, and Kite’s Hang Seng and Nikkei) are internal and only their return gaps appear.

The proxies

One scheme per market. Each is the direct growth plan, or the ETF itself for ETFs, which have a single plan. Where several funds cover a market, the choice favours a passive fund and the longest clean history. Source: Data bank amfi_nav_history (all eras).

Chained codes. Hang Seng BeES moved from Benchmark to Goldman Sachs (2011) and then to Reliance and Nippon (2016), and the JPMorgan offshore funds became Edelweiss’s in November 2016. Each move opened a new AMFI code for the same scheme. Codes are joined only when the gap is at most 7 days and the NAV moves less than 10% across the junction. All five junctions pass, and each move is close to the reference market’s move over the same days (Hang Seng BeES −2.77% against the Hang Seng’s −2.87% in rupees, and +0.46% against +0.53%).

Cleaning. Weekend rows are dropped. Clean 1:k unit splits are undone: MON100 1:10 on 2021-06-21 and Hang Seng BeES 1:10 on 2019-12-23. Isolated bad prints, meaning a step of more than 15% that reverts within five days, are dropped: two days in DSP World Gold Mining in March 2020. The only gaps longer than a week are Lunar New Year closures in the Taiwan and Greater China funds, of 10 to 12 days.

NAV timing

An Indian fund’s NAV for day T is struck on either the same day’s overseas closes or the previous session’s, and funds have switched between the two. The pipeline measures this rather than assuming it. Quarter by quarter, it correlates the fund’s daily NAV returns with a reference market’s returns on the same session and on the previous one. The reference is the matching index, gold for the gold miners, or the FTSE 100 for the miners. A quarter counts as decided when the better lag correlates at least 0.40 and beats the other by 0.15. A timing regime needs two consecutive decided quarters, and each switch day is placed by a changepoint search. Every NAV is then re-dated to the overseas session it prices.

What the data shows:

• Motilal Oswal’s two US funds priced the previous US session until 2023-06-16 and the same session from 2023-06-19. Both switch on the same day, found independently.
• The HDFC Developed World FoF priced the previous session until 2025-04 and the same session since.
• Every other fund prices the same session throughout: the Asian and European funds, Brazil, global REITs, the miners and Mirae’s FANG+ ETF.

Without re-dating, the S&P 500 fund’s daily correlation with the index is 0.36, and its monthly tracking error is 4.3%. Re-dated, they become 0.997 (monthly) and 1.15%.

The US Treasuries fund has no usable reference, so its timing is unresolved (timing_resolved: false, nav_timing: unknown) and its NAV dates are kept. Its daily moves are small (the largest is 1.5%), so a one-day offset barely matters. A switch that shows up in only the latest quarter is ignored until a second quarter confirms it.

Validation against the index, in rupees

Each index is converted to rupees with ECB reference rates. Each fund’s re-dated NAV is then compared with it. The indices are price returns while the NAVs include dividends, so a passive fund should run ahead by the dividend yield, net of withholding tax and costs.

The passive proxies track their indices closely. The Japan fund is a proxy for Japanese equities only in a loose sense. It is active and differs widely from the price-weighted Nikkei, so read it as “a diversified Japan fund”, not “the Nikkei”. The other active proxies have no public index to check against in the warehouse, and the same caution applies to them. These numbers are rebuilt on every run and stored in world/proxies_grid under meta.checks, along with each fund’s timing regimes and continuity record.

What the bundles hold

• world/proxies_grid: one row per proxy. Returns over 1W, 1M, 3M, YTD, 1Y, 3Y and 5Y (3Y and 5Y annualised) in rupees, which is the fund’s own basis, and in local terms. Local terms remove the rupee’s move: they use the market’s own currency for single-market funds and US dollars for multi-market funds (local_basis: usd), and are blank for Brazil and Taiwan, because no public real or Taiwan-dollar reference rate is held. The row also carries the current and worst drawdown in rupees since the proxy’s start, and the latest 1-year correlation with Nifty 50.
• world/proxies_calendar_years: calendar-year returns from 2011, in rupees and local terms. A year needs a value in the last 10 days of the prior year.
• world/proxies_correlation_weekly: Nifty 50’s rolling 52-week correlation with each proxy, using Friday-ending weekly returns with both in rupees and at least 40 weeks with both.

Caveats

• NAV returns are total returns after costs and in rupees. They are not index price returns. A passive fund runs ahead of a price index by its dividend yield (about 2.5 pp a year for Hang Seng BeES), less the expense ratio.
• Active funds are their managers’ portfolios. Japan, Europe, Taiwan, Greater China, ASEAN, Brazil, emerging markets, global REITs and the two mining funds can differ from their market by several points a year.
• Correlations here are in rupees on both sides. Unlike world/correlation_weekly, which uses local currency, they include the rupee’s move against the fund’s currencies.
• FX timing. The fund’s rupee value uses the day’s rupee rate even when the overseas session is the previous day, so daily figures carry some FX timing noise. Weekly and longer figures are little affected.
• Inflow suspensions. Several overseas funds have at times stopped taking new money because of the overseas investment limit. Their NAVs are still struck and remain valid for returns.
• Overseas NAVs arrive a day late. AMFI publishes them the next morning, and the Data bank re-fetches them. A proxy’s as_of can therefore trail the domestic funds by a day; stale turns on after 5 days.