From 1c3ad7043183327f18f25953b4a792caca024f00 Mon Sep 17 00:00:00 2001 From: Joe Jevnik Date: Mon, 13 Jun 2016 18:26:08 -0400 Subject: [PATCH] DOC: docs for checkpoints --- docs/source/whatsnew/1.0.2.txt | 23 +++++++++++++++++++++++ zipline/pipeline/loaders/blaze/core.py | 15 ++++++++++++++- 2 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 docs/source/whatsnew/1.0.2.txt diff --git a/docs/source/whatsnew/1.0.2.txt b/docs/source/whatsnew/1.0.2.txt new file mode 100644 index 00000000..348dc076 --- /dev/null +++ b/docs/source/whatsnew/1.0.2.txt @@ -0,0 +1,23 @@ +Release 1.0.2 +------------- + +:Release: 1.0.2 +:Date: TBD + +Enhancements +~~~~~~~~~~~~ + +- Adds forward fill checkpoint tables for the blaze core loader. This allow the + loader to more efficiently forward fill the data by capping the lower date it + must search for when querying data. The checkpoints should have novel deltas + applied (:issue:`1276`). + +Bug Fixes +~~~~~~~~~ + +None + +Documentation +~~~~~~~~~~~~~ + +None diff --git a/zipline/pipeline/loaders/blaze/core.py b/zipline/pipeline/loaders/blaze/core.py index 4b4d555b..a23d5b70 100644 --- a/zipline/pipeline/loaders/blaze/core.py +++ b/zipline/pipeline/loaders/blaze/core.py @@ -1,4 +1,5 @@ -"""Blaze integration with the Pipeline API. +""" +Blaze integration with the Pipeline API. For an overview of the blaze project, see blaze.pydata.org @@ -81,6 +82,18 @@ actually 3. By pulling our data into these two tables and not silently updating our original table we can run our pipelines using the information we would have had on that day, and we can prevent lookahead bias in the pipelines. + +Another optional expression that may be provided is ``checkpoints``. The +``checkpoints`` expression is used when doing a forward fill query to cap the +lower date that must be searched. This expression has the same shape as the +``baseline`` and ``deltas`` expressions but should be downsampled with novel +deltas applied. For example, imagine we had one data point per asset per day +for some dataset. We could dramatically speed up our queries by pre populating +a downsampled version which has the most recently known value at the start of +each month. Then, when we query, we only must look back at most one month +before the start of the pipeline query to provide enough data to forward fill +correctly. + Conversion from Blaze to the Pipeline API -----------------------------------------