wassname/ETSformer
1
0
Fork 0
mirror of https://github.com/wassname/ETSformer.git synced 2026-07-28 11:15:06 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
multiinput
ETSformer/notebooks/001_001_run_MS2S.ipynb
T
wassname 7c22ba2c1e misc
2022-11-29 06:07:44 +08:00

65 KiB
Raw Permalink Blame History

In [1]:
import os
os.sys.path.append('..')
In [2]:
import warnings
warnings.simplefilter("ignore")

# autoreload import your package
%load_ext autoreload
%autoreload 2
In [3]:
from loguru import logger
logger.remove()
logger.add(os.sys.stdout, level="ERROR", colorize=True, format="<level>{time} | {message}</level>")
# import_dir(ta_dir, verbose=False)
warnings.simplefilter("ignore")
In [4]:
import torch
import pandas as pd
from matplotlib import pyplot as plt
%matplotlib inline

Args

In [5]:
from run import set_seed, get_args, Exp_Main
In [6]:
# mimic cli args to avoid code duplication
argv = """python -u run.py \
  --root_path ../dataset/stocks/ \
  --data_path OXY_2019.csv.gz \
  --checkpoints ./checkpoints/ \
  --model_id Exchange \
  --model ETSformer \
  --data custom \
  --features M2S \
  --seq_len 256 \
  --pred_len 12 \
  --e_layers 3 \
  --d_layers 3 \
  --enc_in 12 \
  --d_model 512 \
  --d_ff 2048 \
  --n_heads 16 \
  --dropout 0.2 \
  --c_out 1 \
  --num_workers 0 \
  --des 'Exp' \
  --K 0 \
  --learning_rate 1e-3 \
  --target RSMKs_18_144_72_2ref_2ref \
  --itr 1
"""
argv = argv.replace("\\n", "").split()[3:]
args = get_args(argv)
args


Exp = Exp_Main
ii=0
set_seed(ii)
# setting record of experiments
setting = '{}_{}_{}_ft{}_sl{}_pl{}_dm{}_nh{}_el{}_dl{}_df{}_K{}_lr{}_{}_{}'.format(
    args.model_id,
    args.model,
    args.data,
    args.features,
    args.seq_len,
    args.pred_len,
    args.d_model,
    args.n_heads,
    args.e_layers,
    args.d_layers,
    args.d_ff,
    args.K,
    args.learning_rate,
    args.des, ii)

#     if os.path.exists(os.path.join(args.checkpoints, setting)):
#         print('skipping exists')
#         continue

exp = Exp(args)  # set experiments
print('>>>>>>>start training : {}>>>>>>>>>>>>>>>>>>>>>>>>>>'.format(setting))
exp.train(setting)

print('>>>>>>>testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
exp.test(setting, data='val')
exp.test(setting, data='test')

torch.cuda.empty_cache()

Args in experiment:
Namespace(K=0, activation='sigmoid', batch_size=32, c_out=1, checkpoints='./checkpoints/', d_ff=2048, d_layers=3, d_model=512, damping_learning_rate=0, data='custom', data_path='OXY_2019.csv.gz', dec_in=7, des="'Exp'", devices='0,1,2,3', dropout=0.2, e_layers=3, embed='timeF', enc_in=12, features='M2S', freq='h', gpu=0, itr=1, label_len=0, learning_rate=0.001, lradj='exponential_with_warmup', min_lr=1e-30, model='ETSformer', model_id='Exchange', n_heads=16, num_workers=0, optim='adam', output_attention=False, patience=5, pred_len=12, root_path='../dataset/stocks/', seq_len=256, smoothing_learning_rate=0, std=0.2, target='RSMKs_18_144_72_2ref_2ref', train_epochs=15, use_gpu=True, use_multi_gpu=False, warmup_epochs=3)
Use GPU: cuda:0
>>>>>>>start training : Exchange_ETSformer_custom_ftM2S_sl256_pl12_dm512_nh16_el3_dl3_df2048_K0_lr0.001_'Exp'_0>>>>>>>>>>>>>>>>>>>>>>>>>>
train 37111
val 5330
test 10668

  0%|          | 0/15 [00:00<?, ?epoch/s]
train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 1, Steps: 1159 | Train Loss: 0.1754474 Vali Loss: 0.0231598 Test Loss: 0.0334731
Validation loss decreased (inf --> 0.023160).  Saving model ...
Updating learning rate to 0.00025

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 2, Steps: 1159 | Train Loss: 0.1607344 Vali Loss: 0.0065583 Test Loss: 0.0085996
Validation loss decreased (0.023160 --> 0.006558).  Saving model ...
Updating learning rate to 0.0005

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 3, Steps: 1159 | Train Loss: 0.1427123 Vali Loss: 0.0055523 Test Loss: 0.0080176
Validation loss decreased (0.006558 --> 0.005552).  Saving model ...
Updating learning rate to 0.00075

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 4, Steps: 1159 | Train Loss: 0.1279594 Vali Loss: 0.0096495 Test Loss: 0.0123530
EarlyStopping counter: 1 out of 5
Updating learning rate to 0.001

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 5, Steps: 1159 | Train Loss: 0.1277149 Vali Loss: 0.0093008 Test Loss: 0.0128422
EarlyStopping counter: 2 out of 5
Updating learning rate to 0.0005

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 6, Steps: 1159 | Train Loss: 0.1275677 Vali Loss: 0.0053658 Test Loss: 0.0075932
Validation loss decreased (0.005552 --> 0.005366).  Saving model ...
Updating learning rate to 0.00025

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 7, Steps: 1159 | Train Loss: 0.1138183 Vali Loss: 0.0042887 Test Loss: 0.0057305
Validation loss decreased (0.005366 --> 0.004289).  Saving model ...
Updating learning rate to 0.000125

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 8, Steps: 1159 | Train Loss: 0.1130605 Vali Loss: 0.0043534 Test Loss: 0.0060191
EarlyStopping counter: 1 out of 5
Updating learning rate to 6.25e-05

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 9, Steps: 1159 | Train Loss: 0.1090457 Vali Loss: 0.0035517 Test Loss: 0.0049039
Validation loss decreased (0.004289 --> 0.003552).  Saving model ...
Updating learning rate to 3.125e-05

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 10, Steps: 1159 | Train Loss: 0.1034120 Vali Loss: 0.0035072 Test Loss: 0.0049077
Validation loss decreased (0.003552 --> 0.003507).  Saving model ...
Updating learning rate to 1.5625e-05

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 11, Steps: 1159 | Train Loss: 0.1109674 Vali Loss: 0.0033732 Test Loss: 0.0047901
Validation loss decreased (0.003507 --> 0.003373).  Saving model ...
Updating learning rate to 7.8125e-06

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 12, Steps: 1159 | Train Loss: 0.1124570 Vali Loss: 0.0034817 Test Loss: 0.0049386
EarlyStopping counter: 1 out of 5
Updating learning rate to 3.90625e-06

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 13, Steps: 1159 | Train Loss: 0.1052720 Vali Loss: 0.0035949 Test Loss: 0.0051179
EarlyStopping counter: 2 out of 5
Updating learning rate to 1.953125e-06

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 14, Steps: 1159 | Train Loss: 0.1118911 Vali Loss: 0.0035167 Test Loss: 0.0050038
EarlyStopping counter: 3 out of 5
Updating learning rate to 9.765625e-07

train:   0%|          | 0/1159 [00:00<?, ?it/s]
Epoch: 15, Steps: 1159 | Train Loss: 0.1101214 Vali Loss: 0.0035256 Test Loss: 0.0050054
EarlyStopping counter: 4 out of 5
Updating learning rate to 4.8828125e-07
>>>>>>>testing : Exchange_ETSformer_custom_ftM2S_sl256_pl12_dm512_nh16_el3_dl3_df2048_K0_lr0.001_'Exp'_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
val 5330
loading model

  0%|          | 0/166 [00:00<?, ?it/s]
test shape: (166, 32, 12, 1) (166, 32, 12, 1)
test shape: (5312, 12, 1) (5312, 12, 1)
mse:0.00336787779815495, mae:0.04273906722664833
test 10668
loading model

  0%|          | 0/333 [00:00<?, ?it/s]
test shape: (333, 32, 12, 1) (333, 32, 12, 1)
test shape: (10656, 12, 1) (10656, 12, 1)
mse:0.004790135659277439, mae:0.051230162382125854

Plot

In [7]:
setting
Out [7]:
"Exchange_ETSformer_custom_ftM2S_sl256_pl12_dm512_nh16_el3_dl3_df2048_K0_lr0.001_'Exp'_0"
In [8]:
ds, dl = exp._get_data('test')
test 10668
In [9]:
preds, trues = exp.test(setting, data='test')
test 10668
loading model

  0%|          | 0/333 [00:00<?, ?it/s]
test shape: (333, 32, 12, 1) (333, 32, 12, 1)
test shape: (10656, 12, 1) (10656, 12, 1)
mse:0.004790135659277439, mae:0.051230162382125854
In [10]:
%matplotlib inline
In [17]:
i=190
(batch_x, batch_y, batch_x_mark, batch_y_mark)  = ds[i]
l1 = batch_x.shape[0]
l2 = batch_y.shape[0]
plt.plot(range(l1), batch_x[:, -1], label='past')
plt.plot(range(l1, l1+l2), batch_y[:, -1], color='blue', ls='--', label='true')
plt.plot(range(l1, l1+l2), preds[i], label='pred');
plt.legend(loc='lower left')
Out [17]:
<matplotlib.legend.Legend at 0x7f3aa8c88490>
In [ ]:

TODO check index

  • s_end = index + self.seq_len
  • r_begin = index + self.seq_len - self.label_len
  • r_end = index + self.seq_len + self.pred_len
In [12]:
f = ds.root_path + '/' + ds.data_path
df = pd.read_csv(f).set_index('date', drop=False)
df = df[ds.cols[1:]]
# df[:] = ds.scaler.transform(df.values)
df
# df
Out [12]:
close ref_close sec_close RVOLs_50_12_2ref_2ref MACDVs_3_10_9_2ref_2ref MACDVs_8_24_9_2ref_2ref RRVOLs_50_12_2ref_2ref RSMKs_3_24_12_2ref_2ref RSMKs_128_8_4_2ref_2ref RRSs_12_12_12_2ref_2ref VRRSs_5_21_12_2ref_2ref RSMKs_18_144_72_2ref_2ref
date
2019-01-02 07:00:00 60.7500 246.3900 56.70 0.634561 0.228822 0.749770 -0.081572 0.754948 0.041974 0.452882 0.094300 -0.141763
2019-01-02 09:30:00 60.2000 246.3000 56.30 0.619485 -0.138271 0.516611 -0.159545 0.690752 0.037310 0.201226 -0.050731 -0.127774
2019-01-02 09:35:00 59.8500 246.5200 56.36 0.689500 -0.535564 0.191501 -0.104752 0.553376 0.027199 -0.217425 -0.069932 -0.117113
2019-01-02 09:40:00 60.0600 246.9001 56.70 0.746596 -0.803242 -0.130051 -0.070761 0.402911 0.015511 -0.473000 -0.089949 -0.109966
2019-01-02 09:45:00 60.2350 247.1400 56.69 0.701637 -0.936439 -0.409001 -0.130581 0.263664 0.003868 -0.679096 -0.096488 -0.105561
... ... ... ... ... ... ... ... ... ... ... ... ...
2020-12-31 17:35:00 17.3300 374.2100 37.90 0.706492 -0.129955 -0.551704 0.197178 -0.842459 -0.216290 -3.579407 -0.088556 -1.118991
2020-12-31 18:30:00 17.3200 374.4000 37.96 0.615936 -0.147461 -0.538942 0.174563 -0.803874 -0.213881 -3.258388 -0.075389 -1.179880
2020-12-31 19:15:00 17.3600 374.3900 37.95 0.605291 -0.105155 -0.508192 0.226633 -0.756699 -0.210656 -2.735903 -0.063731 -1.239256
2020-12-31 19:20:00 17.3501 374.3900 37.95 0.592393 -0.065321 -0.475375 0.268617 -0.720279 -0.207647 -2.416748 -0.053941 -1.297199
2020-12-31 19:55:00 17.4000 374.3900 37.95 0.504877 0.054870 -0.408612 0.198019 -0.664792 -0.201618 -2.009522 -0.045287 -1.352772

53398 rows × 12 columns

In [13]:
from data_provider.data_loader import Dataset_Custom
ds2 = Dataset_Custom(
    ds.root_path,
    flag="test",
#     size=ds.size,
    size=[ds.seq_len, ds.label_len, ds.pred_len],
    features=ds.features,
    data_path=ds.data_path,
    target=ds.target,
    scale=False,
    timeenc=ds.timeenc,
    freq=ds.freq
)

i=99
(batch_x, batch_y, batch_x_mark, batch_y_mark)  = ds2[i]
batch_x[-1]
Out [13]:
array([ 1.24250000e+01,  3.54330000e+02,  3.52700000e+01,  6.24194744e-01,
       -1.20630267e-01,  4.64484086e-01, -3.96678363e-01,  1.04169477e+00,
       -2.88024043e-02, -4.61072831e-01, -9.81798622e-03, -3.57276350e+00])
In [14]:
dt = ds2.index.iloc[i-1]
df.loc[dt]
Out [14]:
close                         12.425000
ref_close                    354.330000
sec_close                     35.270000
RVOLs_50_12_2ref_2ref          0.624195
MACDVs_3_10_9_2ref_2ref       -0.120630
MACDVs_8_24_9_2ref_2ref        0.464484
RRVOLs_50_12_2ref_2ref        -0.396678
RSMKs_3_24_12_2ref_2ref        1.041695
RSMKs_128_8_4_2ref_2ref       -0.028802
RRSs_12_12_12_2ref_2ref       -0.461073
VRRSs_5_21_12_2ref_2ref       -0.009818
RSMKs_18_144_72_2ref_2ref     -3.572763
Name: 2020-09-02 11:20:00, dtype: float64
In [15]:
assert df.loc[dt].close == batch_x[-1, 0], 'index should be right'
'OK'
Out [15]:
'OK'
In [ ]:
In [ ]:
In [ ]:
Reference in New Issue View Git Blame Copy Permalink