...
/Advanced Techniques in LSTM Models
Advanced Techniques in LSTM Models
Discover advanced LSTM configurations and techniques for enhanced time series analysis in this lesson.
We'll cover the following...
LSTM models worked better than MLPs, which was expected because they can learn temporal patterns. The baseline restricted LSTM model beat the best MLP model in the previous chapter. The unrestricted LSTM proved to perform even better. Adding a recurrent dropout for regularization further improved and stabilized the model.
Inspired from other works on sequence modeling, backward and bidirectional LSTM models were developed. The backward model performed below the baseline. They work better for sequence-to-sequence problems like language translations. However, the bidirectional model outperformed the others. This could be attributed to a bidirectional LSTM’s ability to capture temporal patterns both retrospectively and prospectively.
Lastly, owing to the expectation from LSTMs to learn even longer-term patterns, a wider time window of inputs is used. This is done by re-preparing the data by increasing the lookback from 5 to 20. However, contrary to the expectation, the performance degraded, primarily due to the LSTM cell state’s limitation in fusing temporal patterns from wide time windows. Stateful LSTMs are an alternative to learning exhaustively long-term patterns. Besides, the LSTM models constructed here faced the issue of increasing validation loss, which is further touched upon in the exercises. Finally, the chapter concludes with a few general principles.
Data processing
The initial data processing, for example, converting the ...