Lesson 4 - Adding Convolutional Layers to RNNs

Welcome to Convolutional Layers for Sequences#

This lesson shows you how to put a convolutional layer in front of a recurrent network. You will learn what a 1D convolution does to a sequence, why a small sliding kernel is good at spotting local patterns that an LSTM can then summarize over time, and how to keep the whole thing honest with causal padding so the model never looks at future values when predicting the present.

By the end of this lesson, you will be able to:

• Explain what a Conv1D layer does as it slides a kernel along a sequence
• Describe why convolution and recurrence are complementary, and when the combination helps
• Use padding='causal' so a forecasting model never peeks at the future
• Build and compile a Conv1D + LSTM hybrid in Keras and read its parameter count
• Compare the hybrid against a plain LSTM and decide whether the extra complexity is justified

You should be comfortable with the RNN concepts from the earlier lessons in this module: windowing a time series, the SimpleRNN and LSTM layers, and evaluating a forecast with RMSE and MAE. Let’s begin.

Why Combine Convolution With Recurrence?#

So far in this module, you have asked an RNN to do two jobs at once. As it reads a sequence one step at a time, it has to (1) recognize the small, local shapes in the data, like a short dip followed by a sharp rebound, and (2) carry a memory of what happened many steps ago. Recurrent layers are excellent at the second job. They are less efficient at the first, because every local pattern has to be discovered one timestep at a time through the recurrence.

A convolutional layer is the opposite. It is built to find local patterns cheaply and in parallel, but on its own it has no sense of long-range order. So a natural idea is to let each layer do what it is best at:

raw sequence  -->  Conv1D  -->  local motifs  -->  LSTM  -->  long-range memory  -->  prediction
   (12 months)    (find shapes)   (compressed)    (remember)     (one number)

The convolution acts as a feature extractor that runs first. It scans the raw sequence and produces a new, shorter sequence of learned local features. The LSTM then reads that cleaner sequence and handles the long-term dependencies. This division of labor is why the hybrid often trains faster and can generalize better, especially on long sequences with repeating local structure.

What “1D” Means Here#

The “1D” in Conv1D refers to the axis the kernel slides along, which is time. Your data still has more than one number per timestep in general (in image work a convolution slides over two spatial axes). For a univariate series like a stock index there is a single value per month, so each input window has shape (timesteps, 1). The convolution slides along the timesteps axis only.

How a 1D Convolution Reads a Sequence#

The core operation is simple. You take a small window of weights called a kernel (also called a filter), line it up against the start of the sequence, multiply element-by-element, and add the products into a single number. Then you slide the kernel one step to the right and repeat. The string of numbers you produce is the convolution’s output, often called a feature map.

The figure below shows a kernel of size 3 sliding along a sequence. At each position it looks at three neighboring timesteps and emits one value, so it is detecting a local shape, a motif spanning three steps, no matter where in the sequence that shape appears.

Two numbers control the layer:

• kernel_size: how many neighboring timesteps the kernel spans. A kernel of 3 looks at short, three-step motifs; a kernel of 7 captures wider patterns. Odd numbers are conventional so each output is centered on a timestep.
• filters: how many different kernels the layer learns. Each filter learns to detect a different motif (one might fire on sharp upswings, another on slow plateaus), and produces its own feature map. With filters=64, the layer outputs 64 parallel feature maps.

You can do the arithmetic by hand for one filter. Suppose a kernel of size 3 has weights $[1, 0, -1]$ (with a bias of 0) and slides over the values $[2, 5, 9, 4]$. The first output covers the first three values:

Sliding one step right, the second output covers the last three values:

So this kernel turns the 4-value input into the 2-value feature map $[-7, 1]$. This particular kernel responds to whether the sequence is rising or falling across a three-step window, which is exactly the kind of local signal a forecaster cares about. During training, the network learns its own kernel weights instead of you choosing them by hand.

import numpy as np

# One filter, kernel [1, 0, -1], sliding over a 4-step signal (by hand)
signal = np.array([2, 5, 9, 4])
kernel = np.array([1, 0, -1])

out = [int(np.dot(kernel, signal[i:i+3])) for i in range(len(signal) - 2)]
print(out)
# Output: [-7, 1]

Causal Padding: Never Peek at the Future#

Here is a subtle trap that is unique to forecasting. A standard (“same”) convolution centers the kernel on each timestep, so the output at month $t$ depends on months before and after $t$. For images that is fine; pixels have no arrow of time. For a forecast it is a disaster: to predict the value at month $t$, the model would be allowed to look at month $t+1$, which in real life has not happened yet. That is data leakage, and it produces accuracy numbers that collapse the moment you deploy.

The fix is padding='causal'. Causal padding adds zeros only on the left (the past side) of the sequence, so the kernel at each position can only reach backward in time. The output at month $t$ depends on months $t, t-1, t-2, \dots$ and never on the future.

"same" padding (pads BOTH sides — output at t can see t+1):
        [0]  x1  x2  x3  ...  x12  [0]
              \_______/  kernel centered on x2 sees x1, x2, x3   <-- x3 is the future!

"causal" padding (pads LEFT only — output at t sees only the past):
   [0] [0]  x1  x2  x3  ...  x12
         \_______/  kernel at x1 sees only [0],[0],x1 (the past)

Causal padding also keeps the output length equal to the input length, so the sequence handed to the LSTM has the same number of timesteps as the window you fed in.

Loading the Data#

You will work with the same real series you used earlier in this module: the monthly closing level of the S&P 500 stock index. Each row is one month, so the series is a clean, evenly spaced univariate time series, perfect for testing whether a convolutional front-end helps an LSTM forecaster.

import pandas as pd

# download: https://datatweets.com/datasets/sp500_monthly.csv
df = pd.read_csv("sp500_monthly.csv", parse_dates=["date"])

print("Rows:", len(df))
print("From", df["date"].min().date(), "to", df["date"].max().date())
print("Price range:", round(df["close"].min(), 1), "to", round(df["close"].max(), 1))
# Output:
# Rows: 917
# From 1950-01-01 to 2026-05-01
# Price range: 16.9 to 7412.6

There are 917 monthly observations spanning from January 1950 to May 2026, with the index climbing from about 16.9 to over 7400. As in the earlier lessons, you turn this single column into supervised examples by windowing: each input is a block of 12 consecutive months, and the target is the value of the following month.

import numpy as np

def make_windows(series, window=12):
    X, y = [], []
    for i in range(len(series) - window):
        X.append(series[i:i + window])
        y.append(series[i + window])
    X = np.array(X).reshape(-1, window, 1)   # (samples, timesteps, features)
    y = np.array(y)
    return X, y

values = df["close"].values.astype("float32")
X, y = make_windows(values, window=12)

# Chronological split: train on the past, test on the most recent stretch
split = int(len(X) * 0.8)
X_train, X_test = X[:split], X[split:]
y_train, y_test = y[:split], y[split:]

print("train:", X_train.shape, "test:", X_test.shape)
# Output:
# train: (721, 12, 1) test: (184, 12, 1)

Each input window has shape (12, 1): twelve monthly timesteps, one feature per step. That trailing 1 is exactly the axis a Conv1D needs. The split is chronological, not random, because shuffling a time series would let the model train on the future and test on the past.

Building a Conv1D + LSTM Hybrid in Keras#

Now you assemble the hybrid. The architecture follows the division of labor from earlier: a causal Conv1D extracts local motifs, then an LSTM reads the resulting feature sequence and carries the long-range memory, then a small Dense head turns the LSTM’s summary into a single predicted number.

import tensorflow as tf
from tensorflow.keras import layers

model = tf.keras.Sequential([
    layers.Input(shape=(12, 1)),                  # 12 months, 1 feature per month
    layers.Conv1D(filters=32, kernel_size=3,
                  padding="causal", activation="relu"),  # extract local motifs, no peeking
    layers.LSTM(32),                               # long-range memory over the motifs
    layers.Dense(16, activation="relu"),
    layers.Dense(1),                               # one number: next month's level
])

model.compile(optimizer="adam", loss="mse", metrics=["mae"])
model.summary()

A few choices are worth calling out:

• padding='causal' on the convolution is non-negotiable here, for the leakage reason above.
• activation='relu' on the Conv1D lets each filter act as a detector that “fires” only when its motif is present.
• The LSTM has no activation argument, so it uses its sensible defaults. You let it focus purely on memory.
• The final Dense(1) has no activation because this is regression: you want a raw number, not a probability.

Because causal padding preserves length, the convolution hands the LSTM a sequence of the same 12 timesteps, now with 32 feature channels instead of 1. The LSTM then collapses that into a single 32-dimensional summary vector, which the Dense head maps to the forecast.

Training and Evaluating the Hybrid#

Before fitting on the raw price scale, recall the discipline from the previous lessons: scale the inputs using statistics learned from the training set only, then train and evaluate. Here we train the hybrid and report RMSE and MAE on the held-out, most-recent months.

# Train the hybrid (errors below are on the original price scale)
model.fit(X_train, y_train, epochs=60, batch_size=32, verbose=0)

pred = model.predict(X_test, verbose=0).ravel()

rmse = np.sqrt(np.mean((pred - y_test) ** 2))
mae = np.mean(np.abs(pred - y_test))
print(f"Conv1D+LSTM  test RMSE={rmse:.1f}  MAE={mae:.1f}")
# Output:
# Conv1D+LSTM  test RMSE=507.0  MAE=353.1

After 60 epochs the hybrid lands at a test RMSE of about 507 and an MAE of about 353 index points. Given that the index ranges into the thousands over the test window, that is a respectable forecast that tracks the broad trajectory of the market.

Due Diligence: Does the Convolution Actually Help?#

Adding a layer always adds complexity, and complexity has to earn its place. The honest way to judge the hybrid is to compare it against a plain LSTM with the same recurrent and dense structure, just with the convolution removed. If the hybrid does not clearly beat it, the extra layer is not justified for this data.

In the previous lesson you already measured a plain LSTM on this exact split. Here is that baseline alongside the gated models you compared, so you have them side by side:

Read that table carefully. The plain LSTM baseline reaches an RMSE of about 423, while the Conv1D + LSTM hybrid sits at about 507. On this dataset, adding the convolution did not improve the forecast; the simpler model is both leaner and more accurate.

That result is not a failure of the technique, and it is worth understanding why. The S&P 500 monthly series is short per window (12 steps) and dominated by a smooth long-term trend rather than fine repeating motifs. There is little local structure for a convolution to extract that the LSTM was not already capturing on its own. The Conv1D + LSTM hybrid pays off when sequences are long and rich in local pattern, which a 12-month window of a smooth index simply is not.

Practice Exercises#

Try these before checking the hints. Reuse X_train, X_test, y_train, and y_test from the lesson.

Exercise 1: Convolve a Sequence by Hand#

Write a small function that applies a single kernel of size 3 to a 1D NumPy array with no padding, and use it on [1, 3, 2, 6, 4] with the kernel [1, 1, 1] (a moving sum). How long is the output, and why?

import numpy as np

signal = np.array([1, 3, 2, 6, 4])
kernel = np.array([1, 1, 1])

# Your code here

Exercise 2: Confirm Causal Padding Preserves Length#

Build a tiny Sequential model with just an Input of shape (12, 1) and a single Conv1D using padding='causal', kernel_size=5, filters=8. Call .predict() on X_test and check the shape of the output. Then change the padding to 'valid' and compare.

import tensorflow as tf
from tensorflow.keras import layers

# Your code here

Exercise 3: Tune the Convolution#

Rebuild the Conv1D + LSTM model but widen the kernel to kernel_size=5 and double the filters to filters=64 (keep padding='causal'). Train it for 60 epochs and print the test RMSE. Does a wider, wider-channel convolution close the gap with the plain LSTM baseline of about 423?

# Your code here (reuse X_train, X_test, y_train, y_test)

Summary#

Nice work. You added a convolutional front-end to a recurrent network, learned the one padding setting that keeps forecasting honest, and ran a fair comparison to decide whether the extra layer was worth it. Let’s review.

Key Concepts#

1D Convolution

• A Conv1D slides a small kernel along the time axis, turning each local window into one feature value
• kernel_size sets how many neighboring timesteps a motif spans; filters sets how many different motifs the layer learns
• With no padding, a size-$k$ kernel over a length-$T$ sequence outputs $T - k + 1$ values

Why Combine Conv1D With an RNN

• Convolution finds local motifs cheaply and in parallel; recurrence carries long-range memory
• The hybrid lets each layer do what it is best at, which can help on long sequences with repeating local structure (audio, sensor, biomedical signals)

Causal Padding

• padding='causal' pads only the left (past) side so the output at time $t$ never depends on the future
• It prevents data leakage in forecasting and preserves the sequence length for the next layer
• Use it whenever the target is a future value of the same series

Building and Judging the Hybrid in Keras

• Conv1D(filters, kernel_size, padding='causal', activation='relu') then LSTM(...) then Dense(1) for regression
• A final Dense(1) has no activation because you are predicting a raw number
• Always compare against a simpler baseline on a held-out set: on the S&P 500 monthly data the plain LSTM (RMSE ~423) beat the hybrid (RMSE ~507)

Why This Matters#

The real lesson here is not “always add a convolution.” It is the discipline around adding one. You saw that a more elaborate architecture can lose to a simpler one, and that the only way to know is to test both on data the models never saw. That habit, of treating each layer as a hypothesis you must justify, is what separates careful practitioners from people who stack layers and hope.

You also met causal padding, a small setting with outsized consequences. The same instinct that warns you about future leakage in a convolution will protect you across every forecasting problem you ever build, from energy demand to web traffic to financial markets. Knowing when a tool helps, and proving it on held-out data, is the skill that carries into the forecasting workflow you tackle next.

Next Steps#

You can now extract local patterns with a Conv1D, guard against future leakage with causal padding, and judge a hybrid against a baseline. Next, you will assemble these pieces into a complete time-series forecasting workflow.

Keep Building Your Skills#

You have learned to combine two of the most important deep learning building blocks, convolution and recurrence, and just as importantly, to test whether that combination earns its keep. Carry both habits forward: reach for a Conv1D when your sequences are long and locally patterned, always make it causal for forecasting, and never trust a fancier architecture until it beats a simple one on data it has never seen. Those instincts will serve you in every sequence model you build from here.