Enhancing Bitcoin Trading Bots with Technical Indicators for Better RL Performance

Integrating technical indicators into a reinforcement learning (RL) Bitcoin trading bot can significantly improve decision-making and profitability. While earlier versions of the bot relied solely on raw price action, this updated model leverages five widely used technical analysis tools—Simple Moving Averages, Bollinger Bands, Parabolic SAR, MACD, and RSI—to enrich the input data fed into the neural network. By combining these indicators with historical market and order data, the agent gains deeper insights into market trends, momentum, volatility, and potential reversal points.

This article walks through the implementation process, visualization techniques, and performance comparison between models with and without indicators. Whether you're building your own algorithmic trading system or exploring how AI interprets financial signals, this guide provides practical steps for enhancing RL-based crypto trading strategies.

👉 Discover how AI-powered trading bots use real-time indicators to boost performance

Key Technical Indicators Used in the Model

To strengthen the decision-making capabilities of the RL agent, we integrate five proven technical indicators. Each contributes unique insights that complement raw price data:

• Simple Moving Average (SMA): Identifies trend direction by smoothing price data over time.
• Bollinger Bands: Measures volatility and potential overbought/oversold conditions.
• Parabolic SAR: Highlights potential trend reversals using dot-based signals.
• MACD (Moving Average Convergence Divergence): Captures momentum shifts via moving average crossovers.
• RSI (Relative Strength Index): Quantifies price momentum to detect overextended moves.

These indicators are calculated using the ta Python library, which simplifies batch computation and ensures consistency across datasets.

Implementing Indicators Using the TA Library

The ta library streamlines technical indicator calculation, allowing seamless integration into the trading bot’s data pipeline. Below is how each indicator is added to the dataset:

from ta.trend import SMAIndicator, PSARIndicator, macd
from ta.volatility import BollingerBands
from ta.momentum import rsi

def AddIndicators(df):
    # Simple Moving Averages
    df["sma7"] = SMAIndicator(close=df["Close"], window=7, fillna=True).sma_indicator()
    df["sma25"] = SMAIndicator(close=df["Close"], window=25, fillna=True).sma_indicator()
    df["sma99"] = SMAIndicator(close=df["Close"], window=99, fillna=True).sma_indicator()

    # Bollinger Bands
    indicator_bb = BollingerBands(close=df["Close"], window=20, window_dev=2)
    df['bb_bbm'] = indicator_bb.bollinger_mavg()
    df['bb_bbh'] = indicator_bb.bollinger_hband()
    df['bb_bbl'] = indicator_bb.bollinger_lband()

    # Parabolic SAR
    indicator_psar = PSARIndicator(high=df["High"], low=df["Low"], close=df["Close"], step=0.02, max_step=2, fillna=True)
    df['psar'] = indicator_psar.psar()

    # MACD
    df["MACD"] = macd(close=df["Close"], window_slow=26, window_fast=12, fillna=True)

    # RSI
    df["RSI"] = rsi(close=df["Close"], window=14, fillna=True)

    return df

Once applied, these features expand the model’s state space from 10 to 19 dimensions—significantly enriching the context available to the agent during training.

Visualizing Multi-Indicator Market Data

To assess how well these indicators align with actual price movements, we visualize them alongside OHLC candlesticks using Matplotlib. The Plot_OHCL function overlays all five indicators on a single chart:

• SMAs (7, 25, 99 periods) appear as smooth trend lines.
• Bollinger Bands form dynamic support/resistance envelopes.
• Parabolic SAR dots highlight potential reversal zones.
• MACD and RSI are plotted below price for clarity.

This consolidated view helps confirm that the indicators behave as expected across different market phases—trending, consolidating, or volatile.

👉 See how advanced bots visualize live trading signals in real time

Updating the Reinforcement Learning Environment

With new indicators in place, the RL environment must be updated to include them in the state representation. This involves modifying two core components:

State Size Expansion

The input layer now accommodates 19 features instead of 10:

self.state_size = (lookback_window_size, 19)  # 10 original + 9 new indicators

Indicator History Tracking

A new deque stores normalized indicator values at each time step:

self.indicators_history = deque(maxlen=self.lookback_window_size)

During observation generation (_next_observation), both market and indicator histories are concatenated:

obs = np.concatenate((self.market_history, self.orders_history), axis=1) / self.normalize_value
obs = np.concatenate((obs, self.indicators_history), axis=1)

Normalization ensures consistent scale across features—especially critical for MACD (divided by 400) and RSI (divided by 100) due to their distinct ranges.

Training and Performance Evaluation

The enhanced model was trained using identical parameters as the baseline version:

• Model: Dense neural network
• Episodes: 50,000
• Lookback window: 50 steps
• Learning rate: 0.00001
• Batch size: 32

After training, the model was tested on unseen data (720 time steps). Results showed clear improvement:

Despite slightly fewer trades, the indicator-enhanced bot achieved higher profitability and far greater consistency—demonstrating that richer input data leads to smarter decisions.

Frequently Asked Questions

How do technical indicators improve RL trading bots?

Indicators provide structured summaries of price behavior—such as trend strength (SMA), volatility (Bollinger Bands), and momentum (RSI)—that help the agent recognize patterns beyond raw prices. This enables more informed buy/sell decisions.

Why normalize MACD and RSI differently?

MACD values can range widely (±300+), while RSI is bounded between 0–100. Normalizing MACD by 400 and RSI by 100 scales them appropriately so they don’t dominate the learning process due to magnitude differences.

Can I add more indicators later?

Yes. The framework supports adding new indicators like Stochastic RSI or ADX. Just ensure proper normalization and update the state size accordingly.

Does this strategy guarantee profits?

No. While backtesting shows improved results under historical conditions, real-world performance depends on market dynamics, slippage, fees, and unforeseen events. Always test thoroughly before live deployment.

Is this bot suitable for other cryptocurrencies?

Yes. With minor adjustments to data preprocessing and normalization values, the same architecture can be adapted for Ethereum, Solana, or any liquid crypto asset.

How long does training take?

Training took approximately 16 hours on a standard GPU setup. Future improvements will explore parallelized environments to reduce training time significantly.

👉 Learn how top traders optimize their models with real-time market data

Final Thoughts and Next Steps

Adding technical indicators has demonstrably improved the Bitcoin trading bot’s performance—increasing profitability and reducing losing episodes. This confirms that even sophisticated AI models benefit from domain-specific feature engineering.

Future work includes:

• Automating live data fetching from exchanges
• Scaling training with distributed environments
• Exploring hybrid models (e.g., LSTM + attention)

While promising, these systems remain experimental. They should be used strictly for educational purposes—not as financial advice.

As AI continues to evolve in finance, integrating time-tested technical tools with modern machine learning offers a balanced path forward: leveraging both human insight and algorithmic precision.