New Strategy
Sign up

Transformer model

1. Node overview

Transformer is a deep-learning architecture introduced by Google and built around self-attention. Multi-Head Attention dynamically assigns importance to different points in time, allowing the model to capture long-range dependencies in complex financial time series. The Transformer model node performs sequence modelling on factor data from the data-processing node and outputs predictions for the strategy node.

Position in the data flow
Data source→Data processing→AI model→Strategy type→Position management→Risk control→Trading engine
✓ Strengths

Captures global, long-range dependencies

Parallel computation through multi-head attention

Learns temporal importance weights automatically

Fast training on a GPU

Use cases

Complex time-series pattern learning

Multi-feature interaction modelling

Long-horizon trend forecasting

Large-scale data training

⚠ Limitations

Slow training and inference with demanding GPU requirements

Memory grows quadratically with sequence length

Requires substantial training data

2. Interactive configuration

The panel below is the actual Transformer model node configuration. Use it to explore every setting: prediction column name, task type, and parameter configuration mode (adaptive or advanced).

Transformer model configuration

Only A-Z, a-z, 0-9, - and _ are supported; other characters are replaced with _.

Name of the model prediction output column.

2.1 Prediction column name

Set the name of the column that stores model predictions
What is the prediction column name?
After training, the model adds a column containing its predictions. A downstream strategy node references model output through this name. The default is pred.
Tip: when a workflow contains several AI model nodes, assign each one a different prediction column, such as pred_tf and pred_lstm, so the strategy can distinguish them.

2.2 Task type

Select the model's training objective and loss function
What is a task type?
The task type determines how the model learns and how it produces predictions. Different task types use different loss functions, so choosing the appropriate task is essential to model performance. The system supports Ranking, Binary, and Regression. LightGBM, LSTM, and Transformer default to Binary; switch explicitly to Ranking only for cross-sectional selection across multiple instruments.
⚡ Quick selection guide
DimensionRankingBinary ⭐Regression
Primary objectiveLearn relative orderClassify upward or downward directionPredict an exact return
Noise robustness⭐⭐⭐⭐⭐⭐⭐⭐⭐
Generalization⭐⭐⭐⭐⭐⭐⭐⭐⭐
Sensitivity to outliersMediumLow (inherently resistant)High
Fit for cryptocurrencyMulti-asset selectionBest for single-asset timingPosition-management support
Typical strategyLong/short hedge or ranked selectionTrend timing or enhanced gridDynamic positions or risk control
Recommendation: cryptocurrency markets are highly volatile and contain many extreme moves. Binary classification collapses continuous returns into direction, naturally filtering tail noise. Directional accuracy matters more than return-magnitude precision, so Binary is the default recommendation.

2.2.1 Ranking

Learn relative order among samples; suitable for multi-factor asset selection

Ranking does not optimize the absolute prediction value. It learns relative order among samples, placing higher-return samples ahead of lower-return samples. In multi-asset cryptocurrency strategies, it can select the instruments with the greatest upside potential from dozens of pairs and is the primary driver of cross-sectional selection strategies.

PropertyDescription
Label requirementA continuous value such as return: shift(close, -5) / shift(open, -1) - 1
Output rangeAny real number; a larger score ranks higher
Use casesMulti-factor selection, cross-sectional ranking, and long/short hedging
Frontend fieldtaskType: "ranking"
Ranking mode (rankingMode)

Selecting Ranking reveals the ranking-mode option, which determines where optimization is concentrated:

Balanced (balanced)
Optimize both the head and tail
→ Long/short hedging
Head (head)
Prioritize the top-ranked samples
→ Long-only strategies
Tail (tail)
Prioritize the bottom-ranked samples
→ Short-only strategies
💡 Cryptocurrency note: Ranking requires cross-sectional data for several trading pairs, such as BTC, ETH, and SOL features at the same time. The model compares their relative strength within each time slice. For a single instrument, choose Binary instead.

2.2.2 Binary ⭐ Recommended

Predict upward or downward direction; the first choice for cryptocurrency timing

Binary classification turns forecasting into an up-or-down decision. The model outputs signed confidence with a theoretical range of -1 to 1: positive is bullish, negative is bearish, and a greater absolute value means higher confidence. Because cryptocurrency has high volatility and a low signal-to-noise ratio, actual predictions usually cluster around -0.1 to 0.1; theoretical values near ±1 are almost never seen. Begin machine-learning models with entry thresholds around 0.03 / -0.03, inspect the prediction distribution after the first backtest, and recalibrate. Do not start from theoretical thresholds such as 0.2 or 0.5, which may trigger extremely rarely or never. Direction still has a much better signal-to-noise ratio than magnitude, making Binary the most practical task.

PropertyDescription
Label requirementA Boolean value or continuous return; continuous values are converted to 0/1 using > 0
Output rangeTheoretical -1 to 1 confidence; generally -0.1 to 0.1 in practice because financial signal-to-noise is low
Use casesDirection forecasting, timing signals, and trend classification
Frontend fieldtaskType: "binary" (system default)
Label example: label = shift(close, -5) / shift(open, -1) - 1 means buying at the next candlestick's open and selling five periods later. The engine converts returns > 0 to positive class 1 and returns ≤ 0 to negative class 0.
📊 Confidence-to-probability table

Binary output is signed confidence in the range -1 to 1: confidence = 2 × probability - 1.

Probability30%40%50%55%60%
Confidence-0.4-0.200.10.2

Strategy example: pred > 0.2 opens a long position (60% bullish probability), while pred < -0.2 closes it (only 40% bullish probability).

🎯 Why is Binary recommended for cryptocurrency?
• Compressed label space: continuous returns collapse to {0, 1}, so extreme ±20% moves no longer dominate the loss.
• Direction over magnitude: getting direction right can be profitable in a volatile market without predicting the exact return.
• Natural regularization: a classification boundary is smoother than a regression fit and is less likely to overfit isolated abnormal candles.
• Actionable confidence: probabilities can directly represent signal strength for dynamic position sizing.

2.2.3 Regression

Predict an exact return and optimize absolute accuracy

Regression requires the model to predict an exact return value. Unlike Ranking, it minimizes the absolute error between prediction and target. Frequent extreme cryptocurrency moves, such as flash crashes and sudden rallies, can dominate regression training, so generalization is often weaker than Binary. The output nevertheless has a clear physical meaning and is useful as an auxiliary signal for position sizing and risk control.

PropertyDescription
Label requirementA continuous return such as shift(close, -5) / shift(open, -1) - 1
Output rangeAny real number representing predicted return
Use casesPosition sizing, risk assessment, and expected-return calibration
Frontend fieldtaskType: "regression"
✓ Advantage: the output has a clear physical meaning and can be used directly for position calculations and expected-return assessments.

2.3 Parameter configuration mode

Choose system-adaptive or manually configured parameters
What is parameter configuration mode?
The system provides Adaptive mode, which tunes parameters from the data size, and Advanced parameter configuration, which exposes every model parameter. Adaptive mode is the default and is suitable for new users and rapid strategy validation.

2.3.1 Adaptive mode

The system chooses parameters automatically from the data size

The system calculates model parameters from the number of training samples and features. This is suitable for new users and rapid strategy validation and does not require detailed knowledge of Transformer model parameters.

✓ Advantages

No tuning experience required

Start training quickly

Lower overfitting risk

⚠ Limitations

No fine-grained model control

Parameters may not be optimal

Adaptive parameter ranges
The system selects parameters within these ranges: sequence length 10–60, model dimension 32–128, an automatically derived number of attention heads, exactly 2 Transformer layers, a learning rate around 0.001, 30–70 epochs, batch size 4–128, sample dropout 0.10–0.25, and feature dropout 0.10–0.25.

2.3.2 Advanced parameter configuration

Configure every parameter manually for fine-grained tuning

Advanced mode displays the full panel in three parts: model architecture, training parameters, and regularization. It is intended for experienced users performing detailed tuning.

2.3.2.1 Model architecture

These parameters control Transformer network complexity. The model dimension must be divisible by the number of attention heads. Leave a field blank to let the system calculate it adaptively.

ParameterAllowed rangeAdaptive rangeDescription
Sequence length5–144010–60Length of the input time-series window
Model dimension8–51232–128Embedding dimension; must be divisible by the head count
Attention heads1–16Derived automaticallyNumber of Multi-Head Attention heads
Transformer layers1–6Fixed at 2Number of stacked encoder layers
Important: The model dimension must be divisible by the number of attention heads. For example, model dimension 64 with 8 heads is valid (64/8=8), while model dimension 64 with 3 heads is invalid.
Small dataset (<100,000): dModel: 32, nHeads: 2, nLayers: 1–2
Large dataset (>1,000,000): dModel: 128–256, nHeads: 4–8, nLayers: 2
2.3.2.2 Training parameters

These parameters control model training. Transformers generally require a smaller learning rate and a longer training time.

ParameterDefaultAllowed rangeDescription
EpochsAdaptive1–500Training iterations; SWA is disabled below 5 epochs
Learning rateAdaptive0.00001–0.01Parameter update step size
Validation split0.20–0.5Used for early stopping; 0 disables validation
Early-stopping patience51–50Stop after N epochs without improvement
Batch sizeAdaptive1–2048Number of samples per batch
Early stopping: training stops automatically when the validation metric has not improved for the configured number of rounds, preventing overfitting. This setting has no effect when the validation split is 0.
2.3.2.3 Regularization

Regularization prevents overfitting. Higher dropout applies stronger regularization and produces a simpler model. Leave a field blank to let the system calculate it adaptively.

ParameterAllowed rangeAdaptive rangeDescription
Sample dropout0–0.50.10–0.25Random dropout ratio along the time dimension
Feature dropout0–0.50.10–0.25Random dropout ratio along the feature dimension
Overfitting: good training but poor validation → increase dropout
Underfitting: both poor → reduce dropout or enlarge the model

3. Output data structure

The Transformer model node adds a prediction column to its input data and returns the following structure:

ColumnTypeSourceDescription
datetimedatetimePassed throughTimestamp
symbolstringPassed throughTrading pair
open/high/low/close/volumefloatPassed throughRaw OHLCV data
[factor columns…]floatPassed throughFactors calculated by the data-processing node
predfloatAddedModel prediction; the name is controlled by predictionColumn
Meaning of predictions by task type
Ranking
Ranking score; a larger value ranks higher
→ Relative selection
Binary
Bullish confidence in the range -1 to 1
→ Direction
Regression
Predicted return, which may be positive or negative
→ Return forecast

4. FAQ

Q: How should I choose between Transformer and LSTM?
A: Transformer captures global long-term dependencies well, but its training and inference are slower and its GPU requirements are higher. LSTM trains and infers faster and can be trained without a GPU. Choose LightGBM for speed, LSTM for temporal features, and Transformer when you have abundant data and GPU resources.
Q: What is attention, and why is it important?
A: Attention lets the model focus dynamically on different positions in the input sequence and learn which timestamps matter most to the current prediction. In finance, this allows the model to discover important market events and price patterns automatically instead of processing every observation only in chronological order.
Q: How should I configure model dimension and attention heads?
A: The key constraint is that model dimension must be divisible by the number of attention heads. Common combinations include dModel=64 with nHeads=4 and dModel=128 with nHeads=8. More heads let the model attend to several patterns simultaneously, but also increase computation.
Q: How much data does a Transformer need?
A: Transformers have many parameters and generally require more training data to realize their advantages. At least 100,000 samples are recommended. With less data, use a small model such as dModel=32 and nLayers=1, or choose LightGBM.
Q: Is Transformer training faster than LSTM?
A: On a GPU, Transformer is often faster because its computations can run in parallel. Their CPU performance is similar. Transformer memory use grows quadratically with sequence length because of the attention matrix, so very long sequences may make it slower.
    BeeQuant - AI Quantitative Trading Platform