Research by Jesus Rodriguez and Lucas Outumuro of IntoTheBlock
At IntoTheBlock, we have been working on predictive models for different crypto assets. The work ranges from sophisticated quant strategies to more basic directional forecasts. Based on that work, we have learned quite a few lessons about effective methods, challenges, and very peculiar aspects that should be taken into consideration when attempting to build predictive models for crypto assets.
In this article, I would like to explore some of those ideas that, hopefully, shed some light about the magnitude of the challenge of predicting the price of crypto.
“All Models are Wrong but Some Are Useful”
British statistician George Box once said that when it comes to statistical models “all models are wrong, but some are useful”. That phrase has been adopted as a mantra in quantitative finance as an indication that the ever-changing nature of financial markets will cause problems to the most sophisticated predictive models. In the case of crypto assets, this idea couldn’t be more correct.
The dynamic nature of crypto assets, the regular volatility ( yes, it will come back ? ) and the short trading history makes crypto particularly challenging for predictive models. Additionally, different crypto assets can be based on fundamentally different protocols and could behave differently during certain market conditions.
In the context of crypto assets, quants or data scientists working in predictive models should realize that even effective predictive models would have a limited time-span and would be vulnerable to changes in market conditions. From that perspective, it is more important to produce a variety of predictive models for different thesis than trying to nail the perfect predictive strategy.
The Case for Deep Learning in Crypto Asset Predictions
There are many methods that can be used to model predictive behaviors in crypto assets. Taking some liberties, most of those techniques can be grouped in one of the following categories:
- Time-Series Forecasting: Traditional statistical methods that focus on predicting a value in a time series based on existing attributes.
- Machine Learning: Simpler machine learning techniques such as linear regression or decision trees which are very common in quant strategies.
- Deep Learning: Predictive models based on the new school of deep neural networks.
In our experience applying time series forecasting methods to crypto-assets showed that, although they are relatively easy to use, they are not very resilient to the constant fluctuations of the crypto space.
Traditional machine learning methods such as linear regression or decision trees have a strong presence in traditional quant systems and, therefore, one might seem inclined to extrapolate those lessons to the crypto space. However, we found that those methods have a strong challenge generalizing and haven’t proven to be very resilient to market conditions.
Deep neural networks is the latest trend in the artificial intelligence(AI) space and the one that has been developing the fastest. In the last few years, the research body in deep learning has increased drastically. The promise of deep neural networks is that they can uncover complex non-linear relationships between arbitrary variables. In our experience, deep learning models in crypto assets have shown strong resiliency to market conditions and the ability to generalize knowledge. The biggest drawbacks are that these types of models are computationally expensive to produce and very hard to understand and interpret how they produce decisions.
In quantitative finance, the application of deep learning methods is still in very nascent stages compared to other methods. However, the promises are tremendous. In the case of crypto assets, deep learning models exhibit some tangible benefits:
- Complex, Non-Linear Relationships: Deep neural networks can model very complex non-linear relationships between different predictors.
- Resilient: Through constant training, deep learning models have proven to be resilient to the constant fluctuations in the crypto market.
- Large Research Body: Deep learning research in quantitative finance is growing faster than any other discipline providing lots of research ideas that can be adapted to the crypto space.
At the same time, there are a couple of potential drawbacks of applying deep learning models to crypto-asset price forecasts:
- Interpretability: Deep neural networks are complex and, therefore, really hard to interpret.
- Expensive to Build: Deep learning models are computationally expensive to build and maintain.
From the different statistical and machine learning schools in the current markets, deep learning seems particularly well equipped to handle the challenges of building predictive models for crypto assets. However, assuming that most deep learning ideas from traditional capital markets will apply in the crypto space would be a mistake. We certainly experimented with a few of the most accepted deep learning methods in traditional quant finance and encountered a few surprises.
10 Things We Learned While Building Predictive Models for Crypto Assets
Building a robust prediction pipeline is a very difficult task. Many of the models that work great in a lab environment will result hard to operationalize. Here are some lessons that might be relevant when considering applying deep learning models to crypto asset predictions:
- Training Size Matters: When it comes to crypto-assets, the larger the dataset to train models the better.
- Data Quality is a Huge Problem for Crypto Assets: It’s very difficult to find reliable data sources in the crypto space.
- Accurate Predictions do not Mean Actionable Predictions: An indication of directional movement in price is not a trading strategy.
- Operationalizing Real-Time Predictions are Hard: Running deep learning models in real-time require quite a bit of infrastructure.
- Blockchain-Based Predictive Models are Very Fragile: Blockchain predictive models are very vulnerable to exchange manipulations, forks and other runtime changes that can affect their performance.
- Deciding When and How to Retrain Models is Challenging: Retraining predictive models after they are in productions can change their performance in unexpected ways.
- Rapid Experimentation is Key: Building predictive models for crypto assets is a business of failure. Trying different ideas rapidly and iterating is essential for success.
- Data Sources in Crypto are Very Unreliable: Exchanges failures, missing data, wash trading records are some of the elements that affect the reliability of data sources used in crypto predictive models.
- Better Infrastructure Beats Better Models: A robust infrastructure with mediocre models will beat a poor infrastructure with great models in the long term.
- Most Quant Research Fails When Applied to Crypto: Most techniques in research papers were not designed for the dynamics of crypto markets.
Quant strategies are likely to be the dominant form of investing and trading in crypto-assets and, as a consequence, predictive models are likely to play an important role in the evolution of those strategies. The ideas in the field of deep learning applied to crypto are still in a very early stage and there is a huge gap between research and practical applications. Some of the ideas outlined in this article attempt to provide a practical perspective on the challenge of building predictive models for crypto assets. The challenges are many, but the journey is certainly fun.
About the authors
Jesus Rodriguez is the CEO-CTO of IntoTheBlock, a market intelligence platform for crypto assets. He is a computer scientist, a speaker, and author on topics related to crypto and artificial intelligence.
Lucas Outumuro is a Sr. Researcher at IntoTheBlock, a market intelligence platform for crypto assets. His areas of focus include crypto derivatives, DeFi and web 3.0 in general.
