Introduction

A few years ago, we started hearing stories of researchers who were able to thwart image processing systems using a variety of crude hacks.

The most famous version of this story came in a widely-circulated report, which showed just how easy it was to confuse self-driving vehicles. By simply placing a single pixel-like sticker over a stop sign, researchers were able to demonstrate just how easy it was for autonomous cars to ignore human traffic signals and blow right through dangerous intersections.

High Fashion has also gotten into the game of thwarting image recognition systems. The CV Dazzle project collects (awesome, futuristic) hair and makeup looks that are designed to defy facial recognition technologies.

The most extreme example of this trend comes from a recent research study, which found that strategically adjusting a single pixel can thwart the successful labeling of 67.97% of the natural images in the CIFAR-10 test dataset.

Together, these efforts reinforce one of the potential weaknesses of Deep Learning systems, namely — minuscule adjustments can have an outsized influence on its results.

This idea of small perturbations to the feature space heavily distorting the targets isn’t limited to image processing systems. An adversarial actor can use this power to manipulate almost any system in which Artificial Intelligence flourishes.

Case in point: the stock market.

The Rise of the Machines

When you think of the Stock Market, what comes to mind?

For many, it’s that crowded image of loud, impatient brokers, standing on the trading floor, waving slips of paper, and yelling over the crush to get their orders in.

It’s this perennial image that has persisted for over a century. And over that century, the story of the stock market was one of boom and bust.

Most people understand know that markets can’t rise forever. Corrections are an inevitable part of doing business.

Surprisingly, these corrections occur about every ten years. For example, in 1987—on what is now infamously known as Black Monday—the Dow Jones lost a staggering 509 points. A decade later, a mini-crash in Asia rippled around the world and ultimately lead to the internet bust. A decade after that, in 2007, the housing market crashed, sending the global economy reeling.

Based on this pattern, one could reasonably expect that the market would have taken a hit again in 2017. Although there were some rumblings of a slowdown, a full crash never happened.

Economists had several theories as to why not but many concluded that the main reason for such unprecedented stability was that AI had finally saturated the market.
 

A New Financial Paradigm

In recent years, AI-powered systems have come to eclipse traditional investment opportunities. Today quantitative and passive investment funds now control about 60% of all equity assets, double from a decade ago. By contrast, only about 10% of trading volume comes from human discretionary investors.

We have the big banks to thank for this shift. In the last decade, Goldman Sachs has laid off 600 traders and replaced them with over 200 computer engineers. Today, some 9,000 computer engineers work for the organization.

There are many reasons for this shift:
 

1. Stability

Humans are emotional creatures and unfortunately, emotions allow trading mistakes and intemperate decision-making. Emotional trading is such a large concern, that nearly every beginner’s guide to the stock market has a whole chapter on keeping the feelings out of the market.

Beginners aren’t the only ones susceptible to emotional trading. Stockbrokers who have been in the business for years often mislabel emotional trading as intuition. Regardless of what they call it, emotional trading still leads to mistakes.
 

2. Secrecy

By shifting trading decisions away from individuals and over to AI-powered systems, brokerage firms can protect their intellectual property and proprietary algorithms. This differentiates the organization from its competitors and prevents secrets from leaking out when traders transition to a different company.
 

3. Scalability

Another factor in the rise of the machines is scalability. By automating trading decisions, firms can manage more assets without needing to hire more people.
 

An Opportunity for Exploitation

Given the ubiquity of AI-powered systems in the markets, we wondered whether it would be possible to manipulate these bots for personal gain or competitive advantage.

And so we set out to create an experiment that would do just that. Using real-world data in a closed system, we tried to see if an adversarial bot could be used to dampen the success of other bots in the system.

Before getting into the details of the experiment, let us say explicitly that this research should not be replicated for nefarious ends. In most markets, manipulative trading is illegal and we do not condone it.

The Experiment

We developed a simulated and closed training environment to discover how the stock market could be manipulated using AI. Just as security experts try to hack into systems to test vulnerabilities, it’s incumbent upon members of the AI/ML to deploy bots to shore up systemic weaknesses or possible exploits.

One of the things that allows this trading to occur is that bots are capable of picking up on really, subtle, almost imperceptible patterns. This is the invisible world of conditional probability. Algorithms can perceive small, interconnected feature-movements and make a prediction about how things will change. In the case of trading, these algorithms are predicting price movements.

This is similar to our previous discussion about image manipulation.

Just as a single pixel can be changed to thwart image recognition systems, our theory was that it would be possible to make small changes to the market to thwart other financial bots.
 

Our Process

We gathered about 20 years of stock data across several major organizations, including Amazon, Microsoft, and Intel. The data included each stock’s daily open, high, low, and trading volume. We set the adjusted close as the prediction target, to imbue the system with some seasonality.

We scaled them by tanh estimators, by year, by company, and by company year.

Then we fed that data into a series of bots.

We did a random grid search on hyper-parameters to train a group of trading bots that would exhibit different behaviors and tendencies, given different inputs. 

We used a set of 1D convolutional layers with dense pooling to feed higher level merge and prediction layers for each of the companies, training on a uniformly weighted MSE between the predicted and actual stock prices.

Now that we had an environment that gave us a little visibility into how bots interact with a closed financial system, the question became — are bots any better than humans? And what happens when the strengths these bots bring to the marketplace can be exploited?

When Recognizing a Pattern is itself a Pattern

Because bots are good at recognizing patterns, one could reasonably expect that a bot can be instructed to find the pattern of other bots. So for this experiment, we trained an Adversarial Bot (AB) to look at the behavior of the other bots in the system.

Our adversarial bot had the same access to the marketplace as a regular trader had; i.e. it could buy and sell stocks, at a relatively low volume. 

Whereas the previous bots were trained to minimize their MSE loss, our AB was charged with maximizing the MSE loss of the other bots.

So Did the Adversarial Bot Work?

In short, the Adversarial Bot did work. As it looked upon the other bots in the system, it was able to identify some bots that were particularly susceptible to being manipulated. Then the Adversarial Bot went to work, making smaller-scale trades to trick these more susceptible bots into over-reacting to these subtle movements towards financial ruin.
 

Conclusions

Our study into using bots to manipulate the stock market lead us to two significant conclusions:

1. Bots are dumber than humans.
2. Bots are smarter than humans.

On the one hand, humans would not have been so easily fooled. The whole premise of this experiment is that an Adversarial Bot can make subtle moves to trigger an outsized reaction from another bot. These moves would likely be overlooked or ignored by the average human. 

It is, however, this exact attention to detail that make bots so powerful in the first place. To be sure, almost all of the bots we created were able to successfully grow their assets in ways that individual human traders can only dream of.
 

Next Steps

Our study verified what we’d long suspected: that bots could be used nefariously to affect markets. In light of this revelation, we encourage financial institutions to put good checks in place that prevent their pattern recognition superpowers from being so easily manipulated.

To watch the full presentation, check it out here.
To get involved in this project, please visit our Github repo.
To save a copy of this story, please visit our PDF page.