Adversarial Machine Learning (ML) algorithms are designed, in many cases by malicious actors, to deceive AI models used by social listening or trading software systems. The end result is the distortion of data used by such systems, an outcome that is often catastrophic for your analysis and trading needs.
This week, we have interviewed ZENPULSAR’s Co-Founder and Head of Product and Data Science, Paul Dudko, who knows more than a thing or two about Adversarial ML. Read on to find out how Adversarial ML algorithms do their sinister job and how PUMP fights this growing trend in social media and trading analytics.
Q: Paul, before we delve into the finer details of Adversarial ML, how would you actually define it?
PD: Adversarial ML is a specialised and growing research discipline dedicated to ML algorithms that attack and trick the base AI algorithms, which social media analytics or trading software is often based on. Adversarial algorithms might be used for completely legitimate reasons, e.g., to train or test the robustness of the software and its base algorithms.
However, as you might imagine, Adversarial ML is frequently used by malicious actors too. These actors might design adversarial systems to distort the results generated by any type of software that relies on AI models, e.g., social analytics packages, trading analytics software, or weather forecast systems.
Q: How does Adversarial ML work on social media?
PD: There are different types of Adversarial ML attacks. One of the most common types that is frequently used on social media is “poisoning”. Poisoning attacks aim to introduce erroneous or deceptive samples into the training set of a legitimate AI algorithm.
This can be carried out by posting a large number of social media messages with deceptive content. For example, malicious actors who aim to trick social listening algorithms might use a large number of bot-generated tweets. Later on, those examples might be used for training/fine-tuning an ML algorithm by the vendors. The end result is that a social listening tool that has trouble dealing with adversarial attacks will have the wrong social sentiment data registered.
Another popular type of attack is an evasion attack - It refers to designing an input (for example, by changing some words to synonyms), which seems normal for a human but is wrongly classified by ML models. ML models have proven effective in detecting malicious emails, fake news, and harmful data traffic in real-time, but with the help of evasion attacks, malicious actors can bypass those filters.
I must say that the damage caused by adversarial attacks is not limited to wrong data in social media and trading software packages. It goes far beyond it – successful adversarial attacks can potentially distort markets in serious ways and cause a general distrust of AI systems’ capabilities in business and society.
Q: Is Adversarial ML used by malicious actors a big problem now?
PD: It’s a growing field, and we see many cases on social media where adversarial attacks are being orchestrated. There have been numerous research papers dedicated to Adversarial ML, so the area is no longer completely greenfield. I believe the practice of adversarial attacks is already quite widespread, but we will see many more news headlines related to it over the next couple of years.
Q: How does ZENPULSAR tackle the issue of Adversarial ML?
PD: We take the issue of adversarial attacks very seriously and use a comprehensive framework made up of several different processes. The multi-process framework is there to ensure that our SaaS social analytics platform, PUMP, and our API datasets are shielded from adversarial attacks.
Firstly, as part of our framework, we use our own adversarial scenarios to train our NLP models. By being trained on a large number of adversarial scenarios, our algorithms become highly capable of detecting them.
Secondly, we use up to 30 different AI algorithms to process the data we source from social media. The use of multiple algorithms helps ensure that there is no reliance on a single model and, therefore, on a single point of failure. Adversarial attacks are capable of deceiving some models some of the time, but they are going to be identified and cut down when they are against a wall of multiple high-quality AI algorithms.
Thirdly, we use a multi-stage process of data filtering that ensures that adversarial attacks are minimised.
I must note that our team has deep expertise in cybersecurity, which is a crucial discipline for tackling the problem of Adversarial ML. To the best of my knowledge, outside of internal systems used by some financial institutions, we are the only finance-oriented social listening platform that actively uses anti-adversarial measures.
This week, we have interviewed ZENPULSAR’s Co-Founder and Head of Product and Data Science, Paul Dudko, who knows more than a thing or two about Adversarial ML. Read on to find out how Adversarial ML algorithms do their sinister job and how PUMP fights this growing trend in social media and trading analytics.
Q: Paul, before we delve into the finer details of Adversarial ML, how would you actually define it?
PD: Adversarial ML is a specialised and growing research discipline dedicated to ML algorithms that attack and trick the base AI algorithms, which social media analytics or trading software is often based on. Adversarial algorithms might be used for completely legitimate reasons, e.g., to train or test the robustness of the software and its base algorithms.
However, as you might imagine, Adversarial ML is frequently used by malicious actors too. These actors might design adversarial systems to distort the results generated by any type of software that relies on AI models, e.g., social analytics packages, trading analytics software, or weather forecast systems.
Q: How does Adversarial ML work on social media?
PD: There are different types of Adversarial ML attacks. One of the most common types that is frequently used on social media is “poisoning”. Poisoning attacks aim to introduce erroneous or deceptive samples into the training set of a legitimate AI algorithm.
This can be carried out by posting a large number of social media messages with deceptive content. For example, malicious actors who aim to trick social listening algorithms might use a large number of bot-generated tweets. Later on, those examples might be used for training/fine-tuning an ML algorithm by the vendors. The end result is that a social listening tool that has trouble dealing with adversarial attacks will have the wrong social sentiment data registered.
Another popular type of attack is an evasion attack - It refers to designing an input (for example, by changing some words to synonyms), which seems normal for a human but is wrongly classified by ML models. ML models have proven effective in detecting malicious emails, fake news, and harmful data traffic in real-time, but with the help of evasion attacks, malicious actors can bypass those filters.
I must say that the damage caused by adversarial attacks is not limited to wrong data in social media and trading software packages. It goes far beyond it – successful adversarial attacks can potentially distort markets in serious ways and cause a general distrust of AI systems’ capabilities in business and society.
Q: Is Adversarial ML used by malicious actors a big problem now?
PD: It’s a growing field, and we see many cases on social media where adversarial attacks are being orchestrated. There have been numerous research papers dedicated to Adversarial ML, so the area is no longer completely greenfield. I believe the practice of adversarial attacks is already quite widespread, but we will see many more news headlines related to it over the next couple of years.
Q: How does ZENPULSAR tackle the issue of Adversarial ML?
PD: We take the issue of adversarial attacks very seriously and use a comprehensive framework made up of several different processes. The multi-process framework is there to ensure that our SaaS social analytics platform, PUMP, and our API datasets are shielded from adversarial attacks.
Firstly, as part of our framework, we use our own adversarial scenarios to train our NLP models. By being trained on a large number of adversarial scenarios, our algorithms become highly capable of detecting them.
Secondly, we use up to 30 different AI algorithms to process the data we source from social media. The use of multiple algorithms helps ensure that there is no reliance on a single model and, therefore, on a single point of failure. Adversarial attacks are capable of deceiving some models some of the time, but they are going to be identified and cut down when they are against a wall of multiple high-quality AI algorithms.
Thirdly, we use a multi-stage process of data filtering that ensures that adversarial attacks are minimised.
I must note that our team has deep expertise in cybersecurity, which is a crucial discipline for tackling the problem of Adversarial ML. To the best of my knowledge, outside of internal systems used by some financial institutions, we are the only finance-oriented social listening platform that actively uses anti-adversarial measures.