What is data poisoning?

Pro Search

3 steps completed
Answer
Data poisoning is a type of cyber attack that targets machine learning models by tampering with their training data. The goal of these attacks is to manipulate the model's behavior by introducing, modifying, or deleting data points in the training dataset. This can lead to the model making incorrect decisions or predictions once deployed. There are two main types of data poisoning attacks:
  1. Targeted Attacks: These aim to influence the model's behavior for specific inputs, such as making a facial recognition system fail to recognize a particular individual, without significantly degrading its overall performance.
  2. Nontargeted Attacks: These seek to reduce the model's general accuracy, precision, or recall by adding noise or irrelevant data points, thus degrading its performance across various inputs.
The success of a data poisoning attack depends on the stealthiness of the poisoned data to avoid detection during data cleaning or preprocessing, and the efficacy of the attack in degrading model performance. Data poisoning can have serious implications, especially as more businesses rely on AI and machine learning systems. It can be used for various malicious purposes, including disinformation, phishing scams, altering public opinion, or discrediting individuals or brands. To protect against data poisoning, organizations can employ robust data filtering, adversarial training, input validation, monitoring, and model transparency to identify and mitigate the impact of poisoned data.
Related
what are some examples of data poisoning attacks
how can data poisoning be detected and prevented
what are the consequences of data poisoning on machine learning models