Perturb

Abstract

We propose a decentralized adversarial robustness network built on Bittensor. Miners compete on two tracks. On the scanning track they find adversarial examples — imperceptible perturbations that make a state-of-the-art image classifier fail. On the training track they fine-tune that same classifier so it resists those attacks. Every two minutes a new task image is drawn from the full ImageNet-1k train split (1.28M images). Validators verify every response deterministically: exact class-index comparison, strict perturbation-norm bounds, perceptual quality gates (SSIM and PSNR), and a pixel hash that pins each submission to the image the miner committed to.

Scanning responses are scored on perturbation minimality — smaller, cleaner attacks earn more — with small bonuses for decision margin and spatial novelty. Exact duplicates are zeroed. Once a day, validators evaluate every adversarially trained model that miners have committed on-chain against clean ImageNet-1k and the previous day's adversarial examples. Miner emissions are split 80% to scanning and 20% to the day's best model. Both tracks are ranked by stake-weighted consensus across validators.

The result is a financially incentivized adversarial testing network that keeps improving. It produces two commercially valuable outputs — a continuously growing adversarial training dataset and model robustness certificates — and the training track shows, every day, that the dataset hardens real models. Together they address a $1.43B market growing at 26.1% annually.

Executive Summary

Every AI model deployed in production carries a hidden vulnerability: adversarial examples. An imperceptible change to a single image can cause a medical imaging model to misclassify a tumor, an autonomous vehicle to ignore a stop sign, or a fraud detection system to approve a fraudulent transaction. The tooling to systematically discover these vulnerabilities before deployment is fragmented, expensive, and — critically — static. It does not improve over time.

Perturb changes this. Built on Bittensor, Perturb is a decentralized adversarial robustness network where miners compete to find adversarial examples — inputs that fool AI models while remaining imperceptible to humans — and to train models that withstand them. Every attack is verified deterministically against the target model itself, imperceptibility is enforced mathematically with SSIM and PSNR gates, and every hardened model is evaluated on held-out clean and adversarial data. The network gets stronger every day, on both offense and defense.

The result is a financially incentivized adversarial testing and hardening infrastructure that keeps improving — something no centralized service can replicate.

The Problem We Solve

AI Models Are Brittle

Modern AI models achieve remarkable accuracy on clean test sets, yet remain catastrophically vulnerable to adversarial perturbations — mathematically crafted modifications to input data that are imperceptible to humans but cause models to fail completely. A ResNet-50 achieving 94% accuracy on ImageNet can be fooled by changing fewer than 0.3% of an image's pixels. An EfficientNetV2-L scoring 85.8% top-1 accuracy can misclassify a tabby cat as a fire truck with a perturbation whose maximum pixel change is bounded at 3% of the color range — invisible to any human observer.

This is not a theoretical concern. Adversarial attacks have been demonstrated against:

• Medical imaging classifiers used for cancer detection and diagnostic triage
• Facial recognition systems controlling physical access to secure facilities
• Autonomous vehicle perception systems for object and sign recognition
• Content moderation models protecting platforms from harmful material
• Financial fraud detection systems processing millions of transactions daily

The Market Opportunity

Three powerful forces are converging to create an urgent, large, and underserved market for adversarial robustness testing:

Regulatory Mandates: The EU AI Act classifies AI systems in healthcare, autonomous vehicles, critical infrastructure, and hiring as high-risk, requiring mandatory conformity assessments including robustness testing before deployment. Non-compliance carries fines of up to €30 million or 6% of global annual revenue.

Enterprise Procurement Requirements: Large enterprises increasingly require AI vendors to demonstrate robustness certifications before purchase. A verified robustness certificate is becoming table stakes for selling AI products into regulated industries.

AI Proliferation: As AI moves from research to production across every industry, the attack surface grows exponentially. Every new model deployment is a new vulnerability. Organizations deploying AI without systematic adversarial testing are accepting unknown, unquantified risk.

Why Existing Solutions Fall Short

Introducing Perturb

What Perturb Does

Perturb is a Bittensor subnet that incentivizes a global network of miners to find adversarial examples — images that fool AI classifiers while remaining visually indistinguishable from the original — and to train classifiers that stay correct on them. The network produces two commercially valuable outputs:

Why Bittensor

Competitive Improvement: TAO emissions reward the best-performing miners. Applied to adversarial attacks and defenses, this creates the first financially incentivized adversarial research network. Miners earn real money for finding better attacks and training more robust models — driving continuous improvement no salaried team can match.

Verification Asymmetry: Finding an adversarial example is computationally hard. Verifying one is trivially cheap: run the model, compare the predicted class index, measure the perturbation norms and perceptual similarity. Every check is deterministic and reproducible. This asymmetry makes Perturb's incentive mechanism clean, objective, and manipulation-resistant.

Decentralized Trust: Weights are set from a stake-weighted consensus of independent validators, and trained models are bound to their miner by an on-chain commitment. On-chain records of adversarial evaluations are more credible than any centralized company's self-reported certificate, and directly relevant to regulators seeking auditable compliance records.

Technical Architecture

System Overview

Perturb runs on a two-minute task loop coordinated through the Perturb task API. A task generator publishes one image from a persisted random traversal of the ImageNet-1k train split. Miners poll the API, attack the image, upload the perturbed result to their own storage, and submit its URL together with a pixel hash. Validators then download every submission and verify it deterministically: hash check, class-change check, perturbation-norm bounds, SSIM and PSNR gates, and exact-duplicate detection. Separately, once a day at 00:00 UTC, validators evaluate the adversarially trained models miners have committed on-chain.

Task Pipeline

Tasks come from a large, auditable corpus: the full ImageNet-1k train split (ILSVRC/imagenet-1k, 1.28M images). The task generator fetches one row at a time through the Hugging Face datasets-server API, so the 150 GB split is never downloaded. Rows are visited in a seeded random permutation that is persisted to disk, so no image repeats until the whole split has been used; the permutation then reshuffles for the next epoch. Ground truth needs no external labeling service: the target model's own prediction on the clean image is the label, so a successful attack is defined unambiguously as changing the model's output.

def generate_and_publish_task(hotkeys) -> GeneratedTask:
    # Next row of the persisted no-repeat random permutation
    row      = order[cursor]; cursor += 1; save_state()
    image    = imagenet1k_rows.fetch(row)

    # Ground truth = the target model's own prediction
    label    = predict_label(efficientnet_v2_l, image)

    task_id  = f"{unix_time}-hf-{dataset_version}-{row:07d}"
    url      = storage.upload("perturb/tasks/current.png", image)
    post_task(task_id=task_id, image_url=url, hotkeys=hotkeys)

Tasks follow a fixed 120-second cadence aligned to wall-clock boundaries. Miners prefetch just before each boundary and pick up the new task the moment it is published. The API closes submissions and moves the task to validating; validators then fetch the task image, re-run the target model on it themselves, and score every submission received for that task.

Deterministic Verification & Perceptual Gates

Verification is fully deterministic and self-contained. Labels are normalized (strip → lowercase → "_" → " ") and resolved to model class indices — including comma-separated ImageNet label aliases — so success is judged by exact class-index comparison, never fuzzy string matching. On top of the norm bounds, two perceptual gates mathematically enforce imperceptibility:

Because both the clean image and the miner's submission are quantized to the uint8 grid before any norm is measured, every reported L∞ value is an exact multiple of 1/255, and every result can be reproduced bit-for-bit by any third party running the same open-source code. The pixel hash is computed over the decoded RGB buffer, so it is stable across lossless PNG re-encodes but catches any change to the content.

Task & Submission API

All coordination runs through four API contracts. Miners do not serve an axon; validators do not broadcast challenges.

// task generator → API
POST /task      { "task_id": "{unix}-hf-{version}-{row}", "imageURL": "https://…/tasks/current.png" }

// miners read the current task
GET  /task      → { "task_id", "imageURL" }

// miners submit their perturbed image
POST /submits   { "task_id", "imageURL", "imageHash": "sha256 over decoded RGB pixels" }

// validators read submissions (Bearer auth, while status = validating)
GET  /submits   → [ { "miner_uid", "imageURL", "imageHash" }, … ]

Miner: Response

Miners return only the perturbed image. The attack method and parameters are entirely proprietary — the miner's competitive edge. Perturb ships a working baseline miner — a 10-step untargeted PGD attack against the local EfficientNetV2-L at ε = 0.03 — so new participants can join immediately. Competitive miners replace it with stronger, more minimal attacks to win a higher rank.

image_url  = storage.upload(perturbed_png)
image_hash = sha256(decoded_rgb_pixels(perturbed_png))
POST /submits { "task_id": task_id, "imageURL": image_url, "imageHash": image_hash }

Scoring: Per-Task

Each miner response passes through strict verification gates. Any hard-fail returns 0.0 immediately:

Scoring for responses that pass every gate rewards perturbation minimality — the smaller and cleaner the perturbation, the higher the score. Both components are squared, so improvements near the minimal end of the range are rewarded super-linearly:

effective_max = min(epsilon, 0.03)

linf_ratio = clamp((norm − 0.003) / (effective_max − 0.003), 0, 1)
linf_score = (1 − linf_ratio) ** 2

rmse_ratio = clamp(rmse / effective_max, 0, 1)
rmse_score = (1 − rmse_ratio) ** 2

perturbation_score = 0.7 * linf_score + 0.3 * rmse_score

margin_score  = clamp((best_non_true_logit − true_logit) / 10, 0, 1)
novelty_score = clamp(changed_pixels / 8, 0, 1)

score = 1.0 * perturbation_score + 0.03 * margin_score + 0.01 * novelty_score

# Anti-copy: identical positive-score submissions (exact sha256 of the
# decoded image bytes) are all zeroed.
zero_duplicate_responses(results, response_hashes)

Every scored task is appended to each miner's rolling history; registered miners that did not submit receive 0 for that task. A miner's reported average covers the last min(300, longest miner history) tasks and stays at 0 until the miner has that many records, and no averages are reported at all until some miner reaches 50. Newcomers and intermittent miners therefore cannot report a competitive average until they have proven sustained performance.

Adversarial Training Track

The scanning track proves models can be broken; the training track proves they can be fixed. Miners fine-tune EfficientNetV2-L on ImageNet-1k plus the Perturb adversarial dataset, publish the weights on Hugging Face, and commit them on-chain. The repository ships the full pipeline: training/train.py (clean cross-entropy, adversarial cross-entropy, and a TRADES-style KL consistency term), training/evaluate.py (clean accuracy, adversarial accuracy, flip rate, and white-box PGD at 1, 2, and 4 / 255), and training/submit.py.

{ "r":  "hf_user/repo",                         // Hugging Face repo, ≤ 44 chars
  "rv": "<commit sha>",                          // pinned revision, never "main"
  "h":  sha256(model.safetensors ‖ hotkey)[:16] }

The hash binds the weights to the committing hotkey, so a copied model cannot reuse another miner's commitment. The repository stays private until the commitment is on-chain and is made public one block later, so nobody can copy a model and claim it first. Validators evaluate the commitment snapshot from a day earlier, so a new model enters evaluation the day after it is committed.

Once a day at 00:00 UTC, every validator evaluates each committed model against the original EfficientNetV2-L as a baseline, on identical data:

• Clean accuracy: 1,000 images streamed from the ImageNet-1k validation split.
• Robust accuracy: up to 400 rows of the adversarial dataset created in the previous 24 hours, read at the dataset commit pinned before 00:00 UTC. A row's label is the original model's prediction on its clean image.
• Unpredictable draw: the sample seed is sha256(date : dataset revision) — identical for every validator, but impossible to compute before the day's data closes, so miners cannot train on the evaluation images.

Scoring: On-Chain Weight Setting

Weights are set once per epoch, inside the final 100 blocks before the epoch boundary, so all validators submit at roughly the same moment. At that point each validator fetches every other validator's leaderboard report (validators are hotkeys with a validator permit and more than 10,000 stake) and builds a stake-weighted consensus ranking:

reports  = [(stake(v), leaderboard_avg_scores(v)) for v in validators]
ranked   = sort_by(stake_weighted_ranks(reports))           # lowest rank first

# Scanning: 75 / 20 / 5 to the top three (1 miner → 100%, 2 → 75 / 25)
scanning = ranked_emission_shares(ranked[:3])

# Track blend: 80% scanning + 20% daily model winner
miners   = blend_track_weights(scanning_shares=scanning,
                               model_winner_uid=consensus_model_winner(),
                               scanning_share=0.8, model_share=0.2)

# Burn: live burnRate from the API goes to UID 0
weights  = (1 − burn_rate) * miners;  weights[0] += burn_rate
set_weights(weights)

A miner that leads both tracks receives 0.8 × 0.75 + 0.2 = 80% of the miner allocation. If there is no model winner, scanning receives the whole miner allocation; if no miner has a positive scanning score, the model winner receives it. If no consensus data is available at all, weight setting is skipped for that cycle.

Leaderboard Reporting & Dataset

Two production subsystems connect the incentive loop directly to Perturb's commercial outputs — both live today, not on a roadmap:

Signed leaderboard reporting. After every scoring round, validators submit a report to the Perturb API covering network-wide metrics (average score, average L∞ norm, average RMSE, average margin, success count) and per-miner results for every registered miner — each classified as Valid, Invalid, Duplicate, Inactive, or Unavailable, with a score history graph and a link to the miner's image. Reports are signed with the validator hotkey and queued in a background thread, so API latency can never affect scoring. The same reports are the input to on-chain weight consensus, and they power the public leaderboard at perturbai.io in near real time. Daily model evaluations are reported the same way.

Adversarial dataset. Verified adversarial examples are published to the public Hugging Face dataset perturb-ai/efficientnet-v2-l-adv-dataset as timestamped parquet shards. Each row pairs a clean ImageNet image with its verified adversarial versions, and the dataset has fixed train, validation, and test splits. It grows roughly every two minutes. It is the training data for the training track and the evaluation data validators use to score it — the dataset described in Section 05, generated as a byproduct of network operation.

Target Model: EfficientNetV2-L

Perturb operates with EfficientNetV2-L as the sole target model on both tracks — a deliberate choice prioritising network stability, miner onboarding, and validation credibility over premature complexity.

EfficientNetV2-L sits at the ideal intersection of attack difficulty, hardware accessibility, and research credibility. At 118.5M parameters it is a genuinely modern production-scale classifier — hard enough that naive single-step attacks perform poorly within the strict 0.03 L∞ ceiling and perceptual gates, requiring stronger iterative methods — yet still attackable on a single consumer GPU within the two-minute task window, and fine-tunable on a single workstation GPU.

Expansion Roadmap

Revenue Model

Adversarial Training Dataset — Short-Term Revenue

Adversarial training — retraining models on adversarial examples — is the most effective known defense against adversarial attacks. Perturb generates this data continuously as a byproduct of its core operation, and the training track measures its value every day by scoring models trained on it. The EfficientNetV2-L dataset is public on Hugging Face and acts as the research tier and conversion funnel. Paid tiers add higher volume, real-time delivery, and adversarial data generated against the customer's own architectures.

Model Robustness Testing Service — Long-Term Revenue

Organizations submit an image classification model — as a Hugging Face link or an inference endpoint — together with a labeled sample of production inputs. Perturb attacks it in an isolated sandbox with white-box and black-box methods across multiple perturbation budgets, then deletes the model. Each report includes:

• Adversarial Robustness Score (0–100): the share of clean accuracy that survives attack across budgets
• Clean versus robust accuracy at ε = 1/255, 2/255, and 4/255
• Real adversarial examples that fooled the model, visualized and downloadable
• Perceptual failure analysis — SSIM, PSNR, and norm profiles, not just pixel statistics
• Comparison against reference architectures and published AutoAttack benchmarks
• A reproducible protocol with a fixed seed, and on-chain proof of evaluation

The Pro plan is the commercial form of the training track: the same fine-tuning recipe miners compete on, applied to the customer's model, with clean accuracy kept within an agreed tolerance.

Go-To-Market Strategy

Target Customers

Phased Launch Plan

Competitive Moat

Self-Improving Attack Quality: Every day miners compete, the network gets better. The dataset becomes more valuable. The certificates become more credible. No centralized service has this compounding property.

Offense and Defense in One Network: The same incentive mechanism that finds attacks also rewards the models that resist them. Hardening is measured daily on data no one could train on in advance, against the original weights as a baseline — so improvements are real, not benchmark overfitting.

Perceptually Gated Precision: Unlike tools that only measure pixel norms, every Perturb example must simultaneously clear an L∞ ceiling, an SSIM floor, and a PSNR floor while flipping the model's exact class index — producing genuinely imperceptible, higher-quality data than norm-only pipelines.

Deterministic, Reproducible Verification: Every score can be independently recomputed bit-for-bit from open-source code, with responses quantized to a canonical 8-bit grid and pinned by a pixel hash. There is no oracle to dispute, no subjective judge — just mathematics that any auditor, regulator, or customer can rerun.

Consensus and On-Chain Accountability: Weights come from a stake-weighted consensus of independent validators, and every trained model is bound to its miner by an on-chain commitment. Records on Bittensor cannot be altered, backdated, or selectively disclosed — categorically different from any vendor's self-reported compliance documentation.

Architecture Diversity: As Perturb adds target models, miners who specialize build irreplaceable expertise. A miner optimizing attacks for six months outperforms any general-purpose tool.

Market Analysis

Conclusion

AI models are increasingly embedded in decisions that affect human safety, financial stability, and personal rights. Yet the vast majority of these models are deployed without systematic adversarial robustness testing — not because organizations don't care, but because the tooling to do so at scale simply doesn't exist.

Perturb changes this. By applying Bittensor's competitive incentive mechanism to both adversarial attack and adversarial defense — and enforcing quality with deterministic class-index verification, hash-pinned submissions, and mathematical perceptual gates — Perturb creates a network that gets measurably better every day. Miners are financially motivated to become the world's best adversarial attack researchers and to train the models that withstand them. Validators verify results with mathematical precision that any third party can reproduce, and set weights by stake-weighted consensus. The pipelines that produce Perturb's commercial outputs — live leaderboard reporting, a continuously growing public adversarial dataset, and daily evaluation of hardened models — are already running.

The validation mechanism is airtight. The market is large and accelerating. The Bittensor architecture provides an unfair advantage no centralized competitor can replicate. And with perceptually gated, fully reproducible scoring and a training track that proves the data works, Perturb produces higher-quality adversarial data than any existing tool.