Skip to content
Clothsy AI Talk to us
Clothsy AI Talk to us

FABRICVTON · RESEARCH PROGRAMME

Live virtual try-on for every garment and every body.

An eighteen-month research and development programme to build photorealistic virtual try-on for any garment and every shopper, first in photos and then in live video, the programme's end goal.

Start reading How live try-on works

Version 1.2 · 30 September 2026Public edition12 chapters

18
monthsin seven phases
7
work packagesimage, video, live and measurement
15
milestoneseach with its evidence
4
papersplus an optional cost report
2,000+
benchmark pairsat least, across 8+ garment families
119
sourcespapers, model cards and licences

TWO TRACKS, ONE GOAL

Image first, because the image model teaches the video model.

The benchmark comes first, so every decision is measured across garments, photos and bodies. The image model then re-dresses real, consented clips to train video, and the video model becomes the teacher for a live student.

  1. 01Measure

    A licensed, consented benchmark across garment families, regions, skin tones and body shapes.

    Months 1 to 6
  2. 02Image

    FabricVTON's own image try-on model, tuned on human preference and distilled to serve.

    Months 3 to 12
  3. 03Video

    The image model re-dresses real clips to teach an offline video try-on model.

    Months 4 to 14
  4. 04Live

    A causal student draws every camera frame live, at 15+ fps on one GPU.

    Months 12 to 18

THE SYSTEM

Seven work packages, three tracks.

Licensed data becomes a servable image model; the image model teaches video and live; a measurement track scores every checkpoint in both.

Image track

Data sourcesLicensed photoshootsLicensed cataloguesApache-2.0 teachersTry-off images
WP2 Data enginePairs and tripletsQuality filtersProvenance ledgerConsent records
WP3 Image modelQwen-Image-Edit-2511LoRA, rank 32Multi-view garmentsWear instruction
WP4 Tune and distilGlobal rater panelReward or judge4B student modelSpeed lab checks
Photo servingProduction serviceL4 and L40S GPUsCost per try-onmeasured

The image model re-dresses video frames as a teacher, and the same distillation recipe carries over.

Video and live track

Video sourcesConsented shootsLicensed motionPhone clipsWebcam clips
Video data engineReal target clipsRe-dressed inputsTemporal filtersHuman QA
WP6 Video modelOpen video baseGarment memoryPerson video inOffline quality first
WP7 Live modelCausal, frame by frameSelf-forcing training1 to 4 stepsKV cache
Live servingWebRTC streamOwn GPU servers15 to 20 fps targetCost per minute

The evaluation harness scores every checkpoint in both tracks.

Measurement and research-output track

WP1 Benchmark and audit2,000+ image test pairs300-clip video hold-out8+ garment families, 5+ regionsSkin tone and body strata
Evaluation harnessImage and video metricsHuman ratingsLive fps, latency, driftFairness measures
WP5 OutputsPapers 1 to 4Open toolkitBenchmark releaseOwn live model
Figure 1Proposed system overview: the image track, the video and live track, and the measurement track.

READ THE DOCS

Every chapter of the proposal.

From the problem and a review of the state of the art to the work packages, roadmap, budget and expected outputs. Every figure is cited to its source.

Chapter 0Start hereExecutive summaryThe programme, its approach and its outputs on one page.4 min read Chapter 1The problemThe problemWhy try-on still fails for many garments and shoppers, and why it must go live.7 min read Chapter 2AState of the artImage try-on todayArchitectures, reported results, industry systems, base models, datasets and evaluation.11 min read Chapter 2BState of the artVideo and live try-onVideo try-on, real-time generation, live products and the research gaps.10 min read Chapter 3AThe programmeTechnical approachObjectives, design principles, the seven work packages and the image track.9 min read Chapter 3BThe programmeVideo and live modelsThe video model, the live student, evaluation, hypotheses and compute.9 min read Chapter 4The programmeR&D roadmapPhases, schedule, milestones, publication calendar, governance and risks.7 min read Chapter 5The programmeWhat funding enablesWork already done, what funding unlocks, the budget and the funding scenarios.5 min read Chapter 6AThe programmeResearch outputsPapers, benchmark, models, software, indicators and open science.6 min read Chapter 6BThe programmeResponsible useBroader impact, the guardrail plan and live-session controls.4 min read AppendixReferenceGlossaryThe terms used across these docs.3 min read AppendixReferenceReferencesOne hundred and nineteen numbered sources.6 min read

THE END GOAL

A new frame every 33 milliseconds, on one GPU.

Live try-on redraws the shopper on their own camera, frame by frame, with a memory of the garment so a print stays the same print as they turn. How the field got here.

One frame, repeated about 30 times a secondShopper's deviceBrowser cameraConsent screenSession time limitNo photo storedUpstreamWebRTC videoOne-time session tokenNearest GPU regionGPU: live try-on modelCurrent camera frameGarment memory, set onceIts own recent framesDraws the next frameDownstreamSame person and poseWearing the garmentStreamed backAI-generated labelframe shown on screen; the next one is drawn about 33 ms laterGuardrails in the loopAge and consent check at session startSampled output moderation, watermark and label
Figure 2One frame of live try-on, from camera to screen. The loop repeats about 30 times a second.

SUPPORT THE PROGRAMME

Fund data, evaluation and researchers.

Research grants, compute partners and investors: request the full proposal or talk to the founders.

Investors Talk to us