[CDA][SS] DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation

1 minute read

[CDA][SS] DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation

paper : https://arxiv.org/abs/2307.15063
git: https://github.com/fy-vision/DiGA
CVPR 2023 (‘23.09.12 인용수 1회)
downstream task : UDA for semantic segmentation
Contribution
- warm-up stage:
  - Supervised learning뿐 아니라 feature-alignment loss를 teacher-student network끼리 주어 augmentation에 대해 robust한 generalizability를 확보
  - Cross-Domain-Mixture (CrDoMix) data augmentation을 주어 성능 향상
- self-training stage:
  - Bilateral-consensus pseudo-supervision으로 threshold-free self-training 기법을 제안
    - feature-induced labels (pixel-to-centroids)
    - probability based labels (from warm-up model)
Warm-up stage
- Supervised Loss → 기존 CE 와 동일
- Distillation Loss : Class 정보를 알고 있는 source dataset에 대해
  - teacher 한테 weak augmentation / student한테 strong augmentation을 주어 symmetric KD Loss를 줌
    - $F^+, \phi^+$ : teacher의 encoder, teacher의 segmentation head
      - teacher는 Ema update
    - $F, \phi$ : student의 encoder, student의 segmentation head
  - fine-grained label에 의한 ovefitting 방지 목적으로 soft-label 사용
- Cross-Domain-Mixture augmentation (CrDoMix)
  - Pretrained, Fixed Cycle GAN (Source 2 Target)을 활용!
  - binary mask를 사용하여 mixture
    - source data의 기하학적 모양을 헤치지 않음 → No label change!
    - 추가 batch 늘릴 필요가 없음
    - Target domain 정보를 흘려주며 학습 가능!
Threshold-free Self-Training
Centroids
- GAP: Global Average Pooling
- $x_{cdm}$ : 6번식 결과
- $\phi$ : warmup Model의 feature
- $\rho^k$ : k class의 centroid
- Warm-up Model을 활용해서 Class centroid를 추출함 → Noise label voting 시에 활용할 목적
- initial value : warmup model의 class-wise feature
Feature-induced labels
- centroid와 teacher feature 간의 제일 similar한 class로 jk pixel을 label → local structure
Probability-based labels
- pseudo-supervision based on Warmup Model prediction → global structure
Pseudo-supervison Loss
- $p_t$: student prediction
Full objective