S²COPE: A New Paradigm for Annotation-Free Self-Supervised Concept Discovery

The S²COPE (Self-Supervised Concept Discovery via Preference Learning) framework breaks the trade-off dilemma between scalability and interpretability of self-supervised methods in representation learning. It leverages Visual Large Language Models (VLLMs) as active participants in concept discovery, achieves annotation-free structured concept discovery through a self-supervised preference optimization loop, and delivers a 24-percentage-point improvement in downstream classification tasks across multiple domains.

Deep learning has achieved success in visual understanding, but faces challenges in interpretability: self-supervised methods (e.g., contrastive learning, masked autoencoders) can use unlabeled data for pre-training to generate powerful features, but these features lack semantic interpretability; interpretable methods like concept bottleneck models require large amounts of labeled samples, predefined concept vocabularies, and expert knowledge, which limits their scalability and applicability.

The core innovation of S²COPE is redefining the role of VLLMs as active participants, enabling concept discovery through an autonomous hypothesis-verification-reinforcement loop: 1. Hypothesis Generation: VLLMs propose candidate visual attributes from images; 2. Verification and Evaluation: A self-supervised mechanism assesses the consistency and discriminability of hypotheses; 3. Preference Optimization: Reinforce effective concepts based on results; 4. Iterative Refinement: Gradually build a structured concept system. Technical details include a preference learning mechanism (constructing positive-negative example contrast optimization), concept discovery strategies (semantic candidate generation, diversity sampling, progressive refinement), and end-to-end optimization of integrating concepts into the VLLM backbone network.

S²COPE performs excellently in multi-domain experiments: in the natural image domain, it discovers object parts, material properties, and scene features; in the medical imaging domain, it identifies pathological features, imaging patterns, and anatomical structures; in the physical science domain, it detects experimental device features and physical phenomenon patterns. In downstream tasks, compared to standard VLLM methods, it achieves a 24-percentage-point improvement in top-1 classification accuracy on unseen data, and has advantages in cross-domain generalization and data efficiency.

Compared to traditional self-supervised methods (e.g., SimCLR, MoCo), S²COPE provides explicit concept representations while retaining the advantages of self-supervision; compared to concept bottleneck models, it does not require predefined concepts or manual annotations; compared to zero-shot methods (e.g., CLIP), it can adaptively discover relevant concepts for specific datasets without being limited by pre-trained vocabularies.

S²COPE has broad application potential: in scientific discovery, it helps researchers find patterns that are hard to detect with the naked eye; in medical diagnosis, it automatically discovers subtle features to assist diagnosis; in content moderation, it identifies visual patterns of non-compliant content; in creative design, it assists in discovering key elements of visual styles.

Current limitations: Concept quality depends on the capabilities of the underlying VLLM, high computational cost, and limited modeling of hierarchical relationships between concepts. Future directions: Extend to multimodal data, discover hierarchical concepts, refine concepts with human feedback, and study cross-domain concept transfer mechanisms.

S²COPE is an important advancement in the field of explainable AI, proving that interpretability can emerge from raw data through autonomous model interaction without human supervision. It transforms VLLMs into active participants in concept discovery, enabling annotation-free structured concept learning, and provides new ideas for building more interpretable and reliable AI systems, with promising applications in more domains in the future.