在 Apple Silicon 上本地微调大模型：构建零成本金融情绪分析实时管道

This project explores using knowledge distillation and LoRA techniques on Apple MLX framework to fine-tune Qwen-2.5 model, building an end-to-end real-time financial social media sentiment analysis pipeline. It achieves zero API cost for generating structured trading signals, leveraging Apple Silicon's hardware advantages.

Financial market data is fast-changing, and social media discussions contain professional jargon, sarcasm, and implicit hints, leading to ambiguity for general LLMs. Traditional solutions relying on cloud APIs (like GPT-4) face high costs and latency issues when processing millions of real-time data streams.

The pipeline uses a 'small but precise' approach with 500M parameter model. Key components:

1. Data Ingestion: Reddit producer pushes raw text to Kafka's financial_raw_text topic.
2. Stream Processing: PySpark consumes data, extracts stock codes via regex, and calls local model via Spark UDF.
3. AI Inference: FastAPI hosts fine-tuned Qwen model to return structured JSON signals.
4. Storage & Visualization: Elasticsearch stores sentiment data, Kibana provides real-time dashboards. Data flow: Social media posts → Kafka Producer → PySpark → FastAPI+MLX → Elasticsearch+Kibana.

To enable small models to understand complex financial jargon, the project uses teacher-student distillation:

1. Teacher model (GPT-4o-mini) generates high-quality training data from zeroshot/twitter-financial-news-sentiment dataset, outputting formatted JSON (stock code, sentiment score -1.0~1.0, reasoning).
2. Convert data to Qwen's ChatML format.
3. Generate synthetic dataset (500 training,100 validation samples).
4. Optimize token packing (2048-token blocks) to avoid padding waste.

LoRA is used to avoid catastrophic forgetting and reduce memory usage:

1. Freeze Qwen-2.5 0.5B base model weights.
2. Inject small trainable LoRA adapters into 16 Transformer layers.
3. Hardware efficiency: Peak memory 4.85GB (possible on Apple Silicon). Hyperparameters: Batch size=8, iterations=1000, LoRA layers=16, learning rate=1e-5. Training speed: ~900-1000 tokens/sec on M-series chips; integrated W&B for loss monitoring.

Training loss dropped to 0.087 at 1000 iterations, but validation loss was best at 200 iterations (1.243) then rose (overfitting). Early stopping was used: selected adapters from 200 iterations, fused into base model to get fused-qwen-finance (inference latency <100ms).

Environment requirements:

• Hardware: Apple Silicon Mac (M1/M2/M3/M4)
• Containers: Docker Desktop (Kafka, Elastic Stack)
• API key: OpenAI (only for distillation phase)
• Python dependencies: mlx-lm, fastapi, uvicorn, pyspark, confluent-kafka, elasticsearch, openai. Deployment steps: 1. Run distillation to generate data;2. Local LoRA fine-tuning;3. Fuse model;4. Start Docker services (Kafka, ES, Kibana) and FastAPI;5. Launch PySpark stream and Kafka producer.

Key innovations:

1. Zero-cost inference (local execution post-deployment).
2. Deterministic formatted JSON output for downstream systems.
3. Domain specialization (understands financial jargon/sarcasm).
4. Edge optimization (uses Apple Silicon's unified memory, 4.85GB memory usage). Conclusion: Small models via distillation and fine-tuning can outperform large models in specific domains, reducing costs. This architecture can be migrated to other real-time, low-cost, specialized AI scenarios.