Fine-Tuning SLMs vs. Prompt Engineering for LLMs in Finance: A Trade-off Experiment Between Performance and Cost

With the popularity of Large Language Models (LLMs), enterprises and developers face a core question: Do specialized domain tasks require trillion-parameter giant models? A study from the Cracow University of Technology provides an answer: A carefully fine-tuned Small Language Model (SLM) with 8 billion parameters can match or even surpass large commercial models in financial tasks while significantly reducing costs and latency. This post will break down the background, methods, results, and implications of this study.

Research Background

Current AI application development faces a dilemma: Commercial large model APIs are convenient but costly and carry data privacy risks; local deployment of open-source large models requires expensive hardware investment.

Core Hypotheses

Can a fine-tuned 8-billion-parameter model running locally achieve or surpass API-based proprietary LLMs in F1 score while significantly reducing computational overhead, latency, and operational costs?

Focused Tasks

The study targets two core tasks in the financial domain: financial text sentiment analysis and financial question answering (which require high accuracy and involve sensitive data).

Comparative Model Configuration

• Fine-tuned Model (SLM): Meta Llama3.1 8B Instruct, fine-tuned on a single NVIDIA T4 GPU using 4-bit QLoRA technology, with memory optimized via the unsloth library.
• Comparative Models (LLMs): OpenAI GPT-4o and GPT-4o-mini, using prompt engineering techniques such as zero-shot, few-shot, and Chain of Thought (CoT)

Datasets

• Sujet-Finance-Instruct-177k (general financial tasks)
• Financial PhraseBank (AllAgree subset, high-precision sentiment analysis)

Evaluation Metrics

Traditional metrics: weighted F1 score, precision, recall, accuracy; additional metrics: inference latency (milliseconds), inference cost (US dollars)

Fine-tuning Technologies

• QLoRA: 4-bit quantization + low-rank adaptation, reducing memory requirements to a level affordable for consumer GPUs
• unsloth library: Training speed increased by 2-5 times, allowing fine-tuning to be completed on Google Colab's free T4 GPU

Prompt Engineering Strategies

Multi-level schemes designed for commercial LLMs:

• Zero-shot: Test basic capabilities
• Few-shot: Provide examples to guide task understanding
• Chain of Thought (CoT): Show reasoning process to improve accuracy in complex tasks All prompts follow the Llama3.1 Instruct template to ensure cross-model fairness

Data Quality Assurance

• Advanced deduplication algorithm: Prevent cross-contamination between training and test sets
• Stratified sampling: Ensure balanced distribution of positive and negative samples in validation/test sets

Economic Feasibility Analysis

Real-time calculation of token costs and inference latency, constructing a cost-benefit framework to help decision-makers evaluate the cost recovery cycle of SLMs replacing commercial APIs

Result Trends

In the professional financial domain, targeted fine-tuned SLMs can undertake actual production tasks

Core Implications

• Significant value for small and medium-sized enterprises, privacy-sensitive institutions, and low-latency applications
• Provide open-source reproducible workflows (code + Colab notebooks) for cross-domain reference

Application Scenarios

Data-sensitive financial analysis, high-frequency report generation, cost-sensitive deployment, low-latency real-time applications

Limitations

• Large models still excel in general tasks
• Fine-tuning requires technical thresholds
• Performance depends on training data quality

• Original Author: Surgeon24
• Source: GitHub
• Original Title: Comparative Analysis: Fine-Tuned SLMs vs. Prompt-Engineered LLMs in Finance
• Link: https://github.com/Surgeon24/Financial-SLM-FineTuning
• Publication Date: June 12, 2026