Implementation Plan for a Full-Stack AI Customer Service Chatbot Based on Dual-Model Architecture

This article introduces a student project from the Artificial Intelligence course of Bahauddin Zakariya University (Pakistan) in the Spring 2026 semester, demonstrating how to build a dual-model AI customer service system combining Naive Bayes and Artificial Neural Network (ANN). The project covers full-process technical details from NLP preprocessing to production deployment, with a tech stack including Python, NLTK, Scikit-learn, Keras, Flask, Supabase, etc., aiming to provide a flexible and efficient solution for customer service intent recognition.

Project Background: This project is a term project for the Artificial Intelligence course of Bahauddin Zakariya University (Pakistan) in the Spring 2026 semester, developed and maintained by WaleedChughtai-106. The source code is hosted on GitHub (link: https://github.com/WaleedChughtai-106/AI-Customer-Support-Chatbot). Dataset: Adopts Bitext's customer service intent dataset, containing 21,534 user utterances covering 27 customer service scenario intents (e.g., order, account, product, payment-related). The dataset is split into 70% training, 15% validation, and 15% testing to ensure the reliability of model evaluation.

The project implements a complete text preprocessing pipeline:

1. Tokenization: Use NLTK's punkt tokenizer to split words;
2. Cleaning and Standardization: Convert to lowercase, remove punctuation/special characters, filter numbers;
3. Stopword Removal: Use NLTK's stopword list to remove high-frequency low-information words;
4. Lemmatization: Reduce words to their base form via WordNet (e.g., running→run). This pipeline ensures the规范性 of input text and lays the foundation for subsequent feature extraction and model training.

Dual-Model Design:

• Naive Bayes Model: An efficient classification algorithm based on probability theory, with a test set accuracy of 98.58%. Its advantages include fast training/inference, low memory usage, and suitability for resource-constrained environments;
• ANN Model: A multi-layer perceptron built with Keras, with an accuracy of 98.45%. Its advantages include capturing complex non-linear patterns and strong expressive power. Switching Mechanism: Flexibly select models via the environment variable MODEL_TYPE (optional values: nb/ann/knn), supporting A/B testing or canary release in production environments.

Three-Layer Architecture:

• Data Layer: Supabase PostgreSQL stores chat history (user messages, replies, intents, confidence scores, etc.), with a session_id index to improve query efficiency;
• Business Layer: Flask backend provides RESTful APIs:
  • POST /api/chat: Receive messages and return replies, intents, and confidence scores;
  • GET /api/history/<session_id>: Query session history;
  • GET /api/status: System health check;
• Presentation Layer: Native HTML/CSS/JS chat interface, supporting model switching and session statistics display.

Deployment Process:

1. Push code to GitHub;
2. Create a Web Service on Render platform and link to the repository;
3. Configure environment variables (FLASK_SECRET_KEY, SUPABASE_URL, etc.);
4. Complete automatic deployment (Render automatically installs dependencies and downloads NLTK corpora). High Availability: Automatically fall back to Naive Bayes when ANN is unavailable; Testing: 38 automated tests (18 unit tests to verify preprocessing, 20 integration tests to verify API behavior), executed using pytest.

Performance Comparison:

Metric	Naive Bayes	ANN
Accuracy	98.58%	98.45%
Precision	98.59%	98.47%
Recall	98.58%	98.45%
F1 Score	98.56%	98.44%
Summary: The project is an excellent teaching case that demonstrates the construction of a full-process AI system. The dual-model design enhances flexibility, the complete pipeline ensures data quality, and engineering thinking (testing, deployment) helps transform prototypes into products, which has reference value for AI and full-stack learners.