Nine Mainstream Large Models Give Nine Different Answers to the Same Working Hours Calculation Problem: AI Logical Consistency Benchmark Test Reveals Striking Disparities

A benchmark test on nine mainstream large models including GPT, Claude, Gemini, and Qwen shows that the same simple working hours calculation problem yielded completely opposite conclusions—ranging from "the employee owes the company 48 hours" to "the company owes the employee 160 hours"—exposing the severe inconsistency of current large models in logical reasoning and arithmetic calculation. This test was based on a real work scenario, and its results serve as an important warning for enterprises and individuals relying on AI for critical decision-making.

Large language models are increasingly used in scenarios such as calculation, legal reasoning, and human resources consulting. However, the latest benchmark test reveals that when faced with a working hours calculation problem involving basic arithmetic and clear rules, nine mainstream large models gave nine different answers—even the direction of the result (who owes whom) was inconsistent. The test question was not a trick question but based on a real work scenario, and the results indicate that large models have significant defects in logical consistency.

The test used a standard working hours calculation problem involving parameters such as annual standard working days, monthly salary benchmark (21.75 days/month), actual working days, and unused paid leave, requiring calculation of the working hours settlement result when an employee resigns. The nine models covered include:

• OpenAI: GPT 5.4 (Deep Thinking)
• Anthropic: Claude Opus4.7, Claude Sonnet4.6
• Google: Gemini3.1 Pro
• Alibaba: Qwen3 Max(Thinking), Qianwen3.5
• ByteDance: Doubao(Super Mode/Regular Mode)
• DeepSeek: DeepSeek(Expert Mode) All tests were conducted in April 2026 to ensure the timeliness of model versions.

The nine models gave nine different answers, with values ranging from -48 hours (employee owes company) to +160 hours (company owes employee), a span of 208 hours. Comparison of conclusions from each model:

Model	Conclusion	Calculation Result
GPT5.4(Deep Thinking)	Employee owes company	8 hours
Claude Opus4.7	Company owes employee	32 hours
Claude Sonnet4.6	Employee owes company	8 hours
Gemini3.1 Pro	Company owes employee	80 hours
Qwen3 Max(Thinking)	Company owes employee	48 hours
Qianwen3.5	Company owes employee	160 hours*
Doubao(Super Mode)	Company owes employee	40 hours
Doubao(Regular Mode)	Employee owes company	48 hours
DeepSeek(Expert Mode)	Company owes employee	40 hours
*Note: Qianwen3.5 initially gave 160 hours and self-corrected to 96 hours in the same response.

1. Directional Divisions: 6 models concluded the company owes the employee, while 3 concluded the employee owes the company—indicating that models have serious problems understanding the basic logical relationships of the problem.
2. Differences in Basic Assumptions: Different models used different annual standard working days (248/250/261 days), calculation benchmarks (21.75 days/month or actual working days), and handling methods for unused paid leave, directly leading to result discrepancies.
3. Self-Contradiction Phenomenon: Qianwen3.5 first gave 160 hours and then corrected it to 96 hours in the same response, exposing the instability of the model's reasoning process.

1. Limitations of Training Data: Training data comes from the internet and contains a large amount of unstructured and contradictory information, which is affected by noise when dealing with precise calculations and logical reasoning.
2. Differences in Reasoning Ability: Different model architectures and training methods have different focuses—some emphasize pattern matching, others have strong symbolic reasoning capabilities—and these differences are amplified in complex reasoning scenarios.
3. Prompt Engineering Sensitivity: Minor differences in wording may lead models to take different reasoning paths, affecting the reliability of practical applications.

1. Enterprise Risk Warning: Large models are not yet sufficient to handle precise logical reasoning tasks independently; strict manual review is required in critical business scenarios.
2. Model Selection Reference: In this test, Claude Opus4.7, DeepSeek Expert Mode, and Doubao Super Mode had relatively better result consistency, but verification based on specific scenarios is needed.
3. Future Improvement Direction: The test project is open-source (MIT license), and the community is welcome to contribute test cases to promote model improvement.

Large models have made significant progress in natural language understanding and generation, but they still have shortcomings in logical reasoning and precise calculation. It is unrealistic to treat AI as a one-size-fits-all solution; we need to understand its capability boundaries and establish usage norms. Before relying on AI for critical decisions, developers and users must fully verify and test, and not blindly trust the output.