NorthStar LLM API: Privatized AI Inference Service for Australian Enterprises

NorthStar LLM API is a privatized AI inference service tailored for Australian enterprises, built on Google's Gemma3 model. It addresses key pain points like data sovereignty, compliance with local regulations, and vendor lock-in risks. This service ensures all data processing stays within Australia, supporting both on-premises and local cloud deployments while offering OpenAI-compatible APIs to minimize migration costs.

Australian businesses, especially in regulated sectors (finance, healthcare, government), face critical data-related challenges:

• Data Localization: Strict laws require sensitive data to be processed/stored domestically.
• Cross-border Restrictions: Overseas data transfers trigger complex compliance checks.
• Vendor Lock-in: Dependence on foreign APIs risks long-term tech/cost reliance.
• Latency: Cross-border connections cause delays and unstable user experiences.

Developed by NorthStar Holdings Australia, this open-source GitHub project centers on:

• Data Sovereignty First: All processing/inference in Australia.
• Open Source Base: Uses Google's Gemma3 (open license, commercial-friendly, strong performance, flexible scales).
• Enterprise Deployment: Supports privatized setups to keep data under business control.

Flexible deployment modes:

1. Local Data Center: Full physical isolation, custom hardware, network control.
2. Australian Cloud: AWS Sydney/Azure Australia East deployments for compliance and elasticity.
3. Hybrid: Mix local (sensitive) and cloud (general) workloads with smart routing.

API Design: OpenAI-compatible interfaces (chat completions, embeddings, SSE streaming) allow seamless migration of existing apps.

Key measures for data sovereignty and safety:

• Localization: All data processing, storage, backups in Australia.
• Compliance: Aligns with Australian Privacy Principles (APPs) and sector-specific rules; full audit trails and data deletion support.
• Security: TLS1.3 encryption, RBAC access control, request throttling, input filtering against prompt injection.

Ideal for sectors needing strict data privacy:

• Finance: Document processing, compliance checks, risk assessment, secure customer service.
• Healthcare: Medical record analysis, literature retrieval, administrative automation, patient communication.
• Government: Official document processing, policy analysis, citizen services, sensitive info analysis.
• Legal: Contract review, case studies, legal research, private client consultations.

Advantages:

• vs Overseas APIs: Data stays local, lower latency, cost control, autonomy.
• vs Local Solutions: Open-source (no lock-in), API compatibility, enterprise-focused features, community support.

Future Directions:

• Expand to multi-modal (image/audio/video) and RAG integration.
• Add industry templates and system integrations.
• Extend to NZ, Southeast Asia, Europe (GDPR-aligned).