Hook
Over the past 12 months, the cost of inference per token has dropped 47% — but only for those on GPU clouds. A new player is challenging that assumption. Samsung SDS, the IT arm of South Korea's largest conglomerate, just launched NPU-as-a-Service using FuriosaAI's RNGD chip. The service is explicitly aimed at government workloads. No big tech names. No global ambitions. Just a focused bet on efficiency and sovereignty.
Hype fades; structure remains.
Context
FuriosaAI is a South Korean AI chip startup. Its second-generation chip, RNGD, is a domain-specific architecture (DSA) optimized for inference. Target performance: ~100 TFLOPS FP16 with a power envelope of ~65W. For comparison, NVIDIA's H100 consumes 700W for similar FP16 throughput. The efficiency gap is not incremental — it is structural.
Samsung SDS runs multiple data centers across South Korea, already serving government clients through its Samsung Cloud platform. By integrating RNGD chips, it now offers a native inference service that bypasses both NVIDIA's GPU dominance and the public cloud hyperscalers. The service is branded as NPUaaS — Neural Processing Unit as a Service. The customer base is narrow: South Korean central and local government agencies, public enterprises.
This is not a global play. It is a territorial fortification strategy.
Core
The technical narrative: inference efficiency as a competitive moat
Inference workloads are fundamentally different from training. Training requires massive matrix multiplications and high memory bandwidth. Inference, especially for real-time applications like document analysis, chatbots, and image recognition, prioritizes latency per token and energy per inference. NVIDIA's GPUs are overkill for most inference tasks — they burn power for capabilities the workload does not need.
RNGD is a purpose-built inference chip. Based on FuriosaAI's public disclosures, it uses a dataflow architecture that minimizes data movement. Each chip delivers roughly 100 TFLOPS FP16 at 65W. That yields 1.5 TFLOPS per watt — more than triple NVIDIA H100's ~0.4 TFLOPS/watt at FP16. In a government data center running thousands of inference jobs daily, this translates to direct savings on electricity, cooling, and server density.
Over the past 7 days, a protocol lost 40% of its LPs. Not here. Samsung SDS is not fighting for retail liquidity. It is building a walled garden for sovereign AI.
But efficiency is not empathy. The real differentiator is the product integration. SDS is likely bundling NPUaaS with its own AI solutions — document processing, smart city analytics, security monitoring. Government buyers do not buy chips. They buy solutions. SDS offers a single contract for hardware, software, compliance, and maintenance. That is a powerful lock-in.
The data-driven analysis: why NPUaaS beats GPU clouds in this segment
Let us model the total cost of ownership. A typical government inference deployment might require 1,000 concurrent inference streams for a natural language processing pipeline. Using NVIDIA A100 cards (with MIG partitioning), each card handles ~30 streams at 100W per stream (SXM version). Total power: ~3.3 kW for 1,000 streams. Using RNGD, assume each chip handles 20 streams (due to smaller batch sizes typical of real-time inference) at 65W. That requires 50 chips, total power: 3.25 kW — similar. But chip cost is different. NVIDIA A100 retails at ~$15,000 (used). RNGD, as a new entrant, may cost $3,000–$5,000 per chip. Capital expenditure drops 50–70%.
Furthermore, RNGD's lower power density allows for higher rack utilization. A standard 42U rack can hold 40 A100s (with cooling overhead) or 80 RNGDs. That doubles inference throughput per rack.
Based on my experience auditing hardware costs for Web3 infrastructure, the unit economics favor NPU by a factor of 2–3x for inference-heavy workloads. The catch: model compatibility. NVIDIA's CUDA ecosystem is a decade of optimizations. FuriosaAI provides a compiler stack based on LLVM and a proprietary runtime. Migration is not free.
The ecosystem friction
Every government AI model was built on PyTorch or TensorFlow with CUDA kernels. Porting to RNGD requires recompilation and potentially manual kernel rewrites for custom ops. FuriosaAI claims compatibility via ONNX Runtime and TFLite, but edge cases exist. SDS must provide a migration toolkit and support team. The cost of migration — in engineering hours — is a hidden tax on adoption.
Efficiency is not empathy. The chip may be efficient, but the transition is not frictionless.
Contrarian
The narrative of AI chip independence is overhyped
South Korea's government has long pushed for "semiconductor sovereignty." This NPUaaS fits that narrative. But the real bottleneck is not hardware — it is software. Most government AI applications are not compute-bound. They are data-bound. The latency in a government document analysis pipeline is dominated by API calls, database lookups, and human-in-the-loop reviews. The actual inference time is a fraction of total processing time. Optimizing the chip from 10ms to 5ms may not improve user experience if the database query takes 200ms.
The contrarian angle: Samsung SDS is solving a problem that does not exist at scale for its target customers. The government's AI workloads are not pushing the limits of NVIDIA's GPUs. They are underutilizing them. Migrating to NPU will save power and cost, but the operational overhead of managing a two-stack environment (training still on GPU, inference on NPU) may offset the savings.
Code doesn't feel. But IT departments do. The headache of maintaining two different hardware backends, two driver stacks, two monitoring systems — that is a hidden cost.
The blind spot: training is still NVIDIA's domain
Samsung SDS's NPUaaS explicitly targets inference. But every government AI project begins with training. Models are trained on NVIDIA GPUs, often in academic or private cloud environments. After training, the same models are deployed on GPU inference servers for consistency. Switching to NPU for inference introduces a training-inference gap. The FP16 precision or quantization behavior may differ slightly, leading to accuracy drift. Government clients, risk-averse by nature, may resist such a change.
The real disruption would be a chip that can do both training and inference efficiently. Until then, NPUaaS remains a niche — a profitable niche, but a niche nonetheless.
Takeaway
Samsung SDS's NPUaaS is a textbook example of structural alignment: the right chip, the right market, the right regulatory environment. It will likely succeed in capturing Korean government inference spend. But the lessons extend beyond South Korea. Every country with sovereignty concerns and budget constraints will eventually seek alternatives to NVIDIA's monopoly.
The future of AI infrastructure is not just about chips; it is about the stack. The company that makes the migration path seamless — that hides the hardware complexity behind a unified API — will win the next narrative cycle. Samsung SDS has the customer access. Now it needs the software maturity.
Hype fades; structure remains. The structure here is solid. The execution will determine whether this is a footnote or a blueprint.