AI Revolution: VKAE System Boosts GPU Performance Up to 23x

While the race for computing resources continues in the artificial intelligence (AI) industry, the primary focus is shifting from creating new models to increasing the efficiency of existing infrastructure. The VKAE inference acceleration system, introduced by Vidraft, has made a giant leap in this direction. According to the developers, the new technology allows increasing the performance of existing GPU (graphics processors) by up to 23 times in certain scenarios without changing the hardware. This was reported by Ixbt.com news provides.

Interest in this technology is linked to the economics of modern AI services. While training a large language model happens once, the inference process — the stage of generating responses to user queries — is continuous. Inference costs determine the primary operating expenses of cloud services and corporate AI platforms. The VKAE system acts as a specific "software extension" for existing accelerators.

Next-Generation Optimization

While chip manufacturers focus on creating next-generation GPU devices, systems like VKAE strive to maximize existing capabilities by optimizing low-level software. This process involves rethinking computing cores and task scheduling mechanisms. According to ixbt.com, tests were conducted on the NVIDIA B200 graphics accelerator, and the results were higher than expected.

During tests, speeds several times higher than baseline systems were recorded across several models. Most importantly, the developers emphasized that no decrease in response quality or deterioration of model accuracy was observed during measurements. This allows for a drastic reduction in costs while maintaining the reliability of AI systems.

One of the most surprising results was recorded during the demonstration of the Qwen3.5-35B-A3B model. Under high-level parallel load, the system showed a generation performance of over 10 thousand tokens (text units) per second. However, for diverse queries in real-world conditions, this figure was approximately 455 tokens per second. This means that the efficiency metric directly depends on the nature of the load.

Integration and Openness

The distinctive features of the VKAE system include:

• High throughput on modern accelerators such as the NVIDIA B200;
• Full compatibility with OpenAI API interfaces;
• Ability to integrate into existing infrastructure with almost no changes;
• Reproducibility and transparency of results.

According to the project authors, the ability to independently verify results should be the main criterion for trust in such technologies. Therefore, the developers have also provided a special container containing model weights and the optimized environment. For now, the exact working mechanism of the VKAE system remains a secret, but a detailed scientific paper on the technology is expected to be published soon.

Such solutions are very important for regions like Uzbekistan, where AI technologies are still developing. This allows for the deployment of powerful AI services using the existing technical base without purchasing expensive new servers. This, in turn, reduces technological barriers for local startups and IT companies.