Umetna inteligenca

Nazaj na temo. Že zelo kmalu prihaja obdobje pocenitev pomnilnika.

Empirični poskusi podjetja Bytedance kažejo, da bi bila različica modela "Full Bandwidth Transformer" lahko primerljiva z modelom, ki je dvakrat večji, kar pomeni znatne prihranke pri treniranju in sklepanju za enako zmogljivost.
Izračunajte koliko je vredno, da se stroški prepolovijo, če upoštevamo, da naložbe segajo v "trillion $".

Recycler predvsem izboljšuje učinkovitost sklepanja, zlasti pri matematičnih in drugih zahtevnejših nalogah. Ideja je, da model pri napovedi naslednjega tokena ne uporabi samo prejšnjega tokena, temveč v naslednji korak prenese tudi del skritega stanja iz prejšnjega koraka. S tem ohrani več notranje semantične informacije, ki bi se pri običajnem avtoregresivnem generiranju izgubila, ker se koraki povezujejo predvsem prek izbranih tokenov.

To naredi z lahkim adapterjem, zato avtorji trdijo, da je dodatna računska obremenitev pri generiranju zanemarljiva. Poročajo o boljši parametrski učinkovitosti: model z Recyclerjem naj bi v nekaterih primerjavah presegel modele z 2–3-krat več parametri, posebej če je arhitektura vključena zgodaj v učenje. Lahko se doda tudi pozneje, vendar so takrat pridobitve manjše.

Recycler poskuša izboljšati razmerje med kakovostjo sklepanja in velikostjo modela oziroma stroškom generiranja ne pa neposredno hitrosti ali kakovosti vsakega modela v vseh nalogah.

Kitajc Alibaba že ima svoje čipe Xuan Tie C950 na RISC-V arhitekturi v produkciji.

Independent technical analyses suggest that the C950’s performance profile and execution efficiency place it nearly on par with Apple’s transformative M1 processor. This indicates Alibaba’s ability to quickly close a multi-decade gap in architectural engineering. This momentum is reflected in the broader market: By January 2026, RISC-V had reached roughly 25% of global processor share, suggesting that the entrenched duopoly between x86 and Arm is beginning to loosen.

Zdej ima Qwen 3.8-27B z 30 tokens na sekundo brez grafične samo na CPU in ne rabi kupit Nvidije.

The flagship CPU Xuantie C950, released by Alibaba DAMO Academy in March this year, has recently achieved a significant milestone in its development history--successfully running the Tongyi Qianwen 3.8-27B large model natively.

As a 64-bit server-grade processor, the Xuantie C950 is manufactured using TSMC's 5nm process technology and integrates up to 64 computing cores internally. These cores are interconnected via a high-speed AMBA CHI bus, with 8 cores grouped together. The chip can reach a maximum clock speed of 3.2GHz. In terms of hardware architecture, it not only features an 8-instruction decoding width and a 16-stage pipeline design, which increases memory bandwidth by more than four times compared to its predecessor, the Xuantie C920, but also directly integrates matrix acceleration engines and vector acceleration engines optimized for AI inference workloads. Thanks to its outstanding design, its single-core general performance has exceeded 70 points in the SPECint2006 benchmark test, successfully setting a new performance record for RISC-V CPUs worldwide.

In practical performance, when driving the Tongyi Qianwen 3.8-27B model, the Xuantie C950 achieves a decoding speed of 30 tokens per second, with the first token latency stably controlled within 1.9 seconds. Traditionally, running such a large-scale model would require 16GB to 24GB of dedicated GPU memory. However, the Xuantie C950 can complete the entire inference process independently using just the CPU itself and the on-chip acceleration engine, eliminating the need for external dedicated graphics cards.