LLM running on Xbox 360

<aside> <img src="https://pbs.twimg.com/profile_images/1736509360417771520/roB6Fuha_x96.jpg" alt="https://pbs.twimg.com/profile_images/1736509360417771520/roB6Fuha_x96.jpg" width="40px" /> Andrei David @AndreiDavid

With Xenon CPU (3.2GHz PowerPC w/ 3 cores) and 512MB unified RAM. Based on @karpathy's🩷llama2.c, ported to run on Microsoft's powerful console from 2005. Pure C implementation optimized for PowerPC architecture and Xbox memory management.

Inspired by @exolabs🤝

https://video.twimg.com/ext_tw_video/1877791200041738240/pu/vid/avc1/720x1280/9IvlxHTM-W8YxSsZ.mp4?tag=12

</aside>

Key challenge: Like PS3, Xbox 360's PowerPC is big-endian vs x86's little-endian. Had to handle endianness conversion for both model config and weights. Implemented careful byte-swapping and proper 128-byte memory alignment required by Xbox's memory subsystem.

</aside>

Memory management was crucial - had to carefully structure the 60MB model to fit in Xbox's unified memory architecture where both CPU and GPU share the same RAM pool. Different from PS3's split memory model. Each allocation needs to be properly aligned for optimal DMA.

</aside>

Fun fact: Xbox 360's memory architecture was ahead of its time. The unified memory approach is now standard in modern consoles/APUs. But in 2005, managing shared CPU/GPU memory space while keeping performance was a unique challenge. These constraints shaped a generation of games.

</aside>

Porting this LLM reminds me of early Xbox 360 dev days. Same careful memory management, same PowerPC quirks. The platform's constraints really make you think about efficient memory use. Some things never change - proper alignment still matters 18 years later! 🎮

</aside>

Where it started @exolabs @alexocheema @mo_baioumy🫡

https://x.com/exolabs/status/1873103218596811059

</aside>