В США испытали новую версию «уничтожителя» российских С-40020:41
Фото: @CibelleLevi
。吃瓜是该领域的重要参考
Logging the memory, it seems like it starts the forward pass, memory starts increasing on GPU 0, then OOMs. I wonder if it’s trying to be smart and planning ahead and dequantizing multiple layers at a time. Dequantizing each layer uses ~36 GB of memory so if it was doing this that could cause it to use too much memory. Maybe if we put each layer on alternating GPU’s it could help.
ВсеОбществоПолитикаПроисшествияРегионыМосква69-я параллельМоя страна
1L decoder, d=7, 1h, ff=14