reasoning_effort parameter and a corresponding reasoning/tool-calling controller FSM to consume it!🔥 Got any pics of this rig? Would love to see how it's managing thermals.
The way the coolers on these cards are designed is VERY unusual - this photo should come in the box! If you're having trouble in a closed case, see if you have space to add an intake on the bottom beside the PCIe edge - if all air is coming from the front like in this pic, the rear blower steals it all!
I am running 3x140mm 89CFM as intakes on the front and directly underneath the in-blowers of those two 3090FE is the main trick - a dedicated 140mm 140CFM blowing upwards to feed the intake blowers at the front/pcie edge of these coolers. The cross-blower air passes through both cards and then has space to vent their heat on the right.
At 280W load the 'outside' card is bored and sitting at 50C with the blower fans at 50% while the 'inside' card maintains 60-65C with it's blowers closer to 70-80%.
The main thing to be careful about with the FE dual-blower coolers is do not try to pump air into the 'front' (pcie side) of them! I see this configuration on many mining rigs and its fine for air cooled cards but the FEs actually have a blower venting out this pci-slot side so you HAVE to feed them from either the rear or underneath.
When I had the 4-slot bridge installed across them, the rear-feed alone was sufficient.
The top high density rack is successfully hitting 65C targets at 1.15kW TDP.
Really impressive coolers on the FEs, there is barely 10mm of space but the sandwiched one still keeps up to unobstructed air coolers.
We are back, baby 🏇
hidden_size, intermediate_size and num_hidden_layers are all different: not only are there more layers as the model grows larger, the layers themselves grow larger.num_attention_heads and num_key_value_heads are also all different: the larger models split the attention into more groups.
