The short answer A Transformer attends to every token pair, so cost grows quadratically with window length. It also cannot model anything past that window. Mamba instead runs a selective state space model. Its SSM parameters become functions of the…
The short answer Batch normalization and layer normalization do the same job on a different axis. Batch norm averages down a column: one feature, across every sample in the mini-batch. Layer norm averages across a row instead: one sample, across…
The short answer A dense model runs every parameter on every token. A mixture of experts instead routes each token through a few chosen experts. Mixtral’s router selects 2 of 8 experts. So it touches only 13B parameters per token,…
The short answer RLHF and DPO both start from the same preference pairs, one answer marked over another. RLHF trains a separate reward model first, then runs reinforcement learning against it, usually PPO. So DPO skips that separate network and…
The short answer A VANET is not a rival to a MANET. Instead, it is a specialised MANET, one where every node is a vehicle or a roadside unit. A general MANET, by comparison, covers phones, laptops, sensors, and even…
The short answer Self-attention and cross-attention run the same formula. Only the source of Q, K and V differs, though. In self-attention, one sequence supplies all three matrices, unlike cross-attention. The paper puts it plainly: queries, keys and values all…
The short answer 2D transformation uses a 3×3 matrix. 3D transformation uses a 4×4 matrix instead. Both rely on one added coordinate. So a 2D point becomes (x, y, 1). A 3D point becomes (x, y, z, 1). That extra…
The short answer Cache mapping decides where a memory block can sit in the cache. So direct mapping gives each block exactly one legal line. So lookup is fast, but collisions hurt badly. Fully associative mapping lets a block sit…
The short answer DSDV vs AODV comes down to when a route gets built. DSDV builds every route in advance, whether or not anyone needs it yet. So each node keeps a full table, listing every other destination, all the…
You cannot copy content of this page
