The short answer Absolute positional encoding adds a fixed vector to every token embedding, once, at the bottom of the stack. RoPE rotates the query and the key instead, inside every attention layer. The 2017 Transformer paper picked sinusoids partly…
The short answer An encoder collapses many input lines into a short binary code. A decoder expands that code back into many output lines. TI’s SN74HC148 devices “encode eight data lines to 3-line (4-2-1) binary (octal)”, and its SN74HC138 reverses…
The short answer Flow control protects the receiver, while congestion control protects the network path. Both of them run inside the TCP sender, though. RFC 5681 settles the relationship in one line: “The minimum of cwnd and rwnd governs data…
The short answer Merge sort splits, sorts and merges, always in O(n log n), and it needs extra memory. Quick sort partitions around a pivot and sorts in place, averaging O(n log n) with an O(n squared) worst case. The…
The short answer An autoregressive model writes left to right, one token at a time. A diffusion LLM, by contrast, starts from a fully masked answer and unmasks tokens over repeated passes. LLaDA is the model that proved the second…
The short answer Muon does not simply replace AdamW, because a Muon run still contains AdamW. Muon is “an optimizer for 2D parameters of neural network hidden layers”. Everything else keeps the old optimizer, and the scaling paper says so…
The short answer GRPO is not PPO’s replacement. Its own authors call it “a variant of Proximal Policy Optimization (PPO)”. The change is narrow but consequential. PPO trains a separate value model to estimate the baseline. That model is “typically…
The short answer Willison and OWASP disagree about where jailbreaking belongs. Willison, who coined the term, treats prompt injection and jailbreaking as separate attacks. His test is simple: if trusted and untrusted strings were never concatenated, it is not injection.…
The short answer A Transformer attends to every token pair, so cost grows quadratically with window length. It also cannot model anything past that window. Mamba instead runs a selective state space model. Its SSM parameters become functions of the…
