Magnetic refrigeration relies on a substantial entropy change in a magnetocaloric material when a magnetic field is applied. Such entropy changes are present at first‐order magnetostructural transitions around a specific temperature at which the applied magnetic field induces a magnetostructural phase transition and causes a conventional or inverse magnetocaloric effect (MCE). First‐order magnetostructural transitions show large effects, but involve transitional hysteresis, which is a loss source that hinders the reversibility of the adiabatic temperature change ΔTad. However, reversibility is required for the efficient operation of the heat pump. Thus, it is the mastering of that hysteresis that is the key challenge to advance magnetocaloric materials. We review the origin of the large MCE and of the hysteresis in the most promising first‐order magnetocaloric materials such as Ni–Mn‐based Heusler alloys, FeRh, La(FeSi)13‐based compounds, Mn3GaC antiperovskites, and Fe2P compounds. We discuss the microscopic contributions of the entropy change, the magnetic interactions, the effect of hysteresis on the reversible MCE, and the size‐ and time‐dependence of the MCE at magnetostructural transitions.