Boltzmann defined entropy as S = k_B ln Ω, where Ω counts microstates. Clausius gave the thermodynamic definition dS = δQ/T. Shannon later reused the same mathematics for information.
The second law
Entropy of an isolated system tends to increase. Hot coffee cools. Perfume spreads. Eggs don't unscramble. Time has a direction because probability favors disorder.
Entropy in the digital age
Shannon entropy H = −Σ p log p measures information surprise. It sets limits on compression, guides machine learning loss functions, and connects thermodynamics to data science.
Explore interactively
Open equation page