Claude Shannon defined entropy for messages in 1948. More unpredictable data carries more bits per symbol; redundant data compresses further.
Explore interactively
Open equation pageAll articles
information theorycomputer science
Shannon Entropy Explained
H = −Σ p log p measures information surprise — and sets limits on compression and communication.
7 min read · 2026-06-11