How this calculation works
Normalize the entered category weights to probabilities and sum their information contributions in bits. Zero-probability categories contribute zero. The maximum uses all entered category slots, including any zero-weight ones.
pᵢ = weightᵢ/Σweight; H = −Σpᵢlog₂pᵢ; effective category count = 2ᴴA worked example
Weights 1,1,2 normalize to 0.25,0.25,0.5. Entropy is 1.5 bits, and the effective equally likely category count is 2^1.5 ≈ 2.828427. The three-slot maximum is log₂3.
Good to know
A finite discrete distribution, not continuous differential entropy or an entropy rate. At least one positive weight is required. Categories must be mutually exclusive outcomes for a probability interpretation. No password-strength, security or thermodynamic inference.
Reference: SciPy: discrete entropy definition (opens in a new tab) ↗
Normalize category weights before measuring information
Weights 1,1,2 have total 4 and therefore probabilities 0.25,0.25,0.5. Their contributions are 0.5,0.5,0.5 bits, giving total entropy 1.5 bits. Multiplying every weight by the same positive factor leaves those probabilities and the entropy unchanged.
The input entries describe category weights, not a list of measured outcomes. If your raw data contain repeated labels, count or otherwise assign a weight to each distinct category first. Supplying one entry per observation would ask a different question by treating those positions as separate category slots.
The grouped frequency table calculator can form explicit numeric bins before you interpret the resulting counts as mutually exclusive category weights.
Compare entropy with its declared category universe
With k entered category slots, the reference maximum is log₂(k), reached when all slots have equal positive probability. Three equal weights therefore have entropy log₂3; the default 1,1,2 distribution has less uncertainty than that reference.
Zero-weight slots remain in k. Thus 1,1 and 1,1,0 have the same entropy of one bit but different normalized entropies, because the second declares a three-slot universe. A one-slot universe has maximum zero and undefined normalized entropy, while its ordinary entropy is still zero.
An effective count is not an observed count
The effective equally likely category count is 2 raised to H. At 1.5 bits it is about 2.828427. This non-integer number describes the size of a uniform distribution with the same entropy; it does not claim that only 2.828427 categories were recorded.
The grouped frequency table calculator can organize numeric observations into explicit bins before category interpretation. The statistics calculator summarizes numeric magnitudes instead. Entropy ignores the spacing between category labels and does not, by itself, measure password security, randomness of a time sequence or the diversity of an unrecorded population.
A few questions, answered
Must weights sum to one?
No. The page explicitly divides each nonnegative weight by their positive total.
What does a zero probability contribute?
Zero entropy contribution by the limit convention for −p log₂p. The surprisal of that impossible category is not reported as a finite number.
Are the units bytes?
No. Base-two logarithms give bits of information per outcome under this finite distribution model.