Entropy

Definitions: Entropy

Entropy

Noun

1. (thermodynamics) a measure of the amount of energy in a system that is available for doing work; entropy increases as matter and energy in the universe degrade to an ultimate state of inert uniformity.

2. (communication theory) a numerical measure of the uncertainty of an outcome; "the signal contained thousands of bits of information".

Date "entropy" was first used: 1868. (references)

Specialty Definitions: Entropy

Domain	Definitions
Computing	Entropy A measure of the disorder of a system. Systems tend to go from a state of order (low entropy) to a state of maximum disorder (high entropy). The entropy of a system is related to the amount of information it contains. A highly ordered system can be described using fewer bits of information than a disordered one. For example, a string containing one million "0"s can be described using run-length encoding as [("0", 1000000)] whereas a string of random symbols (e.g. bits, or characters) will be much harder, if not impossible, to compress in this way. Shannon's formula gives the entropy H(M) of a message M in bits: H(M) = -log2 p(M) Where p(M) is the probability of message M. (1998-11-23). Source: The Free On-line Dictionary of Computing.
Aerospace	1. A measure of the extent to which the energy of a system is unavailable. A mathematically defined thermodynamic function of state, the increase in which gives a measure of the energy of a system which has ceased to be available for work during a certain process: ds = (du + pdv)/T >= dq/T where s is specific entropy; u is specific internal energy; p is pressure; v is specific volume; T is Kelvin temperature; and q is heat per unit mass. For reversible processes, ds = dq/T In terms of potential temperature , ds = cp (d/) where cp is the specific heat at constant pressure. See third law of thermodynamics. In an adiabatic process, the entropy increases if the process is irreversible and remains unchanged if the process is reversible. Thus, since all natural processes are irreversible, it is said that in an isolated system the entropy is always increasing as the system tends toward equilibrium, a statement which may be considered a form of the second law of thermodynamics.2. In communication theory, average information content. (references)
Chemistry	Function of state whose differential during a reversible transformation is dS = delta Q/T, where delta Q is the thermal energy supplied to the physical system by the external environment at thermodynamic temperature T. Source: European Union. (references)
Energy	A measure of the unavailable or unusable energy in a system; energy that cannot be converted to another form. (references)
Mining	A. A measure of the unavailable energy in a system; i.e., energy that cannot be converted into another form of energy b. A measure of the mixing of different kinds of sediment; high entropy is approach to unmixed sediment of one kind c. Ratio of amount of heat added to air to the absolute temperature at which it is added. Measured in Btu d. Specific entropy is the ratio of entropy to weight of substanc i.e., energy that cannot be converted into another form of energy. (references)
Source: compiled by the editor from various references; see credits.

Top

Specialty Definition: Information entropy

(From Wikipedia, the free Encyclopedia)

Entropy is a concept in thermodynamics (see thermodynamic entropy) and information theory. The two concepts do actually have something in common, although it takes a thorough understanding of both fields for this to become apparent.

Claude E. Shannon defined a measure of entropy (H = - Σ p_i log p_i) that, when applied to an information source, could determine the capacity of the channel required to transmit the source as encoded binary digits. Shannon's measure of entropy came to be taken as a measure of the information contained in a message, as opposed to the portion of the message that is strictly determined (hence predictable) by inherent structures, like for instance redundancy in the structure of languages or the statistical properties of a language relating to the frequencies of occurrence of different letter or word pairs, triplets etc. See Markov chains.

Entropy as defined by Shannon is closely related to thermodynamic entropy as defined by physicists and many chemists. Boltzmann and Gibbs did considerable work on statistical thermodynamics. This work was the inspiration for adopting the term entropy in information theory. There are deep relationships between entropy in the thermodynamic and informational senses. For instance, Maxwell's demon needs information to reverse thermodynamic entropy and getting that information exactly balances out the thermodynamic gain that the demon would otherwise achieve.

In information theory, entropy is conceptually the actual amount of (information theoretic) information in a piece of data. Entirely random byte data has an entropy of about infinity, since you never know what the next character will be. A long string of A's has an entropy of 0, since you know that the next character will always be an 'A'. The entropy of English text is about 1.5 bits per character (Try compressing it with the PPM compression algorithm!) The entropy rate of a data source means the average number of bits per symbol needed to encode it.

Many of the bits in the data may not be conveying any information. For instance it is often the case that data structures store information redundantly, or have sections that are always the same regardless of the information in the data structure.
The amount of entropy is not always an integer number of bits.

Entropy is effectively the strongest non-lossy compression possible, which can be realised in theory by the use of the typical set or in practise using Huffman, Lempel-Ziv or Arithmetic coding. The definition of entropy is based on the Markov model of text. For an order-0 source (each character is selected independent of the last characters), the entropy is:

Where is the probability of . For a second-order Markov source (one in which probabilities are dependent on the preceding character), the entropy rate is:

Where is a state (certain preceding characters) and is the probability of given as the previous character (s).

Source: adapted by the editor from Wikipedia, the free encyclopedia under a copyleft GNU Free Documentation License (GFDL) from the article "Information entropy."

Top

Synonyms: Entropy

Synonyms: information (n), selective information (n). (additional references)

Antonym: ectropy (n). (additional references)

Top

Crosswords: Entropy

English words defined with "entropy": conformational entropy ♦ Heat weight ♦ isentropic, Isentropic lines ♦ second law of thermodynamics ♦ third law of thermodynamics. (references)

Specialty definitions using "entropy": alternate polarity, average information content ♦ Boltzmann equation, Boltzmann relation ♦ isentrope ♦ Proton/Neutron conversion ♦ steam tables ♦ thermodynamic probability. (references)

Etymologies containing "entropy": Entropium ♦ isentropic. (references)

Top

Modern Usage: Entropy

Domain	Usage
Movie/TV Titles	Ecstasy in Entropy (1999)
Source: compiled by the editor from various references; see credits.

Top

Commercial Usage: Entropy

Domain	Title
Books	Complexity, Entropy and the Physics of Information: The Proceedings of the 1988 Workshop on Complexity, Entropy, and the Physics of Information Held (reference) Entropy and Art: An Essay on Disorder and Order (reference) Information, Entropy, and Progress: A New Evolutionary Paradigm (reference) Mother Nature's Two Laws: Ringmasters for Circus Earth--Lessons on Entropy, Energy, Critical Thinking and the Practice of Science (reference) (more book examples)

Theater & Movies	Entropy (reference) (more DVD examples; more video examples)

Music	Entropy Lingua (reference) (more classical music examples; more popular music examples)
Source: compiled by the editor from various references; see credits.

Top

Usage Frequency: Entropy

"Entropy" is generally used as a noun (singular) -- approximately 100.00% of the time. "Entropy" is used about 238 times out of a sample of 100 million words spoken or written in English. Its rank is based on over 700,000 words used in the English language. Some parts-of-speech are not covered due to the samples used by the British National Corpus. (note: percents less than one-hundredth of one percent have been omitted)

Parts of Speech	Percent	Usage per 100 Million Words	Rank in English
Noun (singular)	100%	238	19,410

Source: compiled by the editor from several corpora; see credits.

Top

Expression: Entropy

Expression using "entropy": conformational entropy. Additional references.
Hyphenated Usage
	Beginning with "entropy": entropy-enthalpy, entropy-maximizing.
	Ending with "entropy": enthalpy-entropy.
Source: compiled by the editor from various references; see credits.

Top

Frequency of Internet Keywords: Entropy

The following statistics estimate the number of searches per day across the major English-language search engines as identified by various trade publications. Hyperlinks lead to commercial use of the expression at Amazon.com.

Expression	Frequency per Day	Expression	Frequency per Day
entropy	177	entropy thermodynamic	3
definition entropy	12	entropy service	3
entropy law	12	define entropy	3
maximum entropy	10	0.5 entropy	3
entropy search	6	entropy picture	3
civil engineering entropy phd project	6	entropy renyi	3
0.7 download entropy	6	entropy phd project	3
entropy thermodynamics	5	cost entropy function matlab	2
entropy information	5	entropy function	2
enthalpy entropy	4	entropy increment wong	2
entropy gradient reversal	4	chat entropy home	2
entropy movie	4	aegis entropy	2
entropy shannon	3	entropy evolution	2
banner entropy	3	entropy social	2
		band entropy	2
Source: compiled by the editor from various references; see credits.

Top

Modern Translations: Entropy

Language		Translations for "entropy"; alternative meanings/domain in parentheses.
Chinese		熵 . (various references)

Danish		entropi. (various references)

Dutch		entropie. (various references)

Finnish		entropia, haje. (various references)

French		entropie, néguentropie. (various references)

German		entropie (average information content). (various references)

Greek		εντροπία, £ñ ¿ªºÕα. (various references)

Hungarian		entrópia. (various references)

Italian		entropia. (various references)

Japanese Kanji		エンジン発動機 (angel, angel baby, angelfish, embassy, emblem, emboss, embroidery, empathy, emperor, emphasis, emphasize, empire, Empire Day, Empire State Building, empress, emptiomania, empty, empty nest, empty nest syndrome, end, end curler, end line, end user, -endian, ending, endive, endless, endless tape, endorphin, engine, engine stop, engineering plastics, enhancement, entasis, enter, enterprise, entertainer, entertainment, entitled, entity, entrance, entry, envelope, hit-and-run, two-base entitlement). (various references)

Japanese Katakana		エントロ�"ー , エントロ�" . (various references)

Korean		�"트로�"�. (various references)

Pig Latin		entropyay

Portuguese		entropia. (various references)

Russian		энтропия. (various references)

Serbo-Croatian		entropija. (various references)

Spanish		entropía. (various references)

Swedish		entropi. (various references)

Turkish		entropi, kullanılmaz enerji miktarı. (various references)
Source: compiled by the editor from various translation references.

Top

Ancestral Language Translations: Entropy

Language	Period	Translations
Greek	700 BCE-300 CE	entropia. (various references)
German	100 BCE-Modern	Entropie. (various references)
Source: compiled by the editor from various references.

Top

Misspellings: Entropy

Misspellings
	"Entropy" is suggested in spellcheckers for the following: antropy, enthropy, entopy, entray, entrepy, entripy, entro, entrogy, entrop, entroph, entrophy, entropi, entropic, entropion, entroppy, entropt, entrpy, ethoxy, extropy, Eythrope, intropy, Metropia, Netrope. (additional references)
Source: compiled by the editor, based on several corpora (additional references).

Top

Rhyming with "Entropy"

# of Phoneme Matches	Pronunciation	Word(s) rhyming with "entropy" (pronounced e"ntrupē)
4	-r u p ē	chemotherapy, hydrotherapy, immunotherapy, philanthropy, psychotherapy, therapy.
3	-u p ē	canopy, gossipy, laparoscopy, microscopy, recipe, spectroscopy.
Source: compiled by the editor (additional references); see credits.

Top

Anagrams: Entropy

Scrabble® Enable2K-Verified Anagrams
	Words within the letters "e-n-o-p-r-t-y"
	-1 letter: poetry, pyrone.
	-2 letters: entry, netop, noter, onery, peony, porny, prone, repot, ropey, tenor, tepoy, toner, toney, toper, toyer, trone, trope.
	-3 letters: nope, note, open, oyer, pent, peon, pert, poet, pone, pony, pore, porn, port, prey, pyre, rent, repo, rope, ropy, rote, ryot, tern, tone, tony, tope, tore, torn, tory, trey, trop, troy, tyer.
	Words containing the letters "e-n-o-p-r-t-y"
	+2 letters: operantly, pennywort, prenotify.
	+3 letters: counterspy, decryption, encryption, hypertonia, hypertonic, hypocenter, neuropathy, pennyworth, pennyworts, periphyton, personalty, phenocryst, poultrymen, prepotency, propensity, pycnometer, pyroxenite.
	+4 letters: antileprosy, antipoverty, corpulently, counterplay, counterploy, cryptogenic, cyproterone, decryptions, encryptions, enteropathy, ethnography, explanatory, heterophony, hymenoptera, hypertonias, hypocenters, hypocentral, importunely, lycanthrope, nephrectomy, nephropathy, opportunely, overpayment, pennyworths, perfunctory, periphytons, personality, petitionary, phenocrysts, poltroonery, polycentric, polystyrene, potteringly, premonitory, prepotently, prominently, proselyting, providently, pycnometers, pyrotechnic, pyroxenites, pyroxenitic, rehypnotize, retinopathy, retinoscopy, stenography, stereophony, teenybopper, trypanosome, trypsinogen, tryptophane, typefounder.
Source: compiled by the editor from various references; see credits. SCRABBLE® is a registered trademark. All intellectual property rights in and to the game are owned in the U.S.A and Canada by Hasbro Inc., and throughout the rest of the world by J.W. Spear & Sons Limited of Maidenhead, Berkshire, England, a subsidiary of Mattel Inc. Mattel and Spear are not affiliated with Hasbro.