Information entropy
The information entropy is a measure of the number of accessible states of the system’s components and how equally accessible they are. The more possible states and the more equally accessible they are the larger is the entropy (see Fig. 1). For a given constant number of possible states in which the system can be found, the entropy will be maximal if all of them are equally accessible.
The information is a reduction of the entropy. When one determines the state of the system one gains quantity of information equal to the entropy before the determination of the state. In other words, information is the inverse of entropy. If some states of the system are more available than others then entropy is not maximal. This property can be interpreted as a structure. So, information, in certain sense, speaks about the amount of structure in the system (See Fig. 1)
Fig. 1. A. System’s components (dots) are positioned in a specific pattern. Not any other position or relation (state) is equally available to the dots. This is a case of low entropy and high amount of information.B. System’s components(dots) are dispersed in many possible spatial positions and relations (configurations). Any position or relation (state) is available to any of the dots. This is a case of high entropy and low amount of information.
In isolated and closed systems (see bellow), on a constant temperature, the spontaneous processes flow from high information-low entropy towards low information-high entropy states, that is from A to B. In open systems it is possible to go in the oposite direction i.e. from B to A. When spontaneous, the path B to A is called self-organization. In closed systems, by decreasing the temperature (or/and increasing pressure), the system can also undergo the B to A path (e.g. crystal formation from fluid). In closed systems this is called a self-assembly. Self-organization and self-assembly proceed through phase transitions and the loss of stability principle.
Thermodynamic entropy (isolated, closed and open systems)
In (bio) physics, (bio) chemistry the components of the system are usually material particles such as atoms, ions or molecules. Their collective behaviors are studied by thermodynamics. These systems can be divided in three categories: 1. Isolated; 2.Closed; and 3. Open systems. Isolated systems (e.g. the Universe) are systems which do not exchange energy-mass with the environment; closed systems (e.g. rocks, stones, dust) are those which continuously exchange energy but not mass with the environment; finally, open systems (e.g. biological systems, fire, oceans etc.) are those which exchange energy-mass with the environment. Any interaction of the system by energy-mass transfer with the environment is simultaneously an exchange of information.
The amount of thermodynamic entropy (see here) depends on how evenly the energy-mass is distributed among the components of the system. The more evenly the energy-mass is distributed the larger the entropy is and vice versa (see Fig. 1). If initially, the energy-mass is condensed in a certain restricted region of space (let’s call it source) and its environment shows shortage of it, the energy-mass will tend to disperse and reach those regions. Energy-mass tends to disperse from locations where they are in excess (sources) toward regions where they are in shortage (see Fig. 1). In this case the object-environment system is in a thermodynamic non-equilibrium. In fact, matter disperses due to the energy of mobile particles it consists of. The energy-mass will tend to be more equally distributed. It will flow from the materially-energetically denser to the scarcer region. More and more spatial locations will be occupied by this dispersal. It will tend to materially-energetically equilibrate the system with its environment.
Eventually, the energy-mass state of the environment may become identical to that of the source. That would be the state of maximum entropy. This is also the state of maximum uncertainty. Every spatial point has the same properties and one cannot distinguish between them. Any single component of the system can be anywhere in the space. For an isolated object-environment system this state is the attractor to any initial non-equilibrium state. It seems as if any initial state of the system is being attracted by this final state.
This means that any initial non-equilibrium state will tend to minimize the mass-energy differences between the spatial points (gradients). The gradients of mass-energy are proportional to the work produced. Hence, increasingly less useful work (exergy) can be extracted from such gradients. As the gradients in isolated systems become increasingly smaller, the system approaches the, so called, state of a heat death. The state of maximum entropy and minimum exergy is also a state of maximum symmetry.
Examples:
1. A drop of ink placed into water will initially occupy a very small region of the space available (it will have a role of a source). This is the initial non-equilibrium state. Slowly it will start to disperse spreading in an increasingly larger volume (the environment). Larger number of states will become occupied by ink molecules. During this initial, non-equilibrium phase, the ink can form interesting layered structures. Eventually, after long time, the ink will spread all over the whole volume of the water and will homogenize the color of that volume. This is the final state of the process called diffusion. The final (attractor) state is the state of maximal entropy (see Fig. 1). The spontaneous inverse chain of events is highly unlikely, but it is not forbidden. This spontaneous tendency of isolated systems toward maximum entropy is claimed to form the –arrow of time-. Time passes most likely toward the state of maximal entropy.
2. Initially built house will slowly start to decay and if not renewed it will decay to unrecognizability. Archaeological remains are an example of what happens to a closed system after a long time has passed without input of mass from the environment. Eventually, after longer time scales even these remains will decay and all initial information of the house will be erased (see Fig. 1). When we say: the house should be renewed, we say that the house should be managed as an open system with input of energy-mass from the outside, in order the house to be maintained close to its initial (non-equilibrium) state. Note, however, that the maintaining of the house needs an active effort of human beings, which must use and disperse energy-mass in their environments. So, although the house will be kept close to its initial state, the total entropy of the house-environment system will increase.
3. The printed letters in a book are yet another example of a closed system which tends toward maximal entropy. The act of printing is assigning an initial non-equilibrium state to the system called a book or a hard disc (see Fig. 1). Due to processes similar to the previously mentioned, the initial state will decay slowly and the initial information will be lost and the entropy will increase. The diffusion will make letters to become more blurred and smeared and at the end the text may become utterly unrecognizable. The system tends towards its attractor, the maximum entropy state. In order to maintain the information people from outside should reprint the text or do a work to store the information in more safe forms. By doing this, one part of the energy disperses again and the total entropy of the printed material-environment increases.
4. Our Universe can be considered as isolated system. To our knowledge no mass-energy input to or output from the Universe exists. However, our Universe is still, dominantly, in the state of non-equilibrium. This fact shows that our Universe is still young and full of useful energy (exergy) for enabling work. There are gravitationally formed and maintained clumps of hot mass-energy dispersed in a vast space of cold vacuum. This thermodynamic non-equilibrium enables existence of hotter and colder places and a mass-energy transfer from hotter to colder places, self-organization and evolution of complex forms of existence including life and intelligence. For example, the system of the Sun and its environment are currently in a non-equilibrium state. The energy-mass is mostly located within the Sun and because of that it spontaneously flows outwards (is being dispersed) in the interplanetary space. A tiny part of that flow heats our and other planets. Because of this, complex life forms and intelligence has developed on it.
Planets, thus, receive and emit mass-energy to the outer space. Hence planets are not isolated or closed systems, they are open systems. We, the humans, and all biological systems, are also open systems. We receive energy-mass (e.g. water, food) from our environment and emit energy-mass into it (conduction, radiation, perspiration etc.). What happens in between is the self-organization of our life processes, such as: formation and repair of organelles and cells, metabolizing, acting, perceiving, thinking, digesting etc.
Any biological or social activity, including, running, dancing, building and maintaining of houses, as well as printing and reading books, can be considered as an epic battle of the Life against the tendency towards increasing the total entropy of the Universe. While running itself disperses the stored chemical energy of the body to a heat, radiation, sweat etc., it also helps the body to self-organize itself and become more functional. So, in order to locally decrease the entropy (of the body) one inevitably increases the entropy of the environment. When a living creature dies, the open system turns into a closed system (no mass exchange with the environment) which will increase its entropy and ultimately decay to dust. Although for open systems, locally and on short-term, the energy-mass can lower the entropy and increase the information through self-organization, on a long term and globally, the entropy tends to increase.
Biological reproduction and cultural (including scientific) production and conservation are the ways we humans fight this loss of information tendency.
Аn artistic depiction of the lamentation for the lost battle of human or humanoid organization with respect to the global entropy increase tendency is given in the closing monologue of the character Roy Batty in the 1982 film Blade Runner:
These are his words:
“I’ve seen things you people wouldn’t believe….. Attack ships on fire off the shoulder of Orion[i]…. I watched C-beams glitter in the dark near the Tannhäuser Gate[ii]…..All those moments will be lost… in time,…, like tears in rain…. Time to die.”
As said before, at the moment of dying, organism transits from eminently open non-equilibrium thermodynamic system to a closed thermodynamic system. Memory, together with all bodily structures and functions, is subject to the energy-mass dispersal, without the compensation from the environment which is present when the system is alive. The organic structures (organelles , cells, organs ) start to slowly converge, in time, to the final attractor state of maximum entropy and loss of information. In similar vein, tears and rain-water molecules that diffuse and disperse among each other become an indistinguishable fluid and the information of their separate identity is irreversibly lost.
[i] Constellation of Orion. The constellation bears the name of the mythological hunter Orion whose celestial body is formed by the brightest stars of the constellation. A pat of the constellation is the ‘shoulder of Orion’, which is menationed in the monologue.
[ii] An imagined location somewhere in the outer space.
After an unimaginably long time and many different life phases, the expanding Universe will cool down and slowly approach the state of thoroughly dispersed energy-mass which will not be distinguishable from place to place. The heat death of the Universe will very slowly spontaneously arise when the energy-mass will be more or less homogenously dispersed through it. The energy-mass gradients, on average, will be close to zero, and no useful work (exergy) can be extracted from them. The Universe would be mostly a cold dark vacuum interspersed with rare wandering elementary particles. What are the prospects for the far descendants of Humanity within this context?
Robert Hristovski 15.03.2016
[1] Constellation of Orion. The constellation bears the name of the mythological hunter Orion whose celestial body is formed by the brightest stars of the constellation. A pat of the constellation is the ‘shoulder of Orion’, which is menationed in the monologue.
[1] An imagined location somewhere in the outer space.