aka structure
Order is the Property of a System that has some properties which are Random variables distributed according to a non-uniform Probabililty distribution. The degree of order can be considered to be the degree to which the distribution is away from a uniform one.
Some quantities/concepts that can be used to measure/describe order are:
- Entropy
- Randomness deficit
- Sparsity
- Compressibility
- Simplicity
Structure vs randomness videos
"Everybody knows that because typical signals have some structure, they can be compressed efficiently without much perceptual loss"
"When we have prior knowledge of the smoothness structure we expect to see (e.g. in natural images), we can impose this structure directly through the choice of factor. " See Learning theory (paper on predicting parameters in deep learning)
"many interesting signals or models in practice contain few degrees of freedom relative to their ambient dimension"
The World is Either Algorithmic or Mostly Random
Physical laws are like computer programs.
"In an informational universe, the world would be computing itself, enabling things to remain themselves"
He proposes the Universal probability distribution as an a priori distribution, if the world is to be algorithmic, and they claim there is some evidence that data from natural phenomena agrees with this universal distribution. They argue that the world being algorithmic means it is probably digital/discrete. If it were analog, it would have more possibilities, and it would more probably be random.
A similar thesis is found in A priori probability estimates from structural complexity.
He finds that feeding random inputs to their abstract computing machines, and using machines without a halting state gives results (probability distributions over bitstrings) which agree most with empirical data.
He discusses a bit the issue of quantum mechanics, but I didn't quite get what he meant
Life as Thermodynamic Evidence of Algorithmic Structure in Natural Environments
Complexity and Information: Measuring Emergence, Self-organization, and Homeostasis at Multiple Scales
because of several properties of the physical world (locality, etc), we often get simple effective descriptionseffective field theories, renormalization, sloppiness.
However, sometimes, we don't: LIFE. Criticallity? Chaos. Information which was previously trapped at the microscale, and very implicit (constantly changing its physical representation, as particles move and bounce around), suddenly became more explicit and emerged into a physical representation, localized in space, and persistent in time. Eventually reaching larger and larger spatial scales. From physical information came physical computation, etc.
But before this, there was already computation. Invisible computation and information, that we may not know how to interpret. But what if in that computation, whole worlds are being simulated, created and destroyed. Boltzmann worlds, Boltzmann computation. What is being computed in the air in your room? Random gibberish? Probably.. but..
Why is that computation more gibberish than the computation happening when you look at a flower? To us, it may look more gibberish, but is there anything fundamental distinguishing the two? Well, generally, I think the distinguishing feature is the complexity of the computation. Life is simpler.
Order out of chaos? With a physical description, it appears that life is more orderly than say a gas.
But still, the question that the gas in your room could be computing something very meaningful, is very interesting.
In a sense, science has been about this. Finding the hidden patterns in what seemed uncomprehensible before. But there could always be more.. (after all Kolmogorov complexity (the ultimate description) is uncomputable..)
What if in the apparent chaos, there's a simulated world. And to that world, the information comprising ours, is but apparent chaos. What if the programmers of the Universe programmed it in the representation corresponding to that world, and we are just an unintended and undiscovered byproduct, which to them, lies in the random noise of the fluctuation of particles (or who-knows-what their universe is composed of).