On Randomness

On Randomness

RNG

We start from RNG (random number generator), which can be seen in almost every utility program today. It’s like the representative we employ in computer systems to shows our awe to the powerful reality. And by a better designed RNG, we hope we would be able to predict God’s next move.

I’ve been TA for simulation course for several times and RNG is always a core part in simulation programming. There are dozens of algorithms for random sequence generation, starting from simplest LCG (linear congruential generator) to “unpredictable” trapdoor function. Correspondingly there are dozens of back-test methods to ensure the sequence we generate looks random as well as normative.

But the problem is if the volume of memory is limited, the computer, theoretically, is just a FSM (finite-state machine). Then there is no way to generate a truly random sequence by such a deterministic system. Randomness should be totally memoryless and independent. What we are trying to do is just to make the generated repetitive sequence as long, chaotic, as possible. Anyway, what you need is what you should generate. So I always told the students, the best way is to know the simulation systems, to feed them what they want. That’s why in most cases, LCG is fine enough.

I further go on thinking about where RNG can be improved. Can we break repetitiveness? Yes. Think about the irrational number, square root of 2.

We know it can be approximated by Taylor’s series to any precision as we like. And the coefficients in Taylor’s series regular which means we can express them by finite information.

But you can’t rush to your boss and tell him you just discover a true RNG. Since if your boss is stupid enough to believe you and use your algorithm for some highly confidential data, it would take long to be cracked. The problem is non-repetitiveness is far from randomness. So can all irrational numbers be generated with finite information? That’s a little beyond this article’s scope. Let’s put it aside. The conclusion now is we can’t make a true RNG by computer. Maybe someday, and the day we achieve this, is the day we solve Turing test.

But that’s not the end of the story. Randomness is still like a buzzword to me. I don’t like buzzwords.

What’s Randomness Anyway?

Random, by Webster, means “relating to, having, or being elements or events with definite probability of occurrence”.

This definition should be familiar since that’s what we are taught in probability course. In my view, randomness, belongs to topology more than to pure numbers (of course you can say all mathematics can be cracked down to number theory). It not only is some gift from observation, but also has been widely applied in practices. The jump from deterministic number to functional can bring many unexpected and surprising benefit.

History repeats but never in the same way. Paul Samuelson once nicely put it in this way: we have only one sample of history. We are living in a world we don’t know completely. Even someone declares the world is deterministic, it’s more philosophical instead of “scientific”. But we desperately want to know the world since we are clear about the pay-off of predictability. Unpredictability and predictability, are like twins.

Randomness is our solution. More specifically, we use probability theory to get ourselves some clue. We “price” unpredictability by measures such as Sharpe ratio. We make ourselves a cozy home in the dark cold universe.

But if you pay a little attention to the definition, you can see a word “definite”. How come a definite thing in definition of a certainly indefinite thing?

Look back the way probability theory goes. It starts with gambling and now, has expanded to every area. But some basic principles haven’t changed. It needs preassumed distributions like Gaussian, to generate the whole topological structure, and needs some mapping rules like VaR, to help our decision. Both are strong assumptions and “definite”.

We get those definite things from data. But recent study is questioning both (for example, see Kahneman, Tversky, Mandelbrot’s writings). In my point of view, this is good but still wrong in direction.

Harold Hardy once said compared to physics, mathematics seems more “real” to him. Absolutely right. Given the randomness theory, itself forms a complete and beautiful system. Just like given gravity and other basic rules, we have our real world. In probability universe, we have rules like optimality, equilibrium, etc. Every entity can enter the universe but if it doesn’t show respect to the game rule, it’s going to be eliminated theoretically. To be specific, this is called arbitrage opportunity in finance.

But as Samuelson said, we can’t believe a model simply because it’s beautiful. Does randomness serve its purpose?

Why We Need Randomness?

We need randomness to make a better understanding of the real world, like Milton Friedman said, “prescribe what should be done in the light of what has been done”. Is the beautiful randomness really helpful in our reality?

Assume you are gambling with an idiot. You two pick 0 or 1 each time. And rule is as follows: If you two pick the same number, nothings happens. Otherwise, the 1 guy wins a million. Obviously, in this zero-sum matrix game, only one Nash equilibrium exists: 1-1.

But what if that idiot keeps picking 0? Are you going to be lured by the million reward? In other words, do you have confidence to tell determinacy by past data?

This simple example shows the gap between theoretical and real worlds. The strong assumptions and lack of details specifications make the theory hard and dangerous to apply. What if everyone truly regards the world as random? Is this going to lead to a limit or a never-converged distribution? Is the whole system resulted robust to its rules? By continuing such discussions, you would find it goes more and more theoretical and useless.

As John Milnor said about Nash’s work on game theory: “However, when mathematics is applied to other branches of human knowledge, we must really ask a quite different question: To what extent does the new work increase our understanding of the real world? On this basis, Nash's thesis was nothing short of revolutionary.” From this angle, probability theory is only a limited, micro-level theory. It sets up a set of rules about how people should play the game but doesn’t explain. What we are doing is to prove the completeness of the universe and seek black holes, etc.

In conclusion, there is no such thing called “randomness” in this world. What we face are only known and unknown.

Unknown is Unmeasurable

Randomness is still deterministic. It’s just a tool we use to treat the unpredictable world. Given circumstances, given uses, it’s useful. If people are stuck in it, they may lose the chance to realize the real world.

Unknown is unmeasurable. We have no idea what it is. Sure we can’t handle this kind of stuff. That’s why we turn to things like probability theory. We need to know the prior knowledge, the time, the frequency, the sample size, the utility, et al. But you can’t say a 99.9% VaR portfolio is secure since you don’t know how the system runs. We are dusts in a dessert and we need to see the big picture.

Think Big

My probability theory is poor and only limited to certain areas like finance. The learning of probability theory did improve my realization of the world. But the trading experience tells me to be a non-believer, i.e., probability theory is merely part of the tools we human being has invented along our history.

For financial world, I want to model it as a dynamical system. However, the complexity, theoretically, is beyond my current capacity and I also don’t have enough resources to practise my theory now. Anyway, I can use some intuitive and straight ways to experiment in small scale. Though they seem not so mathematically beautiful and even hard to express by words, they are better than orthodox finance engineering stuff because they are “big”.

Yes, think big and think different. We are not what we are taught but certainly we are what we think.