27.08.2016 - Making shadows
A shadow have a shape of the thing that's casting it. A flat projection, devoid of colors.
Regression from rich and volumetric to monotone and flat.
How often are people making shadows?
There is a point when you look at all the things you made, and see nothing but shadows of real things, existing or emerging.
It sounds like the same idea, but the thing you did is like a schematic to the other thing that is a painting.
With great effort you try to make a thing that can be used, learning all the tiny little details that ornate the core idea and make it usable.
You implement them all, and get something which is almost usable.
But on a closer look, it's still a mere shadow, lacking half the features that are used every day instead of in a test suite.
You jump to something else to work on.
You try to stop comparing, and yet spent days inspecting edges and details of some other, similar, work.
To grasp the core idea, to try to replicate it, and stare at another shadow.
Another thing. You map the parts that must be made to work, you stretch that bridge across the gap.
One day the bridge have reached the other end, and the thing works.
A rare but familiar rush of getting something done, of seeing a thing that does a task you thougt you'll never be able to make.
But wait the rush out, and take another look - that thing is a mere shadow, a bridge that's barely there. It can be crossed at peril, but you find yourself preferring the real one.
Sometimes the things look like they are on their own. A thought, refined over years, of which there doesn't seem to be anything similar around.
But it is just a question of time or scope - one day you find what's shadow your thought was, or someone would make that real thing.
For all the effort spent, the end result is only a shadow.
There is a point when you look at all the things you made, and notice that there is a pattern.
16.05.2016 - Static motion
On static motion:
Or read below if that visual page does not work for you.
Contemplate this line.
The line is a moving slice of the Mandelbrot set.
Finite numbers make it look continuous and produce frames that seem consequential.
Things are happening on it, events unfold.
Objects move, interact, collide, appear and disappear.
It seems to be in motion, and yet it is only a static mathematical pattern.
But if all the motion is already pre-written by the pattern,
Then does it matter if it is computed at all?
The pattern, being a mathematical object, simply exists.
Regardless of whether someone computed it or not.
Our universe is a mathematical object too.
Think about it.
06.05.2016 - On hoarding ideas
To speak an idea aloud means certain death.
Is that a familiar feeling? Does that sound absurd? - Don't close the tab.
There is a choice - to hoard ideas or to spread them.
To keep your secrets to yourself or to talk them out at every opportunity.
It's a choice between a high chance of low benefit to everyone at certainty of zero benefit to yourself and tiny chance of high benefit to yourself at almost no benefit to everyone.
An idea is like a seed.
You can try to grow it yourself, but you can only grow a few at a time and won't know the outcome for years.
Or you can send it out, to be duplicated a hundred times by spoken word and digital magic.
Each then might be tried many times by many people.
Each might scatter against someone's head and turn into more ideas.
Imagine a billiard ball hitting the wall and turning into a dozen balls.
Net benefit to everyone would be higher.
You don't see it that way.
All you see is an idea lost.
Permanently less available.
Not spent on something good, but spent on something like a new iPhone - something you play with for a few hours, and then put into the drawer forever.
You get a few `wow`s and `awesome`s, and it's never heard of again, whether it rings across the world, annoys or vanishes without a trace.
It's unnatural for a human to feel the benefit from long-term investments.
All you can feel is that personal benefit would be certain zero.
If you spread ideas, you stay a nobody, you die.
So you hoard them, to have a chance of making one work at a benefit to yourself, to avoid certain death.
Just something to think about.
07.04.2016 - You've got mail
The user pressed the "Get e-mail" button.
What happens next?
The mail server is a name, the network talks in numbers.
The DNS will a call, and tells the number for the name.
The TCP connection gets established, data split into packets encoded into frames turned into electric signals... There be dragons for another raid.
Over that TCP connection, a plain text hello is sent.
The other side replies with a matching set of algorithms.
It sends a certificate and an invitation.
The certificate is a blob of data in the DER subformat of ASN.1 - type, size, data, type, size, data. In there hides the public key and the signature.
SHA-256 is the hashing algorithm. It takes as many bytes as you have and turns them into a 32 byte string.
We compute it over the certificate.
We then fetch the public key of the authority which granted it and take the signature to it's Dth power modulo N.
The answer is PCKS#1 - a padded SHA256 hash of the certificate.
The two match. Good, the server is genuine!
Now we send our shared secret to it, taken to the Eth power modulo N of it's public key.
The shared secret is a random string.
To get the encryption and hashing keys, we mix the words 'master secret' with the set of random numbers exchanged previously in the hello and invitation.
Just hash is unsecure, and so HMAC was invented after a battle lost and won decades ago.
The mix is the key to HMAC, taken over a recursive set of MD5 and SHA1 hashes to produce a string of bytes.
The string is split into two pairs of 20 bytes for the MAC, then 2x16 bytes for the crypto key and 2x16 more bytes of the initial vectors.
All the prior messages were saved, with the SHA1 and MD5 hashes being taken over them.
Now, we signal the server to begin the coded exchange.
As a confirmation we send the hash above, the 'client finished' string fed to the PRF function with the hash as a key, padded in the PCKS#1, and encoded with the keys above by AES.
Add the protocol version, or someone will fake us into talking on an older, insecure one. Every weird step have a history. A battle lost or won.
The server will reply with the same courtesy, only the 'server finished' line is used.
After the niceties are done, in comes the actual data.
The application data - blocks of AES containing the whatever apps would say.
The signs of ancient battles are seen in them as well - the first block message is just random to confuse potential decoders that figured an in before.
The data follows, completed by a hash.
The MAC, a SHA256 HMAC of the number of the packets sent, the data, it's size and it's purpose.
The end is ceremonially padded, so as not to leave any predictable text that might assist the hackers.
The data goes in and out of a buffer, read and written by the layers over them engaging in further ceremonies.
Ready, says the POP3.
+OK i know you.
+OK You may proceed.
Now, for the actual mail.
LIST, the client asks, +OK the server responds. RECV, RECV, RECV, RECV, DELE, DELE, DELE, DELE...
The client sends a coded ALERT to make the server know not to wait for it at dinner and breaks the TCP connection.
We now have the dumps of data made out of lines with tag: value in them, then the data terminated by two new lines.
What do it say? Where did it came from?
The mess is indescribable, the standards lax, a guess after guess on standard named fields containing data in a free-for-all format.
Americans say Feb 02 2016, the europeans - 02 02 16. Some say +0400, some say PST and EDT.
And that's just the date.
The subject line can be some text. Can be =?UTF-8?B?SGVsbG8ga2l0dHkhCg==?=, can be something even worse. All this must be decoded.
Every language have it's code page, which may not even be provided. We have to guess, to try them all and look for meaning.
The message itself, you might expect, is only text?
Nope, it's MIME. A set of parts, separated by whatever-i-want lines.
In there is a recursive mix of HTML, text versions of it, the files being attached, some signatures, some stuff embedded and some missed never to be seen.
BASE64, QP code, "plain" text in myriads of codepages.
As this final mess is waded through, the status bar approaches 100%...
You've got mail.
01.04.2016 - Hard is easy, easy is hard
Sometimes you do things that are hard and challenging.
Like implementing SSL/TLS stack and fetching google.com over https.
Or making a procedurally generated universe that fits in 1Mb.
And no one cares.
Sometimes you do things that are extremely simple.
Like showing off a 3D printed quadrocopter.
Or reciting something obvious from a podium.
And people are impressed.
The amount of effort put in the first thing is huge, but it's not valued.
The amount of effort you put into the second is about nil, but it's valued.
You, on the other hand, expect to have the payoff be proportional to the effort.
More effort equals more payoff.
It might be tied to the old paradox of doing a good deed.
You spent effort and missed a schedule to help and old lady you found laying across the path to get back home and call a doctor.
You might expect some karmic repayment.
What you get is an existential hurt that there is nothing in the world that would repay you for the deed, no enforcing of fairness like that.
No one cares, other than yourself.
What it should really be:
If you spent a lot of effort it means you did a bad job and learned something.
If you spent no effort, you did a good job, and learned nothing.
Not always, but most of the time.
And this is completely opposite to what your hedonic wiring tells you.
Anything you are good at would seem simple to you, but the fact of the lack of effort at the output makes you feel like that output is worthless.
So you avoid showing it, not wanting to annoy people with "bad work".
Anything that you find impressive would be something that you did for the first time, and so be worthless, but you see it as valuable.
So you show it off, and get hurt by the fact that "even your best" annoys people.
It is a vicious circle of pain and self-improvement that ends up with 99 burnt-out husks for every Napoleon among it's sufferers.
Gonna be a hard bias to compensate for...
22.03.2016 - Hello, thing. What is your name?
When people talk, they use the names of things.
You can't telepathically pass a thought of a thing to someone else.
You can't, in general, show a picture of it.
You can always describe the function, and the other person should get the idea.
That's how many ancestral languages work, synthetic words that combine small patterns into a new concept.
What you often do is to name it, and the other person would know what is being talked about.
To function, names must form a shared knowledge.
But what names a thing?
Suppose you were playing with an air pump, and noticed that it can help you clean up by sucking dust.
Some pipes and bags later, you have a device that sucks air and filters dust out of it.
What is the thing's name?
For you, it would be obvious - it's a "Dust Sucker".
For someone else it will also be obvious - it's a "Vacuum Cleaner".
Another would obviously name in "Antipressure Duster".
Obviously, the three of you don't share a name.
How would you solve that problem?
You might think the easy answer these days is to Google it.
But Google what, exactly?
"Dust Sucker" is an album by some metal band.
"Antipressure Duster" returns something about blowing dust with compressed air, which might be good for outdoors, but inside the house it would just rearrange the mess. Different idea.
Eventually, you might hit the right combination of keywords or stumble upon the description.
What if you made something obscure?
You might call it "Ambient Field Electrostatic Particle Accelerator", while the guys at Berkeley that made it before have called it "Plasma Wakefield Accelerator".
Google away, you'll never find the name without knowing it.
A name does not come with the thing or a concept, it's something people make up when they create it.
How would you know the name of something you thought about or something you made?
It's an open question, not even contemplated by modern search engines.
It might sound irrelevant, but without a name you can't know how your idea was done before, or what reasons prevented it from being done.
Maybe making that thing is dangerous. Maybe the concept you thought of turned out to be flawed.
You can't tell if you don't know the name.
How would you do the "Concept Search", as i would call it?
I don't have a clue, but that sounds like a problem worth solving.
23.02.2016 - Optimization and discovery
There are goals that are imaginable, and there are goals that aren't.
Contemplate going to the Moon.
There it is in the sky, just a light-second away. Enormous distance in a direction that you can't easily walk.
But you can imagine getting there.
There is space between you and the goal, and you know how to move through space.
Maybe you could build a tower, maybe you can fly up there in a balloon, maybe you can fire yourself up from a cannon, or notice the cannon's recoil and use that to build a rocket.
There is the brute-force solution of crossing the space.
HOW is the other question, of optimizing and searching until you come up with a plan that might work.
The key point it - you can imagine getting there.
You can imagine curing cancer - all it takes is removing all the cancer cells from the body, however hard it might be.
You can imagine assassinating a dictator - he is a man, there are that many ways to make one stop existing, there are that many ways to approach him, there are that many ways to get away with it (if necessary).
The rest is an optimization and path finding problem.
But imagine, say, that you want to get to a parallel world.
To Narnia, or Valhalla, or something else entirely.
How can you get there...?
There is no space between you and the goal. No imaginable path to be taken, no matter how difficult.
Not even the brute-force approach.
You can walk in any direction, but there is no walking between universes.
You can think all over the known patterns, but there are no technology to access other worlds.
Here you are, on your map of the universe, and there is the goal - floating off the map, unconnected and unrelated to it.
This is no longer an optimization problem. It is a problem of discovering the existence of a path.
A thousand years ago, curing smallpox might have been like that. There was nothing you could imagine that would lead to the solution.
Making a lightning would have been like that. There were no lightnings in your map of the universe, no imaginable way to make one.
Getting heard on the other side on the planet the instant you said something would be like that. Sound, no matter how loud, can't travel that fast or that far.
And yet, if you know the goal exists, you can do something about that.
Thousands of years ago people knew about static electricity.
Some thinking, some mapping out, some luck with vine and different metals, and you get something that behaves just like a lightning. From there, it's optimization.
Long ago people knew that diseases spread, that they can be contagious. You might think that there was something to transfer it.
You kept looking, and eventually saw it in a microscope you made by accident. Tiny bugs, germs, that spread it.
Someone noticed that having a weaker strand of the virus makes you immune to the big one - vaccination.
Someone noticed that a certain moss kills the germs - antibiotics.
And then, there was space between you and elimination of most diseases.
Hertz, who discovered the radio waves, when asked what practical use there is for them, famously said "there are none", even as he put instant global talking on the map.
Imaginable and unimaginable.
Optimization and discovery.
The first takes work, the second takes an open mind, ability to notice something that is odd, that leads beyond your map.
Not just notice it, but use it.
The easy way is not to trust any opinion you formed longer that a couple years ago. Maybe you recently noticed something that would make the unimaginable problem imaginable, if not solvable?
That is the key of discovery.
18.04.2015 - The Nobody Bias
Let's say you have an idea, or something you made, or something you can write or talk about.
Maybe you make videos of skyscrapers being built in time-lapse, maybe you draw a webcomic, maybe you traveled across Australia on foot taking pictures, maybe you write a story, maybe you make computer games. Anything will do.
The end point is that you want to share it. You set up a site on the internet, you post about it here and there on related forums, you tell people about it.
Well, not interesting to anyone. Let's move on to the other things.
You try something else and get good results. The same itch to share comes back again. You set up a site, or extend the existing one, or start a Youtube channel. Post about it here and there, discuss, etc.
Well, not interesting enough.
Another hobby, another set of good results, another try at sharing.
So on and so forth. After a decade of this you can't help but wonder what is going on.
Now, the strange part.
What you do is good content. How you know this? By targeting individual persons.
The few comments you get, they are all of the "WOW!" variety. You show your work to the right professor - you get a PhD a few years down the line. One of the 10 or so visitors of your sites is impressed enough to offer you a damn good and high paying job. When that job gets boring, you get another offer of a better one the same way. People who get into your hobby electronics lab get their mind blown. Photos you made are hanging on the walls at your employer company's lobby. Time lapse photography you made feature in documentaries on international TV. DIY presents you give out make people envious.
And so on, all at once.
Simply speaking - you target individuals, you win hard. You target an audience, you get cricket sounds.
So, what is the problem, one might ask?
The problem is - you don't notice the "winning" part. All you notice are the cricket sounds. That you fail to engage any audience larger than the number of fingers on your hands. That no one seems to care.
In other words, you feel like you are nobody, no matter how hard you try.
It affects the way you think. It stifles your creativity, since there is no point to bother - no one cares anyway. It makes you wonder about otherwisely rational ideas - "sure, these people will get carefully cryopreserved/given immortality treatment in time, but what about the billions of nobodies like me?".
You might quite naturally want to discuss it somewhere, and try to figure out if that's a thing. But there the problem remains essentially the same. Either it's a big enough place that you get no chance of being heard, or it's a small enough place to get traction, but with little to no point in posting.
A catch-22. You need an audience to get an audience. And the circle goes on.
This appear to be a cognitive bias to me.
For some archaic reason, the desire for recognition tend to outweigh the utility obtained, in the calculations of one's feeling of well being.
I'm not too sure how to correct for it. For one thing, you might try to blog to an empty room, explicitly. What writers call "writing for the table drawer". Or you might play make believe, and pretend there is an audience. To scratch that itch one way or another, and get used to the idea that audience does not matter.
To keep the projects going, regardless.
16.04.2015 - A world full of magic
Social media is like black magic to me.
I can observe a wizard using an incantation, then try it myself.
But nothing will happen.
See, a wizard knows that the incantation should be spoken while facing north at sunset, not south at sunrise, and most certainly not during the day facing a building (since that can backfire).
Also, i couldn't have know that the initial hand waving and the first word wasn't a part of the incantation - the wizard just tripped and swore in his native language.
So when a commenter suggests that "You'll want to add a facebook button to your blog", i google "facebook", look around blogs for such buttons, google "addthis", get the right code, put it on the site so that it looks kind of like on the other blogs, and wait for the magic to happen.
Which it does not.
And i have no clue why - it's an incantation, not understanding.
Contrast this to writing code.
It looks like an incantation - a bit of text which you feed to a compiler, and magic happens.
int main(int argc,char **argv)
But in the code everything makes sense.
Return type, procedure name and parameters, calling to a function that prints, string that is to be printed, job done statement, all intermixed with bits of syntax.
If something does not work, i know exactly where to look and what can be wrong.
If it wrote "Hallo world", then obviously the string given to the print function have a typo.
Here i am the wizard, and the spells make sense...
That happens a lot in life.
Right now i'm in Chile, and i don't know Spanish.
How big of a problem is it? Not as much as you think. You go into a store, pick what you need, go to the check-out, put it all on the conveyor, smile when the lady says Hola, wait for number-sounding words and give her money.
It's an incantation, and it works. Most of the time.
The other times she will ask "Dona [something] [something] cuatro peso?".
The magic fails to happen, and you have no clue what is going on.
The other time your roommate will try to light a gas stove.
She goes through the motions of opening the valve, striking the lighter, moving it to the burner...
And magic fails to happen.
She have no clue why not. It worked before, and now there is no gas.
I take a look, and there are two options that are immediately apparent - either the gas is not paid for, or the master valve is closed for some reason. Turns out it's the second.
For me it's understanding, for her it's incantation.
We live in a world full of magic, and each of us is a wizard in his or her trade, and a spell reciter in the rest.