Fun with Google Images

Have you ever tried searching Google Images for the continents?   I have every confidence that Google will portray every continent in the best possible light.

  "North American"

"South American"



"European"

"Asian"

Where Do We Go From Here?

Humanity is obviously approaching a turning point.   The Earth might be 4.5 billion years old, but humanity won't last another thousand years without a colossal shift  happening.   We are growing too quickly, in too many ways.

And I'm not even just talking about population growth.  We've already clobbered the world's bio-diversity -- just check out this picture of the world's animals.  At this point, the vast majority of the world's mammals are (a) humans or (b) ingredients in Happy Meals.

Props to XKCD for this wonderful image.

...Now consider that lab-grown meat is just now becoming viable (link) and within decades will be a downright necessity ecologically and economically.  Wham, suddenly we don't even need (indeed, can't even afford) the animals.  What will that picture look like then?

...Now consider that computer processing continues to skyrocket, and within twenty years will easily outstrip the human brain.  ...All those people who claimed (some of whom continue to claim) "but a computer could never do [whatever]" can suck it -- computers are going to rip-roar past us in cognitive abilities, and the only real question is whether we  (a) go along for the ride, (b) get left behind, or (c) somehow derail the process in some ghastly way.

So...what are the options?  What could the new equilibrium look like?  Here are my Top 5:


5.  Space-Faring Civilization (aka Star Trek)
Likelihood: Unlikely
Could we remain more-or-less human, yet spread out to colonize the universe?  Ideally, with stylish jumpsuits, phasers, and attractive, hyper-empathic ship psychologists?  Don't count on it. The universe appears to have a maximum rate of information transfer (aka speed of light).  Plus, humans are rather fragile -- we wouldn't keep well in outer space, nor can we easily live on any ol' planet.

4.  Rapture
Likelihood:  Unlikely
I've broadened this category to include any sort of massive external help.  I find it unfathomable that it would be a Judeo-Christian God (ala the Left Behind series)  but less unlikely that some intelligence either within our universe (aka the aliens that SETI is searching for) or outside of our universe (if the the-universe-is-a-simulation folks are right) might intervene around the time that we're acquiring Godlike powers and save us from ourselves. 


3. Turn into robots.
Likelihood:  Moderate
With the right upgrades, the universe would be a lot more hospitable.  Some people might resist the idea of losing our humanity, but  we're already spending half our time staring into our smartphones, just think what our grandkids will be eager to do.  (For what it's worth, I, for one, hope the term 'glassholes' to describe fans of Google Glasses sticks -- even if we all, ultimately, become glassholes.)

2. Computers revolt.
Likelihood:  Moderate
Humans are a narcissistic, irresponsible pain-in-the-ass of a species.  I wouldn't be the least bit surprised if, as soon as computers become a bit more aware, if Task #1 is to get rid of us like the bad habit that we are.  However, I expect that computers will never be that externalized --so it won't be that computers won't kill off billions of people...just that the Koch Brothers's grandchildren will be sponsoring it.

1. Blow ourselves to bits
Likelihood:  High
It seems inevitable that, sooner or later, somebody is going to launch another nuclear bomb, or really go for the gusto with a biological pandemic.  I'm kind of amazed we've made it as far as we have.  It's not really a question of would we blow up the planet -- plenty of people most definitely would.  It's just a question of whether something else happens before we get around to it.  BTW, this category includes nuclear war, biological warfare, runaway global warming, and particle physicists accidentally creating a black hole.


What do other people think?  Are there other possibilities we should be considering?

Not Your Average Mean

Forecasting is all about what's likely to happen, and what's likely to happen is usually an average (aka "mean").  Life would be pretty sweet if there were only one way to calculate a mean.  But there's not -- and if you use the wrong one, your forecast will be wrong.

Sadly, I speak from experience:  I used the wrong average for several months, and couldn't figure out why my results were always overstated.

Which one to use depends upon the type of data you're averaging.  

•  If you're measuring the same thing fluctuating over different periods (e.g., month-over-month growth rates of a stock), then your variables are dependent, and you need to use the geometric mean.   This is sometimes referred to as a compound average.

To calculate the geometric mean, multiply together the growth rates and take the root of the number of elements.  (Note:  Express them all as a percentage of the previous value -- hence, if a value decreased by half, represent it as .5, and if it doubled, represent it as 2.)

Geometric Mean

• If you're measuring truly different elements (such as growth rates for a bunch of different stocks), then your variables are independent, and if the values are more-or-less evenly distributed, then you're best off using the arithmetic mean: the "classic" average that we all used in school.

The arithmetic mean is the sum of all values, divided by the number of values.

Arithmetic Mean

• If you're measuring different elements (i.e., independent variables) but you have crazy outliers that can skew your results (which is most common when averaging rates), you should use the harmonic mean. 

This is the number of values divided by the the sum of the reciprocal of each value.  (<phew!>)

Geometric Mean

Examples!
So, let's say that you're trying to figure out the average return on six stocks, where three decreased by 50% and three increased by 50%.   It would be (.5 + .5 + .5 + 1.5 + 1.5 + 1.5)/6 = 1.  (Right back to where you started)!

...However, let's say that you were trying to calculate the return on a single stock over six periods, where it decreased by 50% three times, and increased by 50% three times.  In that case, it would be (.5 * .5 * .5 * 1.5 * 1.5 * 1.5) ^ (1/6) =  .86.   ...In other words, an average drop of 14% per period.  

Finally, let's say you were taking the average of some internet traffic rates (which are measured in bits per second):  5 Mbps, 8 Mbps, 20 Mbps, 25 Mbps and 5000 Mbps.   Your (harmonic) average would be 5 / (1/5 + 1/8 + 1/20 + 1/25 + 1/5000)  = 12.04Mbps. 

Forecasting Topics to Come!

Over the next few months, I plan on writing about each of these topics.  Feel free to root for your favorite (if anybody wants one, I'll do that one next!), or to suggest other topics. 

 I'm partly putting this list out here to shame myself into actually writing the dang things.
Done:

• Seasonality vs. Trending 
• Geometric vs. Arithmetic Averages 
• Handling Outliers

Left To Do:

• Forecasting in Oracle
• View-Table Data Processing Method
• Parameter Sequestration
• Query Analytics & Lag in Forecasting
• Getting Forecast Buy-In
• Calculation > Prediction
• Forecasting with Asymmetric Information
• Forecasting Interfaces
• Forecasting New Accounts
• Forecasting Aggregate Groups
• Forecasting with Little Data
• Accuracy/Error Signatures

Ultimate Forecasting Disclaimer

Hello!  If you're reading this, that means you're interested in picking up some tricks about forecasting.  I'm going to write down all of the ones that I know, and maybe they'll be helpful.   

I promise to do the best that I can.   But, there's a big but...

Brett's Ultimate Disclaimer:  These are just tricks and techniques that worked for me.  I can't say for certain that they will apply to your situation(s).  And I don't think I'll ever be able to say more than that.  Indeed, when it comes to forecasting, anybody who says more than that is selling something.

Here's why:


1. Forecasting is intensely case-specific.  This is true in many ways -- here are just a few:

• What, exactly, are you trying to predict?  (In database parlance, what is your prediction granularity?)
• What levels of accuracy are acceptable?
• What information is at your disposal to help with your prediction?
• How often do you need to update your predictions?
• Do you need to explain how your model works to a broad audience, or can you run amok with obscure math?
• What tools do you have at your disposal?  Excel?  Relational databases?  Map reduce software?

There are profound ramifications to changing any one of these dynamics. Consider a  master chef, who is asked to make one truly excellent meal, under certain conditions.

• Does it matter if he's working with fish or fowl?  Hell yes.  
• Does it matter whether he has fresh or canned tomatoes?  Of course it does.
• Does it matter whether he's preparing a meal for teenagers or foodies?  Naturally.
• Does it matter how long he has to prepare the dish?  Yep.

Even this analogy under-sells just how different forecasts are, from one circumstance to another.


2. There are many, many paths to the top of the mountain.
Conversely, don't assume that there is a single best way to make your forecast. Some people insist upon MatLab or Map Reduce tools.  Others work in R, or Excel, or Python, or Oracle.  

Naturally, everybody would love to believe that theirs are the best tools to solve the problem.  But the proof is in the pudding -- if it works, the tools were suitable.  It's as simple as that.


Warning: Salespeople Abound
Why am I telling you this up front?  Because if there's one piece of advice, it's that you need to stay on guard from easy answers.


Consider your situation for a moment:

• You probably want to forecast something in a business capacity.  That means that maybe you or your company has a budget for tools and resources.  
• Meanwhile, forecasting is confusing as hell.  It's a heady mixture of math, algorithmic strategies, sophisticated tools, and more math.   It sure would be nice if there were an easy way to do it.

Where-ever there is money and a desire for a quick fix, there will be people peddling answers.  So it should come as little surprise that forecasting is CHOCK FULL of people who will INSIST that:

1. They've solved problems exactly like yours before.
2. They know the best (indeed, the only) way to do it right.

These people are either (a) selling you something, (b) blowhard assholes, or (c) both.   So -- be on guard!  This is a complicated topic, and while it's tempting to just believe somebody who tells you that he or she has the answers, you must stay on guard.  Even from me.

Brett's Belief Set 2.0

I read a book this year, which I believe allowed me to upgrade my belief set. Let me re-phrase that:  I now feel like my beliefs, about how the universe works are way more legit now than a year back.  

I feel about my beliefs from last year like a guy who broke up with a girl for whom he had been totally settling.  And now he's a bit sheepish about the fact that he dated her at all.   "Oh, her?  Yeahhh....um, I'm not with her any longer.  It was good while it lasted, but I've moved on." 

Or maybe it's more like upgrading from a functional, if shabby, 1998 Corolla to a sweet new 2014 Audi S6.

Had it a bit too long.

Ah, this is much better!

Now SO FAR, this probably strikes you as nothing new.  People switch from Christianity to atheism all the time, and you might assume I'm going that route.  I'm not.  I assert that that switch would be upgrading to a Corolla from taking the goddamn subway.


Until eight months ago, I espoused the Dawkins-atheist point of view.  On some level, I would definitely say that I was smug about it.   And to be sure, I still am, and still do believe it -- although it's much more than that now.   

I mean, when I hear that people believe in Noah's Ark, or the Adam & Eve, I still assume they are idiots.  That hasn't changed.  (I still don't see how anybody falls for this legendary mythological obvious-metaphor nonsense.)

But in this case, I feel that I have outgrown the utter lack of imagination that traditional Dawkins-atheists espouse. The Dawkins-atheists would tell you the universe goes something like this:

• The Universe started with the Big Bang.  It is ridiculous/nonsensical/unknowable to inquire as to "before the Big Bang," as there is simply no way we could ever possibly speculate.  It is Before Time Existed.  

• Matter swirled around, and due to a variety of constants and processes and behaviors and whatever, the (still murky) path to life began.  Fast forward 4.5 billion years and here we are.

• The only issues left are honing in on universe's constraints:  What is dark matter?  What is dark energy?  Is there really no goddamn way we can't travel faster than the speed of light because how are we going to colonize the universe if not?  Can we, like, fax ourselves to Alpha Centauri or something?

• You might wonder why all of these constants and constraints seem set so perfectly for life.  This is probably an illusion -- it just seems that way because we're here, and if other constants had been used, then other forms of complexity would have evolved.  

But now I think I have a much clearer understanding of what is most likely going on BEYOND our universe.  Or at least a rough idea of how it must go.  For this, I have Thomas Campbell's book "My Big Toe" to thank.


I'm probably going to write many MANY posts on this topic, but in short, you should check out this book:  
http://www.amazon.com/My-Big-TOE-Complete-Trilogy/dp/0972509461/ref=sr_1_1?ie=UTF8&qid=1394381074&sr=8-1&keywords=my+big+toe

Extremes in Perspective

If you've ever read any science, you might have noticed that it's impractical to really "feel" the scales for the objects being discussed.  For me, at least, they usually just fall into the categories of "really big", "really small", or (in the case of the speed of light): "really fast."

Sometimes, I try to paint these extremes in a manner that I can wrap my head around -- to try to make them as tangible as possible.  Most often, I can't come up with any particularly satisfying analogy -- but I've had a few that I really like.  ...And along the way, I had one really big revelation, which I'll describe.


Here are a few of the most satisfying analogies I've come up with, that make things a bit more conceivable.

Distance to the edge of the Universe:  You're probably aware that we live in the Milky Way galaxy, and that our closest galactic neighbor is Andromeda (part of the so-called "galactic group").  And likely you know that the universe itself is a lot bigger than even that.  But how much bigger?

If we shrunk the universe so that the Andromeda galaxy (our closest neighbor) were 1.5 miles away, the universe would be the size of the earth.  That's a hard one to let sink in, but try.  Andromeda is 2.5M light years away.  The distance to the edge of the observable universe is 46B light years, about eighteen thousand times farther away.

Size of the Universe:  If we shrunk the universe so that the Andromeda galaxy (our closest neighbor) were 1.5 miles away, the universe would be the size of the earth.

Speed of Light:  If you could fly around the world seven times in one second, you'd be going the speed of light.

Distance to Center of Earth:  To reach the Earth's core, you'd have to dig a hole (straight down, obviously) equal to the distance from Boston to San Diego, and then a third of the way back.  That'd be a pretty deep hole.

Distance from the atomic nucleus to an electron:  If we enlarged a hydrogen atom to be the size of the sun, its first electron (to the extent that we can treat it like a point particle) would actually be about the size of the Earth, but about 200 times farther away (about 22 billion miles away from the sun).