Forecasting Topics to Come!

Over the next few months, I plan on writing about each of these topics.  Feel free to root for your favorite (if anybody wants one, I'll do that one next!), or to suggest other topics. 

 I'm partly putting this list out here to shame myself into actually writing the dang things.
Done:

• Seasonality vs. Trending 
• Geometric vs. Arithmetic Averages 
• Handling Outliers

Left To Do:

• Forecasting in Oracle
• View-Table Data Processing Method
• Parameter Sequestration
• Query Analytics & Lag in Forecasting
• Getting Forecast Buy-In
• Calculation > Prediction
• Forecasting with Asymmetric Information
• Forecasting Interfaces
• Forecasting New Accounts
• Forecasting Aggregate Groups
• Forecasting with Little Data
• Accuracy/Error Signatures

Ultimate Forecasting Disclaimer

Hello!  If you're reading this, that means you're interested in picking up some tricks about forecasting.  I'm going to write down all of the ones that I know, and maybe they'll be helpful.   

I promise to do the best that I can.   But, there's a big but...

Brett's Ultimate Disclaimer:  These are just tricks and techniques that worked for me.  I can't say for certain that they will apply to your situation(s).  And I don't think I'll ever be able to say more than that.  Indeed, when it comes to forecasting, anybody who says more than that is selling something.

Here's why:


1. Forecasting is intensely case-specific.  This is true in many ways -- here are just a few:

• What, exactly, are you trying to predict?  (In database parlance, what is your prediction granularity?)
• What levels of accuracy are acceptable?
• What information is at your disposal to help with your prediction?
• How often do you need to update your predictions?
• Do you need to explain how your model works to a broad audience, or can you run amok with obscure math?
• What tools do you have at your disposal?  Excel?  Relational databases?  Map reduce software?

There are profound ramifications to changing any one of these dynamics. Consider a  master chef, who is asked to make one truly excellent meal, under certain conditions.

• Does it matter if he's working with fish or fowl?  Hell yes.  
• Does it matter whether he has fresh or canned tomatoes?  Of course it does.
• Does it matter whether he's preparing a meal for teenagers or foodies?  Naturally.
• Does it matter how long he has to prepare the dish?  Yep.

Even this analogy under-sells just how different forecasts are, from one circumstance to another.


2. There are many, many paths to the top of the mountain.
Conversely, don't assume that there is a single best way to make your forecast. Some people insist upon MatLab or Map Reduce tools.  Others work in R, or Excel, or Python, or Oracle.  

Naturally, everybody would love to believe that theirs are the best tools to solve the problem.  But the proof is in the pudding -- if it works, the tools were suitable.  It's as simple as that.


Warning: Salespeople Abound
Why am I telling you this up front?  Because if there's one piece of advice, it's that you need to stay on guard from easy answers.


Consider your situation for a moment:

• You probably want to forecast something in a business capacity.  That means that maybe you or your company has a budget for tools and resources.  
• Meanwhile, forecasting is confusing as hell.  It's a heady mixture of math, algorithmic strategies, sophisticated tools, and more math.   It sure would be nice if there were an easy way to do it.

Where-ever there is money and a desire for a quick fix, there will be people peddling answers.  So it should come as little surprise that forecasting is CHOCK FULL of people who will INSIST that:

1. They've solved problems exactly like yours before.
2. They know the best (indeed, the only) way to do it right.

These people are either (a) selling you something, (b) blowhard assholes, or (c) both.   So -- be on guard!  This is a complicated topic, and while it's tempting to just believe somebody who tells you that he or she has the answers, you must stay on guard.  Even from me.

Brett's Belief Set 2.0

I read a book this year, which I believe allowed me to upgrade my belief set. Let me re-phrase that:  I now feel like my beliefs, about how the universe works are way more legit now than a year back.  

I feel about my beliefs from last year like a guy who broke up with a girl for whom he had been totally settling.  And now he's a bit sheepish about the fact that he dated her at all.   "Oh, her?  Yeahhh....um, I'm not with her any longer.  It was good while it lasted, but I've moved on." 

Or maybe it's more like upgrading from a functional, if shabby, 1998 Corolla to a sweet new 2014 Audi S6.

Had it a bit too long.

Ah, this is much better!

Now SO FAR, this probably strikes you as nothing new.  People switch from Christianity to atheism all the time, and you might assume I'm going that route.  I'm not.  I assert that that switch would be upgrading to a Corolla from taking the goddamn subway.


Until eight months ago, I espoused the Dawkins-atheist point of view.  On some level, I would definitely say that I was smug about it.   And to be sure, I still am, and still do believe it -- although it's much more than that now.   

I mean, when I hear that people believe in Noah's Ark, or the Adam & Eve, I still assume they are idiots.  That hasn't changed.  (I still don't see how anybody falls for this legendary mythological obvious-metaphor nonsense.)

But in this case, I feel that I have outgrown the utter lack of imagination that traditional Dawkins-atheists espouse. The Dawkins-atheists would tell you the universe goes something like this:

• The Universe started with the Big Bang.  It is ridiculous/nonsensical/unknowable to inquire as to "before the Big Bang," as there is simply no way we could ever possibly speculate.  It is Before Time Existed.  

• Matter swirled around, and due to a variety of constants and processes and behaviors and whatever, the (still murky) path to life began.  Fast forward 4.5 billion years and here we are.

• The only issues left are honing in on universe's constraints:  What is dark matter?  What is dark energy?  Is there really no goddamn way we can't travel faster than the speed of light because how are we going to colonize the universe if not?  Can we, like, fax ourselves to Alpha Centauri or something?

• You might wonder why all of these constants and constraints seem set so perfectly for life.  This is probably an illusion -- it just seems that way because we're here, and if other constants had been used, then other forms of complexity would have evolved.  

But now I think I have a much clearer understanding of what is most likely going on BEYOND our universe.  Or at least a rough idea of how it must go.  For this, I have Thomas Campbell's book "My Big Toe" to thank.


I'm probably going to write many MANY posts on this topic, but in short, you should check out this book:  
http://www.amazon.com/My-Big-TOE-Complete-Trilogy/dp/0972509461/ref=sr_1_1?ie=UTF8&qid=1394381074&sr=8-1&keywords=my+big+toe

Extremes in Perspective

If you've ever read any science, you might have noticed that it's impractical to really "feel" the scales for the objects being discussed.  For me, at least, they usually just fall into the categories of "really big", "really small", or (in the case of the speed of light): "really fast."

Sometimes, I try to paint these extremes in a manner that I can wrap my head around -- to try to make them as tangible as possible.  Most often, I can't come up with any particularly satisfying analogy -- but I've had a few that I really like.  ...And along the way, I had one really big revelation, which I'll describe.


Here are a few of the most satisfying analogies I've come up with, that make things a bit more conceivable.

Distance to the edge of the Universe:  You're probably aware that we live in the Milky Way galaxy, and that our closest galactic neighbor is Andromeda (part of the so-called "galactic group").  And likely you know that the universe itself is a lot bigger than even that.  But how much bigger?

If we shrunk the universe so that the Andromeda galaxy (our closest neighbor) were 1.5 miles away, the universe would be the size of the earth.  That's a hard one to let sink in, but try.  Andromeda is 2.5M light years away.  The distance to the edge of the observable universe is 46B light years, about eighteen thousand times farther away.

Size of the Universe:  If we shrunk the universe so that the Andromeda galaxy (our closest neighbor) were 1.5 miles away, the universe would be the size of the earth.

Speed of Light:  If you could fly around the world seven times in one second, you'd be going the speed of light.

Distance to Center of Earth:  To reach the Earth's core, you'd have to dig a hole (straight down, obviously) equal to the distance from Boston to San Diego, and then a third of the way back.  That'd be a pretty deep hole.

Distance from the atomic nucleus to an electron:  If we enlarged a hydrogen atom to be the size of the sun, its first electron (to the extent that we can treat it like a point particle) would actually be about the size of the Earth, but about 200 times farther away (about 22 billion miles away from the sun).
 

Vegan v. Bacon

I have a friend who is the epitome of a positive lifestyle.  He boxes, in great shape, very environmentally sensitive and quite zen. He claims to love everybody, and I think he actually both means it and does.  It's fascinating.

Anyway, I adore this guy-- everybody does.  Yet sadly, he's a Vegan.  And he lets us know it.  (I seem to recall that he THINKS he's quite subtle.  That makes it doubly bad.)  

But I do not shy from a battle!  ...even one where I obviously do NOT have the moral high ground, by any stretch of the imagination.  

For those of you familiar with the movie Scott Pilgrim Versus The World, you know the stakes are high.  Vegans get Vegan Edge, just as portrayed (100% accurately) in said movie and documented by many reputable scientific studies (including here, here and here).

In this movie, the protagonist has to fight a  bad-guy Vegan, and it gives said baddie psychic powers.  The enemy's girlfriend actually SAYS what all non-Vegans think/fear:   "Being Vegan just means that you're better than most people."  The enemy Vegan in the movie adds "Bingo", his eyes glow white, and he throws the hero (Scott Pilgrim) a half-mile straight upwards, through a psychic force of will.  See for yourself.

This is some heady shit I'm up against here.

When Scott (played capably by Michael Cera) lands in a heap in the alley, he tells his girlfriend, "If I peed my pants, could you just pretend I got wet from rain?" She replies, "It's not raining." ...Anyway, that could be ME if I'm not careful.

So last week, he comes into his office (which he lets me use sometimes) and blammo:  "Did you eat animal flesh in my office?"  I had just finished eating an individual-sized pepperoni pizza from Cosi.   My response below:

 Hm.  Is pepperoni considered flesh?  It is?  Oh.  Then I definitely consumed flesh in here.  It was delicious.  Pepperoni is a form of bacon, and bacon is the best food in the world.
 
Some day, I have no doubt, you will transcend this earth and rise unto the heavens where you will be greeted by angels who can see, on a cosmically crystal-clear level, how you eased the burden and lifted the spirits of those around you, and generally did What Was Right.  The trumpets will sound and honors will be bestowed upon you.

But then you'll be like, "Can I finally get some bacon?"  And they'll be all "No, we don't have bacon here.  If you wanted bacon, you should have eaten bacon down there.  I mean, you really should have, it's absolutely delicious."

And you'll be all like "Fuck."

Awesome Links!

Note from graphic designer:  Yes, I know that's an 'At' sign, which is more appropriate for a blog entry about emails, rather than one about links.  Links don't really use the 'At' sign, I get that. But seriously, what symbol would work?  A cobweb?  Maybe a computer?  A chain link?   You're judging me, aren't you?  Hey, if you don't like my image selection, then you can go screw.  I'm doing the best I can.  I don't deserve this. 

Shut Up!  Jesus.  Somebody seriously needs to talk to the guy who makes these images.  He totally wrecked the mood.  I don't think it's even worth doing a story now.  Just forget it, go surf porn or reddit or whatever.

No, wait.   Wait.

I'm back.  I can do this.   It just shook me up for a minute there.  S'cool.

So here are are some cool links.  They're at least worth clicking on so you can silently judge me and how shitty my tastes are that I was ensnared by such vapid clickbait.

Cool things I've read online lately: 

Deepest Hole!  BTW, the story of how far we've dug into the earth is absolutely fascinating.  The deepest hole, I've learned, was only seven miles deep.   Drive seven miles from Boston, and -- well, depending upon where you started, you might not technically have left the city yet.  Go that far down and you're at the bottom of the deepest hole.  

To get to the middle of the Earth, you'd have to go from Boston to San Francisco, and then drive another thousand miles after THAT.  ..And so far, we've gotten to Cambridge MA.  To wit, we haven't been ANYWHERE underneath the Earth.   (BTW, it got too mushy to continue drilling, and crazy hot, which is why they stopped.)

Check it out:  http://www.damninteresting.com/the-deepest-hole/
 
Warning:  If you're interested, please follow that link.  ...Do NOT Google the term "Deepest Hole," or else it will end in tears, all right?    (...For those of you who didn't get the joke, Deepest Hole is also a rather lurid type of porn.)

Online Scale Model of the Goddamn Solar System 
This site takes you on a truly intimate journey of the solar system.  Amazingly well done.  This needs to be nominated for an award or something.  It shows you just how vast space is, between the Sun and Mercury ...and then continues all the way out to Pluto!  AND IT'S ALL TO SCALE (with a lot of horizontal scrolling which is AMAZINGLY EFFECTIVE as an analogy for distance).  

And then it's overlaid with wonderful writing, starting out with inviting lines like "It sure is empty out here." as you scroll and then chimes in every now and then, as you "journey" outward by scrolling right.

Here is the link:  http://joshworth.com/dev/pixelspace/pixelspace_solarsystem.html

Being Good Is Important

I think that being good is universal, and does matter.  And I think that the lords of the universe (if any exist) would agree with me upon that, for the very reason that being good invites things to succeed, whereas being bad makes them more prone to fail.  And failing just sucks.  

Said another way, being good has short- and long-term consequences seriously beyond our ken, and invite success of things that might, on many many cosmic levels, really matter.

How much of a difference can we really make?  I'm not really sure, although I think it's a question worth pondering.  I mean, how many people could you seriously help  (or impinge upon) in a lifetime?  How many systems (governmental, economic, relationship, community) could a person help to advance, or help to undermine?   And is there value in the positive outcome?  I think the answer is [expletive] of course there is.

I say: Goodness is honoring those around you, and the systems in which you exist.  Wish for their success just as your own, and on some cosmic level, you really ARE doing good that matters.  You're helping advance things. Love it all.  Wish it the best in the world.   

But then again...


Do you ever think that some people kind of deserve bad things to happen to them?  If so, would the destroyer be doing a good deed?   Could it be good to destroy?  Maybe something good would grow in its place or something. My brother is prone to raging against injustice. 

I don't know how to feel about it.  There is a strong argument to be made that we should fight oppression -- indeed, if we are being oppressed.  Maybe he's right, but it's not in my nature. I'm more of the "Can't we all just get along" type.

A Clean-Cut-Boy-Band-turned-Stoner-Hippie-Troupe once penned: 

You say you want a revolution
Well you know
We all want to change the world
You tell me that it's evolution
Well you know
We all want to change the world
But when you talk about destruction
Don't you know you can count me out

I think I support that position, despite having never technically been part of a boy band, myself.