Big Data @Nemo

The best predictor of future behavior is past behavior.

"Big Data" Do a google search and you will get about 2 million hits, yes the term is popular. Basically big data is capturing lots of data points for a process, so that you can reason about the process and make predictions about future events based on historical trends. The stock market, patient response to therapy, crime trends, and shopping trends all examples of how people are using historical data to predict future events. If you are really interested I highly recommend taking Stephen Ng class on Machine Learning. He is a rock start in this world, and will teach you the methods to find the signal in the noise.

Well at nemo we are also on the Big Data bandwagon. We have our super fast computers analyzing large data sets to make predictions. I have chosen the tricky problem of weather. I've been fortunate enough to live in several different climates, Macon, Rochester, and now Boston. I am pretty much temperature agnostic, but I do like the sunshine. So who has more sunshine? Sure I could go out to the data bases and get the number of days of sunshine in each location, but I want to verify they are correct.

Below is our water cooled computer analyzing current weather conditions from these three locations.

Basically it wakes up 6 times a day, goes to Yahoo and asks for the cloud cover at each location. Records this and sends  the data to a free online graphing service. Ok we are one week into the process, so its difficult to draw any conclusion thus far. You should be able to click on this link in the future and see the graph of historical sunshine. As of today, Macon about 50 percent of the days are sunny, Boston about 40 percent, and Huron about 30.

The last graph is the temperature of the computer over the same time period. At a months scale it is hard to see but there is a pattern of when the sun is on the device, when it wakes up and when the heat turns on.  Here is a screen shot of the computer temperature this morning, you can see the slight rise in its temperature when the radiator turned on.