Finding Nemo’s Silver Lining: Using Big Data to Analyze Big Weather
For those of you who live in a data cave on the West Coast like I do, it may come as a surprise that there was a blizzard this past weekend—a BIG one. The ‘Nemo’ blizzard, caused by a merging of two low pressure systems that originated in the central and southeast US, then migrated to the north-eastern seaboard—affected millions of people in the US Northeast, with heavy snow and multiple power-outages.
The crack about using Big Data to analyze Big Weather almost writes itself here, but it is rather astounding how much external events, such as extreme weather and power outages, can influence online activity. Obviously, it’s no surprise that without power, there’s no e-commerce – yet I am surprised with how closely we can track the storm’s influence (and people’s response to it) using RichRelevance’s dataset.
In the animation below, I look at week over week hourly increase in the number of shopping sessions this past Friday and Saturday, for over 50 US retailers (comparing Friday Feb 9th to Friday Feb 1st, and Saturday Feb 10th to Saturday Feb 2nd, both on an hour by hour basis). To look at local effects, I tied our sales data to zip codes, using shape files available through the US census website.
Focusing on the northeast, most of Friday, we see a pronounced week over week increase in activity in the Greater New York City area, Albany, all of Connecticut and Rhode Island, Southeast Massachusetts, Southwest Maine, and most of Vermont. This was when the storm had really started its snow dump and most people were confined to their homes. By mid-day Saturday, when the storm was tapering, much of that same area saw a decrease in week over week activity, likely from a combination of power outages and people working outside to clear their homes.
For all you etailers out there, it just goes to show that while a little bit of bad weather might be a good thing, a lot of bad weather means your customer is behind his snow plow instead of his laptop!
Hello 2D Otters! That’s the most interactive method of data analysis I think I’ve ever heard of!
The video itself was made by generating still images (plots) of the data for each hour interval, then combining those images into a movie using a freeware stop-motion movie maker (basically you set the frame rate and BOOM! magic happens)
I forgot to include a legend, but the dark red means higher activity, and the pale yellow means lower activity. The maximum and minimum range was roughly +20% to -20% relative week over week change in the number of online shopping sessions. Thanks for reading and watching!
My second grade class has watched your video 4 times. When we thought everyone was on the internet, we shouted “Now”. Our teacher then paused the video. We started it again and when we thought the power went out, we shouted “Now”. When we thought the power went back on, we shouted “Now” a third time. We were pretty accurate by the 3rd time we watched the video.
We would like to know how you made this video. How did you turn the data into a video? How did you make the colors change?
The 2D Otters from El Paso, Texas say hi!