BTW: Referencing My 2009 Annual Report.
- Specs. Hardware: Mac Pro / 2.66GHz Quad-Core / 8GB ¶ Software: Excel 2004, Photoshop CS4, After Effects CS4, Final Cut Pro 7.
- Workflow. I sketched an outline on paper, then ordered it in Google Docs and turned that into sixty Photoshop compositions. That took about two weeks. Then I sequenced those compositions into a slideshow of still images and synced them in Final Cut Pro to a Creative Commons track. After Effects doesn’t play nicely with music so I spent the next two weeks working deaf, working exclusively off the timecodes from Final Cut Pro. (ie “Okay, the pie graph needs to finish its rotation at 2:41:20.”) The first day I saw it with music was January 31, the same day I posted it.
- Music. I’m not saying I did anything fantastic to the music track, but I did have to sync the slides to the rhythm, making adjustments for longer segments (any of the “top five + other” bar graphs, the travel maps), cutting and blending the song so it complemented the content of the video. I am saying that Animoto won’t do this automatically. iMovie won’t do this automatically. And teachers consistently overrepresent the capability of those tools.
- Data Sources. I maintained active records in Google Tasks before transferring them to an Excel sheet biweekly which I backed up fastidiously over the course of the year. Perish the possibility I might lose it. ¶ I collected all music records passively through last.fm, which became significantly more accurate after I outfitted my car with a 30-pin iPhone cable and began tracking car audio. ¶ I also collected my mobile phone statistics passively through AT&T’s online billing system, which kindly exports data to Excel.
I don’t see any of my students buying this pitch but here it is anyway: I would have had to release this video somewhere in April if I didn’t have a working knowledge of a) the degree measure of angles, b) proportions, c) percents, d) coordinates, e) 3D space, f) modular arithmetic, and g) linear interpolation. I even calculated an integral.
Here’s just one example. You noticed the little animated counters running all throughout the project? Problem: you want the counter to read “0” at 773 frames into the composition and “44651” at 795 frames:
Solution: a linear equation!
- Guilt. I watched a continuous 20 days and 23 hours of television and movies. I could slap qualifiers all over that statistic but you’re still talking about nearly a month spent proximate to a flickering light.
- Battlestar Galactica. Not worth it.
- Guilt II. 18 gallons sounds like kind of a lot of beer when you put it that way.
- Margin of Error. I’ll put the average margin of error for the project somewhere below 1%. And I’ll wager the sleep statistics are the worst. I had to remember to take a screenshot of my iPhone’s clock wallpaper twice a day at the two times of the day that I was the least likely to remember to take a screenshot of my iPhone’s clock wallpaper.
- 2.5 Minutes. I’m really happy with the length of the piece. That’s, like, 2.4 days per second!
- Editing. For every statistic I included there were two I cut. There were albums and songs; incoming calls, outgoing calls, outgoing messages, fastest rising message recipients as well as fastest falling; places where I drank beer, number of people with whom I drank beer; repeat vs. first time movie viewings, number of people with whom I watched movies; plus a host of Twitter statistics and a Wordle visualization of my 2009 text message content which were, predictably, pretty dull. ¶ The most poignant graph that I cut for length was this one, which features both my father’s cancer diagnosis and, relatedly, the fact that I drove the length of the US in one month without leaving California.
What a strange project.
Doug BelshawFebruary 2, 2010 - 5:38 am -
Right. So all I need is:
A Mac Pro
Final Cut Pro
Some other expensive stuff
A month of my life
I’m there! ;-)
But seriously, Dan, this is absolutely great. A lesser person would have kept how to do it to himself out of a fear of being knocked off his perch.
Well done. :-D
JennyFebruary 2, 2010 - 5:54 am -
Whether or not you decide to share TV and beer data next year you know we will all be curious to see if your reflections on 2009 impact your choices in 2010.
Chris DFebruary 2, 2010 - 8:39 am -
For tracking sleep, a lot of my friends have been finding interesting data with this: http://www.geek.com/articles/mobile/sleep-cycle-alarm-clock-for-iphone-20090914/
Haven’t tried it myself, but might be interesting.
Dan MeyerFebruary 2, 2010 - 9:34 am -
@Doug, video remains frustrating undemocratic and prohibitively expensive. Open source options either suck or don’t exist. I could only justify the cost of the hardware / software I used here because I freelance on it also.
You’ve expressed an interest in Processing, which might work for this kind of data visualization. It would be a good excuse to learn the language, in any case.
@Jenny, I’ve been thinking a lot about the merits of using this as a kind of formative assessment for life. I could set up the charts on 1 Jan 2010 and watch them change over the year, modifying my behavior when I see a spike in my boozing or TV watching. But would that betray the accuracy of the year-end introspection? I don’t really know and I get a headache thinking about it.
Dean ShareskiFebruary 2, 2010 - 11:49 am -
Having taken a photo a day for 2 years and a month, it’s very interesting to compare the two projects. When I watch my photos I see the ebb and flow of the year and are quickly taken away to specific moments that extend beyond the photo. Others are mundane but even they tell the story of mundaneness. For me they weave together in a fairly coherent narrative, meaning I think I’ve represented my year fairly accurately.
You’ve shared a few of your reflections above but does it somehow tell a bigger story for you or is each statistic simply a story on its own?
gbrandtechFebruary 2, 2010 - 12:03 pm -
Never feel guilty about drinking good beer.
Dan MeyerFebruary 2, 2010 - 12:20 pm -
Afraid it’s too hard for me to tell with 2009 looming so large in the rear-view mirror right now. At the moment it all feels disjointed but I suspect over time, I’ll begin to put the scattered pieces together and recall that 2009 was the year I would come home from work, watch the Daily Show, then text Chris, and grab a Stella Artois at his house.
DenisFebruary 2, 2010 - 6:27 pm -
*records in Google Tasks before transferring them to an Excel sheet biweekly which I backed up fastidiously over the course of the year. Perish the possibility I might lose it.*
Hello Dan. If I wrote a tool to automatically backup your Google Tasks would you use it? How much would you be willing to pay (per year)?
Michael K.February 4, 2010 - 8:13 am -
You know I love this. I’d love to see everyone in the world do something like this, with the medium chosen to best represent their idiom, for lack of a better word. Fr’instance, mine would manifest itself as a snarky, hyper-punctuated three-sentence pp. followed by a self-deprecating doodle.
Doug BelshawFebruary 6, 2010 - 2:23 pm -
@Dan: Thanks Dan – did you buy a book to help you get started with Processing or did you use the online guides + trial-and-error?
Dan MeyerFebruary 6, 2010 - 3:56 pm -
Oh, mercy, I have no idea how to code in Processing. I’m just saying, it’s another option for this kind of visualization that has the benefit of being free.
MarkFebruary 7, 2010 - 6:02 am -
Totally blown away…how did you create the animated data visualizations?
Would be a very helpful to learn more about this….
Mark in Cinci
SamFebruary 9, 2010 - 8:46 am -
I so agree about Battlestar Galactica.
Great work. This plugs into the idea that knowledge and data will set you free/change your life. You can see it on a small scale in things like Nike +, which changes the way you go running, to larger ‘crowdsourcing’ environments, like Amazon, which leverages data about what other people have done/bought to provide users with suggestions and recommendations.
Excellent to see someone do it to their life in so much detail.
evinhasFebruary 15, 2010 - 3:40 am -
Nice video and great statistical source of data. Could you talk about how-to you collect your data? I like so much statistical subject, and I’d like to do some similar database (without the video, only with the data), but I can’t understand how-know I can do this. Any suggestion?
Thanks in advance.
Kind regards from Spain.
SimonFebruary 15, 2010 - 3:12 pm -
Wow.. this was just awesome..
I would also like to know more about your routines for data gathering.. Does it take up alot of time or do you have an efficient way of doing it?
Cheers from norway :)
TylerFebruary 15, 2010 - 5:40 pm -
You’re not using After Effects right.
Hold down Ctrl when scrubbing and you’ll hear the audio. Also, you don’t need equations to animate counters, you can use keyframes and the number format.
Really nice work, though.
Dan MeyerFebruary 15, 2010 - 8:36 pm -
@Tyler, thanks for the tips. The numbers effect is pretty slick, but it only counts up to 30,000, which was too low. Is there a way to hear music during RAM preview in AfterEffects, not just scrubbing through?
@Simon & @evinhas, I’m not sure what you’re after specifically. I can mention that, per day, I spend less than a minute tallying up my results. If I watch a movie or drink a beer, I make a note on my phone in Google Tasks, which syncs immediately to the web. After two weeks, I’ll spend an hour dumping all that data into an Excel spreadsheet and then back it up in several places.
I would record as much metadata as I could for each statistic. So, for every beer, I recorded how many people I drank it with, where I drank it, and whether it was from a bottle or off the tap. That’s a very small price to pay for access to several more infographics at the year’s end.
Let me know if you have any specific questions I can answer.
GabeFebruary 16, 2010 - 7:44 am -
How did you create the circular bar chart (used in “mobile call minutes by time of day)? Was this done in excel or one of the other programs? Do you know of a way to do this in excel?
Dan MeyerFebruary 16, 2010 - 11:54 am -
Sorry, Gabe, that was a homebrew.
jaredFebruary 16, 2010 - 2:58 pm -
i this is something i’ve been trying to do for a little while, not so much the movie aspect, but the data aspect. I kept a notebook in my back pocket, but have since upgraded to a “smarter” phone so i really appreciate the tips and behind the scenes look at your process. Have you ever used http://www.daytum.com or seeen the work of Feltron? You guys should probably sit down for a drink so you could each record your encounter. http://feltron.com/index.php?/content/2009_annual_report/
i think people really respond to their own statistics. is video your only medium of choice? would you ever try a printed or other tangible version?
ishak kangFebruary 17, 2010 - 1:20 pm -
you sir, are an inspiration. let’s get your home and work space wired up with sensors for more meta data! we’re trying to figure out how to make data viz exciting for college students in order to affect energy efficiency. i will share this video with my team.
CynthiaFebruary 18, 2010 - 4:53 pm -
Dan, thanks! I stumbled upon this by accident, and just lost an hour’s worth of productive work time (well, maybe not…should have measured it I guess…) Seriously, thanks for doing this, and for explaining how. Hey, did you find that your behavior changed at all since you knew you were measuring it and might be “reporting” it later?
TouzelFebruary 23, 2010 - 9:05 pm -
I’m with Gabe–the coolest graphic, by far, was the “mobile call minutes by time of day”. That one is pretty rad.
MattFebruary 24, 2010 - 6:06 am -
interested to hear what you used to create graphics and video?
PaulFebruary 26, 2010 - 7:37 pm -
Hey Dan, What typefaces are you using for this piece?
As for hearing audio correctly in After Effects, you may want to convert them to .AIF or .WAV files. AE or even FCP don’t play nicely with .mp3’s. When you ram preview (0 on number pad) them or even do a frame skipping(shift + 0) ram preview, you should be able to hear the music just fine
Dan MeyerFebruary 28, 2010 - 8:22 am -
Thanks for the tip, Paul. The primary typefaces here are headthinker and BlaxSlabXXL, both of which are available free through dafont.
@Cynthia, I’ve gone back and forth on the matter of looking at the graphs throughout the year and adjusting my life accordingly. I decided that would fall outside the scope of my goals here which is, first and foremost, a reflection.
AyeshaMarch 1, 2010 - 7:49 am -
goneskiingOctober 14, 2010 - 7:40 am -
Dan- I loved your presentation!
I teach remedial math to 8th graders. One point that I am constantly making to my students is that math is everywhere in our lives. I showed my students your video + they first commented on how much they liked your music. I asked them if they could compose a video of their own lives. They said yes….we will see if they can.
Thanks for sharing your math tips!