Posts

Comments

Get Posts by E-mail

BTW: Referencing My 2009 Annual Report.

Technical

  1. Specs. Hardware: Mac Pro / 2.66GHz Quad-Core / 8GB ¶ Software: Excel 2004, Photoshop CS4, After Effects CS4, Final Cut Pro 7.
  2. Workflow. I sketched an outline on paper, then ordered it in Google Docs and turned that into sixty Photoshop compositions. That took about two weeks. Then I sequenced those compositions into a slideshow of still images and synced them in Final Cut Pro to a Creative Commons track. After Effects doesn't play nicely with music so I spent the next two weeks working deaf, working exclusively off the timecodes from Final Cut Pro. (ie "Okay, the pie graph needs to finish its rotation at 2:41:20.") The first day I saw it with music was January 31, the same day I posted it.
  3. Music. I'm not saying I did anything fantastic to the music track, but I did have to sync the slides to the rhythm, making adjustments for longer segments (any of the "top five + other" bar graphs, the travel maps), cutting and blending the song so it complemented the content of the video. I am saying that Animoto won't do this automatically. iMovie won't do this automatically. And teachers consistently overrepresent the capability of those tools.
  4. Data Sources. I maintained active records in Google Tasks before transferring them to an Excel sheet biweekly which I backed up fastidiously over the course of the year. Perish the possibility I might lose it. ¶ I collected all music records passively through last.fm, which became significantly more accurate after I outfitted my car with a 30-pin iPhone cable and began tracking car audio. ¶ I also collected my mobile phone statistics passively through AT&T's online billing system, which kindly exports data to Excel.

Mathematical

I don't see any of my students buying this pitch but here it is anyway: I would have had to release this video somewhere in April if I didn't have a working knowledge of a) the degree measure of angles, b) proportions, c) percents, d) coordinates, e) 3D space, f) modular arithmetic, and g) linear interpolation. I even calculated an integral.

Here's just one example. You noticed the little animated counters running all throughout the project? Problem: you want the counter to read "0″ at 773 frames into the composition and "44651″ at 795 frames:

Solution: a linear equation!

Math.round(timeToFrames()*2029.59-1568872.77)

Miscellaneous

  1. Guilt. I watched a continuous 20 days and 23 hours of television and movies. I could slap qualifiers all over that statistic but you're still talking about nearly a month spent proximate to a flickering light.
  2. Battlestar Galactica. Not worth it.
  3. Guilt II. 18 gallons sounds like kind of a lot of beer when you put it that way.
  4. Margin of Error. I'll put the average margin of error for the project somewhere below 1%. And I'll wager the sleep statistics are the worst. I had to remember to take a screenshot of my iPhone's clock wallpaper twice a day at the two times of the day that I was the least likely to remember to take a screenshot of my iPhone's clock wallpaper.
  5. 2.5 Minutes. I'm really happy with the length of the piece. That's, like, 2.4 days per second!
  6. Editing. For every statistic I included there were two I cut. There were albums and songs; incoming calls, outgoing calls, outgoing messages, fastest rising message recipients as well as fastest falling; places where I drank beer, number of people with whom I drank beer; repeat vs. first time movie viewings, number of people with whom I watched movies; plus a host of Twitter statistics and a Wordle visualization of my 2009 text message content which were, predictably, pretty dull. ¶ The most poignant graph that I cut for length was this one, which features both my father's cancer diagnosis and, relatedly, the fact that I drove the length of the US in one month without leaving California.

    What a strange project.

  7. 2010. It's been two years and I can't see quitting this kind of introspection. I'm already anticipating my decade retrospective where I hope I'll see a lot of huge life changes reflected in microscopic daily statistics. That'll be great. ¶ My goal for next year is to post my completed annual report video no later than January 1, 2011. I don't think this is impossible. Andrew Kramer recently composed a convincing tutorial explaining dynamic bar graphs in After Effects, where you just enter the final data and Javascript recomposes the entire project. The trick will be extending that process across an entire video and several different infographics. In any case, I need to publicly throw down this gauntlet.

41 Responses to “My 2009 Annual Report — Behind The Scenes”

  1. [...] BTW: My 2009 Annual Report — Behind The Scenes [...]

  2. on 02 Feb 2010 at 5:38 amDoug Belshaw

    Right. So all I need is:

    A Mac Pro
    Final Cut Pro
    Some other expensive stuff
    A month of my life

    I’m there! ;-)

    But seriously, Dan, this is absolutely great. A lesser person would have kept how to do it to himself out of a fear of being knocked off his perch.

    Well done. :-D

  3. on 02 Feb 2010 at 5:54 amJenny

    Whether or not you decide to share TV and beer data next year you know we will all be curious to see if your reflections on 2009 impact your choices in 2010.

  4. on 02 Feb 2010 at 8:39 amChris D

    Dan:

    For tracking sleep, a lot of my friends have been finding interesting data with this: http://www.geek.com/articles/mobile/sleep-cycle-alarm-clock-for-iphone-20090914/

    Haven’t tried it myself, but might be interesting.

    Chris

  5. on 02 Feb 2010 at 9:34 amDan Meyer

    @Doug, video remains frustrating undemocratic and prohibitively expensive. Open source options either suck or don’t exist. I could only justify the cost of the hardware / software I used here because I freelance on it also.

    You’ve expressed an interest in Processing, which might work for this kind of data visualization. It would be a good excuse to learn the language, in any case.

    @Jenny, I’ve been thinking a lot about the merits of using this as a kind of formative assessment for life. I could set up the charts on 1 Jan 2010 and watch them change over the year, modifying my behavior when I see a spike in my boozing or TV watching. But would that betray the accuracy of the year-end introspection? I don’t really know and I get a headache thinking about it.

  6. on 02 Feb 2010 at 11:49 amDean Shareski

    Having taken a photo a day for 2 years and a month, it’s very interesting to compare the two projects. When I watch my photos I see the ebb and flow of the year and are quickly taken away to specific moments that extend beyond the photo. Others are mundane but even they tell the story of mundaneness. For me they weave together in a fairly coherent narrative, meaning I think I’ve represented my year fairly accurately.

    You’ve shared a few of your reflections above but does it somehow tell a bigger story for you or is each statistic simply a story on its own?

  7. on 02 Feb 2010 at 12:03 pmgbrandtech

    Never feel guilty about drinking good beer.

  8. on 02 Feb 2010 at 12:20 pmDan Meyer
    You’ve shared a few of your reflections above but does it somehow tell a bigger story for you or is each statistic simply a story on its own?

    Afraid it’s too hard for me to tell with 2009 looming so large in the rear-view mirror right now. At the moment it all feels disjointed but I suspect over time, I’ll begin to put the scattered pieces together and recall that 2009 was the year I would come home from work, watch the Daily Show, then text Chris, and grab a Stella Artois at his house.

  9. on 02 Feb 2010 at 6:27 pmDenis

    *records in Google Tasks before transferring them to an Excel sheet biweekly which I backed up fastidiously over the course of the year. Perish the possibility I might lose it.*

    Hello Dan. If I wrote a tool to automatically backup your Google Tasks would you use it? How much would you be willing to pay (per year)?

  10. on 04 Feb 2010 at 8:13 amMichael K.

    You know I love this. I’d love to see everyone in the world do something like this, with the medium chosen to best represent their idiom, for lack of a better word. Fr’instance, mine would manifest itself as a snarky, hyper-punctuated three-sentence pp. followed by a self-deprecating doodle.

  11. on 06 Feb 2010 at 2:23 pmDoug Belshaw

    @Dan: Thanks Dan – did you buy a book to help you get started with Processing or did you use the online guides + trial-and-error?

  12. on 06 Feb 2010 at 3:56 pmDan Meyer

    Oh, mercy, I have no idea how to code in Processing. I’m just saying, it’s another option for this kind of visualization that has the benefit of being free.

  13. [...] aiming to surpass himself, Dan Meyer has posted his 2009 annual report (and, usefully, how he did it). Not only did he have the discipline to gather all this data, but just look at the quality of the [...]

  14. on 07 Feb 2010 at 6:02 amMark

    Totally blown away…how did you create the animated data visualizations?

    Would be a very helpful to learn more about this….

    Mark in Cinci

  15. on 09 Feb 2010 at 8:46 amSam

    I so agree about Battlestar Galactica.

    Great work. This plugs into the idea that knowledge and data will set you free/change your life. You can see it on a small scale in things like Nike +, which changes the way you go running, to larger ‘crowdsourcing’ environments, like Amazon, which leverages data about what other people have done/bought to provide users with suggestions and recommendations.

    Excellent to see someone do it to their life in so much detail.

  16. on 15 Feb 2010 at 3:40 amevinhas

    Hi Dan,

    Nice video and great statistical source of data. Could you talk about how-to you collect your data? I like so much statistical subject, and I’d like to do some similar database (without the video, only with the data), but I can’t understand how-know I can do this. Any suggestion?

    Thanks in advance.

    Kind regards from Spain.

  17. on 15 Feb 2010 at 11:01 amWhat did you do last year? | ohmykevin

    [...] Via Boing Boing comes this cool look at a year in the life of high school teacher Dan Meyer. [...]

  18. [...] movie watching, sleep, and so on, and made an entertaining movie from the compiled statistics. Dan Meyer’s 2009 Annual Report (Via The Quantified [...]

  19. on 15 Feb 2010 at 12:42 pmLa Lengua » Estadísticas de 2009

    [...] su blog cuenta con cierto detalle cómo lo llevó a cabo. Vía Boing Boing. Categoría: Curiosidades, Internet | No hay [...]

  20. on 15 Feb 2010 at 3:12 pmSimon

    Wow.. this was just awesome..

    I would also like to know more about your routines for data gathering.. Does it take up alot of time or do you have an efficient way of doing it?

    Cheers from norway :)

  21. [...] Behind the scene del video [...]

  22. on 15 Feb 2010 at 5:40 pmTyler

    You’re not using After Effects right.

    Hold down Ctrl when scrubbing and you’ll hear the audio. Also, you don’t need equations to animate counters, you can use keyframes and the number format.

    Really nice work, though.

  23. [...] of the video. You may also want to check out the post on his blog or another post where he explains how he made the video. SHARETHIS.addEntry({ title: "Dan Meyer is a Badass", url: [...]

  24. on 15 Feb 2010 at 8:36 pmDan Meyer

    @Tyler, thanks for the tips. The numbers effect is pretty slick, but it only counts up to 30,000, which was too low. Is there a way to hear music during RAM preview in AfterEffects, not just scrubbing through?

    @Simon & @evinhas, I’m not sure what you’re after specifically. I can mention that, per day, I spend less than a minute tallying up my results. If I watch a movie or drink a beer, I make a note on my phone in Google Tasks, which syncs immediately to the web. After two weeks, I’ll spend an hour dumping all that data into an Excel spreadsheet and then back it up in several places.

    I would record as much metadata as I could for each statistic. So, for every beer, I recorded how many people I drank it with, where I drank it, and whether it was from a bottle or off the tap. That’s a very small price to pay for access to several more infographics at the year’s end.

    Let me know if you have any specific questions I can answer.

  25. [...] Nice to know: The video takes two and a half minutes. Making that video took quite a while, as you can imagine. 2.4 days per second of the video, to be precise. If you want to learn more about how this video was made visit Mr Meyer’s blog: blog.mrmeyer.com. [...]

  26. [...] all that aside – because, really, who cares? – he’s got a break down of how to do it on his blog, where he’s also got enough math info to make my head spin (sorry kids, you [...]

  27. on 16 Feb 2010 at 7:44 amGabe

    How did you create the circular bar chart (used in “mobile call minutes by time of day)? Was this done in excel or one of the other programs? Do you know of a way to do this in excel?

  28. on 16 Feb 2010 at 11:54 amDan Meyer

    Sorry, Gabe, that was a homebrew.

  29. on 16 Feb 2010 at 2:58 pmjared

    great job!
    i this is something i’ve been trying to do for a little while, not so much the movie aspect, but the data aspect. I kept a notebook in my back pocket, but have since upgraded to a “smarter” phone so i really appreciate the tips and behind the scenes look at your process. Have you ever used http://www.daytum.com or seeen the work of Feltron? You guys should probably sit down for a drink so you could each record your encounter. http://feltron.com/index.php?/content/2009_annual_report/
    i think people really respond to their own statistics. is video your only medium of choice? would you ever try a printed or other tangible version?

  30. on 17 Feb 2010 at 1:20 pmishak kang

    you sir, are an inspiration. let’s get your home and work space wired up with sensors for more meta data! we’re trying to figure out how to make data viz exciting for college students in order to affect energy efficiency. i will share this video with my team.

  31. [...] Gran trabajo visual de Dan Meyer para ofrecernos una porción de lo que sucedió en el año 2009 a puro efecto visual. Pueden ver el minucioso trabajo, paso a paso, en su blog. [...]

  32. on 18 Feb 2010 at 4:53 pmCynthia

    Dan, thanks! I stumbled upon this by accident, and just lost an hour’s worth of productive work time (well, maybe not…should have measured it I guess…) Seriously, thanks for doing this, and for explaining how. Hey, did you find that your behavior changed at all since you knew you were measuring it and might be “reporting” it later?

  33. on 23 Feb 2010 at 9:05 pmTouzel

    I’m with Gabe–the coolest graphic, by far, was the “mobile call minutes by time of day”. That one is pretty rad.

  34. on 24 Feb 2010 at 6:06 amMatt

    interested to hear what you used to create graphics and video?

  35. on 26 Feb 2010 at 7:37 pmPaul

    Hey Dan, What typefaces are you using for this piece?

    As for hearing audio correctly in After Effects, you may want to convert them to .AIF or .WAV files. AE or even FCP don’t play nicely with .mp3′s. When you ram preview (0 on number pad) them or even do a frame skipping(shift + 0) ram preview, you should be able to hear the music just fine

  36. on 28 Feb 2010 at 8:22 amDan Meyer

    Thanks for the tip, Paul. The primary typefaces here are headthinker and BlaxSlabXXL, both of which are available free through dafont.

    @Cynthia, I’ve gone back and forth on the matter of looking at the graphs throughout the year and adjusting my life accordingly. I decided that would fall outside the scope of my goals here which is, first and foremost, a reflection.

  37. on 01 Mar 2010 at 7:49 amAyesha

    Awesome!

  38. [...] If you’re interested in how it was made, he lets you in on all his secrets here. [...]

  39. on 14 Oct 2010 at 7:40 amgoneskiing

    Dan- I loved your presentation!
    I teach remedial math to 8th graders. One point that I am constantly making to my students is that math is everywhere in our lives. I showed my students your video + they first commented on how much they liked your music. I asked them if they could compose a video of their own lives. They said yes….we will see if they can.
    Thanks for sharing your math tips!

  40. [...] Dan Meyer’s 2009 Annual Report from Dan Meyer on Vimeo. [...]

  41. [...] it often as a example of using data to tell storie, I decided to give it a shot. While Dan used a wack sack of tools and does a much more professional job, I wanted to see if I could use Keynote to tell the story of [...]