The SF Women's March... my first 360° doc
If you're reading this on your phone and want to watch the video on your Google Cardboard headset, just tap the title of the video and it'll take you directly to the Youtube app where you can click on the cardboard "glasses" icon to begin watching immersively.
In my quest this year to do as much 360° experimentation as possible, I'm always looking for interesting subjects that would be enhanced by immersion. My friend David Lawrence (one of the most experienced 360 video creators around) recently pointed me to an early Chris Milk video... a piece Milk did for Vice News along with Spike Jonze about the 2014 Millions March in NYC. Here's a link to it... very much worth watching. I was struck by how powerful experiencing a protest march was in 360° compared to a typical 16:9 frame and realized that I had a similar opportunity coming up this month at the Women's March in San Francisco. Milk had a camera operator carry a large 360° rig on a steadicam for that shoot (and you can see him walking behind the camera operator if you look behind you) and he shot it stereoscopically, which really amped up the presence of the scene. I'm still primarily shooting monoscopic 360° video because the constraints required for stereoscopic (both on the shooting and stitching side) are too much for me to deal with on a run&gun shoot like this at this point. My big camera was definitely too cumbersome for me to be nimble as a solo director/operator, and even with the new stabilization code in MistikaVR, I didn't think I could be spontaneous enough with a big heavy rig. But I'd just purchased a GoPro Fusion last week expressly for these kinds of run&gun shoots and was eager to see how good its vaunted stabilization really was. The Fusion is a 5.2k camera, with resolution still above what any of the HMDs can display, and it looked like a great solution. Although you can control it with a phone app, once it's configured you can start it up from power-off and begin shooting in 4 seconds by pressing just one button. (!)
I set the camera to underexpose by .5/stop to make sure I was protecting the highlights a bit, put the camera on a lightweight monopod with fold-out legs, and had a rig that weighed just 2lbs total... perfect for carrying around all day on the streets of San Francisco. As part of shoot prep, I showed the Millions March piece to my friend Debra Schwartz to give her a sense of what we were going after and was confident that Debra could handle the role of guide because of her vast experience interviewing people for the Mill Valley Historical Society.
For any film to be engaging it has to have what the Stillmotion folks call the "four pillars of story"... People, Place, Plot and Purpose. The Women's March had all of these IN SPADES. And with Debra being our guide through the film, I had a person who the audience could relate to as a proxy within the scene... all good. With her friend Tom Miscia as our assistant tasked with slating the audio and video for our double-system sound (using the Zoom H6n with the SGH-6 shotgun mic), we were ready to go.
In addition to capturing beautiful vignettes of the people at the march, I wanted to bring protest music into the film because every political movement is defined by its songs and music. (the film "Amandla" about South African apartheid is my favorite example of this.) And that approach paid off hugely for me... the film begins and ends with music and features 6-7 different songs throughout.
As usual for one of my projects, I had the structure for the piece pre-visualized "holographically" in my head before we started so all we had to do was fill in the blanks with actual footage. I don't want to write about the film itself too much because it speaks for itself, but you can imagine that with 60-70,000 people at an event like that, there are a wide variety of great characters to interview... and we did. I'd try to physically and psychically get out of the way as much as possible... sitting on the ground with the monopod between my legs if we weren't moving, and holding the camera slightly above my head and looking down at the ground if I didn't have the opportunity to sit.
Lessons Learned (oh boy... I learned so many lessons!)
Camera position: It's obvious that anyone on the crew should get as far away from the camera as possible or we'll be creating an episode of "Where's Waldo." Took me a few minutes to get that protocol in place.
The greatest thing about shooting with a camera that doesn't required the videographer to be near the camera itself is that the subject becomes relaxed while filming. Sitting on the ground while Debra was asking questions removed me so much from the scene that the subjects opened up more than they typically would, and in a different way than when they're in front of a traditional camera. This is extremely interesting (!) and I look forward to exploiting it further, although it does come with the caveat that your interviewer has a tremendous amount of responsibility to keep the interview on track.
Interview technique digression: It's possible in a controlled scene to shoot a clean plate in 360 without me in the scene and then composite that into the final shot, effectively removing me from the scene. I'm sure I'll experiment with that in the future, but it still begs the question about interview timing and how to remove the interviewer's audio from the shot without dissolving through those portions when the interviewer is asking questions. The best way to deal with it is to have the interviewer ask questions that lead the subject to answering in a comprehensive (but not overly wordy) way and hoping for answers in the 20-40 second range.
Camera support and the nadir: In a professional production you'd probably want to remove the camera support (and me) from the nadir of the image by either patching or covering with a dark disc. For me, at this point in my experimentation, I'm allowing the viewer to see what's going on below the camera because I'm curious about how the support for a given shot happens. At some point I'll start experimenting with nadir occlusion, but that's still down the road for me.
Different perspectives for the camera: You'll see me experiment with 3 different perspectives in this piece... keeping the camera at eye level (most common), holding the camera 3-4 feet above me for a birds-eye view that forces the viewer to look down at the scene (very effective for a god-like viewer) and in a couple of cases keeping the camera a bit lower than the eyeline, forcing the viewer to look up at the subject (the interview with the Taiko drummer is a good example of this and it gives the subject a lot of gravitas when you have to look up at her). Typically in a 360° shot you'd want to keep the camera slightly below the subject's eyeline to make the viewer/subject "interaction" as natural as possible, but I do like playing with this.
Moving the camera: Here's a big one... As 360° filmmakers we have a responsibility to our subjects to do as much as possible to keep their vestibular systems happy. We don't want to make our viewers sick and that's easy to do when they're wearing a headset and their vestibular system's input doesn't match their visual system's input. Of course the safest way to do this is to just not move the camera. But as I've seen from Chris Milk's films (and others), you can get away with camera movement if it's very slow and even, especially if the shot includes an acceleration from stopped and ends with a deceleration to stopping. You can tell me how well I did with this, but since it was a march I really had no choice but to move the camera sometimes. What's amazing is how well the GoPro Fusion's stabilization works... it looks like I had the camera on a 3-axis gimbal instead of just a stick. VERY impressive!
Benefit of a 2-lens camera: With my bigger 6-lens camera there are 6 stitch lines and 6 opportunities for stitching errors as subjects cross the stitch line. For a shoot like this, with subjects and bystanders close to the camera. it would've been a nightmare to accomplish a 6-camera stitch without lots of tweaking (and living with a lot of errors). The GoPro (and other great inexpensive 2-lens 360 cameras like the Yi360) simplifies the stitching process so there's only one stitch line fusing the two lenses together, and GoPro's optical flow algorithm works quite well to mitigate stitching errors. I think the optimal 360 camera would be something like two high-quality 8k mirrorless cameras (GH6?), each with an Entaniya 250° "Hal" lens. This would be an expensive solution (each lens is almost $5k and the camera bodies would probably cost the same if they existed) but it would sure simplify the stitching solution.
While on the subject of 2-lens cameras and stitching... this march video is essentially a news story and my goal was to turn this piece around within 24 hours of the shoot, and with a larger camera it would've been impossible to stitch and edit so quickly. So, for run&gun situations, the GoPro is amazing. I did achieve my goal of finishing the edit within 24 hours, but then had to upload the 11Gb file to Youtube and wait for it to encode it for 360 video, which took overnight.
The Edit: This piece is long... 14 minutes is about twice as long as I'd usually give to a short doc about a subject like this. I'd use shorter cuts and coverage at different angles to produce a piece that provides a lot of variety in a shorter amount of time. But with 360° video I personally find that when I have time to absorb the scene without being rushed, it's much more compelling and immersive. I certainly could've eliminated a couple of our interviews and it would've provided a very similar feel to the doc, and I did cut the final edit from a first pass of 22 minutes to 14. But I wanted to give each of these interviewees a voice and since I didn't have a client over my shoulder pushing me down to a shorter TRT, I went with the fuller story of the march.
There are no connected clips or cuts to different angles. This is such a big deal... usually I'd cut into an interview with a combination of b-roll or coverage from a different camera, or maybe a push into the scene from the (4k or higher) master shot. I didn't do any of that for this piece although I could've cut away to 360° B-roll during an interview (and will experiment with that in a future film). For this film, and since there's so much going on in each scene, I decided to stay with each subject and allow the viewer to create their own B-roll by looking around the scene during the interview.
Keeping subjects lined up in post: To reduce the amount of spinning around that viewers have to do to watch important action, and to keep them from getting nauseous, one of the most important aspect of the edit is to adjust all of the subjects so that they appear in the same portion of the 360° frame. You'll see that you can watch the entire film by just looking at the same quadrant of the 360° view, which also simplifies my job as director with regards to making sure that the viewer is confident that they're looking at the right place and not missing any critical action. FCPX makes this easy to do with the Reorient control in the HUD, by holding down the Shift key while reorienting so that you're only moving along the X-axis. On a similar subject, it's very important to make sure vertical lines are as vertical as possible so that the viewer doesn't feel that they're falling over, another sure way to make them sick.
Color/grading: I used the GoPro "flat" color profile to maximize dynamic range, adding a little saturation back in during post with a few contrast curves along the way. Nothing fancy and just trying to keep the screen looking as natural as possible. (Luckily we had great sunny weather that day with lots of well-defined shadows, creating a very dimensional scene for the viewer to become immersed in.)
Audio: Slating the shot for double-system sound was a critical part of the production workflow and we used a whiteboard with incrementing shot numbers plus Debra speaking the number after we both were rolling. That made it easy for me to match up shots to synchronize them in Final Cut Pro X 10.4.
But of course the elephant in the room here is Spatial Audio and why I didn't post this in Ambisonic 360° audio on Youtube. Bottom line is that, since FCPX doesn't support spatial audio yet, I have to find a 3rd party solution and teach myself how to do it. At this point I've decided to go with a combination of Reaper and AudioEase and will give myself the next 3-4 weeks to learn those tools and come up with an efficient workflow. My lessons learned during that process ought to be interesting.
FCPX gotchas: 1) 2D image scaling... When inserting a 2D image into the 360 scene, the only scaling control you have is the Depth parameter and that doesn't provide enough flexibility to resize the image to any arbitrary size. I tried making different resolutions but that didn't work so then I brought it into Photoshop, added a transparent background layer and then increased the canvas size so the logo itself occupied only about 10% of the canvas. THEN I was able to get it to scale correctly. The 360° transform ought to be able to scale 2D content more flexibly than it does. Having to prep the artwork in Photoshop just to scale an image is clumsy and most people wouldn't intuitively realize that that was the way to do it. 2) Adjustment layers... I wanted to apply some global color grading to the entire timeline and normally would do that with adjustment layers. Unfortunately there are no 360° adjustment layers for FCPX yet and when you apply a regular Adj. layer it'll instantly create a seam in the shot. I've had the same problem when bringing shots over to grade in Davinci Resolve and will soon be digging deeper into how it may be possible grade in Resolve. So I had to apply individual adjustments to each shot. Stay tuned.
All said and done, I'm happy with how this project turned out and consider it a successful experiment because I learned a lot while creating something that serves as an historical document of the march. Onward to the next project!
Epilogue: The day after I posted this, my friend Kevin dropped by the studio and I asked if he'd ever worn a VR headset before. He hadn't and was eager to try the Vive. He put it on and after watching the Women's March video for 10 minutes I noticed he was weeping. Let him tell you in his own words:
Feel free to reword or edit or throw in the trash or whatever
On monday I stopped off at Gary’s office to do a quick guitar swap.
Gary was scrubbing through some Women’s March 360 video footage he shot over the weekend in SF.
Guitar swap would have to wait — I was going to pull up a stool and watch a sprint-to-the-finish edit.
If you have not watched one of these I urge you to get your hands on some goggles and watch. First 5 minutes were me thinking about the medium. Eventually, though, it crept up on me, much like drifting off to sleep. I was in the middle of it all. And better than that, I was not self conscious or concerned about stepping on a foot or bumping into an elbow. It was the version of me raw and open to input; an invisble guy floating around in the day in the life of
the Womens March. A very very passionate climate it was.
As I lost myself in the stories, overwhelming emotion took me over.. I had to pull off the gear and let my eyes out.
I can read the news, an emotional op-ed or watch a moving documentary, but something in the safety of riding along anonymous and invisible allowed these beautiful, empowered people to hit me right in the gut, and so I chose to weep outside gary’s goggles and regain composure over some duely offered kleenex.
Visualizing things you are afraid to actually do, attempting things you cannot afford to screw up for real — learning tools, therapy methods, or just an enhanced form of entertainment — if you get too lost there is always a box of ultra-soft bleached shredded trees to bring you back to earth.
As a storyteller, you can't ask for an empathetic response greater than that. This is quite a medium.