The scariest difference between 360 and 2D video.
With flat video the director can easily show the viewer exactly what he wants to show, which is usually in a narrow field of view... 65 degrees for a 28mm wide shot, 40 degrees for a 50mm medium shot, and 20 degrees for a 100mm closeup. But 360 degree video shows everything, and that freedom for the viewer to look anywhere is terrifying for a traditional 2D filmmaker.
360 degree filmmaking is much closer to theatrical productions than film productions in that the audience can see everything... but even more so because now the audience is on the stage! Kind of like an "in the round" theatrical production but in reverse. Now we must become choreographers more than cinematographers.
In my own work, I'm interested in using 360 video in documentary storytelling more than in narratives. I've got an interest in exploring creative projects and music videos, but in general I want to use these new tools to tell compelling stories about people and places that expand the experience of the viewer. So in many cases I won't have the luxury of rehearsals, complex scene blocking and other methods of scene preparation.
And I won't have the luxury of cutting to an intense closeup when I want my viewers to feel the subject. This is one of the things that scare me the most... how can I create a strong connection to the subject when I'm only shooting with a wide-angle lens?