With so many VR practitioners focused on building immersive experiences, our team wanted to focus on building a tool to make creative professionals more productive. Specifically, we wanted to explore how virtual reality might make the process of video editing more intuitive and collaborative. Magic Studio is a step in that direction and one of the first apps that recognizes VR's ability to complete complex tasks.
We pay particular attention to the pain points of professional video editors. While today’s editors are privileged by a suite of powerful tools, they're also constrained by two-dimensional surfaces and a collaboration process that stifles creativity. As a result, their quality of product and degree of self-expression often suffer. That means decreased productivity and more expensive post-production services for digital media companies, TV networks, film studios.
Magic Studios seeks to address those issues by building the most intuitive video editing tool possible. Our design is based around the principles of direct manipulation, world-in-miniature perspective, and infinite screen space. We believe that this approach will not only improve the efficiency of professional video editors, but also expand the accessibility of video editing to new consumer markets.
What it does
Magic Studio is a VR experience that allows users to create and collaborate with others inside a 3D environment. Users can drag video clips from a pre-organized folder, set in and out points using the controllers, quickly access and edit clips on the storyboard, add and rearrange multiple clips to a timeline, and preview playback of the sequence. To collaborate with others, users can duplicate the existing timeline for others to make modifications to.
We built Magic Studio for the HTC Vive using Unity, C#, Photoshop and Premier.
Challenges we ran into
We weren’t able to get real time collaboration to work, but we were able to get a version of forking to work.
There are many UI ideas that we weren’t able to implement in time such as layout, transitions, audio within the clips - these are minor changes but would have made the experience better.
We lacked of free video codec that was useful so we had to code our own from scratch. We ended up implementing 2 different kinds of video playback methods using image stacks and we ended up using a mix of both of them for the final demo - twitch play and slow play. We did this because it was impossible to have many clips playing at full frame rate loading from the file system and we didn’t have the time to implement an H264 encoder.
We spent too much of our time benchmarking our video-streaming algorithm to make sure it could handle our project before we built anything.
We experienced difficulty with 3D UI design - there was so much we were able to do but we didn’t know what we should do because we are not used to thinking beyond 2D interface so we have to spent a lot of time figuring out what would make sense for the user.
We experienced challenges optimizing our program so that it would run at an excellent frame rate.
We struggled to show multiple clips at once without losing frame rate, we struggled to find our upper bound with how many clips we can show on the screen at one time.
We had some issues implementing an efficient fork feature, where we could fork our project into a duplicate. Doing deep copies proved expensive and severely impacted frame rate, while shallow copies lacked some of the full features we wanted at first.
We experienced issues with grabbing certain objects and there was an issue with overlapping items so we dragged two at the same time.
From a design perspective, given that we had professional editors on the team as well as people that have never edited before, we struggled to create an MVP that would be satisfactory to both audiences.
We experienced challenges around adding traditional assets to 3D a world and learning new 3D design tools using Unity.
Accomplishments that we're proud of
Working collaboratively with a diverse team of coders, designers, and media professionals to complete the prototype in 2 days in the Media Lab. We are proud of creating something from scratch leveraging our passion in future technologies. While the differing levels of Unity experience could have been a problem on the team, we worked together and learned from each other in an extremely harmonious way by breaking into small teams, dividing up the work and ensuring that everyone had equal voice.
Michael Drachkovitch: Team Lead
Jennifer Hurford: Product
Alyssa Li: Designer/Product
Megan Li: Designer
Xindeling Pan: Designer
Hisham Bedri: Engineer
Wesley Abbey: Engineer
Here's a short demo of the experience!
Ask me about:
· Designing a 3D experience from a 2D product
· Interaction design in VR: 6 DOF manipulation, hand controls
· Collaboration within VR