Hello to all of our creative owners!


In this article, we will guide you on how to create your own custom Augmented Reality content from scratch. 


We have divided the article into 3 parts: Creating the content (the fun part), Post-production (editing and bringing the content to life), and software integration (which will bring everything together). 

For each part, we have a step-by-step written guide as well as a video to demonstrate it.


Ready to get started?!


TABLE OF CONTENTS


1. Creating Content:

Let's start with the essentials stuff:

  • Green/Blue Screen Backdrop
  • Lighting Kit
  • Video capture device (Webcam, DSLR, Arri Alexa)


The first thing is planning, of course, you want to know how the experience will look. From there we will work backward.


In our case, we had to produce, post, and integrate with only a few days on our hands. We invested in shooting exactly what we needed.


To capture the video we used a 4K Webcam: the Logitech Brio. 

To support the illusion of the AR figure being “really there” with you we captured our actors in the same measured space of the final experience.


Step one: Set

We used a Blue Screen backdrop (as the uniforms our actors wore were green). 

We built the backdrop in a way that we will have a substantial “stage” with the backdrop filling the frame. The camera-backdrop distance differs with different cameras. 

Next set the angle and zoom (if available), mark the exact camera position, and lock it. 

In our case, we mounted the Webcam at about eye level on the side of the booth (in a vertical position). 

Finally, test with the actor at the perfect point for the end-user to be standing (framing-wise) and mark it.


Step Two: Lights

Now we are getting to the good part, as the illusion will take place in “Live View” conditions we will have to carefully think and plan for the same light conditions when we are producing the content and also running the experience if the light will not look the same the illusion might not work and the final result will not look believable. 

As we were filming in our offices and also shot the promotional video in the same spot we were in full control of how it will look. 


Lighting the backdrop – we want to make sure the backdrop is evenly lit if you are using a fabric backdrop check to see that there are no wrinkles that might cast shadows. We used a small continuous LED Lights kit with two lights. We positioned the two lights one from each side of the Blue Screen backdrop between the subject and the backdrop, we set the height to the middle of the backdrop so it will be evenly lit.

 


Step Three: Action:

Just before you will be ready to shout “Action!”-


We captured the video using Logitech capture software (and changed the camera field view to 90). With that said, you can use any webcam software that you see fit, as long as the software is capable of controlling the camera settings such as Exposure, White balance, Focus, Etc...  



An important note here is timing- since you will be cutting the video files to the exact length you will use in the experience, think of the last frame of the video. 

In our case we filmed the actors entering the frame, positioning themselves for the photo taking and posing. 

We counted two seconds in pose and then them leaving the frame. In this single video, we cut into two video files so for each actor/player we have an “In” and an “Out” video.




You can watch this video tutorial that shows all the steps above:


 



2. Post-Production:

Time to get your hands dirty, boys and girls, this is the main part of creating your experience, but first, you will need some tools:

  • Powerful computer
  • Adobe After Effects  
  • Adobe After Effects CS6 (Available for CC users - download link) 
  • FLVmeta - (available here - download link)


a. After Effects

1. Import the selected footage into After Effects. 

2. Right-click the footage file and select “New comp from selection”.

3. In the top menu, go to Composition → Composition settings.

4. In the pop-up window change the video ratio from horizontal to vertical using these parameters: Width: 1080 Height:1920

5. In the composition timeline, press on the little arrow on the left of the selected footage and go to: transform → Rotation and set the rotation to 90°

6. If your shot was done in 4K,  Adjust your footage to fit the composition.

7. When the footage is selected on the timeline, select the rectangle tool or the pen tool from the top toolbar, and create a mask around all the parts of the footage that don’t contain a green/blue screen. 

To mask them out, change the mask mode from “Add” to “Subtract”.

8. In the effects pannel's search bar, type “key” and drag the “keylight + key cleaner + advanced spill suppressor” Preset on your footage layer.

9. On the Keylight 1.2 effect (in the effect control), use the Eyedropper tool to select a green/Blue area of the footage.

10. On the Keylight 1.2 effect (in the effect control), change the “View” mode to “Screen Matte” and open the “Screen Matte” tab

11. Refine the effect using the parameters Below Such as: clip black, clip white, etc.

12. On the Keylight 1.2 effect (in the effect control) Change the “view” mode to “intermediate result”.

13. To eliminate green reflection on the subject, in the effect control, turn on “Advanced Spill Suppressor”

14. Before exporting check closely the result to see that it’s clean, in this stage, if needed you can do your color correction and grading using the lumetri color effect

15. Exporting the footage - when the composition is selected press ctrl+m

16. In the render queue open the “output module and select the following adjustment:

Format: AVI

Channels: RGB+Alpha

Press Ok

17. Go to "output to", select the folder you want to export to and rename your file.

18. Press render.



b. After Effects CS6

1. Open a new composition - in the top menu go to Composition and select "New Composition" using these parameters Width: 1920 Height: 1080

2. Import your AVI file

3. Double-click your footage and find your "IN" point (A little before the actor enters the screen poses for the photo and leaves the screen). Mark those "IN" and "OUT" poses.

4. Drag this part to the composition. Rotate the video to -90° and place the subject

5. Duplicate your composition by pressing CTRL + D

6. Rename the two compositions “subject_in” and “subject_out”. With no space between the words

7. Inside your “subject_in” composition, find your "in" point and adjust your clip layer. Your start ("in") point should be a little before the player enters the screen. Your finish ("out") point should be when your subject is posing for the photo.

8. Inside your “subject_out” composition, find your "out" point and adjust your clip layer. Your start ("in") point should be the moment when the subject starts moving again. Your finish ("out") point should be when your subject exits the frame completely. 

Leave a few blank seconds, the whole composition should be around 4 seconds

9. Exporting the footage - when the composition is selected press ctrl+M

10. In the render queue open the “output module" and select the following adjustment:

Format: FLV

Channels: RGB+Alpha

Press Ok

11. Go to "output to" and select the folder you want to export to, rename your file and give it a name without spaces (you can use underline)

12. Press render.



c. FLVmeta

To rotate the file to appear in a vertical position in the Fotomaster software use the FLVmeta.exe program


1. Open the folder

2. Go to your c drive (and it must be on C: drive) create a new folder call it flvmeta

3. Copy the flvmeta.exe file into the new folder

4. Copy the folder path

5. Go to the Windows search barת write "path" and open the Edit the system environment variables

In the popup window click the environment variables then under the user variables for IEUser click Path and then Edit a new screen will popup

Click New and paste the folder path to the new line

Then click OK

6. Copy your FLV files to the folder in the C drive

7. Open the Command Prompt

8. Copy the folder’s path

9. On the Command prompt type cd and paste your copied path and hit enter.

10 Still, in the Command Prompt, enter: 

flvmeta.exe -U FileName.flv -a rotation=90 FileName_1.flv  and press Enter. 

(Because the second file name is with a slight change, it will create a new file which is the one that we will use in the Foto Master Photo Booth software)


Do the same to your "in" and "out" files.

In the path folder, you will find new files. These are the files we will use in the Fotomaster software



To be able to use the file, place it in the Animations folder in the Preset pack: This PC -> C drive -> MirrorMeBooth-> Animations. 

You can use an existing folder or create a custom folder and place the file there



You can watch this video tutorial that shows all the steps above:




3. Software Integration:

 Here we have an example of the workflow we made. We will:

1. Stop the Snapshot animation and Live View (adding both Live View Stop and Snapshot Animation Stop features) before taking a photo using the DSLR camera or the Webcam.


2. In the Photo Layout tab, add the snapshot animation's ("in") frame capture and position it accordingly.




3. In the Features tab > Live View > enable:

- Mirror Live View Image

- Wait on Live View Feed

- Hide Live View on Stop Live View - this will hide the live view presentation, instead of stopping it completely. This will allow a faster reaction of the camera's shutter after taking a photo and going back to the live view presentation.


Note: Since each computer runs a little bit differently, timing issues may occur with the animations such as lags or jumps.. what we did to compensate is to add a transparent button animation and tweak it in a few seconds so that the IN and OUT animations will run smoothly. 

This can be done by adding\subtracting a few seconds.



You can watch this video tutorial that shows all the steps above:





And that's it! your own creation is ready to be enjoyed by everyone!