Major: Demonstration and Brief Evaluation

Strictly a 'Proof of Concept' project to demonstrate what a Kinect and two projectors can be potentially used for in a context of a Theatrical Production

I'm reciting Shakespeare's Henry V Prologue, using the Kinect to play seven separate video segments as I step on "virtual plates" on the floor.


Some quick after-thoughts:

• I should have used more time to prepare the live recital. or I should have hired an actor for the recital.
• The camera should have been positioned more to the right and focused better
• It should have been worth making and recording the second offshoot-context of the theatre concept- in which anyone can step on the plates and have the sequence play out with the appropriate narration

The final sequence (The fade to white) there was a major idea cut. This idea was that the actor reciting the prologue, steps within the screens and becomes King Henry V.



The technology was already set up. I would have been dressed in complete white so no colours from the projection would bleed.

The mask crafted would have covered the face. The 3D model of King Henry V  was also completed, rigged in Maya, and the bones were connected to the Kinect skeleton via Unity (Similar to my previous demo).



The reason for its cut was due to time constraints in fixing the problems it involved-

As the projector is calibrated to the three screens' distance, there is no "dynamic" way to adjust the projection images based on physical objects added to the projection area. Any objects moving closer to the projection distance would cause the projections to intersect.

Major: Technical Paper

Major: Art Of

Major: Demo Reel

Major: 03/05 Henry V Draft Part Two

Major: 30/04 - 02/05 Henry V Draft Part One

Major: 29/04 - Content Concept Art Part Two

Major: 29/04 - Content Concept Art Part One

Major: Update 28/04

Dissecting the script

Major: Update 22/04-27/04

Doing theatre/concert design
Taking forward my most successful past ideas - The third projection mapping test and the Pandora's Box.

More specifically, I'm doing a theatre demonstration sequence adapted from Shakespeare's Henry V

There's restrictions with unity in terms of how I can vary the visuals and the direction I'm hitting at this point.  So I'm cutting that out of the pipeline.

I'm pulling pre-rendered clips and attaching them to user interaction. It'll take away the problem of latency and will make things less complicated then it needs to be.



Fundamentals are



Dialogue Spoken by Chorus, Henry V, Prologue:

[Enter Chorus]

• Chorus. O for a Muse of fire, that would ascend
  The brightest heaven of invention,
  A kingdom for a stage, princes to act
  And monarchs to behold the swelling scene! 5
  Then should the warlike Harry, like himself,
  Assume the port of Mars; and at his heels,
  Leash'd in like hounds, should famine, sword and fire
  Crouch for employment. But pardon, and gentles all,
  The flat unraised spirits that have dared 10
  On this unworthy scaffold to bring forth
  So great an object: can this cockpit hold
  The vasty fields of France? or may we cram
  Within this wooden O the very casques
  That did affright the air at Agincourt? 15
  O, pardon! since a crooked figure may
  Attest in little place a million;
  And let us, ciphers to this great accompt,
  On your imaginary forces work.
  Suppose within the girdle of these walls 20
  Are now confined two mighty monarchies,
  Whose high upreared and abutting fronts
  The perilous narrow ocean parts asunder:
  Piece out our imperfections with your thoughts;
  Into a thousand parts divide on man, 25
  And make imaginary puissance;
  Think when we talk of horses, that you see them
  Printing their proud hoofs i' the receiving earth;
  For 'tis your thoughts that now must deck our kings,
  Carry them here and there; jumping o'er times, 30
  Turning the accomplishment of many years
  Into an hour-glass: for the which supply,
  Admit me Chorus to this history;
  Who prologue-like your humble patience pray,
  Gently to hear, kindly to judge, our play.35

[Exit]

Seven bundles of dialogue, Seven pre-rendered sequences, Seven areas to mark.
Feedback Welcome.

Major: Update 14/04-21/04

Worked on debugging/cleaning the code:

-Researched and made efforts to have the Kinect automatically detect user position without needing to do the calibration pose
-Removed the image jitter by dampening the rotation and transform values of the user that the Kinect gives out.
-Removed the huge sway when the user rotates their body left and right and instead replaced it with the camera kept at a single rotation with only transformations happening. - Therefore when you move around the area, the proper illusion is seen.




Cleaned up my general creative pipeline/workflow for this specific project. - now smarter, faster and easier.

-Optimised the pipeline to allow easier transition from my current toolset to surface projection.
-More Portability, any demonstrations I set up can be easily manipulated to adjust the angle of projection.



Currently need to: Finish Reading up on the history of Trompe L'Oeil some more.
Finish Reading up on the DLSR low-light recording functions properly.
Sort out delay in image
Finish experimenting with animating textures to models

Major: Update 29/03 - 14/04

Initial Demo (very rough)





Demonstrations over the period last week. 8th-14th

Trompe L'oeil


Samuel Dirksz van Hoogstraten's View of a Corridor, 1662



Visualiser - ALL-DYNAMIC RENDER


Visualiser - ALL-COMPOSITED RENDER



Visualiser - Visualiser Visuals

Major: Old Concept Ideas

----------------------------------------------------------------------------------------------------------

Major: Technical Concepts

I'm going through all the my 'For the future' bullet-points from the interim-presentation like a check-list, at this stage I'm currently focusing on the clarification and refinement of concept and physical requirements.

I have a few concepts lined up but in terms of technical execution simplicity is the main goal.


For the technical execution. I'm sticking to the idea that one can touch a surface in order to generate something.










I can't consider the physical area concept final yet, as it will dependent on the space I can work with (working on securing that hopefully by the end of the week)-However, saying that, I'm preferable towards a wide angled area, as one's peripheral vision will be better.


In regards to hardware:










Idea is to extend the projection area to 2560x720, covering the walls three sides and floor.

Next step is narrative concepts.

Major: Interm Crit Presentation

Major: Update 22/02 - 01/03

I'm stuck somewhere within the realisation that I have a technical position down with no clear specific context. This is alarming (bordering on decadence).


Here's what I have so far:

• The method is through projection mapping coupled with dynamics of human interaction
• The research has been through the use of book internet and film
• The experiments have been conducted somewhat extensively, specifically through from concept to form, and through use of trial and error
• The creative understanding has been through learning various code, creating concepts, dissecting other project samples, numerous trial and error experiments as well as most recently researching the history and methodologies of pseudo-3D (specifically trompe l'oeil)
• I've investigated what the potential can be in creating pseudo-3D visuals as well as interaction
• I've reflected on the limitations this can provide in terms of an experience
• I've also reflected on the limitations and outcomes I can potentially provide in terms of an experience through these methodologists based upon my current research and experiments

The context of the project is to create an experience through the users' interaction with the subject, enabling a dynamic audiovisual installation

Major: Interactive Video Experiments

Passed the rigorous task in finding the best method of creating an interactive projection mapped area.

Demonstration of various Kinect experiments over the past week:



----------------------------------------
Demonstration of basic body gestures that change videos on the fly

-Left Arm Flexing
-Right Arm Flexing
-Right Hand Above Head
-Left Hand Above Head
-Left Knee Above Torso
-Right Knee Above Torso

Major: Progress - Misc update 16-20th February

This is a technical post.

I've chosen the interactivity pathway because I want to create an experience that is simple but fully mastered.

I've had a strong push into understanding the Xbox Kinect tech during the last five days, corresponding with a ton of people, figuring out the technical side, and how this can be fully utilised with the experiments I've already done with projection mapping. I'm planning to spend the rest of the week fine-testing the Microsoft Kinect and putting more solidified concepts into a full demonstration video.

Laterally, I've established a solid idea of how the final product will be presented, and am still in the process of constructing concepts to utilise all of this.



Maya or After Effects was initially established as the primary method for constructing the environment but because of its limitations in the Kinect (they are limited to pre-keying motion capture, no real-time rendering is available) I've had to utilise a middleman. Unity3D acts as a bridge to import the rig model and environment assets easily from Maya and hook the Kinect to the model all in fully-rendered real-time. Creating the illusion of the three planes is already established without any hard manual tweaking at this point.




Primary objective is to
-Create a simple model and rig it to the Kinect bone structure.
-Create an interactive environment
-Render all in real-time

Optional Objectives are to
-Create a second or third rig for multiple people to interact with.
-Utilise physics
-Utilise particles/dynamics for further eye-candy
-Utilise a secondary / tertiary projector to remove distortion (unlikely)

Fail-safe objective is to:
-Create a fully animated environment
-Execute the illusion of three-dimensional space on three planes
-None or limited interaction available

Major: Narrative Selection

3d Illusion

The most straight-forward method. A traditional narrative or environment showcase will be projected in the illusion of three-dimensions.


Initial Influence: http://vimeo.com/34025760



3d with real life elements
A branch off the 3d illusion. More limited in theme, but a stronger execution of the illusion.


Initial Influence: http://vimeo.com/19348564



Sculpture Mapping
A single real life element that is projected onto.
Dependant on model, can have a strong variety of projections, i.e a human face can be projected in many art-styles / a cat-looking sculpture can have projections that turn it into a cat/panther/tiger/lioness etc.
Will require creating a model from card/paper/3d-printer. Will need to think what the sculpture should be.



Initial Influence: http://vimeo.com/39697056



Music Visualisation
An environment or set of elements that Sync to the music - Why should it be done, How does it differ from normal visualisers. What will the end product be.



Initial Influence: http://www.youtube.com/watch?feature=player_detailpage&v=Zv1fU9flSp8


Interactive Environment
Environment created with a camera for the viewer to dynamically alter or interact with the scene. Will need to think about how one can interact with it, how interactive should it be, How much time will it allocate to implement.


Initial Influences: http://vimeo.com/45417241
http://www.youtube.com/watch?v=EwrNMhCm5_Q

Major: Experiment III

A final experiment before I move on.

This was to test the following:
-Accurately project onto three connected planes - XY and Z
-Accurately move objects through the planes without obvious distortion
-Create realistic artificial shadows for object to give an illusion of depth
-Create artificial 3D Models with lighting depth and shadow using only Maya


Three planes, three boxes, and three scenes were composited-

A Cartoon Landscape
A Library
A Scene only created in Maya