assignment:

Make a piece of software that edits a short movie out of a longer clip of source footage. The edit should be based on an analysis of the source footage and the editing choices should offer an interesting interpretation of the source footage. Make a short video demonstrating the successful use of your software. You can do this assignment on your own or with one other person.

backgrounds:

This assignment was inspired by the “Automatic Trailer Generation / Semantic Video Patterns” project carried out at the ‘Digital media’ program at the University of Dresden. This group has been studying how to automatically generate movie trailers out of an analysis of the feature film needing to be advertised. Their software is optimized for action movies, since both action movies and their trailers are highly codified forms of cinema in which camera movement, explosion detection and gunshot detection can already come a long way towards decoding the basic structure of the film.

goals of the assignment:

This assignment makes the question behind the “Automatic Trailer Generation” approach more general: how can we automate the making of editing decisions ?
On a practical level this assignment is meant to familiarize you with image handling software and elementary image analysis techniques.
On another level there is a deliberate vagueness in the assignment; this vagueness is meant as an empty space in which to formulate your own goal. There is no pressure to use (or pretend to use) a top-down approach in which you start with a clear idea and your implementation and the resulting video are the consequence of this first idea. If it works like that for you that is no problem, but it is also perfectly fine to start with a small idea and see where that takes you. During this process you’ll probably be adjusting the ‘big picture’ to the detail and the detail to the ‘big picture’ as you go along. Such an approach has the potential to generate truly new ideas, but it is a process that takes more time. So please feel free to adjust or even formulate your goal along the way, as long as you document this research process and make these adjustments explicit.

some suggestions:

– often you don’t need very sophisticated computer vision algorithms to get good results; clever use of simple analysis is more elegant and more interesting. Object following and recognition is easy for humans but very hard for computers, so it is advisable to think of an approach in which knowledge of objects is not necessary.

– it is fine to make software that is designed to handle only a certain type of footage as long as you make this explicit. It is not fine to make software that only works with one particular source clip.

– think about rules for the transition between shots. A good way to become more aware of the rules of video editing is to analyze a commercial or a fragment of a film. Try to locate every cut in the fragment you chose and write down the essence of each shot. Look at the movement around the cuts.

– instead of having jitter to render your video, you can also have jitter produce an edl file. An edl file contains editing instructions that editing packages like Final Cut Pro or Premiere can read. In this way you can use these packages to render your piece.

– please think about the sound part of your video. It is logical to also have your software edit the sound based on an analysis of the source. The sound is not strictly part of this assignment, though.

max/msp/jitter:

For this assignment you are encouraged to use the Max/Msp/Jitter environment. Like Msp, Jitter is an addition to the original Max graphical programming language. Jitter is a collection of objects for matrix manipulations, designed to work with real-time video.
It is possible to use another programming environment, as long as the following requirements are met:
– the programming environment should be powerful enough to do the assignment in it
– you should be able to figure out your implementation without the student assistants
– it should be possible for you to share both a working version of your software as well as a readable (annotated) version the source that enables me to figure out your approach.
Other possible environments include pd/gem (similar to max/msp/jitter, but free and possibly less coherent), vvvv (free, windows only, not sure about file handling), processing (free, java based), nodebox (free, mac only, python based) or ofcourse plain C, C++, objective-C or any language of your choice.

support:

Maarten Wesselius will be the assistant for this course and he will be available to help you with the Max/Msp/Jitter part. He will also give an introduction to programming in Max/Msp/Jitter. Joost Rekveld will give an introduction to Jitter programming, and towards the end of the course he will be available to answer questions, discuss approaches and ideas relating to the assignment.

still from “A Movie” by Bruce Conner (1958)

presentation and grades:

The assignment should result in a short paper, a short video, a working piece of software, and a short presentation.
The paper should clearly explain the concept and the principles behind the implementation. It should be between 2 to 4 pages long, and sent to me via email ( joost.rekveld (at) interfaculty (dot) nl ) in either .doc or .pdf format. Please use the IEEE guidelines to format your paper.
The software should be available for download with all that is necessary to make it function on any machine and with any source material (It is not necessary to have versions for both mac and windows and it is ok to restrict the input material to a few common video formats). If you work in jitter, please export your project as a collective (which will be cross-platform if you use no third-party externals).
Please put your video online and post the url on the forum.
During the presentation on the 4th of June the video should be shown and the software should be working and demonstrated. The concept and the principles behind the implementation should also be explained. I should have the paper and the software by thursday the 5th of June 10am.

still from “Piece Touchée” by Martin Arnold (1989)

references:

There is a rich tradition of so-called ‘found footage‘ films. Filmmakers in the genre include Bruce Conner (who made the classic “A Movie“), Martin Arnold, Craig Baldwin, Abigail Child, Cecile Fontaine, Al Razutis, David Rimmer and Michael Wallin.
Filmmaker and collector Rick Prelinger started his online Prelinger Film Archive with found footage filmmakers in mind.

Good introductions to the subject are:

– William C. Wees, “Recycled Images: The Art and Politics of Found Footage Films”, Anthology Film Archives, New York: 1993
– “The Recycled Cinema“