assignment 2006/2007

assignment for Machine Mediated Vision course 2006/2007:

introduction:

Many people have drawn the parallel between the self-organization of neurons in our brain and the self-organization of ants in a colony. Several artists have built systems where images are produced by colonies of artificial agents, similar to ants, that live and cooperate in an artificial world, be it either in software or in hardware. The ant-metaphor has also been used by scientists to explore agent-based image processing methods. In many of these cases virtual ants live within the image, as it were, and their collective behaviour is their reaction to this image-environment.

mourasp0018.gif
Leonel Moura: Swarm Painting

assignment:

Make an agent-based computer vision system that extracts information that is not directly visible from a webcam input stream and visualizes it. This system should consist of two parts:

  • a colony of virtual ants: these are software-agents that live within the image and which react to their evironment. They move around in the image and are sensitive to a local property of the image that is to be defined by you. The virtual ants should interact with eachother in a way which causes them to extract meaningful information from the image stream. The number of ants should be a parameter of your programme.
  • a visualization of the state of the ant colony. This visualization should be esthetically interesting and should clearly show the information your system is supposed to extract from the webcam input stream.

This assignment will be done in groups of two or three. Because of the large number of participants it is not possible to do this assignment on your own, sorry.

some suggestions:

  • ant sensitivities: Virtual ants can for instance be sensitive to brightness value, to brightness gradients, to the amount of movement flow at their image location, to a certain kind of blue, to colour saturation, a combination of these, or anything else you can imagine (as long as it is a local property of a pixel or pixel edge).
  • ant interactions: They can interact by detecting and reacting to other ants in their vicinity (flocking) or for instance by leaving ‘chemical’ trails along their path and by switching behaviour in reaction to the concentration of this ‘chemical’ in their environment (stigmergy, like real ants do). These ‘chemical’ trails allow ants to store information in their environment.
  • ant properties: ants have a position in the image, but perhaps they can have a speed (and inertia), or perhaps they have another property, like temperature. Their location (and if you think you need them: their other properties) is like a kind of trace of the images recently experienced by the colony.
  • ant intelligence: perhaps in contrast to humans, ants are only intelligent in groups. Their group behaviour shows many properties that are ’emergent’: these can not be predicted on the basis of the behaviour of one ant alone. The challenge of this assignment is to design a system in which one of these ’emergent properties’ is the extraction of specific information from your image data.
  • ant density: how many ants do you need ?
  • ant bookkeeping: the central part of your programme is going to be about keeping records of all the individuals in your colony, and updating these records in each ‘ant time’ cycle.
  • ant visualization: a graph is boring and does not show many properties well. Visualization can for instance be an ‘ant map’ that can be overlaid on the input image, it can show the traces of all ants, it can show only the movement of ants, everything you can imagine which relates to the property you want to show.
  • ant songs: the visualization part can include sound too
  • the LIACS building (where you will present and demonstrate your system) is visually not the most exciting place on the planet; In case you so desire you can design your system for a more interesting place. In that case you should have a webcam-stream (a clip) from that place ready for the demonstration. However, your system should also work with webcam-input.

max/msp/jitter:

For this assignment you will be encouraged to use the Max/Msp/Jitter environment. Like Msp, Jitter is an addition to the original Max graphical programming language. Jitter is a collection of objects for matrix manipulations, designed to work with real-time video.
It is possible to use another programming environment, as long as the following requirements are met:
– the programming environment should be powerful enough to do the assignment in it
– you should be able to figure out your implementation without the student assistants
– it should be possible for you to share both a working version of your software as well as a readable (annotated) version the source
– you should be able to collaborate with the other person(s) in your group. This means that you’ll have to find a partner with an interest in using the same language (or you have to split up the assignment in parts that communicate with eachother for example).

support:

Due to the large number of people who have inscribed for this course, two assistants will be available to help you with the Max/Msp/Jitter part. They will also give an introduction to programming in Max/Msp/Jitter. Joost Rekveld will give an introduction to Jitter programming, and at the end of the course he will be available to answer questions, discuss approaches and ideas relating to the assignment.

presentation and examination:

The assignment should result in a short paper, a working programme, and a short presentation.
The paper should clearly explain the concept and the principles behind the implementation. It should be between 2 to 4 pages long, and sent to me ( joost.rekveld (at) interfaculty (dot) nl ) in either .doc or .pdf format. Please use the IEEE guidelines.
The working programme should be available for download with all that is necessary to make it function on any machine (It is not necessary to have versions for both mac and windows). If you work in jitter, please export your project as a collective (which will be cross-platform if you use no third-party externals).
In the presentation the programme should be working and demonstrated. The concept and the principles behind the implementation should also be explained.

mourasp0024.gif
Leonel Moura: Swarm Painting

literature/links:

background:

Joost Rekveld