Feb 11: Holistic images
Images used here are from Wikimedia commons, and are in the public domain. Thanks go primarily to user Kane5187.
Basics
We can use images in Processing much the same way that we use ellipses and so forth (that's the "holistic" part of the title -- treating an image as a whole, for drawing). Processing provides a class PImage for storing and manipulating images.
In order to use an image we already have, we must first put it in the data folder and then load it into our sketch (much like with fonts). The Processing environment has an "Add file..." menu option under the "Sketch" menu to put a file in the data folder, or it can just be dragged there with the Mac Finder or Windows Explorer. The main types of image file formats supported are gif, jpg, and png; since Processing is case sensitive, be sure to name the file "something.jpg" and not "something.JPG".
Once the image is available in the right place, our sketch can use the loadImage() function, given the filename (in quotes), to load it into Processing. (Images can also be loaded from a URL, but since that doesn't easily work in an applet, I won't be doing it.) The loadImage() function returns an object of type PImage. I got strange errors when I tried to call loadImage() in a variable declaration before setup(). Thus I typically declare the variables at the top, and load the images within setup(). (They only need to be loaded once, which means setup() is the appopriate place.)
We can use a loaded image as the background, passing a PImage rather than a color to the good old background() function. In that case, the window size and image size have to match. We can also draw an image at a selected position on the window, using the image() function. The first parameter is the PImage, and the second and third are the x and y coordinates of its upper-left corner (like a rect). If desired, we can pass two more parameters to set the width and height of the image; it will be scaled accordingly. Play around with these functions for some of your favorite images.
As mentioned, we can use the image() function just like with other drawing commands. Here's an example with a few simple modifications to the "follower" sketch, allowing the user to actually lead Dartmouth. (Right, the portrait is William Legge, 2nd Earl of Dartmouth.)
// Declare the two images PImage green, earl; // The current coordinates of the earl float x=0, y=0; // A parameter controlling how quickly the earl catches up float easing=0.05; void setup() { size(800,463); // Load the two images green = loadImage("green.jpg"); earl = loadImage("legge.jpg"); } void draw() { background(green); // Move towards the mouse x += (mouseX - x)*easing; y += (mouseY - y)*easing; image(earl,x,y, 40,46.8); }
[applet]
As another example, see Reas & Fry 22-1. A number of images are split into two halves, front and back, such that the dividing line looks the same for each. A hybrid is formed by picking a random front half and a random back half. The images are named "1f.jpg", "1b.jpg", "2f.jpg", etc. Thus we can select a random front half by appending a random number (integer) from 1 to 9 and the string "f.jpg". The "+" here is to append strings, as we saw with println.
The tint of an image is somewhat analogous to the fill color of an ellipse. The image already has colors, of course, so the tint just modifies them. The tint() function works the same way as fill() -- gray value or r/g/b values, with an optional transparency; noTint() goes back to normal. We can modify the transparency without the color by using a gray value of 255 (white).
size(320,240); PImage dartmouth = loadImage("dartmouth-hall.jpg"); // Put a big D in the background to illustrate transparency strokeWeight(20); noFill(); stroke(0,128,0); line(width/4,0,width/4,height); arc(width/4,height/2,width,height,-HALF_PI,HALF_PI); // Normal image(dartmouth,0,0,160,120); tint(128); // Grayer image(dartmouth,160,0,160,120); tint(0,255,0); // Greener image(dartmouth,0,120,160,120); tint(255,196); // Somewhat transparent image(dartmouth,160,120,160,120);
[applet]
Gif and png images may have portions that are already transparent. Processing does the right thing. (One difference in the formats is that gif is all-or-nothing, whereas png allows the 256 levels of transparency we've been using.) The Processing example Basics | Image | Sprite illustrates. The gif provided with it only has a little transparency on the outside, so I modified it (below) so that everything that isn't black is transparent. The sketch folder also has a background image, but the sketch doesn't use it unless you edit it appropriately.
Finally (for the basics), inspired by the Reas & Fry Synthesis sketch "collage", here's a sketch to generate a collage of randomly-positioned, oriented, and scaled images of Dartmouth buildings. We use the same approach as above to append strings for the filenames. The images are stored in an array, and for each mouse press a random one is selected. The random translation and rotation use coordinate transformations; scaling can be done directly with the image() command. A random width is chosen, and the height is scaled so as to maintain the aspect ratio. We make use of the width and height fields of PImage to determine the image size. The expression w / img.width computes the scaling factor, and we multiply the height by that same factor
// An array of images int numPix = 9; PImage[] pix = new PImage[numPix]; void setup() { size(800,600); background(255); tint(255, 200); // each will be a bit transparent // The images are named "dart1.jpg", "dart2.jpg", etc. for (int i=0; i<numPix; i++) pix[i] = loadImage("dart"+(i+1)+".jpg"); } void draw() {} void mousePressed() { // Random member of the array PImage img = pix[int(random(numPix))]; pushMatrix(); // Random position and rotation (not too much rotation, though) translate(random(width),random(height)); rotate(random(-PI/4,PI/4)); // Desired width is random; height preserves aspect ratio float w = random(200,400); float h = w*img.height/img.width; // By putting the corner of the image at (-w/2,-h/2), // we center it at the (translated) origin image(img,-w/2,-h/2,w,h); popMatrix(); }
[applet]
Animation
In addition to moving images around the window, we can change which image is drawn for each frame. Thus, by using the same approach as the opening-and-closing mouth from the "State" lecture (cycling through different states), we can create an image-based animation. Reas & Fry 34-2 nicely illustrates. Each frame, we draw the next image in the loop, going back to the beginning when all images have been drawn.
This is good if we have available snapshots that form a nice animation sequence, but what if we want to create our own interactively? To support that, Processing can treat what's been currently drawn as an image (we saw that ability with the save() function). The get() function returns a PImage for the current contents of the window; we can then use that just like any other image. The Processing example Topics | Drawing | Animator lets us draw in the window with the mouse. It gets a snapshot every 100 milliseconds (according to the millis() function) and puts 12 such snapshots into an animation loop. The animation builds on itself once it cycles around -- the current version of the frame is displayed and then drawn on before being used as the next version of the frame.
To allow more control over the frames of the animation, I created an alternative approach in which key presses allow moving forward/backward through the frames, and starting/stopping the animation.
// The array of screen grabs int maxFrames = 100; PImage[] frames = new PImage[maxFrames]; // Which frame we're currently looking at int currentFrame = 0; // How many frames of the array have actually been drawn int numFrames = 0; // Whether cycling through the drawn frames, or drawing them boolean play = false; // Let the user control the frame rate int fr=10; void setup() { size(400,400); smooth(); strokeWeight(4); background(0); stroke(255); fill(255); textFont(loadFont("Kartika-24.vlw")); text(0,0,height); // start on 0th frame } void draw() { if (play) { // Cycle through drawn frames if (numFrames > 0) { image(frames[currentFrame],0,0); currentFrame = (currentFrame+1)%numFrames; } } else if (mousePressed) // Draw in the window line(mouseX,mouseY,pmouseX,pmouseY); } void keyPressed() { if (key=='f') { // forward one frame if (!play) { // Save the current frame frames[currentFrame] = get(); if (currentFrame < maxFrames) currentFrame++; if (currentFrame > numFrames) { // Totally new frame numFrames++; background(0); } else { // Drawing over an existing frame -- show it image(frames[currentFrame],0,0); } text(currentFrame,0,height); // indicate the frame } } else if (key=='b') { // back one frame if (!play && currentFrame > 0) { // Save the current frame frames[currentFrame] = get(); currentFrame--; // Show the previous frame image(frames[currentFrame],0,0); } } else if (key=='p') { // start/stop playing if (play) { // now stop play = false; frameRate(60); // standard rate while drawing } else { // now start play = true; frameRate(fr); // user-defined rate while playing // Save the current frame frames[currentFrame] = get(); if (currentFrame < maxFrames && currentFrame == numFrames) numFrames++; } } else if (key=='r') { // slow down if (fr > 2) { fr--; if (play) frameRate(fr); } } else if (key=='R') { // speed up fr++; if (play) frameRate(fr); } }
[applet]
The Reas & Fry Synthesis example "Chronodraw" gives a different perspective on animation -- it lays out some number of frames side-by-side. In order to get at the heart of the idea, I modified the Animator sketch discussed above to have a similar behavior (though more limited, particularly in that only the left-most panel can be drawn in). This sketch passes parameters to the get() function (coordinates of the corner, along with width and height) to get just a portion of the window -- the leftmost strip. Then each time it advances one frame, it pushes that frame one strip to the right. Thus if frame 28 is in the leftmost strip, 27 is in the next, 26 in the next, and so forth. We keep more frames than there are strips, but after some amount of time they wrap around.
// The array of (partial) screen grabs int numFrames = 100; PImage[] frames = new PImage[numFrames]; // Which frame we're currently drawing (in the leftmost strip) int currentFrame = 0; // Layout of the strips -- how many, and how wide (calculated in setup) int numStrips = 23, stripWidth; // millis() at the last grab int lastTime = 0; void setup() { size(800,200); smooth(); background(0); stripWidth = width/numStrips; // equally sized // Create blank frames for (int i=0; i<numFrames; i++) frames[i] = get(0,0,stripWidth,height); } void draw() { // Get another frame every 100 ms int currentTime = millis(); if (currentTime > lastTime+100) { nextFrame(); lastTime = currentTime; } // Draw in the leftmost strip if (mousePressed && mouseX < stripWidth) { noFill(); stroke(255); line(mouseX,mouseY,pmouseX,pmouseY); } // Bars dividing the strips stroke(64); for (int i=1; i<numStrips; i++) line(i*stripWidth,0,i*stripWidth,height); } void nextFrame() { frames[currentFrame] = get(0,0,stripWidth,height); // Get the strip of the display window currentFrame++; // Increment to next frame if (currentFrame >= numFrames) { currentFrame = 0; } // Draw all the strips -- the current frame at the far left, // the frame _before_ that to its immediate right, // then the frame _before_ that, etc. for (int strip=0; strip<numStrips; strip++) { int frame = currentFrame-strip; if (frame < 0) frame += numFrames; // wrap around image(frames[frame],strip*stripWidth,0); } }
[applet]
Practice problems
- Make a small photo gallery of some of your images. Define a function to drotate, translate, and display a scaled image, and call it with different sets of parameters. [hints]
- Revisit previous lectures and modify another example to draw with an image rather than an ellipse or rect. For example, try draw the random wanderer with your mug shot.
- Extend the stop-frame animation sketch to support copying and pasting frames. This would be even more useful with an extension to allow erasing (e.g., draw in white with left mouse button and black with right). [hints]