CS 2, Winter 2008
Programming for Interactive Digital Arts
About Schedule

Feb 13: Pixels


Grids

We saw in the "more for" section of the repition lecture that a for loop can be used to create various patterns. For example, to create a dotted line, we set up a loop that increments the loop variable by the dot spacing; inside the loop, we plot a point (or a rect, for fun):

for (int x=0; x<100; x+=10)
  rect(x,50,5,5);

In order to move from lines to grids, we nest a for loop inside another for loop. The outer loop (for y) goes from line to line; the inner loop (for x) goes across the current line:

for (int y=0; y<100; y+=10) {
  for (int x=0; x<100; x+=10) {
    rect(x,y,5,5);
  }
}
screenshot[applet]

It's very important to understand how this works, so take some time to step through it. Keep track of the current values of y and x. First y is 0, and we move inside the body of the y-loop. Inside the body, it's just a loop over x, like we've seen before -- x starts off at 0, then 10, then 20, etc. Each time through the x-loop, its body asks to draw a rectangle at (x,y), which is currently 0. Once the x-loop is finished, that's all there is in the body of the y-loop, so now the y-loop repeats. Thus y is incremented to 10, and the y-loop body is repeated. So now we do another x-loop, but this time with y at 10. This process continues -- a whole x-loop for every iteration of the y-loop. Ultimately this yields a rectangle for each (x,y) pair, where y is between 0 and 90 and so is x.

There are all kinds of variations on the structure -- the initial values for x and y, additive vs. multiplicative steps (or something else), step amounts, whether x and y are incremented the same way, what to do for each (x,y) value, etc. See Reas & Fry p. 66 and sketches 6-08 through 6-10 for some examples. Here's an example based on a sketch by Greenberg (6-23).

size(400,300);
background(0);

int xsz=16, ysz=12;    // how big the rectangles are
int xsp=4, ysp=3;      // spacing between them
int xnz=2, ynz=1;      // noise to add to endpoints
int maxW=5;            // maximum stroke weight

for (int y=0; y<height-ysz; y+=ysz+ysp) {
  for (int x=0; x<width-xsz; x+=xsz+xsp) {
    stroke(random(255),random(255),random(255));
    strokeWeight(random(0.1,maxW));
    line(x+random(-xnz,xnz), y+random(-ynz,ynz), 
         x+xsz+random(-xnz,xnz), y+random(-ynz,ynz));
    line(x+random(-xnz,xnz), y+random(-ynz,ynz), 
         x+random(-ynz,ynz), y+ysz+random(-ynz,ynz));
    line(x+xsz+random(-xnz,xnz), y+random(-ynz,ynz),
         x+xsz+random(-xnz,xnz), y+ysz+random(-ynz,ynz));
    line(x+random(-xnz,xnz), y+ysz+random(-ynz,ynz),
         x+xsz+random(-xnz,xnz), y+ysz+random(-ynz,ynz));
  }
}
screenshot[applet]

Getting pixel values

At the lowest level, images (as well as computer screens) are made up of discrete rectangles called "pixels". If you zoom way in to a picture with an image editor, you can see that. An image of size 800x600 has 800 pixels across by 600 down. Thus we can look at each of them by using a nested loop, as above. Note that since pixels are discrete, it is natural to use integers to index them.

We saw last time how to get() a snapshot of the current window contents, or just a region of it. The get() function can even tell us about a single pixel, when passed x and y coordinates (integers, as discussed above), without a width and height (as needed for a region). It then returns the color of the pixel (instead of a PImage for the region). We can use the color just like any other color, or extract its red(), green(), and blue() components (floats) with the functions of those names.

PImage grn;

void setup() 
{
  size(300,116);  // green-small is 200x116
  grn = loadImage("green-small.jpg");
  background(0);
  stroke(255);
  rectMode(CENTER);
}

void draw() 
{
  background(0);
  image(grn,0,0);  // put it at the corner (doesn't fill whole window)

  // Get the moused-over pixel and extract its r,g,b components
  color pixel = get(mouseX,mouseY);
  float r=red(pixel), g=green(pixel), b=blue(pixel);
  // Print it
  println(r+","+g+","+b);
  // Show the pixel extra large
  fill(r,g,b);
  rect(250,height/4,50,50);
  // Show the components
  fill(r,0,0);
  rect(225,3*height/4,25,50);
  fill(0,g,0);
  rect(250,3*height/4,25,50);
  fill(0,0,b);
  rect(275,3*height/4,25,50);
}
screenshot[applet]

While the function get() gets from the main window, there is also a method PImage.get(). It works the same way; e.g., img.get(10,20) gets the color at position (10,20) in img. We can use this technique, within a nested loop, to devise our own function for drawing an image, rendering them differently from the usual image() function. Here's an example that draws each pixel as a 5x5 ellipse, on a grid with spacing 4x4.

size(800,464);  // green-small is 200x116
PImage grn = loadImage("green-small.jpg");
noStroke();
for (int y=0; y<grn.height; y++) {
  for (int x=0; x<grn.width; x++) {
    fill(grn.get(x,y));
    ellipse(4*x,4*y,5,5);
  }
}

Or rather than just rendering the pixels with ellipses, we can use them to create Ball objects at those positions with the appropriate colors.

PImage grn;
Ball[] balls;

void setup()
{
  size(320,184);  // green-tiny is 80x46
  grn = loadImage("green-tiny.jpg");
  noStroke();
  // Create a ball for each pixel
  balls = new Ball[grn.height*grn.width];
  int b = 0;
  for (int y=0; y<grn.height; y++) {
    for (int x=0; x<grn.width; x++) {
      balls[b] = new Ball(4*x,4*y, grn.get(x,y));
      b++;
    }
  }
}

void draw()
{
  background(0,128,0);
  for (int i=0; i<balls.length; i++) balls[i].draw();
  for (int i=0; i<balls.length; i++) balls[i].update();
}
screenshot[applet]

We can use this approach interactively. For example, as the mouse is pressed we can draw random lines that take their colors from the underlying picture, according to where the mouse is.

PImage dartmouth;
int numLines=3;     // how many lines per mouse press
int sz=20;          // size of lines

void setup() 
{
  size(400,300);
  dartmouth = loadImage("dartmouth-hall.jpg");
}

void draw() 
{
  if (mousePressed) {
    color pixel = dartmouth.get(mouseX,mouseY);
    stroke(pixel);
    for (int i=0; i<numLines; i++) {
      strokeWeight(random(sz/4,sz/2));
      line(mouseX+random(-sz,sz),mouseY+random(-sz,sz), 
           mouseX+random(-sz,sz),mouseY+random(-sz,sz));
    }
  }
}
screenshot[applet]

Processing example Basics | Image | Pointillism further illustrates this idea. The map() function does the kind of math we've always done by hand, converting from one range to another. In the example, it converts from the range 0-width to the range 2-18, thereby setting the ellipse size from the mouseX. Can you derive the underlying formula?

As yet another example, we can leave the background the same, and just magnify a rectangle around where the mouse is. The nested for loops here iterate over offsets from the mouse position. As long as the position plus the offset remains within the image, the pixel color is determined and a rectangle is drawn filled with that color. Each rectangle is the same size, so a rectangle's location is computed by scaling up the offset by that size and adding that to the mouse position.

PImage grn;
int magD=10;  // diameter of magnifying rectangle
int rectSz=8; // how big the magnified pixels are

void setup() 
{
  size(800,463);  // size of green
  grn = loadImage("green.jpg");
  noStroke();
}

void draw() 
{
  background(grn);
  // magnify rectangle from mouse-magD to mouse+magD
  // have to make sure don't go out of bound of image 
  for (int y=-magD; y<magD; y++) {
    if (mouseY+y >= 0 && mouseY+y < height) {
      for (int x=-magD; x<magD; x++) {
        if (mouseX+x >= 0 && mouseX+x < width) {
          fill(grn.get(round(mouseX)+x,round(mouseY)+y));
          rect(mouseX+rectSz*x,mouseY+rectSz*y,rectSz,rectSz);
        }
      }
    }
  }
}
screenshot[applet]

We can get kind of a focusing effect by drawing higher-resolution ellipses nearer the mouse and lower-resolution ones further away. This requires the nested loops to be over the radius and angle, rather than x and y. We move from far out inward to the mouse, so that the closer, smaller ellipses overlay the outer ones. For each radius, we move around the circle. The radial steps are multiplicative, while the circular ones are additive; in both cases, a random step is taken rather than a fixed one, giving a kind of motion effect.

PImage grn;
float maxR;  // how far away can the mouse be from a pixel

void setup() 
{
  size(800,463);  // size of green
  maxR = sqrt(800*800+463*463);
  grn = loadImage("green.jpg");
  noStroke();
}

void draw() 
{
  // move from outside in, geometrically (but randomly)
  for (float r=maxR; r>0.01; r*=random(0.7,0.95)) {
    // move around circle (randomly)
    for (float a=random(5); a<360; a+=random(3,5)) {
      // convert to (x,y), and plot ellipse if in-bounds
      float x = mouseX + r*cos(radians(a));
      float y = mouseY + r*sin(radians(a));
      if (x > 0 && x < width && y > 0 && y < height) {
        fill(grn.get(round(x),round(y)));
        ellipse(x,y,r/10,r/10);
      }
    }
  }
}
screenshot[applet]

Finally, let's make a puzzle. We use the PImage get() method to extract a grid of rectangular "pieces" from an image. The pieces are stored in an array, such that the pieces on the first row are followed by the pieces on the second row, and so forth. Thus if there are 10 pieces per row, array elements 0-9 are from the first row, 10-19 from the second row, and so forth. Once we have this array, we shuffle it up by swapping each element with some other element (or maybe itself). We then display it by again going through the nested for loops and drawing the piece from the current position in the array. When someone selects two pieces by clicking first on one and then on another, we swap them. Note the formula for going from an (x,y) index in the grid to a single position in the linear array: x+y*nx. As discussed above, if nx=10, then element (2,3) is at 2+3*10, since we have to skip 3 full rows of 10 elements and then take the second element of the next row.

PImage[] pieces;    // the image fragmented into rectangles
int nx=10, ny=10;   // number of rectangles across and down

int dx,dy;          // rectangle sizes, computed from width,height and nx,ny
int selX=-1,selY=-1;  // if a piece has been selected, its x and y indices (-1 if none)
boolean auto=false; // whether or not to auto swap pieces

void setup()
{
  size(800,463);  // size of green
  PImage grn = loadImage("green.jpg");
  noFill(); 

  dx=floor(width/nx);
  dy=floor(height/ny);

  // Fragment the image into rectangles, and string them into a line
  // piece 0,0; piece 0,1; ...; piece 0,nx-1; piece 1,0; piece 1,1; ...
  pieces = new PImage[nx*ny];
  int p=0;
  for (int j=0; j<ny; j++) {
    for (int i=0; i<nx; i++) {
      pieces[p] = grn.get(dx*i,dy*j,dx,dy);
      p++;
    }
  }
  
  shuffle();
}

void draw()
{
  if (auto && frameCount % 10 == 0)
    swapPieces(int(random(pieces.length)), int(random(pieces.length)));
 
  // Draw the pieces at their positions 
  int p=0;
  for (int j=0; j<ny; j++) {
    for (int i=0; i<nx; i++) {
      image(pieces[p],dx*i,dy*j);
      // Outline in red if selected, black otherwise
      if (i==selX && j==selY) { 
        stroke(255,0,0);
        rect(dx*i,dy*j,dx-1,dy-1); // -1 to make sure red can be seen
      }
      else {
        stroke(0);
        rect(dx*i,dy*j,dx,dy);
      }
      p++;
    }
  }
}

// Mix up all the pieces
void shuffle()
{
  // Swap each piece with some piece later in the array
  for (int i=0; i<pieces.length; i++)
    swapPieces(i, int(random(i,pieces.length)));
}

// Swap the pieces at the two indices
void swapPieces(int i1, int i2)
{
  PImage p1 = pieces[i1];    // remember what's at i1
  pieces[i1] = pieces[i2];   // so that we can put at i1 whatever's at i2
  pieces[i2] = p1;           // and then put at i2 what we remembered
}

void mousePressed()
{
  // Find which piece was clicked on
  int px = mouseX/dx, py = mouseY/dy;
  // Select two pieces to swap; same piece twice to cancel
  if (selX >= 0 && selY >= 0) {
    if (selX == px && selY == py) { // same one twice -- cancel
      selX = -1; selY = -1;
    }
    else { // second selection -- swap
      swapPieces(px+py*nx, selX+selY*nx);
      selX = -1; selY = -1;
    }
  }
  else { // first selection
    selX = px; selY = py;
  }
}

void keyPressed()
{
  if (key == 'a') auto = !auto;
  else if (key == 's') shuffle();
}
screenshot[applet]

Practice problems

  1. Draw a grid of ellipses, with the diameters changing along with position (wider toward the right, taller toward the bottom). [hints]
  2. Draw a grid of randomly-oriented lines. [hints]
  3. Make an image stamper. On the left half of the window, display an image and allow someone to select a rectangle. On the right half, stamp the selected rectangle upon mouse press. [hints]
  4. Make a "scratch-and-play". The window is initially blank, and as the mouse is pressed and moved around, an image is revealed. [hints]
  5. Every so many frames, randomly reveal some portions of an image, before hiding them again. [hints]
Last modified: Thu Feb 14 08:09:18 2008