CS 2, Winter 2009
Programming for Interactive Digital Arts
About Schedule

Feb 18: Pixels


Images used here are from Wikimedia commons (Category: Dartmouth College), and are in the public domain. Thanks go primarily to user Kane5187.

Getting pixel values

At the lowest level, images (as well as computer screens) are made up of discrete rectangles called "pixels". If you zoom way in to a picture with an image editor, you can see that. An image of size 800x600 has 800 pixels across by 600 down.

We saw last time how to get() a snapshot of the current window contents, or just a region of it. The get() function can even tell us about a single pixel, when passed x and y coordinates, without a width and height (as needed for a region). It then returns the color of the pixel (instead of a PImage for the region). We can use the color just like any other color, or extract its red(), green(), and blue() components (floats) with the functions of those names.

PImage grn;

void setup() 
{
  size(300,116);  // green-small is 200x116
  grn = loadImage("green-small.jpg");
  background(0);
  stroke(255);
  rectMode(CENTER);
}

void draw() 
{
  background(0);
  image(grn,0,0);  // put it at the corner (doesn't fill whole window)

  // Get the moused-over pixel and extract its r,g,b components
  color pixel = get(mouseX,mouseY);
  float r=red(pixel), g=green(pixel), b=blue(pixel);
  // Print it
  println(r+","+g+","+b);
  // Show the pixel extra large
  fill(r,g,b);
  rect(250,height/4,50,50);
  // Show the components
  fill(r,0,0);
  rect(225,3*height/4,25,50);
  fill(0,g,0);
  rect(250,3*height/4,25,50);
  fill(0,0,b);
  rect(275,3*height/4,25,50);
}
screenshot[applet]

We can use this approach interactively. For example, as the mouse is pressed we can draw random lines that take their colors from the underlying picture, according to where the mouse is.

PImage dartmouth;
int numLines=3;     // how many lines per mouse press
int sz=20;          // size of lines

void setup() 
{
  size(400,300);
  dartmouth = loadImage("dartmouth-hall.jpg");
}

void draw() 
{
  if (mousePressed) {
    color pixel = dartmouth.get(mouseX,mouseY);
    stroke(pixel);
    for (int i=0; i<numLines; i++) {
      strokeWeight(random(sz/4,sz/2));
      line(mouseX+random(-sz,sz),mouseY+random(-sz,sz), 
           mouseX+random(-sz,sz),mouseY+random(-sz,sz));
    }
  }
}
screenshot[applet]

Processing example Basics | Image | Pointillism further illustrates this idea. The map() function does the kind of math we've always done by hand, converting from one range to another. In the example, it converts from the range 0-width to the range 2-18, thereby setting the ellipse size from the mouseX. Can you derive the underlying formula?

Grids

As we say in the iteration lecture, a for loop can be used to create regular patterns. For example, to create a dotted line, we set up a loop that increments the loop variable by the dot spacing; inside the loop, we plot a point (or a rect, for fun):

for (int x=0; x<100; x+=10)
  rect(x,50,5,5);

In order to move from lines to grids, we nest a for loop inside another for loop. The outer loop (for y) goes from line to line; the inner loop (for x) goes across the current line:

for (int y=0; y<100; y+=10) {
  for (int x=0; x<100; x+=10) {
    rect(x,y,5,5);
  }
}
screenshot[pde]

It's very important to understand how this works, so take some time to step through it. Keep track of the current values of y and x. First y is 0, and we move inside the body of the y-loop. Inside the body, it's just a loop over x, like we've seen before — x starts off at 0, then 10, then 20, etc. Each time through the x-loop, its body asks to draw a rectangle at (x,y), which is currently 0. Once the x-loop is finished, that's all there is in the body of the y-loop, so now the y-loop repeats. Thus y is incremented to 10, and the y-loop body is repeated. So now we do another x-loop, but this time with y at 10. This process continues — a whole x-loop for every iteration of the y-loop. Ultimately this yields a rectangle for each (x,y) pair, where y is between 0 and 90 and so is x.

While the function get() gets from the main window, there is also a method PImage.get(). It works the same way; e.g., img.get(10,20) gets the color at position (10,20) in img. Likewise, a PImage has fields width and height that tell us its size. We can use these, along with a nested loop, to devise our own function for drawing an image.

size(200,116);  // green-small is scaled down
PImage grn = loadImage("green-small.jpg");
for (int y=0; y<grn.height; y++) {
  for (int x=0; x<grn.width; x++) {
    stroke(grn.get(x,y));
    point(x,y);
  }
}

That's not so helpful, but it points the way to the ability to render pixels however we like. For example, we can draw each pixel as a 5x5 ellipse, on a grid with spacing 4x4 (the size of the window needs to be scaled up).

size(800,464);
PImage grn = loadImage("green-small.jpg");
for (int y=0; y<grn.height; y++) {
  for (int x=0; x<grn.width; x++) {
    fill(grn.get(x,y));
    ellipse(4*x,4*y,5,5);
  }
}
screenshot[applet]

Or rather than just rendering the pixels with ellipses, we can use them to create Ball objects (using the same old class we've been using since introducing gravity) at those positions with the appropriate colors.

PImage grn;
Ball[] balls;

void setup()
{
  size(320,184);  // green-tiny is 80x46
  grn = loadImage("green-tiny.jpg");
  noStroke();
  // Create a ball for each pixel
  balls = new Ball[grn.height*grn.width];
  int i = 0;  // which ball to add next
  for (int y=0; y<grn.height; y++) {
    for (int x=0; x<grn.width; x++) {
      balls[i] = new Ball(4*x,4*y, grn.get(x,y));
      i++;
    }
  }
}

void draw()
{
  background(0,128,0);
  for (int i=0; i<balls.length; i++) balls[i].draw();
  for (int i=0; i<balls.length; i++) balls[i].update();
}
screenshot[applet]

As yet another example, we can leave the background the same, and just magnify a rectangle around where the mouse is. The nested for loops here iterate over offsets from the mouse position. As long as the position plus the offset remains within the image, the pixel color is determined and a rectangle is drawn filled with that color. Each rectangle is the same size, so a rectangle's location is computed by scaling up the offset by that size and adding that to the mouse position.

PImage grn;
int magD=10;  // diameter of magnifying rectangle
int rectSz=8; // how big the magnified pixels are

void setup() 
{
  size(800,463);  // size of green
  grn = loadImage("green.jpg");
  noStroke();
  rectMode(CENTER);
}

void draw() 
{
  background(grn);
  // magnify rectangle from mouse-magD to mouse+magD
  for (int y=-magD; y<magD; y++) {
    for (int x=-magD; x<magD; x++) {
      // to be careful, would make sure don't go out of bound of image 
      fill(grn.get(mouseX+x,mouseY+y));
      rect(mouseX+rectSz*x,mouseY+rectSz*y,rectSz,rectSz);
    }
  }
}
screenshot[applet]

We can get kind of a focusing effect by drawing higher-resolution ellipses nearer the mouse and lower-resolution ones further away. This requires the nested loops to be over the radius and angle, rather than x and y. We move from far out inward to the mouse, so that the closer, smaller ellipses overlay the outer ones. For each radius, we move around the circle. The radial steps are multiplicative, while the circular ones are additive; in both cases, a random step is taken rather than a fixed one, giving a kind of motion effect.

PImage grn;
float maxR;  // how far away can the mouse be from a pixel

void setup() 
{
  size(800,463);  // size of green
  maxR = sqrt(800*800+463*463);
  grn = loadImage("green.jpg");
  noStroke();
}

void draw() 
{
  // move from outside in, geometrically (but randomly)
  for (float r=maxR; r>0.01; r*=random(0.7,0.95)) {
    // move around circle (randomly)
    for (float a=random(5); a<360; a+=random(3,5)) {
      // convert to (x,y), and plot ellipse if in-bounds
      float x = mouseX + r*cos(radians(a));
      float y = mouseY + r*sin(radians(a));
      if (x > 0 && x < width && y > 0 && y < height) {
        fill(grn.get(round(x),round(y)));
        ellipse(x,y,r/10,r/10);
      }
    }
  }
}
screenshot[applet]

Finally, let's make a very rudimentary puzzle. We use the PImage get() method to extract a grid of rectangular "pieces" from an image. The pieces are stored in an array, such that the pieces on the first row are followed by the pieces on the second row, and so forth. Thus if there are 10 pieces per row, array elements 0-9 are from the first row, 10-19 from the second row, and so forth. Once we have this array, we shuffle it up by swapping each element with some other element (or maybe itself). We then display it by again going through the nested for loops and drawing the piece from the current position in the array. When someone selects two pieces by clicking first on one and then on another, we swap them. Note the formula for going from an (x,y) index in the grid to a single position in the linear array: x+y*nx. As discussed above, if nx=10, then element (2,3) is at 2+3*10, since we have to skip 3 full rows of 10 elements and then take the second element of the next row.

PImage[] pieces;    // the image fragmented into rectangles
int nx=10, ny=10;   // number of rectangles across and down

int dx,dy;          // rectangle sizes, computed from width,height and nx,ny
int selX=-1,selY=-1;  // if a piece has been selected, its x and y indices (-1 if none)
boolean auto=false; // whether or not to auto swap pieces

void setup()
{
  size(800,463);  // size of green
  PImage grn = loadImage("green.jpg");
  noFill(); 

  dx=floor(width/nx);
  dy=floor(height/ny);

  // Fragment the image into rectangles, and string them into a line
  // piece 0,0; piece 0,1; ...; piece 0,nx-1; piece 1,0; piece 1,1; ...
  pieces = new PImage[nx*ny];
  int p=0;
  for (int j=0; j<ny; j++) {
    for (int i=0; i<nx; i++) {
      pieces[p] = grn.get(dx*i,dy*j,dx,dy);
      p++;
    }
  }
  
  shuffle();
}

void draw()
{
  if (auto && frameCount % 10 == 0)
    swapPieces(int(random(pieces.length)), int(random(pieces.length)));
 
  // Draw the pieces at their positions 
  int p=0;
  for (int j=0; j<ny; j++) {
    for (int i=0; i<nx; i++) {
      image(pieces[p],dx*i,dy*j);
      // Outline in red if selected, black otherwise
      if (i==selX && j==selY) { 
        stroke(255,0,0);
        rect(dx*i,dy*j,dx-1,dy-1); // -1 to make sure red can be seen
      }
      else {
        stroke(0);
        rect(dx*i,dy*j,dx,dy);
      }
      p++;
    }
  }
}

// Mix up all the pieces
void shuffle()
{
  // Swap each piece with some piece later in the array
  for (int i=0; i<pieces.length; i++)
    swapPieces(i, int(random(i,pieces.length)));
}

// Swap the pieces at the two indices
void swapPieces(int i1, int i2)
{
  PImage p1 = pieces[i1];    // remember what's at i1
  pieces[i1] = pieces[i2];   // so that we can put at i1 whatever's at i2
  pieces[i2] = p1;           // and then put at i2 what we remembered
}

void mousePressed()
{
  // Find which piece was clicked on
  int px = mouseX/dx, py = mouseY/dy;
  // Select two pieces to swap; same piece twice to cancel
  if (selX >= 0 && selY >= 0) {
    if (selX == px && selY == py) { // same one twice -- cancel
      selX = -1; selY = -1;
    }
    else { // second selection -- swap
      swapPieces(px+py*nx, selX+selY*nx);
      selX = -1; selY = -1;
    }
  }
  else { // first selection
    selX = px; selY = py;
  }
}

void keyPressed()
{
  if (key == 'a') auto = !auto;
  else if (key == 's') shuffle();
}
screenshot[applet]
Last modified: Wed Feb 18 13:28:32 2009