10.4: Pixels! (The Pixels Array) – Processing Tutorial


Hi, okay, this, this is the video. This is the one that… I don’t know. Maybe you’ve been waiting for. Maybe not. This is the video where I want to talk about pixels. Pixels. The pixels. The pixels themselves. So let’s think about this in context for a second. Something that we do a lot in our lives. This is the lives that we’re stuck living. Is we draw lines to the screen. We say something like line, you know… 100, 50, 300, 250. Processing says: “AH! You have said that I should draw a line I see the coordinates you want me to draw the line to. I am gonna start here and I end over here and I draw a line there. The line appears on the screen but what’s really going on here? The existence of this line didn’t happen by magic. The line, the line is not something inherent to a computer or screen. There is an algorithm. Processing has an algorithm when it gets a two points it knows how to fill in all of the pixels between those two points to create the illusion of that line on the screen. I don’t know if it’s really a line or not but that’s that’s a metaphysical question. We don’t have to answer This is something. This exists because we need to do this stuff all the time in order to draw lines and circles, rectangles and triangles and put images all the time. We don’t want to have to figure out which pixels we need to set, which colors… Every single time we want to draw a shape. That’s what these functions are doing for us. However there comes a time in our lives. When we might want to do that. We might want to say I don’t…. These functions are not good enough for me. They… I have this idea for this kind of shape that doesn’t exist anywhere in the world and I need to set all the pixels individually or another… Another scenario might be I took a picture. A picture’s too dark. I wanna make it brighter. I open Photoshop and make it brighter. I save it. That’s what Photoshop is for. It’s for, you know, image Processing. Making, changing brightness… Contrast. Also color all sorts of things you can do to an image. But if there is something you want to do to an image that isn’t available as part of the tools of Photoshop, you need to invent your own image Processing algorithm. The way to do that would be by reading the pixels and setting the pixels of an image one by one at a time with your own algorithm. With your own piece of software. This is what I want to examine how to do. In this video. And the next one and the next one. And I don’t know if there’s another one after that. In a few videos I want to look at what, what happens when you can, you know, actually just Individually set every pixel on the screen. What, what… there’s nothing you couldn’t draw in theory, if you have access to every single pixel on the screen. So that’s what I want to look at. Now… There’s a kind of quick and dirty way to do this in Processing. There are two functions get() and… set() and get() will retrieve… the color of a particular pixel. I want the color at this xy coordinate and I want to store that in a variable. Set() will set a particular pixel to a certain color. Now, these functions are useful and maybe actually lets demonstrate set() really quickly although in the end we’re going to see is these aren’t very efficient and when we want to actually do something to all of the pixels… then we want to look at the pixel array itself. These are kind of helper functions for, you know, just the moment you know… Here’s a scenario where you can… and I’ll give this as an exercise where you might use get(). This is kind of off a little tangent here but it’s worth it for a second. Let’s say you are working on a program… Where you have a map of the United States I wish I could think of a different country but I’m just gonna use the United States right now and I’m incapable of drawing. That’s like the worst drawing of the United States ever but you know the states are all these misshapen strange-looking things. It’s going to be how do you know when the user has clicked on a certain state? Which state that user has clicked on. What if you had a map of the United States where each state was colored with a highly specific color? If you just looked at the pixel underneath the mouse coordinates to see what color that was you could then know which state the user clicked on. So anyway this is a kind of cockamamie scenario that we’re not looking at right now, but would be a great topic for a video but this is why you might want to just retrieve a single color of a single pixel. Let’s go back and let’s look at set for a second. So if I come over here and I have a Processing sketch already kind of going here here’s a blank screen and… And what I want to do is I want to say, let me set the pixel 300, 200 to the color red. Let me run this and somewhere in there… Can we find it? Is there a… Pause for my… pause this video Okay, I’m back, I found the pixel. So, we can see right there that little pixel is red. We’ve set that one individual pixel red. So, you know, there’s no reason why I couldn’t do something like… I don’t know, for int x equals 0; x is less than width; x plus plus. set x, 200 to red. Look I now have a line of all those pixels going right across the screen. So we can see how there’s a function in Processing set() which says for this particular x for this particular y, stick this particular color in that location and this will actually work, and we could start to build this example out more and more but… set(), the a… There’s a detail about set() which I will explain in a moment and we’ll come back full circle, which makes calling the set() function very very very slow. So what we want to do is do this same exact idea, which is a bit simpler, albeit to see with the set() function using the Pixel Array. And what do I mean by the Pixel Array. So let’s come back over here. So… Secretly and not really so secretly but Processing and… Is keeping track of for you, all the pixels in your window or as we’re going to see in a moment all of the pixels in a PImage. It’s keeping track of it in a data structure. That data structure is an array. Just Processing has built in variables. A built-in variable that you were probably quite familiar with is mouseX. Whenever I say mouseX I get the x coordinate of the mouse. I don’t have to declare it. It’s just there. It’s a variable I get it for free. There’s another variable I get for free. pixels Whenever I say pixels that variable holds all of the pixels, all of the colors on the screen but what is this data structure? I mean the window itself… You can think of as a grid. Right.. If we were zoomed way in every single pixel is a square… Rectangle. I don’t know. You decide. Uh-huh and each each one of them has a number. 0 1 2 3 4 5 6 7 8 9 10 11 12 Etc, etc, etc. Each one of these is in an array. The pixels are actually an array of integers or you can think of them as an array of colors. Intege… A color, the color data type is really just an integer. The integer that has the red, green and blue component. This pixel array is one long array of pixels. So if I wanted to set pixel number nine. I could say pixels index [9] equals color 255, 0, 0. This is just like the set function only I’m accessing the array and a particular index in the array which is a particular… sorry. A…. particular pixel number nine referring to a particular pixel that happens to be on the screen. Blah, okay! Let’s go back and see if we can implement that here. Oh i’m here. I’m here. Okay so let’s comment this out and let’s say, right, there’s some pixel, right, pixels index 9. Equals color 255, 0, 0. This should work right? There’s no reason why this is going to fail. Let’s run this and look I got an error. What error did I get? NullPointerException. Null Pointer Exception. The Pixels were nulled. There are no pixels. So what’s going on here? So something that set() did for us Is set() warned Processing in advance, by the way, I’m about to start looking at the pixels and this is something that you just have to do. Before you access that pixel array you’ve got to give Processing a warning that you’re planning to do that so that it’s not just sitting there empty. Processing is going to load those pixels into memory and give you access to them. And the way that we do that is by saying loadPixels(). So anytime we do an operation with pixels we need to first call loadPixels(). By the way right now I’m speaking about pixels in this sort of general way. Pixels being… The pixels of the display window but soon we’re going to start seeing things like image dot pixels or image dot loadPixels. So right now we’re talking in generic terms about manipulating the pixels of the display window but later as we’re going to see it’s perhaps a bit more flexible and sometimes more powerful to work with PImages and manipulate those pixels which you then draw to the screen. So we’ll see that in a little bit. So now that I’ve done this… This works. I actually, if we search I’m pretty sure we would not be able to find your red pixel because another thing you need to do; I don’t see it in there. Another thing you need to do is after you’re done with the pixels need to tell Processing: “I’m done with the pixels please update your display accordingly…” and the way to do that is with updatePixels(). So now we can see that I can look into that Pixel Array… Specify a particular pixel, give it a certain color and call updatePixels(). Now… I’m not going to pause. There it is, there it is. There’s that red pixel. Pixel number nine. Right up there is red. Beautiful. We have now set a pixel in the window. By the way… This should… This should give you an idea right. Well pixel is an array it must have a length and why couldn’t I write a loop Why couldn’t I write a loop which says do something to every single pixel in the window. Now I have a fully red window. We just broke the algorithm for the background function essentially. Again why does the background function exist? Because it would be a lot of work to have to put this in our code every single time we want to fill the background. It’s such a common task. There’s a background function. But this here is us doing the background manually and if you think about this if I’m setting every pixel individually there’s no reason why every pixel needs to have the same color. You know random 255 random 50, 200 So now I’m going to set each pixel with some random color. So we can see I have this kind of purply bluey random set of colors. So this might actually be your first project here which is our first exercise to think about. You know, this isn’t the most and I want to take this a few steps further. Actually you know, what I think But what might be a different way of thinking about this. You know the first thing that could come up with is just like I don’t know, make some random colors but what if you wanted to have the colors from go get like progressively more and more blue as we went across the screen or all the colors closer to the center be more blue and closer to the edge were red. There are lots and lots of possibilities here. The truth of the matter is, and this is why this video can’t end here, and has to keep going even though it’s… we’re about 12 minutes in; is that, we’re missing a really key, key important piece of information. You can pause now and try to just put this in and try to manipulate this, but there’s a real big problem. Like for example the exercise I just suggested was have the pixels nearer to the center…. Be more red and the pixels to the edge be more blue. Well how do I know which pixel is which? I mean I have this just like numbers. Like this pixel over here is probably like pixel like 722,000. Like how do I know which one is which. I really want to say… I know that that… I know, I know about its x y but there’s nothing, there’s no x y here. So let’s return back to our diagram over here and let’s, let’s… Let’s think about something. This is the kind of key piece of information That you’re going to see in just about every example we do from now until the end of whenever these videos end. This… Are the pixels are in this one dimensional array. Numerically indexed 0 1 2 3 4….. This… They…This list of pixels is arranged on the screen in a grid where the pixels count like this: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23… But we, whenever we work with pixels, we really want to think of them in terms of their x and in terms of their y. Their column number and their row number like: 0 1 2 3 4 5 and the row numbers 0 1 2 3 4 5 So I made a 5 by 5 square this will be nice for making the math kind of work out very nicely but let’s actually let’s pretend that the y was 6. Just so we can prove that if I have an image that’s 5 by 6… you can’t see that – if I have an image that’s 5 by 6 how do I figure out where is pixel 4 comma 3. (4, 3). Now bear with me for a second let’s count, I’m going to be way too long winded about this but lets count. 11 12 13 14 15 16 18 19 20 21 22 23 so this should be pixel 23 It’s pretty obvious to us that this is pixel 4… By the way I… in my head I just, did something wrong which I wonder if you’re doing in your head as well. What’s the width of this image? It’s not… it’s not five by six. I put that over here. I didn’t do it through my head I wrote it out here and it was absolutely wrong. The index value is 0 1 2 3 4 5. 0 to 5 but the width is actually 6 and the height here is actually 7. So this image is a 6 by 7 image. This pixel is pixel number 4. The one below it is pixel number 10 4 plus the width 6 is 10. This pixel is actually 16 and then this pixel… I counted wrong. It should actually be 22. Oh boy this is such a disaster 17 18 19 20 21 22. I counted wrong. This Pixel is definitely 22. How did I get that? We can get… If we know the two-dimensional location of a pixel. It’s x and it’s y. We could get the one dimensional location into this array by saying x plus y times width. 4 is the x y is 3 times the width which is 6 which is 4 plus 18. Which equals I got a different number, no, no that’s right. 4 plus 18 is 22. Oh boy… My mind, my brain is melting. 4 plus 18 is 22. You know… Maybe I’ll rerecord this video later but What’s going on here. This is actually quite a crucial algorithm. This magic little formula you can see why it makes sense. Pixels here: 0 6 12 18. The pixels are internally stored as a long list but we want to think of them as a two-dimensional grid. An X Y grid. So if we’re thinking them as an xy grid the moment that we need to do something to its color we use convert from that one dimensional index with this formula x plus y times width. If this is confusing to you I would suggest pausing this video draw a grid, number everything and number all the rows, number all the columns and try to try to get this formula to work for you. It’ll, it’ll, it’ll make itself clear and otherwise you should e-mail me your complaints. I would love to hear them. Okay so let’s go back over here. Now how do we apply that to this particular algorithm? Instead of looping through all the pixels from 0 to the length of the pixel array what I want to do is write an algorithm that loops through every single x value and loops through every single y value. Whoops and this is y plus plus and this should be x plus plus. Now I’m almost certain that there was another video earlier… Which looked at this idea of a nested loop. This is a kind of a difficult thing to get used to this idea of a nested loop. What I’m saying is: “For every single x. For, every, single, x. When I’m at the first x, go over every y. When I’m at the next x go through every y.” Let’s think about that over here. When I’m at the first, x equals 0. Let me go through every y. Now I’m at the next x equals 1. Let me go through every y. Then I’m at the next x. Let me go through every y. That algorithm of for every x look at every y, that will cover every single pixel on the screen. So this algorithm, if I were to say print x y. We’re going to see in the console the coordinates of every single pixel on the screen but I don’t want to do that. I want to say pixels x plus y times width. Right? Here I was able to say pixels index i because I was just counting through all the pixels. Now that I’m storing them by x&y Now… I have to still look up at the array that one-dimensional index and this is my formula for it. That… This formula is now the formula I’m applying right here in the index of the array. So if I were to say this and I’m going to take a loadPixels() and put it down here and updatePixels() and put it down here and then I’m going to comment out this old way of doing it and save it. Whoops. Pixels 2… and… save it in here. I’ll try to figure out where to put these so you guys can find them. Now I have an entire green window. Again I’ve rewritten the background function but now I have access to every single pixel’s x y location. This is very powerful because what if I said y divided by 2 for example. Now you can see the Pixels get more and more green as I go down to the bottom and what if I set in the blue, like x divided by 2? Now that… Now we can start seeing these gradients of color because I can’t arrange the colors I’m picking according to their x y values. Let’s go ahead and do that distance idea. Let’s say I want to get the distance between that particular x y coordinate and the center of the window. The center of the window being the width divided by 2 and the height divided by 2. What if I set the color… equal to that distance? Now look what I have. When that distance is small I have that black color. When that distance is big I have a white color. What if I did something random? What if I put this in the draw loop and instead of having it just be the center I set the pixels according to the distance from the actual mouse. So there’s lots and lots of steps here. This core algorithm of looking through every x and every y and setting every pixel according to that x and y. This is the basis behind every single image processing algorithm. Image processing, computer vision, all the things we’re going to look. All of the code is always going to have this in it. So this is worth taking some time to getting used to. I would say take this, download it, copy it, whatever makes help and then just try it in a lot of different ways. Use a sine function. Use perlin noise. Use Random. Anything you can think of as a way of coloring the Pixels. Give that a try and try to do as much as you can using those x and y values. One little improvement though I want to make to this is, this is a little bit awkward and I think sometimes it can be a little easier to just put that in a separate variable. So i’m going to take that formula x plus y times width and put it in a separate variable and just put that variable in here. I don’t know… this to me just gives it a little bit, a little easier to read and work with if I’m kind of calculating that and storing it in a variable and actually if you look at a lot of the examples that are in Processing or certainly in the Learning Processing book if you’re following along in that, I often use the variable named loc as in like location of pixels. Okay so this was a particularly long video and hopefully it was helpful. and… That’s that. So in the next video I want to start… This is just kind of start looking at what is… What, right now we’re just kind of making up pixels on the fly, but what if we actually read the pixels from an image. Manipulate those pixels and draw them to the screen. That’s sort of the next piece that we will look at. Okay. But I’m going to stop this video and see you in the next one if I could find the mouse to stop recording.

66 Replies to “10.4: Pixels! (The Pixels Array) – Processing Tutorial”

  1. I completely understand the concept now (after watching this like 3 times). Could you please tell me how to make the pixels close to the center white instead of black? I tried dividing a number by the distance and use it as a color but it didn't work out.

  2. hey, I am trying to create a clone stamp tool where I have a transparent box where it copies the pixels i click on and then paste those pixels on the screen. I am stuck and have no idea what to do. Please Help!

  3. hey dani back once again lol, listen dan im looking to stitch 2 video live feeds together from 2 cameras, now i have already figured out how to display 2 cameras on a single processing screen by doing capture(size/2, size/2, camera); 'you know what i mean' so now im looking to divide both of those live feed screens in half and merge both screens together so that we end up with one single screen but sharing the same vision. Im looking to simulate sorta of a human vision 'you know two eyes one perspective ? how would one go about that?

  4. I don't know if you said this already but what language is this? Also, don't you have to define a class, interface, or enum?

  5. Sometimes, without loading an image from an external file, you can include an image in processing by using pixel array (nice if you are coding a webpage and not make any reference to external files) , once saw it but cannot remember how it is done. Any idea?

  6. Thanks alot for your video, it is very useful, do you might have heard about something called maxwell triangle or barycentric interpolation ?. I want to program this issue and i was wondering if you can help me with some references thank you .

  7. It looks like you have always been complaining about video being long since 2015. Please don't, it is hard to come by someone so passionate about programming and can keep talking about the subject forever. I am really inspired watching you so please keep up the good work. Thanks!

  8. Suppose I wanted to -watch- every pixel plot on the screen, one-by-one. Can (is it valid?) "updatePixels()" be placed inside the main loop so that it is executed after EACH pixel operation?? I realize I will have to insert a time delay between each pixel set to slow things to a "watchable" speed.

  9. I'm actually using vpn to watch this video in China. Tbh, you are so passionate and cool. I learnt a lot from it! 😀

  10. you are such a amazing teacher! do u know how to code this staff in javabluej instead of using processing 3 cuz all i am using now is javabluej. Thanks!!!

  11. OMG, what did I watch!) Thank you so much for this, and every other video you've posted! it's so helpful and interesting. .and so much to learn

  12. I couldn't stop thinking about how a nested loop has to be slower than a single loop for looping through every pixel.
    So i tested it in processing, and on my computer it can take up to 2s to iterate 10M times through the pixels using a nested loop, and just 6-7ms using a single loop.
    Code for the benchmark here: https://gist.github.com/YoDevil/b396d3624a6c302123cea86378907dc3
    Thoughts?

  13. Adds a small circle around the cursor:

    void setup() {
    size(600, 430);
    }

    void draw() {
    loadPixels();

    for (int x = 0; x < width; ++x) {
    for (int y = 0; y < height; ++y) {
    int pos = x + y * width;
    int d = int(dist(x, y, mouseX, mouseY));
    pixels[pos] = color(random(0, 200), d*4, random(0, 200));
    }
    }

    updatePixels();
    }

  14. Thank you very much for your series of videos! It's really helpful for newbies as me!
    Hope that you can continue supporting us like this.

  15. Thank you! Your videos are great! it is well explained from the fundamental part and that's the most meaningful part of learning. It's essential for a beginner!

  16. hey Dan !!! i'm getting a nullPointerexception error in processing fireworks sketch in your GitHub pository.. could you please explain to me what the problem could be? thanks..

  17. your mic sounds like it's storming.
    And actually real question is not How do i draw a line? the algorithms do that.
    The question is how can i recognize that an image HAS a line. Or: what pixel colors are on a given line in an image.

    12:00 what you really want is to know Hey I've this array, but I see corners in my display window; where are those corners, i.e. what pixel index is a corner?

    17:00 you complicate stuff needlessly, better to do for y then for x.

    and what language do u use?

  18. Hello sir i am working on a project based on image processing in which i store image pixels in 2D array and retrieve the image pixels with the help of the index value .please help me sir

  19. What if you play this example with file protocol (file:///…) and not on the server?
    Will be still working?
    I'm just curious if will work to you.

  20. If you are only getting a small black box with

    loadPixels();
    for (int x = 0; x < width; x++) {
    for (int y = 0; y < height; y++) {
    float d = dist(x,y,width/2,height/2);
    pixels[x+y*width] = color (d);
    }
    }
    updatePixels();

    just place
    size(600,400);
    background(0);
    before everything else. Cheers

  21. //Colors Magic
    void setup() {
    size(456, 456);

    }
    void draw() {
    loadPixels();
    for(int x = 0;x < width;x++){
    for(int y = 0; y < height; y++) {
    int d = int(dist(x,y,width/2,height/2));
    int d2 = int(dist(x,y,mouseX,mouseY));
    int d3 = int(dist(x,d2,y,d));
    int d4 = int(dist(x,d3,d2,d));
    int d5 = int(dist(d4,d3,d2,d));
    pixels[x+y*width]=color(d3,d4,d5,d2);

    }
    }
    updatePixels();

    }

  22. i think instead of writing the calculation of finding the pixel index for a particular x,y . making a function that returns that value is better … especially when you need these calculations many time
    a function that would be like :
    int locPix (float x,float y){
    int pixelIndex = int(x + width*y);
    return pixelIndex;
    }

  23. I have an image and what to get pixel values of dat image . And store in 2d array can I use below code in Java
    Buffered image img = new buffered image;
    int a [][] = img.get pixel("img.jpg");
    Plss help me…

  24. Please don't be hard on yourself. your videos (and books) are my favorites. It took me way to long to realize you wrote my favorite book, "The nature of code". Is it strange that I want your autograph. Keep it up.

  25. Coding Train, is the mvp of programming. You and khan make it so accessible, I have tried countless times to learn and have never been able to, reading books is dull, and certain online resources just give you obscure code that doesnt work, and offer no help. You give us code that works, with a huge range of examples, and you explain the concepts easily and succinctly. Then Khan academy gives me a platform where people of all levels can ask questions and have a professional answer quickly and directly. You guys are awesome!!

  26. around 1980 i did the following. directly of the equivalent of a pixels array. in this case direct peek and poke of memory, the section of it that was the bitmap of the monochrome screen of this particular computer. and unlike Processing i could watch each and every non-prime that was not already weeded out, get weeded out.
    https://sites.google.com/site/rayjeromejobs/geocachinghobbies

  27. How could I make the inverse way?
    Get the X,Y coordinates from a given index in pixels array?
    In pure Javascript the ImageData.Data has no getPixel method.

  28. Greet video!
    and thanks for all that greet videos
    I found that maybe it easy to get the X and Y of the pixel like that
    loadPixels();

    for(int i = 0 ;i < pixels.length;i++)

    {

    x = i % width;

    y = i / width;

    d =dist( x, y, width/2, height/2 );

    pixels[i] = color(d);

    }

    updatePixels();
    it is instead of nested loop

  29. Thank you so much for these excellent videos.

    I know this is not the place to ask this, but I'll give it a try anyway.
    so I am a beginner to programming and I've just started learning java. I've decided to dedicate a solid amount of time in the next few months to follow The Coding Train videos for learning and educational purpose.

    Currently I've started with Java and I'm following the videos on p3/Java. But I've also notices that most of the recent content here is focused on p5.js, although I am very much interested to learn p5.js, the reality is I am still new to Java, so I don't want to confuse between Java and Javascript. therefore, I am facing a tough time deciding if I should skip Java/P3 videos and directly jump to p5.js videos.

    or do I finish with the p3 videos and then take some time, get some experience and the start with p5.js.

    mainly because I want to be following The Coding Challenge series of videos, asap, where most of the content is in p5.js.

    Thanks again

Leave a Reply

Your email address will not be published. Required fields are marked *