Phone Vision – 02 Extracting Color

6 01 2011

Last time we set up a very simple project that will allow us to display and manipulate image data using a WriteableBitmap.  Before we can do anything cool though, we need to understand a few things about the WriteableBitmap class.

Pixel Collection

The WriteableBitmap has an integer array called Pixels that represents the 2D texture of the bitmap.  In most cases we will be looping through Pixels (using PixelWidth and PixelHeight) to perform our operations.  You can find the data for a pixel located at (x,y) in the WritableBitmap wb  with:

int color = wb.Pixels[x + y * wb.PixelWidth];

Without knowing how the color is stored we really can’t do much with this data.  Let’s take a peak under the hood.

Pixel Format

The format used by the Silverlight WriteableBitmap is ARGB32 (premultiplied RGB – we’ll cover this later).  This means that the color is represented by a 32-bit integer with the following format:

image

As you can see, each channel (alpha, red, green, and blue) uses 8 bits, giving a possible 28 or 256 intensities for each (0-255). 

 

Extracting Color Components

Extracting each component intensity can be done by masking the color value with 0xFF then shifting the number right by 8 bits.  The heck you say?

Bitwise Operators

And

As their name implies, bitwise operators operate on one bit at a time.  The operator we are concerned with for now is & (AND).

operator bit one bit two output
& 0 0 0
& 0 1 0
& 1 0 0
& 1 1 1

Notice in the & operator when a bit is 0 the output bit will always be 0.  Wherever there is a 1 in a bit, the result is always the value of the other bit.  This trait will allow us to create what is called a mask (or bitmask).  We cover the bits we don’t care about with 0.  So the mask 0xFF above in binary is…

image

& this with the pixel and we will get the value for the last 8  bits of the color.

Let’s work with an example.  I randomly picked #FFDB91D6.  #FFDB91D6 in binary:

image

&

image

=

image

 

Voila.

  

 

Right Shift

 

If we were to use the & alone we could only recover the values for the blue channel.  If only we could shift the color bits to match our mask…   Turns out there is another handy bitwise operator for doing just that: >> (RIGHT SHIFT)

 

operator input shift output
>> 10001000 1 01000100
>> 10001000 2 00100010
>> 10001000 3 00010001
>> 10001000 4 00001000

 

Notice how the bits shift right?  This is equivalent to dividing by powers of 2.  Don’t let the leading zeros confuse you.  If you divide 1120 by 10 (in base 10) you get 0112.  If you divide again you get 0011 (drop the remainder).  See?

 

image

>> 8

=

image

 

Now the green bits match our mask.  Hooray!

 

Full Example

 

Let’s run through our example from above (#FFDB91D6) in its entirety:

 

1.) First, let’s get the blue bits:

 

image

&

image

=image

 

 

2.) Shift the bits right 8 places:

 

image

>> 8

=

image

 

 

3.) Recover the green bits:

 

image

&

image

=

image 

 

 

4.) Shift the bits right another 8 places:

 

image

>> 8

=

image

 

 

5.) Extract the red bits:

 

image

&

image

=image

 

 

6.) Last shift!!!

 

image

>> 8

=

image

 

 

7.) Extract the alpha bits:

 

 image

=image

 

 

We don’t have to mask the last value because it’s already exactly what we need.  Let’s see how this plays out in code.

 

int pixel = (int)0xFFDB91D6;

byte blue = (byte)(pixel & 0xFF);

pixel >>= 8;

byte green = (byte)(pixel & 0xFF);

pixel >>= 8;

byte red = (byte)(pixel & 0xFF);

pixel >>= 8;

byte alpha = (byte)(pixel);

Quite elegant and very efficient.

 

Summary

Recovering colors from the WriteableBitmap is pretty straightforward once you understand the pixel structure and how to mask them.

Download Code

http://cid-88e82fb27d609ced.office.live.com/embedicon.aspx/Blog%20Files/PhoneVision/PhoneVision%2002%20-%20ExtractingColors.zip

Note:  This code doesn’t actually modify the image in any way.  We will do that in the next installment.

 

Up next: Encoding Color

 





Phone Vision – 01 Acquiring the Image

3 01 2011

All image processing projects have one thing in common: an image (or multiple images).  We need to first acquire an image, then we need to get it into a format that is conducive to editing.  Let’s get cracking.

Setting up the Project

We are going to set up a very simple project to show the original image, the processed image, and an application bar button to start the image capture.  This will be the basis for future projects.

1.) Create a new Silverlight for Windows Phone Project

image

2.) Modify the LayoutRoot to have two images:

    <Grid x:Name="LayoutRoot"

          Background="Transparent">

        <Grid.RowDefinitions>

            <RowDefinition />

            <RowDefinition />

        </Grid.RowDefinitions>

      

        <Image x:Name="OriginalImage"

            Grid.Row="0" />

        <Image x:Name="ProcessedImage"

            Grid.Row="1" />

    </Grid>

3.) Create the application bar:

<phone:PhoneApplicationPage.ApplicationBar>

        <shell:ApplicationBar IsVisible="True" >

            <shell:ApplicationBarIconButton

                x:Name="AcquireImageButton"

                IconUri="/Images/appbar.feature.camera.rest.png"

                Text="acquire"

                Click="AcquireImageButton_Click" />

        </shell:ApplicationBar>

    </phone:PhoneApplicationPage.ApplicationBar>

Note: Windows Phone 7 icons can be found here: C:\Program Files (x86)\Microsoft SDKs\Windows Phone\v7.0\Icons

Acquiring the Image

Now for the meat of the post.  First, we need to acquire an image then convert it to a WriteableBitmap.  In the wild, we are going to want to actually take a picture, but in the lab we simply want to load an image from the project.

1.) When we press the Acquire button we want to launch the Camera Capture Task.  Once it’s complete, we can start processing.

private void AcquireImageButton_Click(object sender, EventArgs e)

{

    CameraCaptureTask cct = new CameraCaptureTask();

    cct.Completed += new EventHandler<PhotoResult>(cct_Completed);

    cct.Show();

}

2.) Create a BitmapImage and a WriteableBitmap from the results.

void cct_Completed(object sender, PhotoResult e)

{

    // if we are working with the actual device

    // we actually want to take a picture

    // otherwise we are going to use a jpg

    // I have included in the project

    if (Microsoft.Devices.Environment.DeviceType == DeviceType.Device)

    {

        // TaskResults can be OK, Cancel, or None

        // Working with the device here is rather

        // painful because it cancels every time

        // for me when I’m connected so I have to

        // run disconnected from the PC

        if (e.TaskResult == TaskResult.OK)

        {

            BitmapImage bmp = new BitmapImage();

            bmp.SetSource(e.ChosenPhoto);

            OriginalImage.Source = bmp;

            WriteableBitmap processedImage = new WriteableBitmap(bmp);

            // you could put some automated processing here

            ProcessedImage.Source = processedImage;

        }

    }

    //else we are running from the emulator

    //in this case the CameraCaptureTask "works"

    //but it doesn’t really return anything of value

    //so we are going to use an image added to the

    //project

    else

    {

        //Creating a WriteableBitmap from a

        //BitmapImage that was created using a

        //Uri can be a little tricky.

        //If you don’t follow these steps you

        //might get a cryptic exception

        BitmapImage bmp = new BitmapImage();

 

        //Options here are None, DelayCreation (default), and

        //IgnoreImageCache (in Silverlight).  Mainly, we don’t want

        //DelayCreation because it waits to initialize it until

        //necessary (i.e. when it’s shown).  Since we are setting

        //the source of OriginalBitmap to bmp, we don’t actually

        //need to perform this step in this case.  In general,

        //however, for my projects, I often want to process the

        //image before I display it.

        bmp.CreateOptions = BitmapCreateOptions.None;

 

        //note how we are waiting for the image to

        //finish opening before we try to create the

        //WriteableBitmap

        bmp.ImageOpened +=

            new EventHandler<RoutedEventArgs>(bmp_ImageOpened);

               

        //I added this test image as Content to the Images folder.

        //I recommend taking a photo with the phone then adding it

        //to your project.  This ensures the image will be the correct

        //format and consistent dimensions (2592×1944) for processing

        bmp.UriSource =

            new Uri("Images/test_image.jpg", UriKind.Relative);

        OriginalImage.Source = bmp;

    }

}

void bmp_ImageOpened(object sender, RoutedEventArgs e)

{

    WriteableBitmap processedImage =

        new WriteableBitmap(sender as BitmapImage);

    // you could put some automated processing here

    ProcessedImage.Source = processedImage;

}

3.) Run it

image

You can really see the HTC HD7 pink color issue in this photo.

Summary

This is a fairly trivial task, but getting the WriteableBitmap functioning properly is the basis for everything from here on out.

 

Download Code

http://cid-88e82fb27d609ced.office.live.com/embedicon.aspx/Blog%20Files/PhoneVision/PhoneVision%2001%20-%20AcquiringImages.zip

 

Up next: Extracting Colors