A generic error occurred in GDI+

This is a common exception you'll get when trying to delete or rename an image file. The problem is that the method to load the image from file will put a lock on the file, so you can't do anything with it until you dispose the image.
There are two ways around that: either load the image from a stream or dispose the image before deleting or renaming the file.
Loading the image from stream works, but the actual loading is a bit slower and you need to keep the stream open until you don't need the image anymore. You can't just load it with a stream, close it, and then work with the image. It might work with some file format, but the documentation itself tells you to keep the stream open. Here's how to do it:

using (FileStream stream = new FileStream(@"path\to\image", FileMode.Open, FileAccess.Read))
{
    pictureBox1.Image = Image.FromStream(stream);
}

The problem here is that after assigning the image to the picturebox the stream will close ("using" statement). You could do this instead:

FileStream stream = new FileStream(@"path\to\image", FileMode.Open, FileAccess.Read);
pictureBox1.Image = Image.FromStream(stream);

But then you'll have an open stream forever, until you close it, which is not really a good solution. The real solution comes from loading the image from file, like this:

pictureBox1.Image = Image.FromFile(@"path\to\image");

And then dispose it correctly.
I'm sure you know that we should always dispose everything disposable after we're done with it, so it shouldn't be a surprise, but the problem in disposing and then deleting or renaming soon after is that the image has not enough time to be disposed. So, if you do this:

pictureBox1.Image.Dipose();
File.Delete(@"path\to\image");

You'll most likely incur in the "A generic error occurred in GDI+" exception, because the lock on the file is not yet released.
The REAL and FINAL solution to this, is to call the garbage collector before deleting the file.
You can do that by writing this:

GC.Collect();

After disposing the image.
There's still a problem though, the Collect() method is called asynchronously, that means you could and probably will incur in the same error as before, since the file could still be locked. You need to tell the program to wait for the garbage collector to finish, by adding this:

GC.WaitForPendingFinalizers();

After the GC.Collect().
So, the final code would be:

pictureBox1.Image.Dipose();
GC.Collect();
GC.WaitForPendingFinalizers();
File.Delete(@"path\to\image");

And that's how you delete or rename an image file safely in C#.

Resize an image in C#

Resizing an image is something that can be very useful, even if you aren't writing a specific application that works mostly with images. I'll post three methods to resize an image, in different ways, all of them useful. All three are really easy to understand and to implement.

Resing an image, regardless of the aspect ratio: this is the easiest, can be useful in some situations. Even though you might want to preserve the aspect ratio most of the time, it's a good thing to have the option of resizing without constraints.

public static Image resize(Image img, int width, int height)
{
    // Resize image with the size from the parameters
    Image bmp = new Bitmap(width, height);
    Graphics g = Graphics.FromImage(bmp);
    // The interpolation mode is optional, you can choose whatever you like
    g.InterpolationMode = InterpolationMode.HighQualityBicubic;
    g.DrawImage(img, 0, 0, width, height);
    g.Dispose();
    return bmp;
}

Resizing an image, maintainig the aspect ratio, based on percentage. Been able to resize an image sending only the image and a percentage as parameters is really useful, because sometimes that's all you need.

public static Image resize_percentage(Image img, int percentage)
{
    // Resize the image from the percentage parameter
    // Calculate the new dimensions
    int resizedWidth = (img.Width * percentage / 100);
    int resizedHeight = (img.Height * percentage / 100);
    // Make a new bitmap with the new dimensions
    Image bmp = new Bitmap(resizedWidth, resizedHeight);
    // Create the Graphics object to draw on the image
    Graphics g = Graphics.FromImage(bmp);
    g.InterpolationMode = InterpolationMode.HighQualityBicubic;
    g.DrawImage(img, 0, 0, resizedWidth, resizedHeight);
    // Always remember to Dispose!
    g.Dispose();
    return bmp;
}

Resizing an image maintaining the aspect ratio, based on actual dimensions. This is one of the most useful methods you'll use when dealing with images.

public static Image resizeWithAR(Image image, int maxWidth, int maxHeight)
{
    // Calculate the ratio to maintain aspect ratio
    double ratioX = (double)maxWidth / image.Width;
    double ratioY = (double)maxHeight / image.Height;
    double ratio = Math.Min(ratioX, ratioY);

    // Calculate the new dimensions
    int newWidth = (int)(image.Width * ratio);
    int newHeight = (int)(image.Height * ratio);

    // Draw the new image
    Image returnImg = new Bitmap(newWidth, newHeight);
    Graphics g = Graphics.FromImage(returnImg);
    g.InterpolationMode = InterpolationMode.HighQualityBicubic;
    g.DrawImage(image, 0, 0, newWidth, newHeight);
    g.Dispose();

    return returnImg;
}

Get thumbnails without reading the entire image

I just thought this could be useful to someone, i was searching for a method to read the thumbnail data of a Jpeg file without loading the entire file into memory, and i stumbled upon this code written by Kourosh Derakshan (Extract thumbnail image from a file without reading the whole file). The only problem was that i needed the thumbnails to have the correct orientation based on the EXIF data of the photo, so i modified it to implement that feature.

private const int THUMBNAIL_DATA = 0x501B;
private const int ORIENTATION_DATA = 0x0112;

public static Image GetThumbnail(string path)
{
    try
    {
        // Please do not remove :)
        // Written by Kourosh Derakshan
        // Modified by Phate for automatic exif orientation
        FileStream fs = File.OpenRead(path);
        Image img = Image.FromStream(fs, false, false);
        // Last parameter tells GDI+ not the load the actual image data

        // GDI+ throws an error if we try to read a property when the image
        // doesn't have that property. Check to make sure the thumbnail property
        // item exists.
        bool propertyFound = false, orientFound = false;
        for (int i = 0; i < img.PropertyIdList.Length; i++)
        {
            if (img.PropertyIdList[i] == ORIENTATION_DATA)
            {
                orientFound = true;
            }
            if (img.PropertyIdList[i] == THUMBNAIL_DATA)
            {
                propertyFound = true;
                break;
            }
        }

        if (!propertyFound)
        {
            fs.Dispose();
            img.Dispose();
            return null;
        }

        PropertyItem o = null;
        if (orientFound)
            o = img.GetPropertyItem(ORIENTATION_DATA);

        PropertyItem p = img.GetPropertyItem(THUMBNAIL_DATA);
        fs.Close();
        img.Dispose();


        // The image data is in the form of a byte array. Write all 
        // the bytes to a stream and create a new image from that stream
        byte[] imageBytes = p.Value;
        MemoryStream stream = new MemoryStream(imageBytes.Length);
        stream.Write(imageBytes, 0, imageBytes.Length);

        Image temp = Image.FromStream(stream);

        if (o != null && Form1.ini.IniReadValue("settings", "automaticEXIF") == "true")
        {
            switch (int.Parse(string.Format("{0}", BitConverter.ToInt16(o.Value, 0))))
            {
                case 1:
                    temp.RotateFlip(RotateFlipType.RotateNoneFlipNone);
                    break;
                case 2:
                    temp.RotateFlip(RotateFlipType.RotateNoneFlipX);
                    break;
                case 3:
                    temp.RotateFlip(RotateFlipType.Rotate180FlipNone);
                    break;
                case 4:
                    temp.RotateFlip(RotateFlipType.Rotate180FlipX);
                    break;
                case 5:
                    temp.RotateFlip(RotateFlipType.Rotate90FlipX);
                    break;
                case 6:
                    temp.RotateFlip(RotateFlipType.Rotate90FlipNone);
                    break;
                case 7:
                    temp.RotateFlip(RotateFlipType.Rotate270FlipX);
                    break;
                case 8:
                    temp.RotateFlip(RotateFlipType.Rotate270FlipNone);
                    break;
                default:
                    temp.RotateFlip(RotateFlipType.RotateNoneFlipNone);
                    break;
            }
        }
        return temp;
    }
    catch (Exception) { return null; }
}