What Picture Types do you allow?

[doubledee]

For User Profile Pictures, what File-Types should I allow?

 

Is it too rigid to just accept JPEGs?

 

Should I ban GIFs since they can contain scripting?

 

Other ideas?

 

Thanks,

 

 

Debbie

 

[litebearer]

jpg, png, gif

[gizmola]

I'm not sure what your concern is about the .gif format.  Gif does allow for animation, where there are multiple frames inside one gif image, and that causes a problem for thumbnailing routines, but I know of no other concern. 

 

There is code out there you can find that will show you how to detect an animated gif and you can reject it, if that's what you're worried about. 

 

Most people support .jpeg, .gif and .png.

 

Also it has become common practice recently for people to try and automate the process by looking for http://en.gravatar.com/ or if the site implements open id integration with facebook, youtube, or twitter -- to pull a default avatar from those services.

[doubledee]

I'm not sure what your concern is about the .gif format.  Gif does allow for animation, where there are multiple frames inside one gif image, and that causes a problem for thumbnailing routines, but I know of no other concern. 

 

GIFs can contain hidden code that can be executed...

 

 

There is code out there you can find that will show you how to detect an animated gif and you can reject it, if that's what you're worried about. 

 

Most people support .jpeg, .gif and .png.

 

Okay, but GIFs are supposed to be for Logos and things where you are relying on 256 colors versus pictures which can have tens of thousands of colors.

 

My point being, why would you save a photo of yourself as a GIF?!

 

 

Also it has become common practice recently for people to try and automate the process by looking for http://en.gravatar.com/ or if the site implements open id integration with facebook, youtube, or twitter -- to pull a default avatar from those services.

 

May be a good idea for my next release.

 

Is there a reliable way to know what File Type a File truly is?

 

I mean just because a file is labeled "nowWhatISeem.jpeg" doesn't mean it is really a JEPG, right?

 

Or someone could do "lookOut.exe.jpg" with the hopes that it gets executed by your system, right?

 

I've been reading up on $_FILES but there is virtually no info in the PHP Manual.

 

Not getting off to a very fast start... 

 

 

Debbie

 

[litebearer]

In a link (http://advosys.ca/viewpoints/2009/04/sanitizing-php-file-uploads/) I provided in one of your other threads, there is this quote...

After a bit of investigation we settled on simply converting images to an intermediary format, then converting back to JPEG.

 

It resulted in a minor loss in quality, but guaranteed the images viewed by other users were free of crafted bytes, metadata, appended data like GIFAR (which also works with JPEGs), and even most steganography.

 

You can easily convert any format to another thereby accomplishing what the author  suggests.

 

(about the author - Advosys Consulting is an information security and systems management company: we help clients protect their information, implement systems and provide technical support. We have a broad range of experience in information security, Internet technologies and open source server software.

 

We are a privately held Canadian corporation with head office in Ottawa, Canada. We have been providing solutions to private sector and government clients since 1990.)

[gizmola]

GIFs can contain hidden code that can be executed...

 

Proof/source of this claim?

 

 

[doubledee]

In a link (http://advosys.ca/viewpoints/2009/04/sanitizing-php-file-uploads/) I provided in one of your other threads, there is this quote...

 

After a bit of investigation we settled on simply converting images to an intermediary format, then converting back to JPEG.

 

It resulted in a minor loss in quality, but guaranteed the images viewed by other users were free of crafted bytes, metadata, appended data like GIFAR (which also works with JPEGs), and even most steganography.

 

I read the article.

 

I also have no clue what his code says.  (Is it even in PHP?!)

 

 

Debbie

 

[doubledee]

GIFs can contain hidden code that can be executed...

 

Proof/source of this claim?

 

 

Google: Maliciously coded GIF

 

(Um, this is pretty old news...)

 

Also...

http://ha.ckers.org/blog/20070604/passing-malicious-php-through-getimagesize/

 

 

Debbie

 

[KevinM1]

A .jpg can also contain code....

 

EDIT: When you can, get yourself a copy of this book: http://www.amazon.com/Pro-PHP-Security-Application-Implementation/dp/1430233184/ref=sr_1_2?s=books&ie=UTF8&qid=1329096463&sr=1-2  It will answer all of your security concerns, and likely inform you of things you never thought of.

[doubledee]

GIFs can contain hidden code that can be executed...

 

Proof/source of this claim?

 

Gizmola, you might want to check this out...

http://www.blogtips.org/avoid-users-uploading-malware/

 

I just ran this test using "test.php.jpg" on my website and I got the phpinfo() page in full display?!  Holy Sh*t!!! 

 

Any security experts out there who want to help me make sure my system is safer with my File Upload module, feel free to dive in...

 

 

 

Debbie

 

 

[doubledee]

A .jpg can also contain code....

 

EDIT: When you can, get yourself a copy of this book: http://www.amazon.com/Pro-PHP-Security-Application-Implementation/dp/1430233184/ref=sr_1_2?s=books&ie=UTF8&qid=1329096463&sr=1-2  It will answer all of your security concerns, and likely inform you of things you never thought of.

 

I've seen that book, and it has been on my "To-Do List".

 

The more I think about Security on my website, the more I cringe.

 

If a person spent time learning about and chasing down all of the currently security exploits out there, you would never have the Time, $$$, or Resources to run a website.  Right?! 

 

I'm not sure how to balance things...

 

And as far as my current sub-project - which is creating the ability to Upload Pictures to my website, I have been UN-impressed with the articles and code samples out there.  (Especially since most of them clearly don't know what they are talking about!)

 

 

Debbie

 

[doubledee]

Forgot to send this.

 

Another article for Gizmola...

 

http://stackoverflow.com/questions/1044229/i-found-a-php-script-in-a-gif-file

 

 

Debbie

 

[kicken]

I just ran this test using "test.php.jpg" on my website and I got the phpinfo() page in full display?!  Holy Sh*t!!! 

 

The article shows you how to guard against that using a .htaccess file to correct the bad server configuration.  As I mentioned in the other thread, it's always a good idea to flat out disable any script handlers in a directory that receives user uploads.

 

If you can't use the .htaccess for some reason, you can simply ensure that no file has '.php' anywhere in it's filename with a simple str_replace

while (strpos($filename, '.php')!==false){
  $filename = str_replace('.php', '', $filename);
}

 

[scootstah]

GIFs can contain hidden code that can be executed...

 

I believe this only affects older browsers. And it's not limited to gif's.

[doubledee]

I just ran this test using "test.php.jpg" on my website and I got the phpinfo() page in full display?!  Holy Sh*t!!! 

 

The article shows you how to guard against that using a .htaccess file to correct the bad server configuration.

 

I don't follow you.  Are you saying that the code I added to the bottom of my .htaccess file was just "clean-up" for something else?

 

If so, what is the root problem on my server?  And how do I fix that?  (I have a Linux VPS.)

 

 

As I mentioned in the other thread, it's always a good idea to flat out disable any script handlers in a directory that receives user uploads.

 

What is a "script handler"??

 

And how do I disable them??

 

 

If you can't use the .htaccess for some reason, you can simply ensure that no file has '.php' anywhere in it's filename with a simple str_replace

while (strpos($filename, '.php')!==false){
  $filename = str_replace('.php', '', $filename);
}

 

 

Debbie

 

[scootstah]

I don't follow you.  Are you saying that the code I added to the bottom of my .htaccess file was just "clean-up" for something else?

 

If so, what is the root problem on my server?  And how do I fix that?  (I have a Linux VPS.)

 

Fix it in the apache config.

 

 

What is a "script handler"??

 

And how do I disable them??

 

http://httpd.apache.org/docs/1.3/handler.html

 

I'll post this again for public scrutiny but I'm pretty sure this is all you need:

php_flag engine off
Options -ExecCGI

[gizmola]

Forgot to send this.

Another article for Gizmola...

 

At the risk of sounding pedantic, I knew all about these when i asked the question.  There is a difference between an image that allows "scripting" and one that includes script.  Quite simply, this is a problem involving misconfiguration of a website -- not with the gif image itself.  You would also be just as exposed if you allowed someone to upload something that your client claimed was an image, and your website accepted it and deposited it somewhere.  A lot of things have to go wrong including:

 

- the site "executing" an image as a .php file (it shouldn't)

- uploads going into a directory where the "execute" bit is set (also shouldn't be setup that way)

 

The point is, that you need to run down these assertions and understand them fully -- not just react to something someone wrote on a blog, or claims someone made.  In no way, is the gif executing code.  The problem here is php running a file that contains executable code.  By itself, even a gif that contains code, poses no particular danger if your site is configured properly.  If it's not configured properly, I don't need to even hide the code inside the image.

 

 

 

 

[doubledee]

Forgot to send this.

Another article for Gizmola...

 

At the risk of sounding pedantic, I knew all about these when i asked the question.  There is a difference between an image that allows "scripting" and one that includes script.  Quite simply, this is a problem involving misconfiguration of a website -- not with the gif image itself.  You would also be just as exposed if you allowed someone to upload something that your client claimed was an image, and your website accepted it and deposited it somewhere.

 

Well, I uploaded a test.php.jpg file - via FTP - and it did just that.

 

 

A lot of things have to go wrong including:

- the site "executing" an image as a .php file (it shouldn't)

 

And how do I prevent that?

 

 

- uploads going into a directory where the "execute" bit is set (also shouldn't be setup that way)

 

And how do I prevent that?

 

 

The point is, that you need to run down these assertions and understand them fully -- not just react to something someone wrote on a blog, or claims someone made.

 

That is why you are an Admin and I am a newbie.  If I knew all of this I wouldn't be here.

 

 

In no way, is the gif executing code.  The problem here is php running a file that contains executable code.  By itself, even a gif that contains code, poses no particular danger if your site is configured properly.  If it's not configured properly, I don't need to even hide the code inside the image.

 

Which the article I posted above clearly showed.

 

 

Debbie

 

[gizmola]

The particular problem being pointed out in that article isn't showing an issue with images, but rather an issue with apache, and multiple extension files.  The reason it works is that if you allow for:

 

"somefile.php.jpg"  then apache will decide to run the file.  This same issue exists for somefile.php.foo.bar.

 

There are ways to combat this, but it's really missing the point -- you are in control of what a file is named in the upload process.  I don't want to get too far afield here, so let's go back to what the initial issue was.

 

-You were concerned about .gif files that could run code.

-I stated that there is not a reason to be worried about that.

-You showed articles with people purposefully exploiting an apache hole (has nothing to do with .gif).

 

Rather than completely get off the rails here the answer to this is:  "Control the naming of your avatar files".

 

I'm sure you know that handling of uploaded files is a multistep process with php which involves move_uploaded_file().

 

When you call move_uploaded_file you specify the destination (path + name) of the file that will be moved.

 

Under no circumstances should the name be *anything other than what you specify it to be".  Typically people will name these avatars something predicatable, and there is no real cause for that to be a secret, because the avatar name will be displayed in normal use anyways.  If you have a user id system, the name of the avatar might be:

 

"a1234.png" which might correspond to the picture uploaded by the user with id "1234".  You are probably going to store this name in your database anyways, so it could be a random series of characters --- it really doesn't matter whatsoever.  All that does matter is that you will set the name to something you control, and what you certainly will not do is name the file:  "something.php.png". 

 

So long as you do this, there is nothing to worry about, and the same thing goes for other types of file uploads or image uploads.  Simply do not allow for multiple extensions, and this apache issue will not be a problem for you.

 

There are more complicated ways to insure this, like this htaccess rule for example:

 

deny from all

order deny,allow
allow from all

 

This is complex because you really need this rule to be in the parent directory of the image directory in question.

 

I also am not going to talk about the permissions issues I mentioned previously because that is really an unrelated concern.  In linux there is (for files) an execute bit that tells the linux operating system it can treat the file in question as being "executable".  When you write shell scripts you have to set this bit to make them run.  You don't want to have a directory where people upload scripts, set this execute bit on files, since it then becomes that much easier for an attacker to try and get your webserver to run code.  That is not a php specific concern however, and exists outside of apache.  What that also tells you is that you should never set all the files AND directories that contain your php scripts to have the permission 777.  This is a common thing that people do when they are having problems getting their scripts to run, and that is never a good idea.

[doubledee]

I'm feeling exhausted (and frustrated) but wanted to write back while all of this is fresh in your mind...

 

 

The particular problem being pointed out in that article isn't showing an issue with images, but rather an issue with apache, and multiple extension files.  The reason it works is that if you allow for:

 

"somefile.php.jpg"  then apache will decide to run the file.  This same issue exists for somefile.php.foo.bar.

 

One thing that confuses me is how a file can be named with a .jpg extension but run as a .php file.  I mean I know my code inside the file was...

<?php
phpinfo();
?>

 

...but I would still expect that to trip things up.

 

 

There are ways to combat this, but it's really missing the point -- you are in control of what a file is named in the upload process.  I don't want to get too far afield here, so let's go back to what the initial issue was.

 

But the issue is not what the file was named, it is that when the file was run, the code inside the file was executed, right?

 

I mean if I renamed that suspect file to goodfile.jpg then it would still run unless I changed Apache's settings, right?

 

 

-You were concerned about .gif files that could run code.

-I stated that there is not a reason to be worried about that.

-You showed articles with people purposefully exploiting an apache hole (has nothing to do with .gif).

 

Well, I thought I had heard in the past that a .GIF could be turned into executable malware, but I'll assume that any sources I posted earlier are around this issue of Apache running a file when it shouldn't.

 

 

Rather than completely get off the rails here the answer to this is:  "Control the naming of your avatar files".

 

It seems to me the issue should be Ensuring that a file is what it says it is...  (Don't let a file called "reallyIAmAPicture.jpg" actually contain executable PHP?!)

 

 

I'm sure you know that handling of uploaded files is a multistep process with php which involves move_uploaded_file().

 

When you call move_uploaded_file you specify the destination (path + name) of the file that will be moved.

 

Well, there is little in the PHP Manual on this.  And for the life of me I can't find the "attributes" for a File in $_FILES.

 

Here is some code I have, but it doesn't work...

 

if ($_SERVER['REQUEST_METHOD']=='POST'){
	// Form was Submitted (Post).

	// Initialize Variables.
	$errors = array();

	// Connect to the database.
	require_once(WEB_ROOT . 'private/mysqli_connect.php');

	$uploadDir = BASE_URL . 'uploads/';								// Relative path under webroot
	$uploadFile = $uploadDir . basename($_FILES['userPhoto']['name']);
	if (move_uploaded_file($_FILES['userPhoto']['tmp_name'], $uploadFile)){
		echo "File is valid, and was successfully uploaded.";
	}else{
		echo "File uploading failed.";
	}
}

 

 

And later in the same file I have...

<form id="uploadPicture" enctype="multipart/form-data" 
				action="<?php echo $_SERVER['SCRIPT_NAME']; ?>" method="post">
	<fieldset>
		<legend>Upload a Picture</legend>
		<label for="userPhoto">Filename:</label>
		<input id="userPhoto" name="userPhoto" type="file" />
		<!-- Submit Form -->
		<input type="submit" name="uploadPicture" class="button" value="Upload Picture"/>
	</fieldset>
</form>

 

But when I try to upload a file I get this...

Warning: move_uploaded_file(http://local.debbie/uploads/1.png) [function.move-uploaded-file]: failed to open stream: HTTP wrapper does not support writeable connections in /Users/user1/Documents/DEV/++htdocs/05_Debbie/members/upload.php on line 30

 

**Where line 30 is "if (move_uploaded_file("

 

 

Under no circumstances should the name be *anything other than what you specify it to be".  Typically people will name these avatars something predicatable, and there is no real cause for that to be a secret, because the avatar name will be displayed in normal use anyways.  If you have a user id system, the name of the avatar might be:

 

"a1234.png" which might correspond to the picture uploaded by the user with id "1234".  You are probably going to store this name in your database anyways, so it could be a random series of characters --- it really doesn't matter whatsoever.  All that does matter is that you will set the name to something you control, and what you certainly will not do is name the file:  "something.php.png".

 

Right, but again, I also need to ensure that a Picture is really a Picture.  (Seems like there are lots of ways that do NOT successfully check this.)

 

Not sure how to be 100% sure that a GIF or PNG is really an Image File?

 

 

So long as you do this, there is nothing to worry about, and the same thing goes for other types of file uploads or image uploads.  Simply do not allow for multiple extensions, and this apache issue will not be a problem for you.

 

You lost me.

 

What do you mean "Do not allow for multiple extensions"?

 

When people say that I think jpg|JPG|jpeg|JPEG

 

You mean "some.name.like.this.php.jpg" ??

 

If so, then that is just using PHP String Functions to check that there is only one period, right?

 

 

There are more complicated ways to insure this, like this htaccess rule for example:

 

deny from all
<Files ~ "^\w+\.(gif|jpe?g|png)$">
order deny,allow
allow from all
</Files>

 

This is complex because you really need this rule to be in the parent directory of the image directory in question.

 

What does that say?

 

 

I also am not going to talk about the permissions issues I mentioned previously because that is really an unrelated concern.  In linux there is (for files) an execute bit that tells the linux operating system it can treat the file in question as being "executable".  When you write shell scripts you have to set this bit to make them run.  You don't want to have a directory where people upload scripts, set this execute bit on files, since it then becomes that much easier for an attacker to try and get your webserver to run code.  That is not a php specific concern however, and exists outside of apache.

 

So in Apache (??) I should set my "uploads" directory to "Read-Only"?  (What is that in hex or whatever?)

 

 

What that also tells you is that you should never set all the files AND directories that contain your php scripts to have the permission 777.  This is a common thing that people do when they are having problems getting their scripts to run, and that is never a good idea.

 

Can you explain what you mean?

 

Sorry for the questions, but I don't know Linux or Apache, and am trying to learn...

 

Thanks,

 

 

 

Debbie

 

 

[RobertP]

i have been working on my fileManager class for a few days now, and i think i have solved your issue, checking what a file is, without relying on the extension.

 

$ext = getFileExt($file['name']);
if(!is_uploaded_file($file['tmp_name']))
return 'Seems your file was lost in transmission, please try again.';
$mimeType = mime_content_type($file['tmp_name']);
if(!startsWith($mimeType,'image')||!in_array($ext,explode(',',$config['allowedUploadExtensions'])))

 

ps: at the moment i only accept image type files.

[gizmola]

I'm at the end of my day here, so I'll try and hit the main points as I see them:

 

What do you mean "Do not allow for multiple extensions"?

When people say that I think jpg|JPG|jpeg|JPEG

You mean "some.name.like.this.php.jpg" ??

 

Yes, that is exactly the problem.  Apache tries to treat these types of files in a special way, and the upshot of it is that it will find the .php inside the filename and apply it's php handler to it, no matter what the rest of the filename looks like.

 

-Like your previous test, make a "testfile.jpg" that has <?php phpinfo();  in it like you did before.  Upload that file and load it throug the url.  If properly configured your server will not run it -- you'll get an error.  If you get the code again, then you have a configuration issue.

 

-Debugging your upload script is something I can't do right now.  Please make another thread for it, but I have to say that if you follow the example in the php.net manual for file uploads, you should not have any problem: http://us3.php.net/manual/en/features.file-upload.php

 

Just read those pages carefully, as they go into great detail, and show you exactly what you have to do.  Speaking generally, you will get the "user's" filename as one of a number of different items in the $_FILES[] array.  It's an array of arrays, so that it can support multiple file uploads.  Do a var_dump() on the variable and examine it until you understand what it contains.

 

For detecting whether or not a file is actually an image:  http://www.php.net/manual/en/function.exif-imagetype.php

 

The crux of the htaccess rule is this:

 

 

That is a regex expression:

 

^ matches the start of the filename

\w+ matches a filename with characters, numbers and underscores in it.

\. matches one period.

(gif|jpe?g|png) matches gif or jpg or jpeg or png

$ matches the end of the line.

 

In short, this regex along with the allow deny rules, will simply not serve any file that does not match the regex, and that regex only matches files with a single period and one of the 3 image extensions.  It will not allow the serving of a file named "something.php.jpg". 

 

Regex is a super powerful tech that permeates a lot of core web dev stuff.  I seem to recall not long ago you were asking a questions that involved the use of regexes.  I know they are confusing, but you have to put in the time and study them.  There are some great tools for testing like the regex coach (pc), or similar tools if you have a mac that help.

 

 

 

 

 

 

 

 

[kicken]

One thing that confuses me is how a file can be named with a .jpg extension but run as a .php file. 

 

Apache understands multiple extensions, and for each extension it will look up the mime type, language, and handlers for that extension.  In mis-configured servers the php handler is applied to the .php extension in a way that does not require it to be the end extension.  It only has to exist in the name somewhere.  If it does not appear in the name, apache will not run the PHP handler on that file so the code will not run.  You can read the small bit in the apache docs about multiple extensions for some further details and information on why it behaves this way.

 

 

 

But the issue is not what the file was named, it is that when the file was run, the code inside the file was executed, right?

No, the issue revolves entierly around what the file was named.  If '.php' does not appear anywhere in the name, it will not be executed.  Try it your self, rename your test file to just somefile.jpg and load it.  All you'll get is a broken image (or error page, depending on how your browser handles invalid images).

 

It seems to me the issue should be Ensuring that a file is what it says it is...  (Don't let a file called "reallyIAmAPicture.jpg" actually contain executable PHP?!)

Unless I have some particular need to maintain the file name, I almost never keep the original name.  For an avatar for example, I just name them with the user's ID number.  so I'd have a directory say /images/avatars and it has all the various files which are just for example 1383.jpg or 1893.png, whatever that user's ID number is.  The only part I keep is the very last extension so that it is served with the proper mime type.  I also always verify that the final extension is one that is expected and allowed.  In the case of an image upload it has to be .jpg, .png, or .gif.

 

Even if you do need to preserve the actual file name, you can do so by saving it in your DB record as an additional column, then name the actual file something random with a harmless extension.  Rather than link directly to the file, you would use a PHP script to serve the file and it can restore the name by sending special headers to the browser before offering to download the file.  By serving the file via PHP yourself rather than via apache you also prevent PHP code from running in any case.

 

 

But when I try to upload a file I get this...

Warning: move_uploaded_file(http://local.debbie/uploads/1.png) [function.move-uploaded-file]: failed to open stream: HTTP wrapper does not support writeable connections in /Users/user1/Documents/DEV/++htdocs/05_Debbie/members/upload.php on line 30

 

The destination path for move_uploaded_file has to be a local file system path, not a URL. eg, /home/yoursite/uploads/

 

 

Right, but again, I also need to ensure that a Picture is really a Picture.  (Seems like there are lots of ways that do NOT successfully check this.)

 

getimagesize will do this.  It will return false if the image is invalid.  Your sample file that just contained the php code would not pass that check as it's not a valid image.  Some image formats allow meta-data though and that meta-data is plain text.  For example .gif has fields for a comment to indcate what program created it, or .jpg has fields for things like date, author, camera, etc.  Someone could take a valid image, and throw PHP code into these meta-data fileds.  This is why in addition to a getimagesize() format validation, you have to ensure the file cannot be run, using steps mentioned previously.

What do you mean "Do not allow for multiple extensions"?

Basically, ensure the file name only contains one '.' character.

 

If so, then that is just using PHP String Functions to check that there is only one period, right?

Yes, substr_count, but if you do like mentioned above and generate your own random file name, then just make sure you only append one extension to your file name.  What I do in the case of images is use the return value of getimagesize to determine the extension.  In the array it returns there is a field that will tell you if the image is a gif, jpg, or png so I just have a few if statements to set a $ext variable appropriately based on that field.

 

 

So in Apache (??) I should set my "uploads" directory to "Read-Only"?  (What is that in hex or whatever?)

 

It needs to be writable for your PHP scripts to save the files there.  You should just ensure no files have their executable bit set.  After the upload you can do

chmod($file, 0644);

to ensure the bit is not set.  Note the leading 0 is important and must be kept.

 

 

[scootstah]

Well, I thought I had heard in the past that a .GIF could be turned into executable malware, but I'll assume that any sources I posted earlier are around this issue of Apache running a file when it shouldn't.

 

You used to be able to embed Javascript in image files. Again, it was not limited to .gif files. This was a vulnerability in certain browsers that I am quite sure has been fixed quite a while ago.

 

You have to understand that vulnerabilities effect different things differently. Especially when you're talking something as broad as images. Lots of applications deal with images. Just because you can embed some malicious code in a file that exploits one application, it does not mean it will exploit every application it comes in contact with. You may have heard about a vulnerability in some other application involving .gif files.