Validating variables

[DaveyK]

I am trying to write a function that will do all the string validation on my project. I am doing this so that I can validate a charset and string length in one place, which makes it easier to maintain for me on the long run. Someone told me to do it like:

 

 

$var = mysql_real_escape_string(trim(nl2br(htmlentities($var))));
if (strlen($var) < 3 || strlen($var) > 400)
{ // do something
} 

That is roughly how I used to do it.

 

A Freaker (not sure who) told me this is wrong and that I should use htmlentities() (and probably also nl2br()) only before displaying the strings. So here I am, confused and not knowing how I should be validating strings prior to adding them to the DB.The situation I want to go to (I THINK) is something like...

 

$var = mysql_real_escape_string(trim($var));

if (!validate_string($var))
{
  // do something
}

 

As I said, I dont know if thats correct or not.

 

How do you Freakers do it? What is "best practise"? Ideally, I would just want to have UTF-8 valid strings in my DB, but I have very very little experience with this.

 

ALSO: I know Im not supposed to be using MySQL, but converting is not an option at this point. Please leave that be

Edited April 2, 2013 by DaveyK

[requinix]

That is correct.

 

There are two stages to dealing with data: getting it and displaying it. When you get the data from the user you should only try to correct for "errors" with the input. Such as there being leading or trailing space - not so much an error per se but very likely undesirable. You may trim() in that stage.

 

Then when you're displaying the data you might need to do some special things to it:

* Unless told otherwise HTML will collapse whitespace down into a single space character: if I type blank lines in a textarea (which will show up as blank lines to me, such as the one before this paragraph) and then you display it in regular HTML, all those blank lines will disappear. If you don't want that then you can use nl2br() to "fix" it. To be clear, that fixes how the text is being displayed.

* Meanwhile if that text contains HTML and you simply output it, the browser will very happily treat it like real HTML. You very likely don't want that either and that's what htmlentities() can help with: it will make sure the text that I entered is literally what I will see on the page. If I type "old" then I will see exactly "old" again - not a bolded word. If you do want people to use HTML then you should use BBCode instead: it's too difficult to provide a way to allow people to use HTML safely. (You can make the BBCode tags look like HTML though...) Again, this is a matter of how the input was being displayed which is why you also don't use this function until the last possible moment.

 

Of course between getting and displaying the data you have to store it somewhere. Since you can't use mysqli/PDO (for now ) then you need to make sure the data goes into the database safely. That's the only purpose for mysql_real_escape_string() and thus the only time you should use it.

 

So,

- trim() for the input

- mysql_real_escape_string() when you put unknown/unsafe strings into SQL queries

- htmlentities() and nl2br() (in that order if you need both) for the output

Edited April 2, 2013 by requinix

[DaveyK]

Okay, I get that. But what function would you use to validate the char set and string length?

 

Also, I dont think htmlentities(nl2br()) should be in that order, because you will create <br /> tags and then create htmlentities of the tags...

[requinix]

For the character encoding mb_detect_encoding() or at worst a regex can work. Use mb_strlen() for the string length in a multibyte encoding, otherwise the binary strlen() is fine.

Also, I dont think htmlentities(nl2br()) should be in that order, because you will create

tags and then create htmlentities of the tags...

Right. I said htmlentities() first, nl2br() second. You're demonstrating the reverse order.

[Psycho]

To expand on Requinix's post.

 

It *most* circumstances you should store data in its "original" format. In other words, do not transform the data based upon how you think you will be outputting it.

 

So, you should do things such as:

Use trim() to remove uneeded spaces

Provide error handling for invalid content (It is a bad idea to just remove invalid data without the user knowing it)

Escape data to make it safe for use in a query or other means of handling (this all depends on how you will use it. If storing in a DB then use mysqli_real_escape_string(), prepared statements, etc. If storing in a comma separated file, then you need to properly escape for commas as part of the date, etc.)

 

You should NOT modify the data before you store it based upon an intended output. So, if you plan to display the data on a web page you should normally not use htmlentites() or any other process to make the data safe for that output method before you store it. Instead, do that transition after you retrieve it and before creating the output. The reason for this is that you lose the ability to re-purpose the data for another need. For example, if you need to send the data as part of a JSON request, put into a plain-text email, etc. It would become "corrupt" for those uses.

 

So, if you do not use htmlentities() or nl2br() before storing the data (which you shouldn't), then they would not affect any string length checks you perform on the data when it is submitted.