<

Magic Quotes (a bad idea from day one)

Published on
46,102 Points
7,302 Views
8 Endorsements
Last Modified:
Awarded
Foreword
This article was written many years ago, in the days when PHP supported the MySQL extensionToday you would not use MySQL examples, instead choose MySQLi or PDO.  When you see mysql_function() examples here, or anywhere else, be aware that the examples are getting old, and should be considered with an historical understanding of the times, context, and modern alternatives.

Both Easy and Powerful
How easy is PHP? http://lmgtfy.com?q=how+easy+is+php  Very easy.  It has been described as "a programming language even my grandmother can use."

How powerful is PHP?  http://en.wikipedia.org/wiki/PHP  Very powerful.  But also very complex.  My grandmother is pretty smart, but I don't think she has been thinking about class abstraction or late static bindings very much.

At the confluence of easy, powerful, and complex, we find a lot of assumptions.  We assume certain things just happen automatically, like the contents of the URL GET arguments showing up in the $_GET array when we start our script.  $_GET is always there, always set, as dependable as gravity.  Unfortunately the contents of $_GET is not as dependable.  In fact, the exact same PHP script, with the exact same URL arguments can produce two different outputs.  How could this confusion have happened?  We tried to make PHP too easy.

Special Characters
The crux of the problem was this: certain characters, including quotation marks, have special meanings to certain software.  If these characters are to be inserted into a data base, they must be "escaped" (marked in a way that causes software to ignore the special meanings). Failure to do so could cause problems including failed queries, and security exposures like SQL injection.
http://php.net/manual/en/security.database.sql-injection.php

Paved With Good Intentions
We wanted a powerful and easy way to prevent these problems and the solution put forth was Magic Quotes.  It seemed like a good idea at the time.  All ' (single-quote), " (double quote), \ (backslash) and NULL characters would be escaped with a backslash automatically.  It was as if your script had used the addslashes() function on all the external data.  And it was made a default setting in PHP.  But that idea, while great for programmers who were blissfully unaware of risks like SQL injection, created portability and performance problems for others.  It also created security problems.  Programmers who had magic quotes in the development environment assumed that this was "just the way it worked."  When their code was ported to an environment without magic quotes, it suddenly became hackable and often malfunctioned.

That Sounds Really Dangerous -- How Do I Find Out If My Scripts Are At Risk?
It is really dangerous.  You can run the phpinfo() function.  Scan the output page and look for the word "magic" to see if your setting is on or off.  At run-time you can use get_magic_quotes_gpc() and you may also want to read the relevant security section of the PHP manual!

The Wrong Solution
PHP's implementation of magic quotes mungs the input data.  Example: When the URL says something like ?name=Ray, then your script finds $_GET["name"] => "Ray".  But what if your URL says ?name=O'Reilly?  With magic quotes Off what you see is what you get.  But with magic quotes On, $_GET["name"] => O\'Reilly.  An extra character has been inserted to escape the apostrophe.  This mung occurs before your script gets control.  Or else it does not, if magic quotes is off.  The implementation created a situation in which an environmental variable could change the contents of the input data.  And that is how we got into the situation that the exact same PHP script, with the exact same URL arguments could produce two different outputs.

If you're a knowledgeable programmer, you know that it is important to escape external input before using it in a query.  So you might write something like this:
$name = mysql_real_escape_string($_GET["name"]);

Open in new window

and now $name => O\\'Reilly.  When this is inserted into the data base, one of the escape characters is removed and the resulting string contains O\'Reilly.  Now your data base is polluted with escape characters.

So maybe there is a different solution?  Before calling mysql_real_escape_string() you can call the PHP function get_magic_quotes_gpc() to see if you need to call mysql_real_escape_string().  But that would entail large changes to a code base.  Automatic escape with magic quotes became a "catch-22" -- if you depended on having an environment with magic quotes, your script was in danger.  If you depended on having an environment without magic quotes, your data could be damaged.

The Right Solution
Here is what PHP has to say about magic quotes, "Warning... This feature has been DEPRECATED as of PHP 5.3.0. Relying on this feature is highly discouraged. Magic Quotes is a process that automagically escapes incoming data to the PHP script. It's preferred to code with magic quotes off (emphasis added) and to instead escape the data at runtime, as needed."
http://php.net/manual/en/security.magicquotes.php

But Not the Easy Solution
When PHP deprecated certain functions like ereg() it had a good tool to show you that you were using outdated code.  PHP issued a message of the E_DEPRECATED level.  You could find and fix those instances pretty easily.  But with magic quotes, it's not so easy.  There is nothing in your code that definitively shows that you have magic quotes on or off.  The data is munged (or not munged) before your script even starts running.  And since magic quotes was on by default, it would hardly make sense to issue DEPRECATED messages before your script started -- that would break millions of scripts.  It's a quandary.

Let's look at this part: code with magic quotes off.  How do we deal with that?  The solution is not pretty, since it requires us to change working code.  Fortunately we can simulate an environment with magic quotes Off and we can write our code to work correctly whether magic quotes is on or off in the current environment.

The first step is the hardest one for most programmers who relied on magic quotes.  You must use a code scanner or text editor to find all references to external variables and you must verify that the external variables are correctly escaped before the variables are used in a data base query.  You can use mysql_real_escape_string() or a similar function to escape the special characters.

The second step is the easier one.  You can write your code in a way that is agnostic about magic quotes.  The keys to this are to avoid any assumptions about magic quotes.  The script needs to test for magic quotes and remove magic quotes from the external inputs. Here is a script that shows how to do that.
<?php // RAY_magic_quotes.php
error_reporting(E_ALL);
echo "<pre>";

// MAGIC QUOTES ARE DEPRECATED.  MAN PAGES CONTAIN REQUIRED READING
// ALL LINKS HERE: http://php.net/manual/en/security.magicquotes.php

// A FUNCTION TO REMOVE MAGIC QUOTES AT RUN-TIME
function no_magic_quotes()
{
    // IF MAGIC QUOTES ARE NOT SET
    if (!get_magic_quotes_gpc()) return FALSE;

    // PROCESS THE EXTERNAL SUPERGLOBAL ARRAYS
    $process = array(&$_GET, &$_POST, &$_COOKIE, &$_REQUEST, &$_FILES);
    while (list($key, $val) = each($process))
    {
        foreach ($val as $k => $v)
        {
            unset($process[$key][$k]);
            if (is_array($v))
            {
                $process[$key][stripslashes($k)] = $v;
                $process[] = &$process[$key][stripslashes($k)];
            }
            else
            {
                $process[$key][stripslashes($k)] = stripslashes($v);
            }
        }
    }
    // END OF MAGIC-QUOTE REMEDIATION
    return TRUE;
}

// PREPOPULATE THE FORM - THIS COULD COME FROM A DATA BASE, OR MIGHT BE EMPTY
$my_INPUT_Field = 'ORIGINAL DATA';

// MAKE SURE WE HAVE CLEAN INPUT DATA (TRY REMOVING THIS CODE BLOCK)
if (no_magic_quotes()) echo 'MAGIC QUOTES REMOVED' . PHP_EOL;

// IF THE FORM WAS FILLED IN, COPY THE INPUT INTO OUR VARIABLE
if (!empty($_POST["my_INPUT_Field"]))
{
    $my_INPUT_Field = $_POST["my_INPUT_Field"];
}

// HAS ANYTHING BEEN POSTED - IF SO, $_POST IS SET AND CONTAINS THE DATA
if (!empty($_POST))
{
    // SHOW THE POST ARRAY
    var_dump($_POST);
}
// THIS IS THE END OF THE ACTION SCRIPT

// THIS IS THE FORM SCRIPT - FOR CONVENIENCE WE DROP OUT OF PHP INTO HTML
?>
<form method="post">
TYPE SOMETHING HERE:
<input type="text"   name="my_INPUT_Field"   value="<?php echo htmlentities($my_INPUT_Field); ?>" />
<input type="submit" name="my_SUBMIT_Button" value="go" />
</form>

Open in new window

You can take the no_magic_quotes() function out of this demonstration script and add it to your own web page scripts.  You probably have a common script that is called at the top of every page script, maybe something that establishes the session and data base connections.  You can add no_magic_quotes() to that script, and you will have created an environment that achieves what PHP recommends, to code with magic quotes off.

Double Escape and StripSlashes()
So you're a knowledgeable programmer and you know that you need to use mysql_real_escape_string() on all your external input.  And you run your script on a server that has magic quotes, and a client puts in O'Reilly.  Magic quotes turns this into O\'Reilly.  MySQL_real_escape_string() turns this into O\\'Reilly and you wind up with O\'Reilly in your data base.  Now what?  You can use stripslashes() to remove the extraneous escape character.  In fact, I've seen programming that used stripslashes() unconditionally on all fields that came out of the data base.  Probably the programmers had experienced a run-in with magic quotes and had come up with an expedient solution to data base pollution.

I have never seen an instance of stripslashes() that caused an error.   To put it simply, we don't use the backslash character in normal text.  So stripping out all of the backslash characters, while it adds a tiny amount of overhead, is almost always a good thing.  And once the data is clean and free of extraneous escape characters you can give it a single trip through mysql_real_escape_string() to prepare it for use in a query.

Future Proof and Learning Resources
One day your hosting company will upgrade your PHP software and magic quotes will suddenly go away.  If you've taken these steps now, your software will be future proof against the removal of magic quotes.

http://php.net/manual/en/security.variables.php
http://php.net/manual/en/security.magicquotes.php
http://php.net/manual/en/security.magicquotes.whynot.php
http://php.net/manual/en/security.magicquotes.disabling.php
http://php.net/manual/en/function.mysql-real-escape-string.php
http://php.net/manual/en/function.filter-var.php

Please give us your feedback!
If you found this article helpful, please click the "thumb's up" button below. Doing so lets the E-E community know what is valuable for E-E members and helps provide direction for future articles.  If you have questions or comments, please add them.  Thanks!
 
8
Comment
Author:Ray Paseur
2 Comments
 
LVL 70

Expert Comment

by:Jason C. Levine
Ray,

Excellent description of the headache that has been magic quotes and nice resolution and sample code.
0
 
LVL 1

Expert Comment

by:Imaginx
Ray - Great article.

I really like the foreach pointer to go through the super globals.

I've always used something like this that I typically just through into my database class:

 
<?php

class MySQLDatabase {
	
	private $magic_quotes_active;
	private $new_enough_string_exists;
	
	function __construct(){
		$this->magic_quotes_active = get_magic_quotes_gpc();
		$this->new_enough_string_exists = function_exists("mysql_real_escape_string");
	}
	public function escape_value($value){
		if($this->new_enough_string_exists){
				if($this->magic_quotes_active){$value=stripslashes($value);}
				$value=mysql_real_escape_string($value);
			}else{
				if(!$this->magic_quotes_active){$value=addslashes($value);}
		}
	  return $value;
	}
}

$db=new MySQLDatabase();

?>
<form method="post">
TYPE SOMETHING HERE:
<input type="text"   name="my_INPUT_Field"   value="<?php echo htmlentities($db->escape_value($_GET['name'])); ?>" />
<input type="submit" name="my_SUBMIT_Button" value="go" />
</form>

Open in new window

0

Featured Post

Upgrade your Question Security!

Add Premium security features to your question to ensure its privacy or anonymity. Learn more about your ability to control Question Security today.

Join & Write a Comment

The viewer will learn how to count occurrences of each item in an array.
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).
Suggested Courses
Course of the Month17 days, 3 hours left to enroll

Keep in touch with Experts Exchange

Tech news and trends delivered to your inbox every month