<

Encryption for passing data securely in URLs

Published on
8,821 Points
2,121 Views
7 Endorsements
Last Modified:
Approved

Summary

In this article I outline a method of passing data in URLs as an alternative to SSL or MD5 hashes or other methods that are difficult or insecure. It offers a method of being sure that URLs being presented for processing are from recognised addresses and stops hackers or search engines from "bookmarking" data.
 

Am I reinventing the wheel?

Over the years I have come across many EE questions where someone wants to be able to pass data via a URL but is concerned about security. The URLs are often GETs as these are simpler to manage and construct than POSTs but they are also the most insecure. Often the advice is "Use an SSL certificate" but this can be expensive or difficult if a mix of domains or subdomains are used.

One alternative that I have used over many years is to encrypt my commands using a PHP class that wraps the mcrypt family of functions. I can vary the key sizes to whatever I need but I generally keep them smallish for speed and convenience. A 256 bit key will ward off most attackers and if it is not secure enough then I can up the key size to 512 or more.

This method is not intended for truly sensitive data. It is intended to provide a good level of security but like all encryption it will have its flaws.
 

Method

The key to this is the class eeEncrypt. The innards of the function need not concern us for the purposes of this article other than to say that it uses the Blowfish method of symmetric encryption so the same key is used for encryption and decryption. I chose this because it is small, fast and widely supported and is secure enough for the purposes I use it for. If you wish to know more about it there is an article about Blowfish on Wikipedia. It has two successors - Twofish and Threefish which allow more security but do not seem to be as widely supported.

Blowfish also requires an Initialisation Vector to initialise the encyrption and this does not have to be secure and can be sent with the data.

For our example, we will use a preset key of "4g23c85dMBPo4NVJ3aDPVUOPwDeaJwcL"
 
<?php

// Include the class and set the encryption key
//
include_once 'eeEncrypt.php';

$obj = new eeEncrypt();
$obj->setKey("4g23c85dMBPo4NVJ3aDPVUOPwDeaJwcL");


// Test text
//
$plainText = "The quick brown fox jumped over the lazy dog";


// Encode the data and send it to the receiving page
//
$data = $obj->urlEncrypt( $plainText );
$iv   = $obj->getInitVector();

$urlData = "data=$data&iv=$iv";

header("Location: eeTest.php?$urlData");

Open in new window

and the receiving code is in a script called eeTest.php and looks like this
 
<?php

// Include the class and instantiate it
//
include_once 'eeEncrypt.php';

$obj = new eeEncrypt(true);


// For testing, use this key
//
$key = "4g23c85dMBPo4NVJ3aDPVUOPwDeaJwcL";


// Collect the data from the URL. This will automatically
// urldecode() it so we will not be doing that manually
//
$iv = $_GET['iv'];
$data = ($_GET['data']);


// Decrypt and display the data
//
$clearText = $obj->urlDecrypt( $data,  $key, $iv );

echo "The original text was<br/><br/>$clearText";

Open in new window


Running these two scripts shows that the test string "The quick brown fox jumped over the lazy dog" survived the encryption, encoding, transmission via HTTP, decoding and decryption. This now opens the way to simple, secure data passing.

[NOTE: In the second script the constructor takes a parameter 'true' to stop it wasting time generating keys and initialisation vectors as the key is preset and the initialisation vector is supplied in the URL by the calling script]
 

So, how does it all work?

Let us say that you wish a PHP script to receive a command to delete data in a given table. It may be that the originating script issuing the request is on a different server or it may be on the same server. It makes no difference. The command to do the delete might look like this

  command=delete; id=12345

but after encryption it looks like this

  cpUxX%2F7QaW%2FeGBzA2taa9W3x6QMIG%2B%2BGKGx%2BxEnJARQ%3D

Any data can be passed like this via an HTTP GET and the nice thing is that it can be trusted because anyone attempting to fiddle with the data will corrupt the decrypted string to garbage. If you get cleartext then the you can have some confidence that the string is safe.

I often pass my data as XML as using SimpleXML makes it possible to retrieve many values and the XML is easy to construct. It lets me coordinate data across different servers where various webservices are running and I need a way of updating information via simple GETs

   http://example.com?cmd=cpUxX%2F7QaW%2FeGBzA2taa9W3x6QMIG%2B%2BGKGx%2BxEnJARQ%3D

When I decode this I make sure that it contains the correct elements with permitted values and the correct numbers of them. If there are any variations or discrepancies then I regard the data as tainted and ignore it. One useful trick is to embed a time limit on the data which automatically expires it after a certain date or time

   command=delete; id=12345; expires=12:22:00

You can prevent other systems from "bookmarking" the URLs by embedding the originating IP address in the data and comparing it with the $_SERVER[]'REMOTE_ADDR'] value and disregarding anything that differs. Because these are all encoded it is very, very difficult for a hacker or anyone else to circumvent the built-in protections.
 
$from    = "10.2.3.4";
$expires = strtotime("+5 MINUTES");
$id      = 123456;
$cmd     = "DELETE";

$data = "<packet>
            <expires>$expiry</expires>
            <from>$from</from>"
            <command>$cmd</command>
            <id>$id</id>
         </packet>";


$urlData = $obj->urlEncrypt( $data );

Open in new window

 

The downsides....

Encrypting data like this is useful but caution is still needed. In particular:
 
1) Just because it decrypted does not exempt you from being careful in your programming. If it was interfered with then it may decode to mush/garbage and your program must be able to handle that.
 

2) The more often you use a key, the less secure it becomes. Because this method is symmetric both the encrypting and decrypting program must know the key. This means that is has to be accessible to both. Any piece of text which is the correct size will do. Instead of the key I used in the example, I could have used

"Hello campers. here is a new key"

This opens the possibility for a key to change on an hourly or daily basis as it could be generated on a schedule as long as it was the correct length and across servers as long as a set method was used

3) To make this code work, you must have the mcrypt PHP modules installed and the Blowfish cipher codes.
 

The Initialisation Vector (the IV)

An important part of this encryption method is the initialisation vector. If you do not supply one then the class will generate its own. This must be sent with the data or you will be unable to decrypt the data. It is NOT a key nor does it need to be secret. It sets the encryption engine into a pre-determined state so that the decryption works. You can think of it as being like a public key even though it is not one.

In the examples above, I let the class generate its own IV and I then added it to the URL
 
$iv   = $obj->getInitVector();
$urlData = "data=$data&iv=$iv";

Open in new window


and then in the receiving code I extracted it and passed it to the class to enable the decryption to be done correctly.
 
$iv = $_GET['iv'];
...
$clearText = $obj->urlDecrypt( $data,  $key, $iv );

Open in new window


If you do not do this then the encryption will fail for two of the modes available - Cipher Block Chaining (CBC) and Cipher Feedback (CFB) - but Electronic Code Book (ECB) seems not to be bothered by it. Nonetheless I always pass the IV and thus avoid any problems.

ECB is very good at small amounts of data which is why I use it for URL data whereas CBC is better for large blocks of data such as files. CFB is used for streaming or small bursts of data.
 

And finally....

The key software to all this is the class that does all the hard work. This code is slightly re-engineered from my working code but it works and was used in the testing of the examples above.

The class was originally designed to allow it to generate its own keys if none where supplied to it, but I have never found the need to use this as I regularly autogenerate keys using a different routine. The constructor takes a boolean parameter that indicates if keys are being supplied automatically. The example key I used earlier in the article was generated using the class.

The class has four encryption functions which fall into two groups of two:
 
  • encrypt and decrypt - these were the original functions n the class and return binary strings that are not suitable for passing in URLs. However this data can be passed internally via function calls or stored as binary data (BLOBs) in MySQL tables.
     
  • urlEncrypt and urlDecrypt - these use encrypt and decrypt but make them safe for URL usage. If you are passing the data around as URL data then use these.
 
<?php
/**
 * Simple symmetric encryption and decryption
 * 
 * @author Beverley Portlock
 *
 */


class eeEncrypt {

     /**
      * The algorithm to use (default is Blowfish)
      * 
      * @var string
      * @see http://uk3.php.net/manual/en/mcrypt.ciphers.php
      */
     protected $algorithm;

     
     /**
      * The keysize
      * 
      * @var int
      */
     protected $keySize;  
     
     
     /**
      * The initialisation vector (aka the "IV")
      * 
      * @var string
      */
     protected $initVector;
     
     
     /**
      * The encryption / decryption key
      * 
      * @var string
      */
     protected $key;
     
     
     
     // internal stuff
     //
     private   $handle;                 // Internal handle for the encryption routines
     private   $ivSize;                 // Initialisation vector size (chars)
     private   $blockSize;              // The block size used by the encryption cypher

     

     // ----- Constructor -----------------------------------------------------


     /**
      * The Constructor
      * 
      * @param bool $alreadyHaveKeys Set to TRUE if you are using a pregenerated Encryption key
      * 
      */
     function __construct( $alreadyHaveKeys = false ) {

          // Defaults
          //
          $this->algorithm = MCRYPT_BLOWFISH;
          $this->keySize   = 256;
          $this->ivSize    = 8;
          $this->blockSize = 8;


          // If you are using pre-generated keys or IVs
          //
          if ( $alreadyHaveKeys ) {
               $this->initVector = "";
               $this->key = "";
          }
          else{
               // Create a new initialisation vector. This does NOT need to be a secure
               // piece of information. It does not matter if the IV is sent with the data
               // as it often is. It is used to seed the generator and has no other
               // purpose
               //
               $this->setInitVector();
     
               // Create the encryption key. This WILL need to be kept secure and should
               // NOT travel with the data
               //
               $this->setKey();
          }

     }



     // ----- Setter and getter methods ---------------------------------------

     /**
      * Set the encryption algorithm. 
      * 
      * @param int $p A symbolic constant defined in PHP
      * @see http://uk3.php.net/manual/en/mcrypt.ciphers.php 
      */
     function setAlgorithm( $p ) {
          $this->algorithm = strip_tags( $p );

     }


     /**
      * Set the IV to the supplied value, else make a new one
      * 
      * @param string $iv The initialisation vector to use (may be blank)
      * 
      * @return string The initialisation vector to use
      * 
      */
     function setInitVector( $iv="" ) {

          // If no init vector is presented then make one up
          //
          if ( $iv == "" )
               $iv = $this->makeRandomString( $this->ivSize );

               
          // Compare the init vector with the preset size. If they differ
          // then throw an error and stop
          //
     	if ( strlen($iv) != $this->ivSize )
               die("In " . __FILE__ . " at " . __LINE__ . " - Initialisation vector is wrong size (" . strlen($iv) . ") $iv");

               
          $this->initVector = $iv;
     }

     

     /**
      * Get the initialisation vector
      * 
      * @return string The currently defined IV
      * 
      */
     function getInitVector() {
     	return $this->initVector;
     }

     

     /**
      * Set the encryption key to the supplied value, else make a new one
      * 
      * @param string $k The key to use (may be blank)
      * 
      * @return string The key to use
      * 
      */
     function setKey( $k="" ) {
          
          // If no key was supplied then make one
          //
          if ( $k == "")
               $this->key = $this->makeRandomString( ( $this->keySize / 8 ) );
          else
               $this->key = $k;
     }


     /**
      * Get the currently defined key
      * 
      * @return string The encryption key
      */
     function getKey() {
     	return $this->key;
     }



     // ----- Private methods -------------------------------------------------


     // Makes a sting full of random characters
     //
     private function makeRandomString( $size ) {

          if ( ! is_numeric( $size ) )
               die( __FILE__. " at " . __LINE__ . " - Parameter is not numeric ($size)");

          $result = "";
          $data = "0123456789ABCDEFGHIJKLMNOPQRSTUVWYXZabcdefghijklmnopqrstuvwxyz";
          $dataLen = strlen( $data ) - 1;
          for ( $i=0; $i < $size; $i++ )

               $result .= $data[ mt_rand( 0, $dataLen )];

          return $result;
     }



     // ----- Public methods --------------------------------------------------



     /**
      * Encrypt some data
      * 
      * @param string $data The data to encrypt (clear text)
      * 
      * @return string The encrypted string
      * 
      */
     function encrypt( $data ) {

          // if the data's length is not a multiple of the block size then pad it out
          //
          $dataLen = strlen( $data );
          if ( ( $dataLen % $this->blockSize ) > 0  ) {
               $extra   = $this->blockSize - ( $dataLen % $this->blockSize );
               $data    = $data . str_pad(" ", $extra);
          }

          // The cypher mode is set to 'ecb'. Settings are as follows:
          //
          // - ECB (electronic codebook) is suitable for encrypting small amounts of data, such as credit card numbers.
          // - CBC (cipher block chaining) is suitable for encrypting large amounts of data, such as files.
          // - CFB (cipher feedback) is suitable for encrypting extremely small amount of data or encrypted streams
          //
          $this->handle = mcrypt_module_open($this->algorithm, '', 'ecb', '');

          mcrypt_generic_init( $this->handle, $this->key, $this->initVector );
          $encData = mcrypt_generic($this->handle, $data);
          mcrypt_generic_deinit( $this->handle );
          mcrypt_module_close( $this->handle );

          return $encData;
     }


     
     
     /**
      * Encrypt data and make it safe for URL transport
      * 
      * @param string $data The data to encrypt (clear text)
      * 
      * @return string The encrypted data encoded for HTTP
      * 
      */
     function urlEncrypt( $data ) {
          return urlencode( base64_encode( $this->encrypt( $data ) ) );
     }


     
     

     
     /**
      * Decrypt encoded data
      * 
      * @param string $encData The encoded data to convert
      * @param string $key The encryption key
      * @param string $iv The initialisation vector
      * 
      * @return string The original clear text
      * 
      */
     function decrypt( $encData, $key, $iv ) {

          $this->initVector = $iv;
          $this->key = $key;


          // Sanity checks
          //
          if ( strlen( $this->initVector) != $this->ivSize )
               die( __FILE__ . " at " . __LINE__ . " - initialisation vector corrupt ($this->initVector)");

          if ( strlen( $this->key ) != ( $this->keySize / 8 ) )
               die( __FILE__ . " at " . __LINE__ . " - decryption key corrupt ($this->key)");


               
          if ( trim($encData) != "" ) {
               $this->handle = mcrypt_module_open($this->algorithm, '', 'ecb', '');
               mcrypt_generic_init( $this->handle, $this->key, $this->initVector );
               $data = mdecrypt_generic($this->handle, $encData);
               mcrypt_generic_deinit( $this->handle );
               mcrypt_module_close( $this->handle );
          }
          else
               $data = "";

          return $data;
     }

     
     
    /**
      * Decrypt encoded data passed via HTTP
      * 
      * @param string $encData The base64 encoded data to convert
      * @param string $key The encryption key
      * @param string $iv The initialisation vector
      * 
      * @return string The original clear text
      * 
      */
     function urlDecrypt( $encData, $key, $iv) {
          return $this->decrypt( base64_decode( $encData ), $key, $iv );
     }
     

} // End of class eeEncrypt

Open in new window



 
7
Comment
  • 3
  • 3
7 Comments
 
LVL 37

Expert Comment

by:gr8gonzo
Just a thought - it's fairly common for the salt and IV parameters to be part of the data string itself (although in this case, the salt doesn't apply). You could simplify the coding by simply prepending the IV to the data string and then extracting it automatically on the receiving side (because it's a known length). This would simplify the usage of your class.
0
 
LVL 37

Expert Comment

by:gr8gonzo
As a side note, public / private keys (PKI) do not have to be expensive. You really only need a commercially-issued one when you're trying to secure a public-facing site for browser usage. You can easily generate free ones that work just as well. It might be worth a little time to develop a second class that uses the PKI methodology to encrypt/decrypt.

That said, PKI is computationally expensive the longer the data is, so in most cases, the actual encryption is AES or 3DES or Blowfish anyway, and the PKI portion is simply for encrypting/decrypting the key. So it would basically be like the class above, but your encryption key portion would involve the certificate/key files.
0
 
LVL 34

Author Comment

by:Beverley Portlock
@gr8gonzo - I had not thought about including the IV as you suggest. It is an interesting suggestion and would certainly be very simple to implement.

As for PKI I wanted keys that I could generate very simply. The original usage of this was to allow me to let differing servers communicate simple information with a key that was generated on a schedule on separate servers, sometimes in different datacentres. The main purpose was to be sure that data being exchanged was unobservable and reasonably secure from hacking. Simplicity of key generation was my driving force. Then I discovered lots of other uses.....
0
Cloud Class® Course: SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

 
LVL 37

Expert Comment

by:gr8gonzo
Out of curiosity, why didn't you just use POSTs? Long GET requests are a little more susceptible to being blocked by overzealous firewalls or security appliances, and query strings are saved into the web server's access logs, which can increase the attack surface.
0
 
LVL 34

Author Comment

by:Beverley Portlock
I needed more security than a POST would give me.

At the time I had machines scattered across a number of providers and each one needed to send and receive data that was potentially "dangerous". As one example, the accounting system would receive payments from people paying for features and would then send an update to the relevant server which would be received and queued as a text file in a secured folder restricted to a profile that could run system commands. This text file would then be processed via a BASH script which would execute system commands based on the information in the text file.

Neither GET, POST nor SSL gave enough security since they could all be used by anyone. With the encrypted data and constantly changing keys and the data packets having originating IPs and expiry times built in, I had higher confidence in the data being passed around. The last thing I wanted was data - even random stuff attempting hacks - causing the systems to alter configurations.  

Given the number of domains, subdomains and other stuff, managing SSL Certs or even self certifying was a lot of bother. I did not want to purchase extra IP addresses and manage those either such for traffic on port 443. It was much simpler to come up with a few classes and a couple of finite state machines (FSMs) and then implement those on the relevant equipment.

Data moved in the other direction as well. The central domain/account controller would regularly query each machine to obtain information on space used, changes to website data and so forth. The receiving system would receive a request and reply to its originator with encrypted information.

I did use POSTs for some of the data requests - such as obtaining an entire machine's quota information. The code was utilised by POSTs using cURL but that was not shown here simply because I was talking about encrypting URL data for GETs.

Here is a working fragment from a host that sends update information to the central accounts system and then gets information back to update the customer's information.

$x    = new BlowfishEncrypt();
$data = $x->getEncryptedXml( BlowfishKey() );
$iv   = $x->getInitVector();


// Create POST transaction via cURL
//
$fields = array(
                    "encryptedData" => urlencode( $data ),
                    "iv"   => urlencode( $iv )
               );

$ch = curl_init();

curl_setopt( $ch, CURLOPT_URL, CreateInvoiceUrl()    );
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $fields  );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_HEADER, false );


// Execute and decode the accounts system response
//
$result = curl_exec($ch);
curl_close($ch);

$parts = explode("=", $result);

$invoiceNumber = $parts[0];
$clientEmail   = $parts[1];
$amountBilled  = $parts[2];

Open in new window



That is probably more info than you wanted but I do ramble when I get started. I am sure that there are other ways of achieving the outcome we did of a distributed architecture that was independent and robust whilst keeping costs and management to a minimum, but this network, although consolidated somewhat since its peak, has proven very reliable so I am happy enough with the outcome.
0
 
LVL 35

Expert Comment

by:Slick812
greetings Beverley Portlock, , you have several statements in your presentation that do not seem correct to me, You say that the IV is used in the encryption, however in the use of the 'ecb' mode access, there is never any use of any IV, by definition the 'ecb' module does not vary from the code book (the algorithm code progression), so does not vary and change the algorithm code progression output, with an IV or anything. . . . . but In CBC mode and others, after the  algorithm code encryption, the IV is used to change the output by adding a "Random" unpredictable Initialize (IV) mode.
this line of code has the mode as  'ecb', so there is never any use of the IV at all in the output encryption.
$this->handle = mcrypt_module_open($this->algorithm, '', 'ecb', '');
you do not need to send or use an IV with the ecb mode. If you doubt me, then try it yourself.

as to effective encryption, you pad the block length with spaces, but when you decrypt, I do not see any code that removes the added spaces as you have this line -
$data = mdecrypt_generic($this->handle, $encData);

but no line that I see to remove the added spaces, the usual default for padding a block size is to add null bytes as
$data    .=  str_pad("\0", $extra);

although the PHP mcrypt will do this automatically (pad the input with null bytes to block length)
and you can use this -
$decrypted = rtrim($decrypted, chr(0));
to remove the null bytes, however, this is completely ineffective if you have non ascii input (as encrypt a binary string-file, that may have a null byte on the end).
But maybe I did not read your code correctly.
0
 
LVL 34

Author Comment

by:Beverley Portlock
@slick812 said:

You say that the IV is used in the encryption, however in the use of the 'ecb' mode access, there is never any use of any IV,

I did mention that "If you do not do this then the encryption will fail for two of the modes available - Cipher Block Chaining (CBC) and Cipher Feedback (CFB) - but Electronic Code Book (ECB) seems not to be bothered by it. Nonetheless I always pass the IV and thus avoid any problems.". It seems simpler just to pass the IV then even if I change encryption mode I do not have to worry about remembering to pass the IV as well. I just always pass it. If it is not needed then no harm is done.


....as to effective encryption, you pad the block length with spaces, but when you decrypt, I do not see any code that removes the added spaces as you have this line -

When I first wrote this some years ago, I had a lot of trouble with padding and the method I finally arrived at was to pad it with spaces. Since it worked I kept it here. As far as stripping data I routinely trim data before I use it so excess spaces never bothered me. I have never investigated altering the code much because after 7 years of use it seems stable and alteration might make it less stable.

Nonetheless, thank you for the comments. People can take your points and alter the code to improve it   further.
0

Featured Post

Introducing Cloud Class® training courses

Tech changes fast. You can learn faster. That’s why we’re bringing professional training courses to Experts Exchange. With a subscription, you can access all the Cloud Class® courses to expand your education, prep for certifications, and get top-notch instructions.

Join & Write a Comment

This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

Keep in touch with Experts Exchange

Tech news and trends delivered to your inbox every month