How can I use a $_SESSION[EMail] variable for the send to email

I'm using a php script to send out an email.
on the line:" $mail->addAddress('josh@example.net');  // Add a recipient", I want to use the $_SESSION[EMail] variable value to replace "josh@example.net".
 I've tried  $mail->addAddress('$_SESSION[EMail]') and the email gets sent to the cc and bcc recipients but not the main recipient which just shows up empty in email.

Here is an example code:

$mail = new PHPMailer;

$mail->isSMTP();                                      // Set mailer to use SMTP
$mail->Host = 'smtp1.example.com;smtp2.example.com';  // Specify main and backup server
$mail->SMTPAuth = true;                               // Enable SMTP authentication
$mail->Username = 'jswan';                            // SMTP username
$mail->Password = 'secret';                           // SMTP password
$mail->SMTPSecure = 'tls';                            // Enable encryption, 'ssl' also accepted

$mail->From = 'from@example.com';
$mail->FromName = 'Mailer';
$mail->addAddress('josh@example.net');  // Add a recipient
$mail->addAddress('ellen@example.com');               // Name is optional
$mail->addReplyTo('info@example.com', 'Information');
$mail->addCC('cc@example.com');
$mail->addBCC('bcc@example.com');