Solved

Where can I find a JavaDKIM Example integrating MimeMessage

Posted on 2010-08-14
29
1,306 Views
Last Modified: 2012-05-10
Can anyone provide a more comprehensive example using BadPenguin's JavaDKIM (http://www.badpenguin.co.uk/dkim) to sign an outgoing JavaMail message? By more comprehensive I mean that the example Signer in the BadPenguin JavaDKIM documentation only shows signing a text input stream. I am trying to clearly understand exactly what I need to do from start to finish, including how to integrate the signed message with the MimeMessage. Am I simply signing the body of the message and passing that to mimeMessage.setText(...)? If so what parameters are used for the character set and subtype, etc.? Exactly what do I need to do differently with the MimeMessage for a DKIM signed message than for an unsigned message?

Note that I have been able to sign outboud e-mail with Agitos's DKIM for JavaMail, which more clearly integrates with the MimeMessage, but particularly larger messages signed by it are not passing on the receiving end (body hash doesn't verify); shorter messages pass. I seem to find more evidence of real implementations using BadPenguin's JavaDKIM so I thought I would try that route.
0
Comment
Question by:tandtsoftware
  • 13
  • 13
  • 2
29 Comments
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>Can anyone provide a more comprehensive example using BadPenguin's JavaDKIM (http://www.badpenguin.co.uk/dkim) to sign an outgoing JavaMail message?

Please post the example you've seen so far
0
 

Author Comment

by:tandtsoftware
Comment Utility
The examples I have seen are the one with the BadPenguin's JavaDKIM documentation (attached here) and another Q&A based on that example here within experts-exchange. Neither one addresses how you would use it in the context of the JavaMail MimeMessage though.


try {

   msg = new FileInputStream("/home/mark/mymail.txt");

   DkimSignature dksig = new DkimSignature("savage", "badpenguin.co.uk");

   dksig.setMethod(CanonicalMethod.RELAXED);

   dksig.setItag("dkim@badpenguin.co.uk");

   dksig.addHeader("received");

   Signer signer = new Signer(dksig,key);

   String header = signer.signMail(msg);

   System.out.println(header);

   msg.close();

} catch (DkimException d) {

      

      if ( d.getError().equals(DkimError.TEMPFAIL)) {

         // message failed, but may succeed later

      } else if ( d.getError().equals(DkimError.PERMFAIL)) {

         // message failed and will never verify

      } else {

         // domainkey error

   }

} catch (Exception e) {

      e.printStackTrace();

}

Open in new window

0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
0
 
LVL 92

Assisted Solution

by:objects
objects earned 167 total points
Comment Utility
you would do it the same as in the example, and generate the header from the text used to build the message

Then set the header in the message before sending

message.setHeader("DKIM-Signature", header);



0
 

Author Comment

by:tandtsoftware
Comment Utility
When you say "...the text used to build the message," I don't understand exactly what text that is.

From my example text is "msg" in the line: String header = signer.signMail(msg);
I have tried passing the body content of the message (i.e. the same thing I pass to mimeMessage.setText()), but I get DkimException: "PERMFAIL: Broken Mail Header encountered, Message rejected." From the same example "msg" is defined as new FileInputStream("/home/mark/mymail.txt"); but then what is the content of mymail.txt? I thought it would have simple been the body content of the message, but apparently not.
0
 

Author Comment

by:tandtsoftware
Comment Utility
Let me present my question another way. Given this segment of code, does it look accurate and what exactly do I pass to signMail?
import javax.mail.Session;

import javax.mail.Transport;

import javax.mail.internet.MimeMessage;

import badpenguin.dkim.Signer;

	        	

Session session;  

Transport transport;

RSAPrivateKey privateKey;

 

...

    

MimeMessage mimeMessage = new MimeMessage(session);



mimeMessage.setFrom( new InternetAddress("fromEmail@myOrganization.com") );



mimeMessage.addRecipients(RecipientType.TO, new InternetAddress[] { new InternetAddress("toEmail@yourOrganization.com") } );

	

mimeMessage.setSubject( "The Subject" );

	

mimeMessage.setText( "The message content." );

	

Signer signer = Signer("mySelector", "myOrganization.com", "rsa-sha256", privateKey);

	

String signature = signer.signMail( ? WHAT GOES HERE ? );

	

mimeMessage.setHeader("DKIM-Signature", signature);

	

transport.sendMessage(mimeMessage, mimeMessage.getAllRecipients());

 

Open in new window

0
 
LVL 92

Expert Comment

by:objects
Comment Utility
I would have thought you'd use "The message content."
whats the stack trace for the exception?

as an alternative the following handles javamail integration
http://www.agitos.de/dkim-for-java-mail-open-source-library-2.html
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>String signature = signer.signMail( ? WHAT GOES HERE ? );

Try

InputStream in = mimeMessage.getInputStream();

String signature = signer.signMail(in);

in.close();

Open in new window

0
 

Author Comment

by:tandtsoftware
Comment Utility
I have been testing with Agitos' DKIM for some time, but eecipients don't pass (fail body hash) some of its larger signed messages. That is why I am trying BadPenguin's JavaDKIM in the first place.

Using mimeMessage.getInputStream for the signMail parameter doesn't work. It is only the body content.

According to BadPenguin's documentation for Signer.signMail(msg), msg "must contain the raw email message: That means both the headers and the body as transfered in SMTP data, excluding the ending CRLFCRLF." So I am still trying to figure out how to get that from the MimeMessage.
0
 

Author Comment

by:tandtsoftware
Comment Utility
Ok, I finally got BadPenguin's signer working but it too has a problem with less-than-simple body content causing the receiving end to fail the body hash. In either case (using BadPenguin or Agitos) you have to "simulate" the header + body content of the message that is going to be sent, sign it, and add a DKIM-Signature header.

So now I am wonder whether or not the whole concept of adding a DKIM-Signature to the MimeMessage *before* the Transport (or Provider? -- whoever is actually creating the SMTP message from the MimeMessage object) has ever created the actual SMTP message is a flawed idea in the first place?? How do I know that the Transport isn't going to create that different than my DKIMSigner? Instead don't I really need to somehow intercept what the Transport actually created, calculate the hash from that, and add the DKIM-Signature?
0
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 333 total points
Comment Utility
>>So now I am wonder whether or not the whole concept ...

Your concerns are legitimate and are covered, afaics, under the topic of 'Canonicalization' in the DKIM spec. If you want to handle header canonicalization too, try message.writeTo(byteArrayOutputStream) and you can collect the headers
0
 

Author Comment

by:tandtsoftware
Comment Utility
Agitos converts a MimeMessage object to a text stream that it then then canonicalizes and then signs. BadPenguin wants to operate on text already intended to be the SMTP message, so the MimeMessage-to-text conversion I did myself (similar to what Agitos is doing internally). But both implementations, BadPenguin and Agitos, ARE doing header and body canonicalization as defined by the DKIM spec (supporting both simple and relaxed methods). My concern is even while following the canonicalization rules, whether there is still too much room for error at "guessing" what the SMTP message is going to look like BEFORE the Java SMTP Provider has ever produced a real, original SMTP message. I am suspicious because both BadPenguin and Agitos are incorrectly calculating the body hash on some (but not all) messages, but I remain open to the possibility that I am still doing something wrong.

With that in mind, is it possible to put a hook or listener that would canonicalize and sign the real SMTP message? I think such a hook would need to be AFTER the call to Transport.send(mimeMessage) but BEFORE it is actually sent out to the mail server.
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>> I think such a hook would need to be AFTER the call to Transport.send(mimeMessage) but BEFORE it is actually sent out to the mail server.

Even then, and after it's gone on the first hop, headers can be, and are, set on the message such that when it arrives at its destination, the signature will only be partial
0
 

Author Comment

by:tandtsoftware
Comment Utility
Yes but I am only signing a subset of headers (i.e. the required ones). I should add that when I don't sign with a Java solution and instead use the Perl add-on to our smtp server everything works just fine. The reason I want a Java solution is so that it won't matter which smtp server the message is originated through and I won't have to maintain DKIM as an add-on to our mail server. But so far the Java solutions have not worked.
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>Yes but I am only signing a subset of headers (i.e. the required ones).

Which ones are you using? They should normally appear if you use the approach i suggested at http:#33472952
0
 

Author Comment

by:tandtsoftware
Comment Utility
At this point with the BadPenguin implementation I am signing the headers they sign by default, from:to:subject:messager-id. I believe Agitos signs more than that by default. The problem really isn't with the headers though; it is the body content that is failing the hash on the receiving end, and I emphasize only with some messages, and that those same messages failing with the Java signers pass when signed by the Perl signer.
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>it is the body content that is failing the hash on the receiving end

That could be a normalization issue
0
 

Author Comment

by:tandtsoftware
Comment Utility
Ok, I reviewed #33472952. I am currently calling mimeMessage.getMatchingHeaderLines() and adding mimeMessage.getDataHandler().writeTo(byteArrayOutputstream) to obtain the content. Are you saying that mimeMessage.writeTo(byteArrayOutputStream) will obtain the headers plus the content as it should appear in the real SMTP message? I have not tried that call specifically yet.
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>Are you saying that mimeMessage.writeTo(byteArrayOutputStream) will obtain the headers plus the content as it should appear in the real SMTP message?

Yes
0
 

Author Comment

by:tandtsoftware
Comment Utility
Ok, I will go back and try mimeMessage.writeTo(byteArrayOutputStream). Should I expect that to have applied any normalization, or would I need to do that before I set the text into the MimeMessage object? Incidentally, does the character set specified when the content is set into the MimeMessage have any influence on normalization? I have tried specifying various character sets (e.g. UTF-8, US-ASCII, ISO-8859-1) that don't affect the outcome of the messages failing body hash.
0
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 333 total points
Comment Utility
>>Incidentally, does the character set specified when the content is set into the MimeMessage have any influence on normalization?

Yes, it does. This is covered in 5.3 of rfc4871. Try base64 at all times
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
At this point, it would be worth finding out what the Perl is doing right
0
 

Author Comment

by:tandtsoftware
Comment Utility
>> At this point, it would be worth finding out what the Perl is doing right
Perl (or any implementation on the mail server itself) has the advantage of signing the real SMTP message. It wouldn't hurt to see if the Perl implemenation is doing anything differently. If I assume the BadPenguin (or Agitos) signer is accurate, the solution may be to figure out how to mirror any encoding the SMTP Provider is doing or, even better, making sure that content is encoded well enough before it is set into the MimeMessage so that the SMTP Provider won't perform any additional encoding on it.
So far I have been using the MimeUtillity.encode methods on the headers and content before signing with BadPenguin (and Agitos does pretty much the same thing internally), however I the SMTP Transport must be doing some additional or different encoding.
It sounds like mimeMessage.writeTo(byteArrayOutputStream) will NOT have applied any such encoding but that is unclear?
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>Perl (or any implementation on the mail server itself)

Sorry - i assumed you were using Perl on the client, as with Java

>>It sounds like mimeMessage.writeTo(byteArrayOutputStream) will NOT have applied any such encoding but that is unclear?

It will only use the encoding in force at the time it's called. It won't apply any encoding i don't think
0
 

Accepted Solution

by:
tandtsoftware earned 0 total points
Comment Utility
Well, I have been able to get BadPenguin's JavaDKIM to successfully sign a MimeMessage (see the attached class). I did have to make two "fixes" to BadPenguin's source:
1. The Canonicaliser will crash with a NullPointerException if one of the headers specified to sign (h field) does not exist in the MimeMessage. According to RFC4871 section 3.5 missing headers should be ignored.
2. dkim will pass on the receiving end ONLY IF I add a preceeding CRLF to the body content being signed (this is I am "hacking" a CRLF in front of the body content in the MailMessage class but not the actual message). It seems that the JavaMail SMTP implementation is adding this precceding CRLF to the real message; JavaMail release notes claim this was fixed in release 1.4.1 and I am using release 1.4.3, so I am a bit confused about what is up with this.
So, despite getting it working, I remain concerned about this approach of trying to sign using a MimeMessage object instead of the literal stream of text that JavaMail's SMTP implementation is actually transmitting to the server. It seems to work though, for now. To emphesize my concern I point to the last paragraph in RFC4871 Section 5.3: "More generally, the signer MUST sign the message as it is expected to be received by the verifier rather than in some local or internal form."

 

public class EmailSigner extends Signer

{



	public EmailSigner(String selector, String domain, PrivateKey key) throws DkimException

	{

		super(selector, domain,

			"from:sender:reply-to:subject:message-id:to:cc:mime-version:content-type:content-transfer-encoding",

			"rsa-sha256",

			key);

		init();

	}



	private void init() throws DkimException

	{

		getDkimSignature().setHeaderMethod(CanonicalMethod.RELAXED);

		getDkimSignature().setBodyMethod(CanonicalMethod.RELAXED);

	}

	

	public void signMail(MimeMessage mimeMessage) throws IVOException

	{

		try

		{

			if (!(mimeMessage.getEncoding().equalsIgnoreCase("7bit") || 

					mimeMessage.getEncoding().equalsIgnoreCase("base64") ||

					mimeMessage.getEncoding().equalsIgnoreCase("quoted-printable")))

			{

				throw new DkimException( DkimError.PERMFAIL, "MimeMessage to sign must be encoded as one of: '7bit', 'base64', or 'quoted-printable'." ); 

			}

			

			// write the MimeMessage in its SMTP form to a byte stream

			ByteArrayOutputStream msgOutputStream = new ByteArrayOutputStream();

			mimeMessage.writeTo(msgOutputStream);

			msgOutputStream.close();

			

			// get a DKIM-Signature header for the byte stream and add it to the MimeMessage

			ByteArrayInputStream msgInputStream = new ByteArrayInputStream( msgOutputStream.toByteArray() );

			String dkimSigHeader = signMail( msgInputStream );		

			msgInputStream.close();	

			

	        mimeMessage.addHeaderLine(dkimSigHeader);

		}

		catch (Exception e)

		{

			throw new RuntimeException( "Failed to sign message.", e );

		}

	}

}

Open in new window

0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
That was a lot of work ;)
0
 

Author Comment

by:tandtsoftware
Comment Utility
I should add that the body of the MimeMessage being signed either needs to have no lines longer than around a little less than 1000 characters (I would have said 998 but it appears to need to be a little shorter than this sometimes) or have the encoding set to base64 (i.e. mimeMessage.setHeader("Content-Transfer-Encoding", "base64") prior to signing). The character set (e.g. us-ascii, utf-8, etc.) does not seem to matter so long as the transfer encoding is 7-bit.

Why there are not more and better examples of using DKIM out there I don't understand. Why is it an add-on and not a standard feature on e-mail servers? Is e-mail signing technology really being applied that often? Is anyone really rejecting or classifying e-mail as spam because it isn't signed or fails the header or body hash? For example, however Yahoo decides to block, delay, or classify e-mail as spam doesn't seem to relate to whether or not e-mail is signed, but that's a topic for another discussion. Thanks everyone for the input on this mini R&D project. :-)
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
You probably should feed all this back to BadPenguin

>>Is e-mail signing technology really being applied that often?

afaik all outgoing GMail is DKIM-signed

>>Is anyone really rejecting or classifying e-mail as spam because it isn't signed

It will probably have to gain more adoption before that happens. If MUAs did that right now, nearly everything would be marked as spam

0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
strCount chalenge 3 50
json format text only 4 65
thymeleaf natural templating vs JSP 2 23
Java JRE greater than 1.6 5 0
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now