Where can I find a JavaDKIM Example integrating MimeMessage

Can anyone provide a more comprehensive example using BadPenguin's JavaDKIM (http://www.badpenguin.co.uk/dkim) to sign an outgoing JavaMail message? By more comprehensive I mean that the example Signer in the BadPenguin JavaDKIM documentation only shows signing a text input stream. I am trying to clearly understand exactly what I need to do from start to finish, including how to integrate the signed message with the MimeMessage. Am I simply signing the body of the message and passing that to mimeMessage.setText(...)? If so what parameters are used for the character set and subtype, etc.? Exactly what do I need to do differently with the MimeMessage for a DKIM signed message than for an unsigned message?

Note that I have been able to sign outboud e-mail with Agitos's DKIM for JavaMail, which more clearly integrates with the MimeMessage, but particularly larger messages signed by it are not passing on the receiving end (body hash doesn't verify); shorter messages pass. I seem to find more evidence of real implementations using BadPenguin's JavaDKIM so I thought I would try that route.
tandtsoftwareAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

CEHJCommented:
>>Can anyone provide a more comprehensive example using BadPenguin's JavaDKIM (http://www.badpenguin.co.uk/dkim) to sign an outgoing JavaMail message?

Please post the example you've seen so far
tandtsoftwareAuthor Commented:
The examples I have seen are the one with the BadPenguin's JavaDKIM documentation (attached here) and another Q&A based on that example here within experts-exchange. Neither one addresses how you would use it in the context of the JavaMail MimeMessage though.


try {
   msg = new FileInputStream("/home/mark/mymail.txt");
   DkimSignature dksig = new DkimSignature("savage", "badpenguin.co.uk");
   dksig.setMethod(CanonicalMethod.RELAXED);
   dksig.setItag("dkim@badpenguin.co.uk");
   dksig.addHeader("received");
   Signer signer = new Signer(dksig,key);
   String header = signer.signMail(msg);
   System.out.println(header);
   msg.close();
} catch (DkimException d) {
      
      if ( d.getError().equals(DkimError.TEMPFAIL)) {
         // message failed, but may succeed later
      } else if ( d.getError().equals(DkimError.PERMFAIL)) {
         // message failed and will never verify
      } else {
         // domainkey error
   }
} catch (Exception e) {
      e.printStackTrace();
}

Open in new window

CEHJCommented:
Introduction to Web Design

Develop a strong foundation and understanding of web design by learning HTML, CSS, and additional tools to help you develop your own website.

Learn More!
Mick BarryJava DeveloperCommented:
you would do it the same as in the example, and generate the header from the text used to build the message

Then set the header in the message before sending

message.setHeader("DKIM-Signature", header);



tandtsoftwareAuthor Commented:
When you say "...the text used to build the message," I don't understand exactly what text that is.

From my example text is "msg" in the line: String header = signer.signMail(msg);
I have tried passing the body content of the message (i.e. the same thing I pass to mimeMessage.setText()), but I get DkimException: "PERMFAIL: Broken Mail Header encountered, Message rejected." From the same example "msg" is defined as new FileInputStream("/home/mark/mymail.txt"); but then what is the content of mymail.txt? I thought it would have simple been the body content of the message, but apparently not.
tandtsoftwareAuthor Commented:
Let me present my question another way. Given this segment of code, does it look accurate and what exactly do I pass to signMail?
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.MimeMessage;
import badpenguin.dkim.Signer;
	        	
Session session;  
Transport transport;
RSAPrivateKey privateKey;
 
...
    
MimeMessage mimeMessage = new MimeMessage(session);

mimeMessage.setFrom( new InternetAddress("fromEmail@myOrganization.com") );

mimeMessage.addRecipients(RecipientType.TO, new InternetAddress[] { new InternetAddress("toEmail@yourOrganization.com") } );
	
mimeMessage.setSubject( "The Subject" );
	
mimeMessage.setText( "The message content." );
	
Signer signer = Signer("mySelector", "myOrganization.com", "rsa-sha256", privateKey);
	
String signature = signer.signMail( ? WHAT GOES HERE ? );
	
mimeMessage.setHeader("DKIM-Signature", signature);
	
transport.sendMessage(mimeMessage, mimeMessage.getAllRecipients());

Open in new window

Mick BarryJava DeveloperCommented:
I would have thought you'd use "The message content."
whats the stack trace for the exception?

as an alternative the following handles javamail integration
http://www.agitos.de/dkim-for-java-mail-open-source-library-2.html
CEHJCommented:
>>String signature = signer.signMail( ? WHAT GOES HERE ? );

Try

InputStream in = mimeMessage.getInputStream();
String signature = signer.signMail(in);
in.close();

Open in new window

tandtsoftwareAuthor Commented:
I have been testing with Agitos' DKIM for some time, but eecipients don't pass (fail body hash) some of its larger signed messages. That is why I am trying BadPenguin's JavaDKIM in the first place.

Using mimeMessage.getInputStream for the signMail parameter doesn't work. It is only the body content.

According to BadPenguin's documentation for Signer.signMail(msg), msg "must contain the raw email message: That means both the headers and the body as transfered in SMTP data, excluding the ending CRLFCRLF." So I am still trying to figure out how to get that from the MimeMessage.
tandtsoftwareAuthor Commented:
Ok, I finally got BadPenguin's signer working but it too has a problem with less-than-simple body content causing the receiving end to fail the body hash. In either case (using BadPenguin or Agitos) you have to "simulate" the header + body content of the message that is going to be sent, sign it, and add a DKIM-Signature header.

So now I am wonder whether or not the whole concept of adding a DKIM-Signature to the MimeMessage *before* the Transport (or Provider? -- whoever is actually creating the SMTP message from the MimeMessage object) has ever created the actual SMTP message is a flawed idea in the first place?? How do I know that the Transport isn't going to create that different than my DKIMSigner? Instead don't I really need to somehow intercept what the Transport actually created, calculate the hash from that, and add the DKIM-Signature?
CEHJCommented:
>>So now I am wonder whether or not the whole concept ...

Your concerns are legitimate and are covered, afaics, under the topic of 'Canonicalization' in the DKIM spec. If you want to handle header canonicalization too, try message.writeTo(byteArrayOutputStream) and you can collect the headers
tandtsoftwareAuthor Commented:
Agitos converts a MimeMessage object to a text stream that it then then canonicalizes and then signs. BadPenguin wants to operate on text already intended to be the SMTP message, so the MimeMessage-to-text conversion I did myself (similar to what Agitos is doing internally). But both implementations, BadPenguin and Agitos, ARE doing header and body canonicalization as defined by the DKIM spec (supporting both simple and relaxed methods). My concern is even while following the canonicalization rules, whether there is still too much room for error at "guessing" what the SMTP message is going to look like BEFORE the Java SMTP Provider has ever produced a real, original SMTP message. I am suspicious because both BadPenguin and Agitos are incorrectly calculating the body hash on some (but not all) messages, but I remain open to the possibility that I am still doing something wrong.

With that in mind, is it possible to put a hook or listener that would canonicalize and sign the real SMTP message? I think such a hook would need to be AFTER the call to Transport.send(mimeMessage) but BEFORE it is actually sent out to the mail server.
CEHJCommented:
>> I think such a hook would need to be AFTER the call to Transport.send(mimeMessage) but BEFORE it is actually sent out to the mail server.

Even then, and after it's gone on the first hop, headers can be, and are, set on the message such that when it arrives at its destination, the signature will only be partial
tandtsoftwareAuthor Commented:
Yes but I am only signing a subset of headers (i.e. the required ones). I should add that when I don't sign with a Java solution and instead use the Perl add-on to our smtp server everything works just fine. The reason I want a Java solution is so that it won't matter which smtp server the message is originated through and I won't have to maintain DKIM as an add-on to our mail server. But so far the Java solutions have not worked.
CEHJCommented:
>>Yes but I am only signing a subset of headers (i.e. the required ones).

Which ones are you using? They should normally appear if you use the approach i suggested at http:#33472952
tandtsoftwareAuthor Commented:
At this point with the BadPenguin implementation I am signing the headers they sign by default, from:to:subject:messager-id. I believe Agitos signs more than that by default. The problem really isn't with the headers though; it is the body content that is failing the hash on the receiving end, and I emphasize only with some messages, and that those same messages failing with the Java signers pass when signed by the Perl signer.
CEHJCommented:
>>it is the body content that is failing the hash on the receiving end

That could be a normalization issue
tandtsoftwareAuthor Commented:
Ok, I reviewed #33472952. I am currently calling mimeMessage.getMatchingHeaderLines() and adding mimeMessage.getDataHandler().writeTo(byteArrayOutputstream) to obtain the content. Are you saying that mimeMessage.writeTo(byteArrayOutputStream) will obtain the headers plus the content as it should appear in the real SMTP message? I have not tried that call specifically yet.
CEHJCommented:
>>Are you saying that mimeMessage.writeTo(byteArrayOutputStream) will obtain the headers plus the content as it should appear in the real SMTP message?

Yes
tandtsoftwareAuthor Commented:
Ok, I will go back and try mimeMessage.writeTo(byteArrayOutputStream). Should I expect that to have applied any normalization, or would I need to do that before I set the text into the MimeMessage object? Incidentally, does the character set specified when the content is set into the MimeMessage have any influence on normalization? I have tried specifying various character sets (e.g. UTF-8, US-ASCII, ISO-8859-1) that don't affect the outcome of the messages failing body hash.
CEHJCommented:
>>Incidentally, does the character set specified when the content is set into the MimeMessage have any influence on normalization?

Yes, it does. This is covered in 5.3 of rfc4871. Try base64 at all times
CEHJCommented:
At this point, it would be worth finding out what the Perl is doing right
tandtsoftwareAuthor Commented:
>> At this point, it would be worth finding out what the Perl is doing right
Perl (or any implementation on the mail server itself) has the advantage of signing the real SMTP message. It wouldn't hurt to see if the Perl implemenation is doing anything differently. If I assume the BadPenguin (or Agitos) signer is accurate, the solution may be to figure out how to mirror any encoding the SMTP Provider is doing or, even better, making sure that content is encoded well enough before it is set into the MimeMessage so that the SMTP Provider won't perform any additional encoding on it.
So far I have been using the MimeUtillity.encode methods on the headers and content before signing with BadPenguin (and Agitos does pretty much the same thing internally), however I the SMTP Transport must be doing some additional or different encoding.
It sounds like mimeMessage.writeTo(byteArrayOutputStream) will NOT have applied any such encoding but that is unclear?
CEHJCommented:
>>Perl (or any implementation on the mail server itself)

Sorry - i assumed you were using Perl on the client, as with Java

>>It sounds like mimeMessage.writeTo(byteArrayOutputStream) will NOT have applied any such encoding but that is unclear?

It will only use the encoding in force at the time it's called. It won't apply any encoding i don't think
tandtsoftwareAuthor Commented:
Well, I have been able to get BadPenguin's JavaDKIM to successfully sign a MimeMessage (see the attached class). I did have to make two "fixes" to BadPenguin's source:
1. The Canonicaliser will crash with a NullPointerException if one of the headers specified to sign (h field) does not exist in the MimeMessage. According to RFC4871 section 3.5 missing headers should be ignored.
2. dkim will pass on the receiving end ONLY IF I add a preceeding CRLF to the body content being signed (this is I am "hacking" a CRLF in front of the body content in the MailMessage class but not the actual message). It seems that the JavaMail SMTP implementation is adding this precceding CRLF to the real message; JavaMail release notes claim this was fixed in release 1.4.1 and I am using release 1.4.3, so I am a bit confused about what is up with this.
So, despite getting it working, I remain concerned about this approach of trying to sign using a MimeMessage object instead of the literal stream of text that JavaMail's SMTP implementation is actually transmitting to the server. It seems to work though, for now. To emphesize my concern I point to the last paragraph in RFC4871 Section 5.3: "More generally, the signer MUST sign the message as it is expected to be received by the verifier rather than in some local or internal form."

 

public class EmailSigner extends Signer
{

	public EmailSigner(String selector, String domain, PrivateKey key) throws DkimException
	{
		super(selector, domain,
			"from:sender:reply-to:subject:message-id:to:cc:mime-version:content-type:content-transfer-encoding",
			"rsa-sha256",
			key);
		init();
	}

	private void init() throws DkimException
	{
		getDkimSignature().setHeaderMethod(CanonicalMethod.RELAXED);
		getDkimSignature().setBodyMethod(CanonicalMethod.RELAXED);
	}
	
	public void signMail(MimeMessage mimeMessage) throws IVOException
	{
		try
		{
			if (!(mimeMessage.getEncoding().equalsIgnoreCase("7bit") || 
					mimeMessage.getEncoding().equalsIgnoreCase("base64") ||
					mimeMessage.getEncoding().equalsIgnoreCase("quoted-printable")))
			{
				throw new DkimException( DkimError.PERMFAIL, "MimeMessage to sign must be encoded as one of: '7bit', 'base64', or 'quoted-printable'." ); 
			}
			
			// write the MimeMessage in its SMTP form to a byte stream
			ByteArrayOutputStream msgOutputStream = new ByteArrayOutputStream();
			mimeMessage.writeTo(msgOutputStream);
			msgOutputStream.close();
			
			// get a DKIM-Signature header for the byte stream and add it to the MimeMessage
			ByteArrayInputStream msgInputStream = new ByteArrayInputStream( msgOutputStream.toByteArray() );
			String dkimSigHeader = signMail( msgInputStream );		
			msgInputStream.close();	
			
	        mimeMessage.addHeaderLine(dkimSigHeader);
		}
		catch (Exception e)
		{
			throw new RuntimeException( "Failed to sign message.", e );
		}
	}
}

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CEHJCommented:
That was a lot of work ;)
tandtsoftwareAuthor Commented:
I should add that the body of the MimeMessage being signed either needs to have no lines longer than around a little less than 1000 characters (I would have said 998 but it appears to need to be a little shorter than this sometimes) or have the encoding set to base64 (i.e. mimeMessage.setHeader("Content-Transfer-Encoding", "base64") prior to signing). The character set (e.g. us-ascii, utf-8, etc.) does not seem to matter so long as the transfer encoding is 7-bit.

Why there are not more and better examples of using DKIM out there I don't understand. Why is it an add-on and not a standard feature on e-mail servers? Is e-mail signing technology really being applied that often? Is anyone really rejecting or classifying e-mail as spam because it isn't signed or fails the header or body hash? For example, however Yahoo decides to block, delay, or classify e-mail as spam doesn't seem to relate to whether or not e-mail is signed, but that's a topic for another discussion. Thanks everyone for the input on this mini R&D project. :-)
CEHJCommented:
You probably should feed all this back to BadPenguin

>>Is e-mail signing technology really being applied that often?

afaik all outgoing GMail is DKIM-signed

>>Is anyone really rejecting or classifying e-mail as spam because it isn't signed

It will probably have to gain more adoption before that happens. If MUAs did that right now, nearly everything would be marked as spam

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.