Solved

Strange Characters in email sent via CDOSYS

Posted on 2013-10-24
13
354 Views
Last Modified: 2013-12-06
Hi Guys,

I have written a script which sends an email to a nominated person - similar to most send to a friend scripts - it sends the email as html - the script works fine except these "strange characters" appear at the very beginning - 

The email comprises html text stored in a text file - which is opened using the filesystemobject OpenTextFile method and stored into a variable

the resulting the variable is then merged with data from a recordset using the following function.


The file merge code:
'*****************************************************************
'Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
' params:
' fn = fn = Server.MapPath("sendtoafriend.htm")
' sql = a valid sql statement
' CustomFieldNames is a valid array with information not available in the recordset formatted a follows
'             dim customFieldNames
'             redim CustomFieldNames(2, 3)
'  	
'             CustomFieldNames(0, 0) = "SenderName"
'             CustomFieldNames(0, 1) = SenderName
'             CustomFieldNames(0, 2) = ""
'
'             CustomFieldNames(1, 0) = "HorseLink"
'             CustomFieldNames(1, 1) = qs
'             CustomFieldNames(1, 2) = ""
'*****************************************************************
Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
    Dim htmlASP
    Dim objFSO, rx  
    Dim objTemplateFile
    Dim strTemplateText
    Dim strTextToDisplay, strHtml 
    dim rs
        ' run the sql string
    call RunSQL(sql, rs)

    ' error checking on resultant recordset
    if rs.eof or rs.bof then
        ' no records
        response.write "Nothing to report"
        response.End
    end if

    ' Read in our template from a file - there's really no reason your
    ' template files have to be plain text... HTML, XML, and other text-based
    ' formats all work just as well if you've got the need to use them.
    Set objFSO = Server.CreateObject("Scripting.FileSystemObject")
    ' check and make sure the file exists
    If Not objFSO.FileExists(fn) Then
        Response.Write "Merge File Does not exist <br> " & fn
        Response.End
        Exit Function
    End If
    'open the file
    Set objTemplateFile = objFSO.OpenTextFile(fn,1,true,0) ' Server.MapPath("default.txt")
    strTemplateText = ""
    ' read the file into our variable
    strTemplateText = objTemplateFile.ReadAll
    ' close the file
    objTemplateFile.Close
    ' release the memory
    Set objTemplateFile = Nothing
    Set objFSO = Nothing
    ' grab our fieldnames and data from the recordset passed in
    Dim vFieldNames()
    Dim i
    i = 0
    ReDim vFieldNames(rs.Fields.Count, 2)
    For i = 0 To rs.Fields.Count - 1
        vFieldNames(i, 0) = rs.Fields(i).Name
        vFieldNames(i, 1) = rs.Fields(i).Value
        vFieldNames(i, 2) = rs.Fields(i).Type
    Next 

	rs.close
	set rs = nothing
	
    ' Using our loaded array
    ' replace our variables in the template file
    ' with the data from recordset
    ' variables are formatted {@FieldName} in the template 
    For i = 0 To UBound(vFieldNames)-1
          sFieldName = "{@" & vFieldNames(i, 0) & "}"
          vValue = Trim("" & vFieldNames(i, 1))
          vType = vFieldNames(i, 2)
          if dDebug = 1 then
            response.write "Number = " & i & " <b>Field</b> = " & sFieldName & " = " & vValue & " = " & vType & "<br>" 
          end if
          If vValue <> "" Then
            ' get the formatted data
             vValue = GetFormattedData(vType, vValue)
          else
      	    vValue = ""
          End If
	       ' replace vbcrlf with a <br>	
	       vValue = replace(vValue, chr(13) & chr(10), "<br>")	
          strTemplateText = Replace(strTemplateText, sFieldName, vValue, 1,-1)
    Next

     ' ok - now we do our custom fieldnames
    For i = 0 To UBound(CustomFieldNames)
      sFieldName = "{@" & CustomFieldNames(i, 0) & "}"
      vValue = Trim("" & CustomFieldNames(i, 1))
      vType = CustomFieldNames(i, 2)
      If vValue <> "" Then
        ' get the formatted data
         vValue = GetFormattedData(vType, vValue)
      End If
      strTemplateText = Replace(strTemplateText, sFieldName, vValue)
    Next    

    htmlASP = strTemplateText
    GrabHtmlFromFile = htmlASP
    htmlASP = ""

End Function


Function GetFormattedData(vType, vValue)
'	response.write vValue & " = vValue / " & vType & " = vType <br>"
	if isnumeric(vType) then
		select case clng(vType)
			case 5, 6
				GetFormattedData = vValue 'Format(vValue, "currency")
'			case 11 ' boolean
			
			case 135
				GetFormattedData = MediumDate(vValue)
'			case 202 ' string
			
			case 203 ' memo
				GetFormattedData = replace(vValue, vbcrlf, "<br>")
			case else
				GetFormattedData = vValue
		end select
	else
		GetFormattedData = vValue
	end if
End Function

Open in new window


When viewing the source of the email those characters appear at the very beginning of the text - I have viewed the email and its source in a number of different email clients and it seems that although the characters are there, only Ms Outlook displays them as that client seems to insert or convert a large part of the text to mso formatting, which otherwise doesn't affect anything else.

I have tried sending the email as plain text and the characters still present.

What I have ascertained, is if I create the merge file in plain text, such as in notepad, the characters do not appear - so I amconvinced the characters are added by my editor (MS Visual Web Developer 10 Express) and I believe those characters are hidden (ie I cant see them in the file)

How do I identify those characters so I can parse the string removing those them?

MTIA

DWE
0
Comment
Question by:dwe0608
  • 6
  • 6
13 Comments
 
LVL 21

Expert Comment

by:Dale Burrell
ID: 39599304
The most likely cause is a mis-match between your the character set used to store your file and the codepage you are using to process them.

http://msdn.microsoft.com/en-us/library/ms525789(v=vs.90).aspx
0
 
LVL 58

Expert Comment

by:Gary
ID: 39599318
The template file you are using was saved with a BOM (Byte Ordered Mark) - you need to save the template file without BOM - most text editors (like Notepad++) allow you set this to off.
0
 
LVL 1

Author Comment

by:dwe0608
ID: 39599334
the headerlines in the merge file are:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<html style="background-color:#F2F2F2;">
<head>
	<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
</head>

Open in new window


I am fairly sure that hidden characters are present though as I saved the file on my local machine in notepad and then used it and no strange characters appeared after that for that file. I created a new file on the server through MS Web Dev and the same problem occurred again ..
0
 
LVL 58

Expert Comment

by:Gary
ID: 39599338
Can you attach the file
0
 
LVL 1

Author Comment

by:dwe0608
ID: 39599339
Thanks Gary123 that pretty much confirms what I was thinking without knowing the terminology - do you know if I can turn that off in MS Dev ?
0
 
LVL 58

Expert Comment

by:Gary
ID: 39599341
Do Save As and you should have the option for UTF-8 without signature
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 1

Author Comment

by:dwe0608
ID: 39599370
Copy of file attached.

MTIA

DWE
TextFile.txt
0
 
LVL 58

Expert Comment

by:Gary
ID: 39599374
Just use the save as - see comment above.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39599375
Without BOM
TextFile.htm
0
 
LVL 1

Author Comment

by:dwe0608
ID: 39599387
is there anyway to strip the BOM through code?
0
 
LVL 58

Accepted Solution

by:
Gary earned 500 total points
ID: 39599405
You could try this.
http://www.bdragon.com/lair/2011/05/remove-byte-order-mark-bom-in-classic-asp-vbscript/

Normally you wouldn't save web files with a BOM to start with so it's not something you would really code to remove it.
0
 
LVL 1

Author Comment

by:dwe0608
ID: 39599538
Gary123 thanks for the assistance

below is the altered code that I am now using without a problem arising

'*****************************************************************
'Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
' params:
' fn = fn = Server.MapPath("sendtoafriend.htm")
' sql = a valid sql statement
' CustomFieldNames is a valid array with information not available in the recordset formatted a follows
'             dim customFieldNames
'             redim CustomFieldNames(2, 3)
'  	
'             CustomFieldNames(0, 0) = "SenderName"
'             CustomFieldNames(0, 1) = SenderName
'             CustomFieldNames(0, 2) = ""
'
'             CustomFieldNames(1, 0) = "HorseLink"
'             CustomFieldNames(1, 1) = qs
'             CustomFieldNames(1, 2) = ""
'*****************************************************************
Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
    Dim htmlASP
    Dim objFSO, rx  
    Dim objTemplateFile
    Dim strTemplateText
    Dim strTextToDisplay, strHtml 
    dim rs
        ' run the sql string
    call RunSQL(sql, rs)

    ' error checking on resultant recordset
    if rs.eof or rs.bof then
        ' no records
        response.write "Nothing to report"
        response.End
    end if

    ' Read in our template from a file - there's really no reason your
    ' template files have to be plain text... HTML, XML, and other text-based
    ' formats all work just as well if you've got the need to use them.
    Set objFSO = Server.CreateObject("Scripting.FileSystemObject")
    ' check and make sure the file exists
    If Not objFSO.FileExists(fn) Then
        Response.Write "Merge File Does not exist <br> " & fn
        Response.End
        Exit Function
    End If
    'open the file
    Set objTemplateFile = objFSO.OpenTextFile(fn,1,true,0) ' Server.MapPath("default.txt")
    strTemplateText = ""
    ' read the file into our variable
    strTemplateText = objTemplateFile.ReadAll
    ' close the file
    objTemplateFile.Close
    ' release the memory
    Set objTemplateFile = Nothing
    Set objFSO = Nothing

    'remove BOM if present http://unicode.org/faq/utf_bom.html
    If (Len(Trim(strTemplateText)) > 0) Then
      Dim AscValue : AscValue = Asc(Trim(strTemplateText))
      If ((AscValue = -15441) Or (AscValue = 239)) Then : strTemplateText = Mid(Trim(strTemplateText),4) : End If
    End If

    ' grab our fieldnames and data from the recordset passed in
    Dim vFieldNames()
    Dim i
    i = 0
    ReDim vFieldNames(rs.Fields.Count, 2)
    For i = 0 To rs.Fields.Count - 1
        vFieldNames(i, 0) = rs.Fields(i).Name
        vFieldNames(i, 1) = rs.Fields(i).Value
        vFieldNames(i, 2) = rs.Fields(i).Type
    Next 

	rs.close
	set rs = nothing
	
    ' Using our loaded array
    ' replace our variables in the template file
    ' with the data from recordset
    ' variables are formatted {@FieldName} in the template 
    For i = 0 To UBound(vFieldNames)-1
          sFieldName = "{@" & vFieldNames(i, 0) & "}"
          vValue = Trim("" & vFieldNames(i, 1))
          vType = vFieldNames(i, 2)
          if dDebug = 1 then
            response.write "Number = " & i & " <b>Field</b> = " & sFieldName & " = " & vValue & " = " & vType & "<br>" 
          end if
          If vValue <> "" Then
            ' get the formatted data
             vValue = GetFormattedData(vType, vValue)
          else
      	    vValue = ""
          End If
	       ' replace vbcrlf with a <br>	
	       vValue = replace(vValue, chr(13) & chr(10), "<br>")	
          strTemplateText = Replace(strTemplateText, sFieldName, vValue, 1,-1)
    Next

     ' ok - now we do our custom fieldnames
    For i = 0 To UBound(CustomFieldNames)
      sFieldName = "{@" & CustomFieldNames(i, 0) & "}"
      vValue = Trim("" & CustomFieldNames(i, 1))
      vType = CustomFieldNames(i, 2)
      If vValue <> "" Then
        ' get the formatted data
         vValue = GetFormattedData(vType, vValue)
      End If
      strTemplateText = Replace(strTemplateText, sFieldName, vValue)
    Next    

    htmlASP = strTemplateText
    GrabHtmlFromFile = htmlASP
    htmlASP = ""

End Function


Function GetFormattedData(vType, vValue)
'	response.write vValue & " = vValue / " & vType & " = vType <br>"
	if isnumeric(vType) then
		select case clng(vType)
			case 5, 6
				GetFormattedData = vValue 'Format(vValue, "currency")
'			case 11 ' boolean
			
			case 135
				GetFormattedData = MediumDate(vValue)
'			case 202 ' string
			
			case 203 ' memo
				GetFormattedData = replace(vValue, vbcrlf, "<br>")
			case else
				GetFormattedData = vValue
		end select
	else
		GetFormattedData = vValue
	end if
End Function

Open in new window

0
 
LVL 1

Author Closing Comment

by:dwe0608
ID: 39599540
Thanks for the help

DWE
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Have you ever needed to get an ASP script to wait for a while? I have, just to let something else happen. Or in my case, to allow other stuff to happen while I was murdering my MySQL database with an update. The Original Issue This was written…
Over the past decade, as Internet security has become a chief concern of IT professionals, one of the most common questions administrators and users ask is, “Which is more secure, SFTP or FTPS?” In short, both file transfer protocols offer a high…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now