asked on

Strange Characters in email sent via CDOSYS

Hi Guys,

I have written a script which sends an email to a nominated person - similar to most send to a friend scripts - it sends the email as html - the script works fine except these "strange characters" appear at the very beginning - ï»¿

The email comprises html text stored in a text file - which is opened using the filesystemobject OpenTextFile method and stored into a variable

the resulting the variable is then merged with data from a recordset using the following function.

The file merge code:

'*****************************************************************
'Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
' params:
' fn = fn = Server.MapPath("sendtoafriend.htm")
' sql = a valid sql statement
' CustomFieldNames is a valid array with information not available in the recordset formatted a follows
'             dim customFieldNames
'             redim CustomFieldNames(2, 3)
'  	
'             CustomFieldNames(0, 0) = "SenderName"
'             CustomFieldNames(0, 1) = SenderName
'             CustomFieldNames(0, 2) = ""
'
'             CustomFieldNames(1, 0) = "HorseLink"
'             CustomFieldNames(1, 1) = qs
'             CustomFieldNames(1, 2) = ""
'*****************************************************************
Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
    Dim htmlASP
    Dim objFSO, rx  
    Dim objTemplateFile
    Dim strTemplateText
    Dim strTextToDisplay, strHtml 
    dim rs
        ' run the sql string
    call RunSQL(sql, rs)

    ' error checking on resultant recordset
    if rs.eof or rs.bof then
        ' no records
        response.write "Nothing to report"
        response.End
    end if

    ' Read in our template from a file - there's really no reason your
    ' template files have to be plain text... HTML, XML, and other text-based
    ' formats all work just as well if you've got the need to use them.
    Set objFSO = Server.CreateObject("Scripting.FileSystemObject")
    ' check and make sure the file exists
    If Not objFSO.FileExists(fn) Then
        Response.Write "Merge File Does not exist <br> " & fn
        Response.End
        Exit Function
    End If
    'open the file
    Set objTemplateFile = objFSO.OpenTextFile(fn,1,true,0) ' Server.MapPath("default.txt")
    strTemplateText = ""
    ' read the file into our variable
    strTemplateText = objTemplateFile.ReadAll
    ' close the file
    objTemplateFile.Close
    ' release the memory
    Set objTemplateFile = Nothing
    Set objFSO = Nothing
    ' grab our fieldnames and data from the recordset passed in
    Dim vFieldNames()
    Dim i
    i = 0
    ReDim vFieldNames(rs.Fields.Count, 2)
    For i = 0 To rs.Fields.Count - 1
        vFieldNames(i, 0) = rs.Fields(i).Name
        vFieldNames(i, 1) = rs.Fields(i).Value
        vFieldNames(i, 2) = rs.Fields(i).Type
    Next 

	rs.close
	set rs = nothing
	
    ' Using our loaded array
    ' replace our variables in the template file
    ' with the data from recordset
    ' variables are formatted {@FieldName} in the template 
    For i = 0 To UBound(vFieldNames)-1
          sFieldName = "{@" & vFieldNames(i, 0) & "}"
          vValue = Trim("" & vFieldNames(i, 1))
          vType = vFieldNames(i, 2)
          if dDebug = 1 then
            response.write "Number = " & i & " <b>Field</b> = " & sFieldName & " = " & vValue & " = " & vType & "<br>" 
          end if
          If vValue <> "" Then
            ' get the formatted data
             vValue = GetFormattedData(vType, vValue)
          else
      	    vValue = ""
          End If
	       ' replace vbcrlf with a <br>	
	       vValue = replace(vValue, chr(13) & chr(10), "<br>")	
          strTemplateText = Replace(strTemplateText, sFieldName, vValue, 1,-1)
    Next

     ' ok - now we do our custom fieldnames
    For i = 0 To UBound(CustomFieldNames)
      sFieldName = "{@" & CustomFieldNames(i, 0) & "}"
      vValue = Trim("" & CustomFieldNames(i, 1))
      vType = CustomFieldNames(i, 2)
      If vValue <> "" Then
        ' get the formatted data
         vValue = GetFormattedData(vType, vValue)
      End If
      strTemplateText = Replace(strTemplateText, sFieldName, vValue)
    Next    

    htmlASP = strTemplateText
    GrabHtmlFromFile = htmlASP
    htmlASP = ""

End Function


Function GetFormattedData(vType, vValue)
'	response.write vValue & " = vValue / " & vType & " = vType <br>"
	if isnumeric(vType) then
		select case clng(vType)
			case 5, 6
				GetFormattedData = vValue 'Format(vValue, "currency")
'			case 11 ' boolean
			
			case 135
				GetFormattedData = MediumDate(vValue)
'			case 202 ' string
			
			case 203 ' memo
				GetFormattedData = replace(vValue, vbcrlf, "<br>")
			case else
				GetFormattedData = vValue
		end select
	else
		GetFormattedData = vValue
	end if
End Function

Open in new window

When viewing the source of the email those characters appear at the very beginning of the text - I have viewed the email and its source in a number of different email clients and it seems that although the characters are there, only Ms Outlook displays them as that client seems to insert or convert a large part of the text to mso formatting, which otherwise doesn't affect anything else.

I have tried sending the email as plain text and the characters still present.

What I have ascertained, is if I create the merge file in plain text, such as in notepad, the characters do not appear - so I amconvinced the characters are added by my editor (MS Visual Web Developer 10 Express) and I believe those characters are hidden (ie I cant see them in the file)

How do I identify those characters so I can parse the string removing those them?

MTIA

DWE

Dale Burrell

The most likely cause is a mis-match between your the character set used to store your file and the codepage you are using to process them.

http://msdn.microsoft.com/en-us/library/ms525789(v=vs.90).aspx

Gary

The template file you are using was saved with a BOM (Byte Ordered Mark) - you need to save the template file without BOM - most text editors (like Notepad++) allow you set this to off.

dwe0608

ASKER

the headerlines in the merge file are:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<html style="background-color:#F2F2F2;">
<head>
	<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
</head>

Open in new window

I am fairly sure that hidden characters are present though as I saved the file on my local machine in notepad and then used it and no strange characters appeared after that for that file. I created a new file on the server through MS Web Dev and the same problem occurred again ..

Gary

Can you attach the file

dwe0608

ASKER

Thanks Gary123 that pretty much confirms what I was thinking without knowing the terminology - do you know if I can turn that off in MS Dev ?

Gary

Do Save As and you should have the option for UTF-8 without signature

dwe0608

ASKER

Copy of file attached.

MTIA

DWE
TextFile.txt

Gary

Just use the save as - see comment above.

Gary

Without BOM
TextFile.htm

dwe0608

ASKER

is there anyway to strip the BOM through code?

ASKER CERTIFIED SOLUTION

Gary

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

dwe0608

ASKER

Gary123 thanks for the assistance

below is the altered code that I am now using without a problem arising

'*****************************************************************
'Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
' params:
' fn = fn = Server.MapPath("sendtoafriend.htm")
' sql = a valid sql statement
' CustomFieldNames is a valid array with information not available in the recordset formatted a follows
'             dim customFieldNames
'             redim CustomFieldNames(2, 3)
'  	
'             CustomFieldNames(0, 0) = "SenderName"
'             CustomFieldNames(0, 1) = SenderName
'             CustomFieldNames(0, 2) = ""
'
'             CustomFieldNames(1, 0) = "HorseLink"
'             CustomFieldNames(1, 1) = qs
'             CustomFieldNames(1, 2) = ""
'*****************************************************************
Function GrabHtmlFromFile(fn, sql, CustomFieldNames)
    Dim htmlASP
    Dim objFSO, rx  
    Dim objTemplateFile
    Dim strTemplateText
    Dim strTextToDisplay, strHtml 
    dim rs
        ' run the sql string
    call RunSQL(sql, rs)

    ' error checking on resultant recordset
    if rs.eof or rs.bof then
        ' no records
        response.write "Nothing to report"
        response.End
    end if

    ' Read in our template from a file - there's really no reason your
    ' template files have to be plain text... HTML, XML, and other text-based
    ' formats all work just as well if you've got the need to use them.
    Set objFSO = Server.CreateObject("Scripting.FileSystemObject")
    ' check and make sure the file exists
    If Not objFSO.FileExists(fn) Then
        Response.Write "Merge File Does not exist <br> " & fn
        Response.End
        Exit Function
    End If
    'open the file
    Set objTemplateFile = objFSO.OpenTextFile(fn,1,true,0) ' Server.MapPath("default.txt")
    strTemplateText = ""
    ' read the file into our variable
    strTemplateText = objTemplateFile.ReadAll
    ' close the file
    objTemplateFile.Close
    ' release the memory
    Set objTemplateFile = Nothing
    Set objFSO = Nothing

    'remove BOM if present http://unicode.org/faq/utf_bom.html
    If (Len(Trim(strTemplateText)) > 0) Then
      Dim AscValue : AscValue = Asc(Trim(strTemplateText))
      If ((AscValue = -15441) Or (AscValue = 239)) Then : strTemplateText = Mid(Trim(strTemplateText),4) : End If
    End If

    ' grab our fieldnames and data from the recordset passed in
    Dim vFieldNames()
    Dim i
    i = 0
    ReDim vFieldNames(rs.Fields.Count, 2)
    For i = 0 To rs.Fields.Count - 1
        vFieldNames(i, 0) = rs.Fields(i).Name
        vFieldNames(i, 1) = rs.Fields(i).Value
        vFieldNames(i, 2) = rs.Fields(i).Type
    Next 

	rs.close
	set rs = nothing
	
    ' Using our loaded array
    ' replace our variables in the template file
    ' with the data from recordset
    ' variables are formatted {@FieldName} in the template 
    For i = 0 To UBound(vFieldNames)-1
          sFieldName = "{@" & vFieldNames(i, 0) & "}"
          vValue = Trim("" & vFieldNames(i, 1))
          vType = vFieldNames(i, 2)
          if dDebug = 1 then
            response.write "Number = " & i & " <b>Field</b> = " & sFieldName & " = " & vValue & " = " & vType & "<br>" 
          end if
          If vValue <> "" Then
            ' get the formatted data
             vValue = GetFormattedData(vType, vValue)
          else
      	    vValue = ""
          End If
	       ' replace vbcrlf with a <br>	
	       vValue = replace(vValue, chr(13) & chr(10), "<br>")	
          strTemplateText = Replace(strTemplateText, sFieldName, vValue, 1,-1)
    Next

     ' ok - now we do our custom fieldnames
    For i = 0 To UBound(CustomFieldNames)
      sFieldName = "{@" & CustomFieldNames(i, 0) & "}"
      vValue = Trim("" & CustomFieldNames(i, 1))
      vType = CustomFieldNames(i, 2)
      If vValue <> "" Then
        ' get the formatted data
         vValue = GetFormattedData(vType, vValue)
      End If
      strTemplateText = Replace(strTemplateText, sFieldName, vValue)
    Next    

    htmlASP = strTemplateText
    GrabHtmlFromFile = htmlASP
    htmlASP = ""

End Function


Function GetFormattedData(vType, vValue)
'	response.write vValue & " = vValue / " & vType & " = vType <br>"
	if isnumeric(vType) then
		select case clng(vType)
			case 5, 6
				GetFormattedData = vValue 'Format(vValue, "currency")
'			case 11 ' boolean
			
			case 135
				GetFormattedData = MediumDate(vValue)
'			case 202 ' string
			
			case 203 ' memo
				GetFormattedData = replace(vValue, vbcrlf, "<br>")
			case else
				GetFormattedData = vValue
		end select
	else
		GetFormattedData = vValue
	end if
End Function

Open in new window

dwe0608

ASKER

Thanks for the help

DWE