Solved

What Could Be Changing the Charset from utf-8 to iso-8859-1 ?

Posted on 2014-09-03
12
187 Views
Last Modified: 2014-09-11
We have a production server where the output in the TCP-IP headers is utf-8 for this URL (checked with URL Valet)

http://www.himalayanacademy.com/view/the-history-of-hindu-india_part-1_ta

But this same URL from our development server

http://dev.himalayanacademy.com/view/the-history-of-hindu-india_part-1_ta

is delivered with this in the header, and the Unicode script is not displayed correctly.

Content-Type: text/html; charset="iso-8859-1"

Now, we have  checked our frame work "RevIgniter" and other pages on the dev site are  delivered as expected with  

Content-Type: text/html; charset="utf-8"

e.g. http://dev.himalayanacademy.com/readlearn/basics

is delivered with headers: charset="utf-8"   from the same domain, using the same httpd.conf directives and the same  dynamic page assembly/delivery framework.

So... what on earth could be causing

http://dev.himalayanacademy.com/view/the-history-of-hindu-india_part-1_ta

to switch to:

Content-Type: text/html; charset="iso-8859-1"

when the production server which is a mirror of the dev site ships the same page with

Content-Type: text/html; charset="utf-8"

??

This one is a real "gremlin" (hard to debug)
0
Comment
Question by:Brahmanatha
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
  • 2
12 Comments
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 40302648
It is definitely coming from the server in the Response header.  Look in the directory that that file comes from and see if there is an .htaccess file there.  The Content-Type and Charset could be being set there.
0
 
LVL 58

Expert Comment

by:Gary
ID: 40302654
Agree with Dave, the server is sending  a page charset for the dev server
Content-Type: text/html; charset=iso-8859-1

But for the other site - which is what you would expect
Content-Type: text/html; charset=utf-8

It maybe as simple as adding to your httpd.conf
AddDefaultCharset utf-8
0
 

Author Comment

by:Brahmanatha
ID: 40302661
But this same URL from the same domain/server/public_html folder

http://dev.himalayanacademy.com/readlearn/basics

has charset=utf-8 in the response header.

I'm running a deep grep... see what pops out, this will take a while...

[root@sat ~]# cd /home/devhap/public_html
[root@sat public_html]# grep -R "charset=iso-8859-1"
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 
LVL 58

Expert Comment

by:Gary
ID: 40302666
Whats your httpd.conf setup? Something is sending the iso-8859-1 and it's not the php page
0
 

Author Comment

by:Brahmanatha
ID: 40304448
Directives for this domain are:

SuexecUserGroup "#501" "#501"
ServerName dev.himalayanacademy.com
ServerAlias www.dev.himalayanacademy.com
ServerAlias webmail.dev.himalayanacademy.com
ServerAlias admin.dev.himalayanacademy.com
DocumentRoot /home/devhap/public_html
ErrorLog /var/log/virtualmin/dev.himalayanacademy.com_error_log
CustomLog /var/log/virtualmin/dev.himalayanacademy.com_access_log combined
ScriptAlias /cgi-bin/ /home/devhap/cgi-bin/
DirectoryIndex index.irev index.shtml index.html index.htm index.php index.php4 index.php5
<Directory /home/devhap/public_html>
Options -Indexes +Includes SymLinksifOwnerMatch +ExecCGI
allow from all
0
 

Author Comment

by:Brahmanatha
ID: 40305054
My deep grep did not turn up anything out of the ordinary that could be trigger the header switch to  iso-8859-1

But one of my team notes:

http://dev.himalayanacademy.com/looklisten/videos 
# view source for the above... see the blank line at the beginning

vs.

http://www.himalayanacademy.com/looklisten/videos 

source has no blank line at the beginning.

If one of our controllers on the dev site is throwing a CRLF into standout before sending out the page...would this cause the header to default to:

Content-Type: text/html; charset=utf-8

??

It's going to take me a bit of research to discover where this blank line is coming from and exactly what is being pushed to standout before the page goes out... mysterious!
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 40305064
The fact that your response headers are including the 'charset' tells me that the server is doing this.  If it was being set in one of your HTML files, it would not show up in the response header.  This is a problem I've seen before and it is always something on the server that is doing it.  If it wasn't being done by the server, you would see just "Content-Type: text/html" without any charset.

I just checked 5 of my websites on different servers and None of them are sending a 'charset' in the response headers.  As far as I'm concerned, that's how it should be.
0
 

Author Comment

by:Brahmanatha
ID: 40305084
I use VirtualMin/WebMin for all our domains on the same box.. they are just virtual servers served from the same /home directory. There is a single httpd.conf file



/home/devhap/public_html  # where the problem is coming from
/home/himalayan/public_html  # theoretically a mirror of the above that does not have the problem

I tried adding

AddDefaultCharset utf-8

so the .htaccess file for devhap... but it doesn't help.

so when you say "the server" I'm not sure what else it could be.. the fact that there is a CRLF output on the top of only some pages at the dev site and not all pages, would suggest it might be a big in one of our controllers
0
 
LVL 83

Accepted Solution

by:
Dave Baldwin earned 500 total points
ID: 40305128
The server means Apache, the web server.  The CRLF output may indicate some other problem but it has nothing to do with this.  If Content-Type: text/html; charset="iso-8859-1" is not present in any of the Apache files, the only other place would be some server side programming file like PHP that has the ability to send headers out.  For response headers, the last one 'wins'.  So even if you put AddDefaultCharset utf-8 in .htaccess, it is possible for a PHP file (for example) to overwrite that.
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 40305133
For what it's worth, I put up a demo showing what I described at http://www.gammaboa.com/UTF8/ .
0
 

Author Comment

by:Brahmanatha
ID: 40318445
Well, yes, I can see that PHP "server side programming" -- in our case, LiveCode/RevIgniter -  is somehow generating Content-Type: text/html; charset="iso-8859-1"

Of  course this is not a solution... I have to do some back end lo-level analysis of the MCV framework we use (think CodeIgniter, but just uses a different language), but I know EE does not like just exiting the question as "no solution" so i will give Dave the points on this (EE's auto monitor is begging me to do something)

thanks for the attention give... wish me luck!
0
 

Author Closing Comment

by:Brahmanatha
ID: 40318449
C given only because there's no way for the "expert" to peer into our backend... as a result the "solution" was actually just pointing out that both Apache, httpd.conf and .htaccess are obviously not the source.
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Validating number not work with decimal 4 42
A responsive image gallery using flexbox 6 71
Chrome SSLException for localhost 1 41
Error Handler 8 38
Is your Office 365 signature not working the way you want it to? Are signature updates taking up too much of your time? Let's run through the most common problems that an IT administrator can encounter when dealing with Office 365 email signatures.
When crafting your “Why Us” page, there are a plethora of pitfalls to avoid. Follow these five tips, and you’ll be well on your way to creating an effective page.
In this tutorial viewers will learn how to code links for mobile sites that, once clicked, send a call or text to a specified number. For a telephone link (once clicked, calls a number), begin with a normal "<a href=" link tag. For the href, specify…
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question