MySQL not outputting all Emojis in text field when using UTF8MB4
We have a project where we're storing Facebook and Twitter posts in a Mysql database, as first almost all Emojis were being stored as ?. We've since gone ahead and made some configuration changes to the database server, and since then we're starting to see more Emojis saving and appearing correctly, however some Emojis are still showing as ?, sadly I'm not sure which ones they are. I know one of them was a basket ball.
When I execute the following commend on MySQL;
SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%';
I think I've narrowed the issue down to the server not applying the init-connect that's defined in the server configuration. If I open mysql workbench and query the database, I see question marks in place of emojis, however if I run the SET NAMES query first, then the subsequent results come back showing the emojis as I expect.
MySQL Server
Last Comment
SheppardDigital
8/22/2022 - Mon
gheist
It is not executed if connecting user has SUPER privilege.
Steve Bink
As noted by gheist, init_connect does not have any effect on non-SUPER users. I very much doubt you want to give that privilege to your application.
Moreover, your client-based settings will not be enforced on every connecting client. It will only impact clients using that .cnf file. And you do need the UTF8MB4 set - some extended characters will not render in vanilla UTF8.
The solution is to have your client (i.e., the application) run SET NAMES as it initializes the database connection.
SheppardDigital
ASKER
Thanks Steve,
We're using Eloquent ORM so I'm looking to see if there's a way to call SET NAMES on every database connection.
I did speak with Rackspace and they confirmed that the user didn't have SUPER privileges, so I suspect that you are correct, the client will need to call SET NAMES.
Setting the charset and collation in Laravel's database.php configuration file to utf8mb4 (charset) and utf8mb4_unicode_ci (collation) seems to have done the trick.