the-miz
asked on
PHP MySQL Large process
I have a website that has over 700 registered users and we attach specific users to certain jobs. After that process, we have a form that we send all these specific users an email notifying them that they have been attached to a job.
If the list is large, 100+ users, the process takes a long time and many times returns a timed out page. We are on a dedicated server and I have been told it might be best to create a cron job that is triggered every minute and checks to see if a flag is set calling for a mass email to be processed to these users.
My first question, will it degrade the performance of the server if a cron job is being ran every minute on a server even if it is not processing a large request?
Secondly, is there a way a php script can be created that calls for a cron job to be initiated? For example, currently we add the users we need emailing to a separate database list, when we are ready to send to them... a push of a button will start the cron process. Is that possible?
Third and final, other than third party mass-emailers, what is the best way of sending emails through php to a great number of users. I have tried simply cycling through a list of emails and sending one at a time and I tried batching all the emails into one BCC. Am I missing some way that is easier and more efficient on a Linux server?
If the list is large, 100+ users, the process takes a long time and many times returns a timed out page. We are on a dedicated server and I have been told it might be best to create a cron job that is triggered every minute and checks to see if a flag is set calling for a mass email to be processed to these users.
My first question, will it degrade the performance of the server if a cron job is being ran every minute on a server even if it is not processing a large request?
Secondly, is there a way a php script can be created that calls for a cron job to be initiated? For example, currently we add the users we need emailing to a separate database list, when we are ready to send to them... a push of a button will start the cron process. Is that possible?
Third and final, other than third party mass-emailers, what is the best way of sending emails through php to a great number of users. I have tried simply cycling through a list of emails and sending one at a time and I tried batching all the emails into one BCC. Am I missing some way that is easier and more efficient on a Linux server?
Wait a minute... You're looking at numbers in the hundreds (or thousands) and you're seeing slow results? Something else is wrong. Maybe the queries need to be optimized! Check my colleague's article here:
https://www.experts-exchange.com/Database/MySQL/A_1250-3-Ways-to-Speed-Up-MySQL.html
A CRON job running once a minute will only degrade server performance if the CRON job itself is slow. A once-a-minute event is nothing to modern servers.
If you need to send a lot of emails, consider using phpMailer
https://www.experts-exchange.com/Database/MySQL/A_1250-3-Ways-to-Speed-Up-MySQL.html
A CRON job running once a minute will only degrade server performance if the CRON job itself is slow. A once-a-minute event is nothing to modern servers.
If you need to send a lot of emails, consider using phpMailer
ASKER
Hundreds, currently we are getting a list with checkboxes. Normally we select all and submit form. It cycles through the list of emails and sends one email to each user through the php sendmail protocol and then adds the user to a database for later reporting (we can see what mail was successfully sent out).
Should we be doing it another way?
PHPMailer? How does this work better for mass emails?
Should we be doing it another way?
PHPMailer? How does this work better for mass emails?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
@Jonathan: You should write that answer into an EE article! Cheers, ~Ray
ASKER
That was a lot to take in gr8gonzo.
So is it possible to run a cron every minute that accesses this database of users needing emailing with a flag that equals 0, the cron first changes all users flag to 1 so the next time it fires of in 60 seconds it will not reattempt the same users. The first cron job takes all users with flag equalling 1 and begins emailing. When each is complete, it is deleted from list and added to a new list for tracking purposes later.
Sounds a bit unorthodox but would it work?
Ray,
Sounds like I would have to adjust the code in our live environment so it writes its output to a file for viewing only by administrators.
So is it possible to run a cron every minute that accesses this database of users needing emailing with a flag that equals 0, the cron first changes all users flag to 1 so the next time it fires of in 60 seconds it will not reattempt the same users. The first cron job takes all users with flag equalling 1 and begins emailing. When each is complete, it is deleted from list and added to a new list for tracking purposes later.
Sounds a bit unorthodox but would it work?
Ray,
Sounds like I would have to adjust the code in our live environment so it writes its output to a file for viewing only by administrators.
... writes its output to a file for viewing only by administrators.Yes. You can probably use error_log() to get a quick implementation.
That was a lot to take in gr8gonzo.Yeah, but a good mass mailing process isn't exactly the easiest thing in the world to build. I didn't even cover things like opt-out compliance, tracking, drip, etc... :) It's been very rare that I've seen someone want to do "just a simple mass mailing" that hasn't turned into a monster later.
I think what you've suggested with a simple 0/1 flag is probably fine. Just remember that having one flag means that you can only do one mailing to those users at a time.
Ray also correctly pointed out that your script really shouldn't time out with 700 users, but if you have low-end hardware or are on a shared server or low-end VPS, you might have some slow performance issues. You can set a higher time limit for your script by adding this to the top of your script:
set_time_limit(###);
where ### is the number of seconds until the script times out. For example, set_time_limit(600); will set the time limit to 10 minutes (600 seconds).
You should write that answer into an EE article!Or just build it out and throw it on SourceForge...
Or set up shop to compete with ConstantContact ;-)
ASKER
I tried ConstantContact and JangoMail, both gets expensive. I will let you know how my progress goes.
Second one is while you are working on mass mailing why not use a mass mailing system understanding it's features as you required.