Solved

Concatenate wav files, encode to mp3 on the fly, and create streaming download link

Posted on 2014-04-12
9
693 Views
Last Modified: 2014-05-06
Ok, this is a big question to me but on IRC I got some people who made it sound very easy, but it's going to take some hand-holding in my case.

My requirement is a functionality that allow users to take a series of small audio files that are stored on the server in wav format, and allow them to select the files they want to use, and put them in the order that they prefer. Then, they click a download link and I start sending them an mp3 of the concatenated wav files.

So basically, files are created on-the-fly, and then sent to the browser.

I'm developing in a LAMP stack, and I'm only 6 months in to programming at all, so php has become my drug of choice.

This code is my first experimentation with creating download links.

<?php
 
$filename = 'Programming for Dumbasses.pdf'; 
header('Pragma: public');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Cache-Control: private', false); // required for certain browsers
header('Content-Type: application/text');

header('Content-Disposition: attachment; filename="'. basename($filename) . '";');
header('Content-Transfer-Encoding: binary');
header('Content-Length: ' . filesize($filename));

readfile($filename);

exit;

?>

Open in new window


and that's just something I pulled off of stackoverflow. So here's my question:

I'm assuming that I can change the content type to an mp3. Also i'm going to have to call some external binaries like sound exchange or the LAME mp3 encoder using php's exec() function.

Furthmore, I'm going to have to pull the order and identity of each sound file from the user's browser. So if he has a list in front of him, how can i send that list using ajax, while still using href = download.php, where download.php is similar to the code described above?

Here's the sequence of events as I am thinking of them.

1. the list of files is acquired, through post or get.
2. the files are confirmed, and sent to the lame or sound exchange binary with a filename for concatenation and conversion
3.the header is sent, and an ajax response  for loading is returned
4. the file is created by the external process, confirmed to exist by php, and sent to the user
5. delete the file, and smash my head on the keyboard.

Now if that sounds stupid to you, believe me it sounds stupid to me too. Ajax returns certain strings and document types etc, but I've never seen it continuously monitor the progress of a download or process outside of itself. At least not that I'm aware of.  Also, creating the file then reading it from php seems super weighty at best. On IRC I got the suggestion to "pipe to concatenated wav files into LAME, then pipe the output to a streaming download link". That sounds beautifully perfect. How the hell do I do that?

Any help whatsoever will earn you my firstborn son.
0
Comment
Question by:ChipmunkRumbleStud
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 33

Expert Comment

by:Slick812
ID: 39996241
greetings  ChipmunkRumbleStud, , ?? ?, I read your statement and question, WOW, what you describe is an entire web page development plan, to do a very complicated and maybe tricky server side operation ( several wave files together, then LAME mp3 encoder). I would say that even if you were a very very experienced web app programmer, with superior PHP, AJAX, and Javascript knowledge, along with experience in your server platform (Window, Linux, Apple) for editing Wave files (an AUDIO application command line, to do file combination) and then More experience in using a MP3 encoder (on command line) for the bitrate and the many MP3 encoding parameters, You would have a long and difficult development time and effort to do this even if you get help here and-or other places. You say - "I'm only 6 months in to programming at all", I do not feel you have the experience to tackle a difficult development task such as this.

But since you are asking here, my advice for development, is to Chop this up, and break it into "logical" development sections-parts, . For instance, you can not expect a web PHP expert to help you with the "make a mp3 of the concatenated wav files", you should take that out of consideration here, and for a Web-server development, just ask about the browser AJAX PHP interaction for your finished "on server" mp3 file. You may should break down the browser-PHP-server development into parts, like HOW to set up an effective HTML GUI to show the many "wave" files and be able to pick and order them in a user interaction (there are MANY different ways to do this), and then another development section for the browser-server communication using the AJAX exchange, (there is NOTHING easy and simple about using AJAX effectively), and maybe another development section about how to arrange the browser to get and play the server mp3 file (using a FLASH player, or HTML5 audio play, or other),. My main point here is that this is an extremely difficult undertaking, even for a very experienced Web developer, as many coding suggestions you may get here or on IRC will be way over your head and beyond your experience to use.
0
 
LVL 19

Expert Comment

by:MerijnB
ID: 39997403
From a 'functionality' point of view, concatenating wave files and sending those to lame isn't that hard to do, but I have no php experience so I don't know how to do anything of this in php.

However, with only 6 months of development under your belt, it might be somewhat difficult :)

I totally agree with Slick812, chop it up in small parts. I can give advice on the audio part, but not on how to do this in php ;)

If you're done chopping and come to preparing the audio, I'm happy to try to advice in that.
0
 

Author Comment

by:ChipmunkRumbleStud
ID: 40008289
Well, let me synthesize the question a little bit, because this is a question about which techniques to use for a service that can support say 1000 users.

There are many way to accompish this task in my view. At least on paper. For instance,  when I started to do this i did it in the way i knew how. Because I knew how use ajax,, create blobs in js, and create download URLs in js, I started by reading the wav files into binary strings, translating them into blobs in js, and the plan was to concatenate them in js into a single blob and create a download URL. I found that his took a long time just running locally on my laptop, and realized it was a very roundabout way of doing this.

If calling an external process from php to concatenate the files (which I've managed to do in the terminal on mac, but not in the xampp setup from php that i'm developing on) is about as efficient as i can expect to get, then I guess the question of which technologies to use has been solved by default, since no one thinks I'm barking up the wrong tree.

If there's no problem with that, then i guess I'll just close the question. But I think there must be issues ahead.
0
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 40008308
The first issue is that WAV files aren't just audio.  They have headers and metadata in them and they are also not all the same bit rate or compression.  http://en.wikipedia.org/wiki/WAV
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 19

Expert Comment

by:MerijnB
ID: 40008342
@Dave Badwin: I think one of the prerequisites is that all audio you want to concatenate is in a set sample rate and bit width. If you need to convert sample rates that's fine, but not on the web server ;)
If you convert up front, you (can) also eliminate headers and meta data, if these two are met, it makes it a lot easier.
0
 
LVL 33

Expert Comment

by:Slick812
ID: 40008946
????  ? ?  What?
@ChipmunkRumbleStud,  I read your last comment, and I can not ever see any reason to try and do a Browser-Javascript binary-wav file retrieve for files ON YOUR SERVER, and then try to use javascript (combine Data-Strings in your case) to produce a new binary-wav file. Javascript was NOT meant, or set up for this kind of opp, so the "took a long time" can be expected.
My view is if the wav section files, are on the server, do ALL file work (combine Data files) On the Server, , this is the ONLY way you can even hope to get this done.

u say - "But I think there must be issues ahead"
OHH, I can guarantee that!

As I said before, you need to hammer out (with frustrating head banging), one section of this development at a time.

You have done a Mac command line (terminal) "Concatenate wav files", which is a fantastic start, however getting this same operation to happen when called from PHP exec( ) , is not a given. As using the PHP to do operating system functions and executables, depends on the PHP settings and the OS settings, sometimes just getting a "User Permission" to allow a operating system to grant PHP access is a hassle. Almost all PHP default installs do NOT allow exec( )  and other dangerous OS calls, so that's something else to deal with. I have never seen a Web Host that runs the Mac OS, so you will need to deal with a Real Web PHP Host, instead of the  xampp, because on a Linux or Windows Host, your WAV executable will need to be set up, and the PHP will have to be compatible to have the exec( ) do the job for you. I really try and never to use any executables on the Host server, because of some of the difficulties you have to wrangle with. I have seen many, many questions here on EE about problems that PHP can have for dealing with OS access. I am NOT much experienced in the "Set Up" changes you need to have for PHP ini, to allow PHP OS access. So you might need to ask another question for that.
I hope some of what I have said is relevant and helpful, but without specifics (which I may know nothing about), it's difficult to talk in "generalities".
0
 

Author Comment

by:ChipmunkRumbleStud
ID: 40021607
@Slick182 I told the embarassing story of my misadventures in sending binary data through ajax in order to impress upon you my lack of understanding of what tools are used to fix what problems. I can see my message was received and understood.

That being said, I've gotten this working.

As has been previously stated, wav files can have different sampling rates, channels, and other attributes. Since all the wav files on the server had been previously entered from the browser using Recorder.js's exportWav function, I am assuming that a fair amount of variance in those attributes has been neatly avoided. Nevertheless, I did experience difficulty with the number of channels that the files had (mono or stereo), probably because I sometimes used my headphones mic and other times I did not, but also I tested this from different devices and browsers, so I havent isolated the cause yet.


Further, some wav files that had not been created with Recorder.js had different sampling rates, and I was able to create a new wav with the appropriate sampling rate with sox.

Heres the normalizing command .
exec('sox /unprocessed/one.wav -c 2 -r 44100 /processed/one.wav' )

Open in new window


except escaped of course.


I've also gotten this working in Ubuntu LAMP environment. Doing so was not difficult, the only required setting change was allowing folder permissions, which I have done for everyone, since I havent looked up what user/ process was executing the php code yet. php not currently in safe mode. If the only gripe people have with this is that it's not secure enough, please tell me why and where to look for fixes.

For now, tts still a two step process for the user: When they click "set" in the page I loop through a sortable list in jquery whose elements contain data attributes that reference primary keys in the database (which holds aliases for the files and the directory they are stored in). Then I create a POST url query in js based on those attributes, and set the url as the href for the download link. Then js enables the download link button, and step 2: the user clicks it, and sox creates their mp3. I'm not happy with the sequence yet but it does get from a to z.

Regardless, I still have know of way of knowing how to objectively look at the strengths and weaknesses of this solution, and whether implementing it in the way I have was a mistake. I opened this question to get some practical advice about what technologies to use, what to avoid, etc. I still havent gotten any indication on how I can use the sox library to perform these tasks WITHOUT calling it from php exec(). I've escaped the commands and arguments... what else am I missing? More to the point, what are my alternatives??

I also learned that Amazon has a media conversion API. While sox is doing the trick for now, this is something that people who are working with different media types on their server should investigate. I haven't been able to see if it would suit my needs yet.

If anyone could give some indication of an implementation that is more helpful than "being really careful using exec() because that's dangerous." I would love to hear it.

My code has a bunch of extraneous stuff wrapped up in it but if anyone wants to see it i can extract the relevant sections just say the word
0
 
LVL 33

Accepted Solution

by:
Slick812 earned 500 total points
ID: 40023437
You say - "php not currently in safe mode", the PHP "safe mode" has little to do with a site's security, but safe mode is used for beginner PHP coders, to help prevent beginner mistakes, I never use it and always turn it off.

As to the wav sample rates - I have found that in specific "audio" created for a site (or any other custom audio) that using only ONE audio executable to produce and edit the files, and create ALL files in the exact SAME wav format specifications, this ensures that you will have FEWER problems later in "editing" (combining wav in your case), and more importantly in using (making the mp3 for you) the files. I always use Mono for voice, even if there is more than one person talking, it makes smaller files and is easier (faster) to edit. Unfortunately there is a very large selection of bit rates and formats for wav, and a Large selection for the MP3 formats. For non-music audio, I use less bit rate, and can default to smaller bit-rate formats in MP3,and get the sound to be very acceptable. In a GUI audio editing, I find I can change the bit rate and apply audio enhancements (filters) to get clearer voice only audio and hear the new edit to see if it works.

as to - "I still havent gotten any indication on how I can use the sox library to perform these tasks WITHOUT calling it from php exec()"

the system commands like php exec() , can cause problems if incorrectly set up for it's use  , However, many do use exec(), with out problems, and you may not have another "practical" way to do this using sox. You must insure that NO UNTESTED STRING from the POST can go into the exec() command line, for security reasons. If there is a POST value that changes the  exec() command line, if you have to get the "string" from the POST choice, then you should use a pre-made array of acceptable command line strings, to insure that the POST strings are not used to "Hack" the command line.
My main point about any OS system use like  php exec() , is that it CAN be Hacked into if you allow UNTESTED STRING additions from the Browser. But you also must know enough to make sure NOT to delete or alter essential files (system or your app files). It is dangerous because of developer ignorance, but it is also useful, even necessary, for more evolved web site apps.
= = = = = = = =

you say - "Then I create a POST url query in js based on those attributes, and set the url as the href for the download link. "
What you describe sounds like a way to do this, but downloading a MP3, is different that "Playing" a MP3 on a web page.

Using a GUI as "sortable list" for the audio file section and ordering, is likely the best way, , , just my hint, avoid any "technical audio words and language" if possible, most do not know that much about it. You might not even refer to them as files, but as "statements" or "phrases".

Without any specifics, I do not know anymore to add. I can see that you have done much on this already, and have success so far, I have not used sox so I can not help you there. If you have any browser GUI, jquery, or CSS problems, then may can help.
0
 

Author Comment

by:ChipmunkRumbleStud
ID: 40046323
Aside from the security concerns that arise from calling these executables from php, I dont know of any other way to accomplish this. I guess the questions  on EE are intended to be more troubleshooting-oriented, but if anyone wants to see some of the code that I used PM me.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Author Note: Since this E-E article was originally written, years ago, formal testing has come into common use in the world of PHP.  PHPUnit (http://en.wikipedia.org/wiki/PHPUnit) and similar technologies have enjoyed wide adoption, making it possib…
OverviewThis article demonstrates a simple search form using AJAX. The purpose of the article is to demonstrate how to use the same code to render a page and javascript (JQuery) and AJAX to make subsequent calls to refine the results. The princip…
The viewer will learn how to dynamically set the form action using jQuery.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now