PHP CLI > Remove a leading block of text from string > Delete from string

Posted on 2011-10-05
Last Modified: 2012-08-13
So I have a huge 5GB string and I want to remove some leading text (from the start of the string).  How do I do that?  

For example:
$big_string     ='really_super_huge_string_but_do_not_remove_this_really_or_this_really';
$text_to_remove ='really_';

$big_string = str_replace($text_to_remove, "", $big_string, 1);

Open in new window

I thought the above would work but cannot use $big_string twice in str_replace() or we get, PHP Fatal error:  Only variables can be passed by reference

I don't want to make a copy of the $big_string variable because then I'm eating up 10GB of RAM.

Question by:Geoff Millikan
    LVL 48

    Expert Comment

    If the string is in memory, there is no way to avoid the 10GB memory usage due the internal structure how php handles variables.
    If the content is in a file you can use the fget and fwrite function to strip that part of the string with low memory usage.

    Author Comment

    by:Geoff Millikan
    ...there is no way to avoid the 10GB memory usage due the internal structure...

    Dang.  You sure?  Not possible with

    Anyway, I rewrote it so that instead of erasing data in $big_string, I just crawl forwards and backwards over it using substr().  This works well for awhile but consistently crashes while accessing the same place in $big_string with the error "zend_mm_heap corrupted" which is not really fixable per open bug report at:

    I'll try using fread() and fseek() to crawl the file on disk however past experience has shown fseek'ing past PHP_INT_MAX is really a pain.

    Are you saying I can use fwrite() to remove data from the front of the file?



    PS.  The function truncate() would work great here but it removes data from the trailing end of the file, not the front.
    LVL 10

    Assisted Solution

    $length = strlen($text_to_remove);
    $big_string = substr($big_string,$length);

    I may not understand the problem as I've never had a single variable that's taking up that much resources, but it seems like this should work without copying the large variable.
    LVL 48

    Accepted Solution

    No you read from one file and write to another file which excludes the part you don't want.

    References do not work for str_replace or substr as in the C implementation of those function the string is copied.
    LVL 13

    Assisted Solution

    by:Hugh McCurdy
    If the problem isn't solved yet, could you explain more about it?

    Why do you have this big string?  Is this a one time edit of the string or something you'll do every day (with different big strings)?

    Context is important.

    (Aside: A customer asked me to quote a project.  I looked at the project and said, that will take 2 to 3 months.  I asked what he was trying to accomplish.  He told me.  I said that I could accomplish the same business effect by doing the project a different way and I'd be done in 6 hours.  Context is important.)

    Featured Post

    Threat Intelligence Starter Resources

    Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

    Join & Write a Comment

    Both Easy and Powerful How easy is PHP? (  Very easy.  It has been described as "a programming language even my grandmother can use." How powerful is PHP?  http://en.wikiped…
    Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
    Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
    The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

    754 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    20 Experts available now in Live!

    Get 1:1 Help Now