FAQ
Is there a limit to the length of a string in perl? I'm combining about
200 pieces of data into a single string and writing the string to a file
and am getting some weird behaviors every once in a while. Does perl
have a limit on the length of a string it writes to a simple file?

Search Discussions

  • Rob Coops at Apr 8, 2010 at 12:17 pm

    On Thu, Apr 8, 2010 at 2:26 PM, Chris Coggins wrote:

    Is there a limit to the length of a string in perl? I'm combining about 200
    pieces of data into a single string and writing the string to a file and am
    getting some weird behaviors every once in a while. Does perl have a limit
    on the length of a string it writes to a simple file?

    --
    To unsubscribe, e-mail: beginners-unsubscribe@perl.org
    For additional commands, e-mail: beginners-help@perl.org
    http://learn.perl.org/

    Hi Chris,

    Perl has no string length limit. You are only limited by the amount of
    memory that is available.

    If your program is misbehaving then I fear it is the programs error (or well
    the person that wrote it ;-) rather then perl or any limit on the length of
    a string.

    On an average machine these days I would say you can store the equivalent of
    all the characters in a small library in your string before you start
    running out of memory. So I would not worry about 200 pieces of data. The
    last time I ran into a memory problem I was working with several sets of a
    few million strings stored in hashes and arrays that I was mixing and
    matching in a well not very elegant way (emergency solution needed to work
    now rather then in 5 minutes, at least that is my story and I'm sticking to
    it)... a few hundred even when they are very long should not cause an issue
    at all.

    Regards,

    Rob
  • Rene Schickbauer at Apr 12, 2010 at 11:51 am
    Hi!
    Perl has no string length limit. You are only limited by the amount of
    memory that is available.

    If your program is misbehaving then I fear it is the programs error (or well
    the person that wrote it ;-) rather then perl or any limit on the length of
    a string.
    And as for the current implementation, i think it's 2 or 4 GB (uint/sint
    index into string length?).

    But basically, if you have a single scalar of that size, the perl
    interpreters limits should be the least of your problems. If you're
    going this way, please redesign (a single, accidental copy of a string
    has the potential to bring the system to a standstill).

    But a few megabytes won't be any problem...
    ...until you do something unwise like split// and turn you scalar into a
    multi-million elements array.

    LG
    Rene
  • Owen at Apr 12, 2010 at 10:11 pm

    On Mon, 12 Apr 2010 13:52:16 +0200 Rene Schickbauer wrote:

    Hi!
    Perl has no string length limit. You are only limited by the amount
    of memory that is available.

    If your program is misbehaving then I fear it is the programs error
    (or well the person that wrote it ;-) rather then perl or any limit
    on the length of a string.
    And as for the current implementation, i think it's 2 or 4 GB
    (uint/sint index into string length?).

    But basically, if you have a single scalar of that size, the perl
    interpreters limits should be the least of your problems. If you're
    going this way, please redesign (a single, accidental copy of a
    string has the potential to bring the system to a standstill).

    But a few megabytes won't be any problem...
    ...until you do something unwise like split// and turn you scalar
    into a multi-million elements array.


    I have a file of 1s and 0's that is 15689303 bytes;

    running the program below takes about 10 seconds on reasonably fast dual
    core with 4 GByte of RAM

    ===============================================================

    #!/usr/bin/perl

    use strict;
    use warnings;

    my $rng = "/home/owen/rng_formatted";
    open (my $RNG, "<", "$rng") or die "$!\n";
    while (<$RNG>) {
    my @bits = split //;
    my $nr_bits = @bits;
    print "Number of bits in the file is $nr_bits\n";
    }

    ===============================================================
    owen@owen-desktop:~/P/Perlscripts$ perl beg1.pl
    Number of bits in the file is 15689303

    Not the best way to get the file size :-)



    Owen
  • John W. Krahn at Apr 12, 2010 at 10:26 pm

    Owen wrote:
    On Mon, 12 Apr 2010 13:52:16 +0200
    Rene Schickbauer wrote:
    Perl has no string length limit. You are only limited by the amount
    of memory that is available.

    If your program is misbehaving then I fear it is the programs error
    (or well the person that wrote it ;-) rather then perl or any limit
    on the length of a string.
    And as for the current implementation, i think it's 2 or 4 GB
    (uint/sint index into string length?).

    But basically, if you have a single scalar of that size, the perl
    interpreters limits should be the least of your problems. If you're
    going this way, please redesign (a single, accidental copy of a
    string has the potential to bring the system to a standstill).

    But a few megabytes won't be any problem...
    ...until you do something unwise like split// and turn you scalar
    into a multi-million elements array.
    I have a file of 1s and 0's that is 15689303 bytes;
    All files on a binary computer are composed of 1s and 0s. :-)

    running the program below takes about 10 seconds on reasonably fast dual
    core with 4 GByte of RAM

    ===============================================================

    #!/usr/bin/perl

    use strict;
    use warnings;

    my $rng = "/home/owen/rng_formatted";
    open (my $RNG, "<", "$rng") or die "$!\n";
    Why are you copying $rng to a string?

    perldoc -q quoting

    while (<$RNG>) {
    my @bits = split //;
    my $nr_bits = @bits;
    If you want a bit count then that should be:

    my $nr_bits = 8 * @bits;

    Although you don't need to create an array to count bytes:

    my $nr_bytes = tr///c;

    print "Number of bits in the file is $nr_bits\n";
    }
    If you want the size of a file just use the -s operator:

    print "Number of bytes in the file is ", -s $rng, "\n";

    ===============================================================
    owen@owen-desktop:~/P/Perlscripts$ perl beg1.pl
    Number of bits in the file is 15689303
    So there are no newlines in your file?

    Not the best way to get the file size :-)


    John
    --
    The programmer is fighting against the two most
    destructive forces in the universe: entropy and
    human stupidity. -- Damian Conway
  • Thomas Bätzler at Apr 8, 2010 at 12:43 pm
    Hi,

    Chris Coggins <cacoggins@cox.net> asked:
    Is there a limit to the length of a string in perl? I'm combining about
    200 pieces of data into a single string and writing the string to a
    file and am getting some weird behaviors every once in a while. Does perl
    have a limit on the length of a string it writes to a simple file?
    In Perl, string size is limited to the amount of memory available to your process, which may be less than the amount of memory installed in your computer.

    Care to elaborate on "weird behavior" for us?

    Could it be a file systems problem, i.e. writing more than 4GB to a single file on a FAT32 filesystem?

    Does the weirdness happen in the context of a web application? If yes, could it be a file locking issue?

    If it's none of the above, did you run a memtest yet?

    HTH,
    Thomas

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbeginners @
categoriesperl
postedApr 8, '10 at 12:08p
activeApr 12, '10 at 10:26p
posts6
users6
websiteperl.org

People

Translate

site design / logo © 2022 Grokbase