XS question

Hi,

perhaps this is not the right list but I know the knowledge is here. So,
please be lenient.

I have a bunch of C-strings with or without a trailing \0. I want to
make them available in Perl as read-only variables. When the perl SV is
destroyed the string must not be freed obviously.

Do I need to make those variables magic? Or is there a simpler way?

If I need magic, do I need to implement all of the 8 MGVTBL members?
What if some of them are NULL?

I think I need only an svt_free function that does:

SvPVX(var) = NULL;
SvCUR(var) = 0;

Is this correct?

I have read somewhere that perl strings do not need the trailing \0
byte. But for some reasons it is normally allocated. Why? Is it
necessary? What can happen if there is no trailing \0?

Thanks,
Torsten

To answer myself, I think this does the trick:

      sv=newSV(0);
      SvUPGRADE(sv, SVt_PV);
      SvPOK_only(sv);
      SvPV_set(sv, pointer);
      SvLEN_set(sv, 0);
      SvCUR_set(sv, length);
      SvREADONLY_on(sv);

It depends on the fact that Safefree is not called if SvLEN(sv)==0. Can
I rely on that in the future?

Torsten

I think tt may be safer to also set SVf_FAKE, so that your sv will
appear as a shared SV. As such, it's pretty unlikely perl will ever try
to free the content buffer.

Vincent

I thought so but I haven't found any documentation about that flag. Is
there any? BTW, is SVf_FAKE available in 5.8.8?

Torsten

Not sure about that.  SvLEN(sv) being 0 should be enough.  Setting SVf_FAKE
marks it as a shared hash key, so some other code may later assume that it
exists in that pool when it actually doesn't.

As far as the trailing '\0' are concerned, they are not necessary for
Perl internals, but some XS modules mistakenly use SvPV() values with
C APIs that assume '\0' terminated strings (strcpy, strcmp). Perl
internals tend to provide the '\0' just outside SvCUR(len) to help
with this, without guaranteeing that you can rely on it.

It is no longer done as diligently in 5.10 as it was before; I did find
some crashes in proprietary XS code that were due to missing '\0's in
5.10 that used to be there in 5.8.  Unfortunately I didn't figure out
which parts of the core were responsible for this, so all I could do was
fix the XS code instead.

Cheers,
-Jan

Actually SvFAKE_on(sv) leads to

Assertion he->shared_he_he.hent_hek == hek failed: file "hv.c", line
2257.

Torsten

2009/7/2 Jan Dubois <jand@activestate.com>:

> As far as the trailing '\0' are concerned, they are not necessary for
> Perl internals, but some XS modules mistakenly use SvPV() values with
> C APIs that assume '\0' terminated strings (strcpy, strcmp).

Hi Jan,

I've been through that, writing "some proprietary XS code"... :)
For example, I have used strchr of a SvPVX().
If that's a mistake, what is the correct (fast?) way to,
say, search into a SV?

I would probably just use Perl_ninstr().  Unfortunately it doesn't
have a perlapi.pod entry; you have to look it up in util.c.

I doubt it makes a difference, but if you think this is still too
slow then you can always write your own strnchr() function.

Cheers,
-Jan

On Thu, Jul 02, 2009 at 07:42:08PM +0200, Torsten Foertsch wrote:

> It depends on the fact that Safefree is not called if SvLEN(sv)==0. Can
> I rely on that in the future?

Yes.

Nicholas Clark

Setting SVf_FAKE is a bad idea. It has a lot of special meanings:

http://perl5.git.perl.org/perl.git/blob/HEAD:/sv.h#l323

and isn't needed for the "Perl own the memory that SvPVX() points to"

> As far as the trailing '\0' are concerned, they are not necessary for
> Perl internals, but some XS modules mistakenly use SvPV() values with

Except there seem to be some parts deep in the regexp engine which do assume
a trailing '\0'. I consider these as bugs that need fixing. I suspect that
every use of string comparison macros and functions (as distinct from memory
comparison functions) needs auditing, to see if they should be converted to
memcmp() or similar. Which, unfortunately is a big job. But fortunately is
an incremental job.

Nicholas Clark