FAQ
Hi everyone,

Leo Lapworth (Ranguard) is working on the www.cpan.org website in the
same way he skinned many of the perl.org websites. He's a bit shy so he
asked that I introduce him since I dumped the task on him. :)

Right now the goal is merely to apply the new skin, get the existing
site into source control with the Perl NOC setup, and deploy it on the
new home whenever that is ready.

Once we've made the switch, we are argue about what the site should be,
but during the switch let's not distract too many people with too many
tasks. :)

------

Now, having said to hold your horses...

Here are some things for the next step, though. Some of these things
people can work on locally and integrate after the switch. Instead of
dreaming about what could be, delivering working code will win the day.

I think David Golden and Adam Kennedy already have the stuff to handle
the mirror discovery, reporting, lists, etc.

I don't know what is generating the http://www.cpan.org/RECENT page
right now, but PAUSE now has a lot of files to show recent activity for
various time slices. Anyone interested in doing something with that,
maybe even twitter-feed like? Someone had a "subscribe to this distro"
feature. I seem to recall that it might have been part of Perlbuzz.

I'm interested in updating and handling the CPAN FAQ.

Who's going to take care of the donations for the gift basket for
Jarkko and Elaine? Does anyone have a tractor trailer we can use to
deliver it?

Search Discussions

  • David Golden at Sep 27, 2010 at 12:40 am

    On Sun, Sep 26, 2010 at 4:55 PM, brian d foy wrote:
    Leo Lapworth (Ranguard) is working on the www.cpan.org website in the
    same way he skinned many of the perl.org websites. He's a bit shy so he
    asked that I introduce him since I dumped the task on him. :)
    Ranguard++

    Really, Leo (are you on this list now?) your work is awesome. You
    have no need to be shy about it.
    Here are some things for the next step, though. Some of these things
    people can work on locally and integrate after the switch. Instead of
    dreaming about what could be, delivering working code will win the day.

    I think David Golden and Adam Kennedy already have the stuff to handle
    the mirror discovery, reporting, lists, etc.
    As usual, I think we have slightly different variations. I'm more
    interested in getting a proper tiering structure in place and a way
    for people to register themselves on a tier. How we syndicate is (a)
    easy if we just use the old MIRRORED.BY that old clients support and
    (b) half-easy if we have a new way to publish information that future
    clients can support.
    I don't know what is generating the http://www.cpan.org/RECENT page
    right now, but PAUSE now has a lot of files to show recent activity for
    various time slices.  Anyone interested in doing something with that,
    maybe even twitter-feed like? Someone had a "subscribe to this distro"
    feature. I seem to recall that it might have been part of Perlbuzz.
    There are lots of these out there already. Can we just steal good
    stuff and consolidate? E.g. http://frepan.64p.org/about

    -- David
  • Ask Bjørn Hansen at Sep 27, 2010 at 2:31 am

    On Sep 26, 2010, at 17:40, David Golden wrote:

    I'm more interested in getting a proper tiering structure in place and a way
    for people to register themselves on a tier.
    Just start with:

    Tier 1 = handpicked mirrors using the instant sync thing; tier 2 = everything else.
    How we syndicate is (a) easy if we just use the old MIRRORED.BY that old clients support and (b) half-easy if we have a new way to publish information that future
    clients can support.
    For new clients I'd really like if we put the discovery mechanism on a separate hostname from the regular mirrors (a copy of the files should be on the mirrors, too).

    Related notes: would it make sense to sign the (timestamped) list of mirrors?
  • Adam Kennedy at Sep 27, 2010 at 2:37 am

    On Mon, Sep 27, 2010 at 12:31 PM, Ask Bjørn Hansen wrote:
    Related notes: would it make sense to sign the (timestamped) list of mirrors?
    I added some heuristic anti-hijacking stuff to a previous version of
    Mirror::JSON so it would at least provide a basic level of projection.

    Basically, if it sees the master server change from the last version
    it has stored (or the default one it is bundled with) it will try to
    contact additional mirrors to confirm the master change.

    That wouldn't fix a DNS hijack of course, but it would hopefully mean
    you have to compromise several mirrors and not just one.

    WRT to signing, I fear the crypto bootstrapping issues we'd suffer.

    However, in the same vein as index multi-formatting, perhaps we could
    make it optional.

    One signed, and one unsigned. Client picks the level of safety they want.

    Adam K
  • David Golden at Sep 27, 2010 at 2:37 am

    On Sun, Sep 26, 2010 at 10:31 PM, Ask Bjørn Hansen wrote:
    On Sep 26, 2010, at 17:40, David Golden wrote:

    I'm more interested in getting a proper tiering structure in place and a way
    for people to register themselves on a tier.
    Just start with:

    Tier 1 = handpicked mirrors using the instant sync thing; tier 2 = everything else.
    Tier 1 are the handpicked mirrors that Andreas has authorized for fast
    rsync. Tier 2 is everything syncing from them. Then "Tier X" is
    everything else that is slow. :-)
    How we syndicate is (a) easy if we just use the old MIRRORED.BY that old clients support and (b) half-easy if we have a new way to publish information that future
    clients can support.
    For new clients I'd really like if we put the discovery mechanism on a separate hostname from the regular mirrors (a copy of the files should be on the mirrors, too). +1
    Related notes: would it make sense to sign the (timestamped) list of mirrors?
    No. I'd rather avoid signing things since there are easier CPAN
    attack vectors anyway and checking signatures requires extra, non-core
    crypto stuff. (Unless you mean stick MD5/SHA1 of the mirror list on
    an "authoritative" URI somewhere, in which case I have no objections.)

    -- David
  • David E. Wheeler at Sep 27, 2010 at 4:45 am

    On Sep 26, 2010, at 7:31 PM, Ask Bjørn Hansen wrote:

    How we syndicate is (a) easy if we just use the old MIRRORED.BY that old clients support and (b) half-easy if we have a new way to publish information that future
    clients can support.
    For new clients I'd really like if we put the discovery mechanism on a separate hostname from the regular mirrors (a copy of the files should be on the mirrors, too).

    Related notes: would it make sense to sign the (timestamped) list of mirrors?
    For PGXN I basically took MIRRORED.BY and turned it into a JSON file.

    David
  • Leo Lapworth at Sep 27, 2010 at 6:44 am

    On 27 September 2010 01:40, David Golden wrote:
    On Sun, Sep 26, 2010 at 4:55 PM, brian d foy wrote:
    Leo Lapworth (Ranguard) is working on the www.cpan.org website in the
    same way he skinned many of the perl.org websites.
    Really, Leo (are you on this list now?) your work is awesome. You
    have no need to be shy about it.

    Thanks :)

    I've got an INITIAL design (really more a structure for cleaning up
    content), based off of Robrts http://cpan.perl.org page:

    http://skitch.com/ranguard/d2x6q/the-comprehensive-perl-archive-network

    I've gone through the dozen or so pages just sorting out the HTML etc and
    porting into the Perl.org web system. As has been mentioned, there are far
    more significant sysadmin issues than the web front end, so I'm not
    currently looking for feedback as I know it can be improved :)

    I'm seeing this as a step on a migration path. This steps goal being have
    something which we can go live with.

    If there are any glaring errors do drop me a line and I'll be happy to look
    into it.

    Cheers

    Leo
  • David Golden at Sep 27, 2010 at 11:26 am

    On Mon, Sep 27, 2010 at 2:44 AM, Leo Lapworth wrote:
    As has been mentioned, there are far
    more significant sysadmin issues than the web front end, so I'm not
    currently looking for feedback as I know it can be improved :)
    Despite waving off feedback, I'll give you mine anyway. :-)

    As a short-term cleanup, I think you're on the right path. Longer
    term, I'd love to see more convergence between *all* the sites, e.g.
    the "download/explore" bar from news.perlfoundation.org, which rocks.

    -- David
  • Nadim khemir at Sep 27, 2010 at 9:46 am
    Leo has done a terrific job before and we all know he's going to do it again.
    Ranguard++ indeed.

    N.
  • Chris 'BinGOs' Williams at Sep 27, 2010 at 2:41 pm

    On Sun, Sep 26, 2010 at 01:55:36PM -0700, brian d foy wrote:

    I don't know what is generating the http://www.cpan.org/RECENT page
    right now, but PAUSE now has a lot of files to show recent activity for
    various time slices. Anyone interested in doing something with that,
    maybe even twitter-feed like? Someone had a "subscribe to this distro"
    feature. I seem to recall that it might have been part of Perlbuzz.
    RECENT is a symlink to indices/RECENT-print

    I'll just point out that my CPAN Testing infrastructure relies on
    the RECENT file as is to find stuff to smoke test.

    ( Not that it can't be changed, just pointing out so I can have some
    warning so I don't have a WTF moment if it does ).

    Cheers,

    --
    Chris Williams
    aka BinGOs
    PGP ID 0x4658671F
    http://www.gumbynet.org.uk
    ==========================
  • David Cantrell at Sep 27, 2010 at 3:17 pm

    On Mon, Sep 27, 2010 at 03:41:24PM +0100, Chris 'BinGOs' Williams wrote:
    On Sun, Sep 26, 2010 at 01:55:36PM -0700, brian d foy wrote:
    I don't know what is generating the http://www.cpan.org/RECENT page
    right now, but PAUSE now has a lot of files to show recent activity for
    various time slices. Anyone interested in doing something with that,
    maybe even twitter-feed like? Someone had a "subscribe to this distro"
    feature. I seem to recall that it might have been part of Perlbuzz.
    RECENT is a symlink to indices/RECENT-print
    I'll just point out that my CPAN Testing infrastructure relies on
    the RECENT file as is to find stuff to smoke test.
    Mine used to, then it changed, so I switched to using RECENT.recent,
    then it changed, now I use RECENT-1W.yaml.

    Hopefully that's nice n stable!

    While we're on the subject of RECENT*, what is RECENT-Z.yaml? And will
    it ever be updated? At a glance, it appears to be a log of all uploads
    and deletes from the beginning of time up to some point in recent
    history, at which point it stops. It would be Really Useful to have it
    be updated a few times a year.

    --
    David Cantrell | semi-evolved ape-thing

    Irregular English:
    ladies glow; gentlemen perspire; brutes, oafs and athletes sweat
  • Ask Bjørn Hansen at Sep 27, 2010 at 3:59 pm

    On Sep 27, 2010, at 8:17, David Cantrell wrote:

    While we're on the subject of RECENT*, what is RECENT-Z.yaml? And will
    it ever be updated? At a glance, it appears to be a log of all uploads
    and deletes from the beginning of time up to some point in recent
    history, at which point it stops.
    It continues in the RECENT-1Y file. It's used by File::Rsync::Mirror::Recent.


    - ask


    [root@mirrors.la authors]# ls -lt RECENT*yaml
    -rw-r--r--+ 1 mirrors mirrors 4614 Sep 27 08:56 RECENT-1h.yaml
    -rw-r--r--+ 1 mirrors mirrors 7793 Sep 27 08:56 RECENT-6h.yaml
    -rw-r--r--+ 1 mirrors mirrors 763623 Sep 27 08:41 RECENT-1M.yaml
    -rw-r--r--+ 1 mirrors mirrors 207896 Sep 27 08:41 RECENT-1W.yaml
    -rw-r--r--+ 1 mirrors mirrors 32697 Sep 27 08:41 RECENT-1d.yaml
    -rw-r--r--+ 1 mirrors mirrors 1984910 Sep 24 07:41 RECENT-1Q.yaml
    -rw-r--r--+ 1 mirrors mirrors 9833609 Sep 3 00:50 RECENT-1Y.yaml
    -rw-r--r--+ 1 mirrors mirrors 15545888 Jun 24 01:41 RECENT-Z.yaml
  • David Cantrell at Sep 27, 2010 at 4:33 pm

    On Mon, Sep 27, 2010 at 08:59:27AM -0700, Ask Bj?rn Hansen wrote:
    On Sep 27, 2010, at 8:17, David Cantrell wrote:
    While we're on the subject of RECENT*, what is RECENT-Z.yaml? And will
    it ever be updated? At a glance, it appears to be a log of all uploads
    and deletes from the beginning of time up to some point in recent
    history, at which point it stops.
    It continues in the RECENT-1Y file. It's used by File::Rsync::Mirror::Recent.
    ...
    -rw-r--r--+ 1 mirrors mirrors 9833609 Sep 3 00:50 RECENT-1Y.yaml
    -rw-r--r--+ 1 mirrors mirrors 15545888 Jun 24 01:41 RECENT-Z.yaml
    Yes, I realise that. Can we assume that in 9-ish months time when
    RECENT-1Y rolls over, RECENT-Z will be updated so that there's no gap in
    the history?

    --
    David Cantrell | Enforcer, South London Linguistic Massive

    Compromise: n: lowering my standards so you can meet them
  • David Golden at Sep 27, 2010 at 5:26 pm

    On Mon, Sep 27, 2010 at 12:32 PM, David Cantrell wrote:
    On Mon, Sep 27, 2010 at 08:59:27AM -0700, Ask Bj?rn Hansen wrote:
    On Sep 27, 2010, at 8:17, David Cantrell wrote:
    While we're on the subject of RECENT*, what is RECENT-Z.yaml?  And will
    it ever be updated?  At a glance, it appears to be a log of all uploads
    and deletes from the beginning of time up to some point in recent
    history, at which point it stops.
    It continues in the RECENT-1Y file.   It's used by File::Rsync::Mirror::Recent.
    ...
    -rw-r--r--+ 1 mirrors mirrors  9833609 Sep  3 00:50 RECENT-1Y.yaml
    -rw-r--r--+ 1 mirrors mirrors 15545888 Jun 24 01:41 RECENT-Z.yaml
    Yes, I realise that.  Can we assume that in 9-ish months time when
    RECENT-1Y rolls over, RECENT-Z will be updated so that there's no gap in
    the history?
    No -- RECENT-Z is stuff not in any of the other files. Stuff older
    than a year rolls to Z.

    David
  • Andreas J. Koenig at Sep 27, 2010 at 6:02 pm

    On Mon, 27 Sep 2010 17:32:57 +0100, David Cantrell said:
    Yes, I realise that. Can we assume that in 9-ish months time when
    RECENT-1Y rolls over, RECENT-Z will be updated so that there's no gap in
    the history?
    Yes, overlapping is always guaranteed. The File::Rsync::Mirror::Recent
    modules are constructed around the idea. There's a little program I use
    to sanity-check whether the roll-over times are following the plan that
    is made up for them:

    % rrr-overview /home/ftp/pub/PAUSE/authors/RECENT-1h.yaml
    Ival Cnt Max Min Span Util Cloud
    1h 57 1285609921.43 1285606962.91 2958.53 82.2% ^ ^
    6h 230 1285609921.43 1285589306.26 20615.18 95.4% ^ ^
    1d 411 1285609321.88 1285525747.73 83574.15 96.7% ^ ^
    1W 2074 1285602058.99 1284997293.30 604765.70 100.0% ^ ^
    1M 7614 1285602058.99 1283010164.75 2591894.24 100.0% ^ ^
    1Q 19780 1285339261.48 1277566019.09 7773242.38 100.0% ^ ^
    1Y 97275 1283500141.57 1251943652.99 31556488.58 100.0% ^ ^
    Z 158266 1277368802.31 0.12 1277368802.19 - ^ ^

    The ascii art in the last column displays how the time intervals
    overlap.

    --
    andreas
  • Elaine Ashton at Sep 27, 2010 at 3:13 pm

    On Sep 26, 2010, at 4:55 PM, brian d foy wrote:

    I'm interested in updating and handling the CPAN FAQ.
    I will send you an email wrt that and the history stuff.
    Who's going to take care of the donations for the gift basket for
    Jarkko and Elaine? Does anyone have a tractor trailer we can use to
    deliver it?
    One should always beware perl people bearing gift baskets...:)

    Also, you should mention Ben Hockenhull (Webster.edu) for hosting gargoyle, the WU CSEng lab (Wustl.edu) for hosting TUCS/Search for over 3 years, and Harri Salminen (FUNET) for hosting the master mirror all these years, Alan Burlison (Sun) for donating the 3 enterprise class systems and the storage array I drove out to St. Louis and Jos for hosting the DNS. It's always the way that memories are short and if you haven't done something today, the 10 or more years of service are forgotten which is a pale, if customary for perl, reward. Do not forget the contributions of the many.

    e.
  • Brian d foy at Sep 27, 2010 at 4:06 pm
    In article <577141B9-FA60-4ABE-9726-CEDCD5384D71@mac.com>, Elaine
    Ashton wrote:

    Also, you should mention Ben Hockenhull (Webster.edu) for hosting gargoyle,
    the WU CSEng lab (Wustl.edu) for hosting TUCS/Search for over 3 years, and
    Harri Salminen (FUNET) for hosting the master mirror all these years, Alan
    Burlison (Sun) for donating the 3 enterprise class systems and the storage
    array I drove out to St. Louis and Jos for hosting the DNS. It's always the
    way that memories are short and if you haven't done something today, the 10
    or more years of service are forgotten which is a pale, if customary for
    perl, reward. Do not forget the contributions of the many.
    It's not that I forget, but most of those I never knew about.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcpan-workers @
categoriesperl
postedSep 26, '10 at 8:55p
activeSep 27, '10 at 6:02p
posts17
users11
websitecpan.org

People

Translate

site design / logo © 2021 Grokbase