FAQ
Hi Serega,
I am not very familiar with Cobbler but here are a couple of thoughts:
1. Can you please verify that the name of the repository is correct?
In line with what's mentioned at
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_30.html,
the repo name you use in your reposync command should be
"cloudera-impala". And, you would need to download the cloudera-impala
repo file from archive.cloudera.com/impala/redhat before you can run
that command.

I tried the command on "reposync -r cloudera-impala" on my CentOS6 box
and it worked.

2. The above only downloads the RPMs. If you want to create your own
personal repository for parcels, you need to copy down the parcels
from the parcel repo
(http://archive.cloudera.com/impala/parcels/latest/) along with the
manifest.

A command like this should do the trick:
wget -r --cut-dirs=3 --accept "*.parcel" --accept="manifest.json"
http://archive.cloudera.com/impala/parcels/latest/

Please let me know how it goes!

Mark

On Wed, May 15, 2013 at 7:21 AM, Serega Sheypak
wrote:
Hi, we have a local miror for CDH parcels.
We would like to create the same mirror for Impala parcels. It doesn't work:

task started: 2013-05-15_181756_reposync
task started (id=Reposync, time=Wed May 15 18:17:56 2013)
hello, reposync
run, reposync, run!
creating: /var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
running: /usr/bin/reposync -l -m -d
--config=/var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
--repoid=Impala --download_path=/var/www/cobbler/repo_mirror -a x86_64
received on stdout:
received on stderr: Error setting up repositories: Cannot retrieve
repository metadata (repomd.xml) for repository: Impala. Please verify its
path and try again

Exception occured: <class 'cobbler.cexceptions.CX'>
Exception value: 'cobbler reposync failed'
Exception Info:
File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
raise CX(msg)


The same thing for CDH parcels works fine. What do I do wrong? How can I
create mirror for impala parcels?

Search Discussions

  • Serega Sheypak at May 17, 2013 at 6:49 am
    I'm doing something wrong :(
    Here are settings for CDH parcels:
    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/cdh4/parcels/latest/

    And it works.

    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/impala/parcels/latest/

    It woesn't work.

    *There is no problem with connection:*

    [sheypak@utility1 ~]$ curl /archive.cloudera.com/impala/parcels/latest/
    curl: (3) <url> malformed
    [sheypak@utility1 ~]$ curl archive.cloudera.com/impala/parcels/latest/
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
    <html>
      <head>
       <title>Index of /impala/parcels/latest</title>
      </head>
      <body>
    <h1>Index of /impala/parcels/latest</h1>
    <pre><img src="/icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>
                                 <a href="?C=M;O=A">Last modified</a> <a
    href="?C=S;O=A">Size</a> <a href="?C=D;O=A">Description</a><hr><img
    src="/icons/back.gif" alt="[DIR]"> <a href="/impala/parcels/">Parent
    Directory</a> -
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el5.parcel">IMPALA-1.0-1.p0.371-el5.parcel</a>
       28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el6.parcel">IMPALA-1.0-1.p0.371-el6.parcel</a>
       28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-lucid.parcel">IMPALA-1.0-1.p0.371-lucid.parcel</a>
       28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-precise.parcel">IMPALA-1.0-1.p0.371-precise.parcel</a>
    28-Apr-2013 23:37 228M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-sles11.parcel">IMPALA-1.0-1.p0.371-sles11.parcel</a>
      28-Apr-2013 23:37 212M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-squeeze.parcel">IMPALA-1.0-1.p0.371-squeeze.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="manifest.json">manifest.json</a> 28-Apr-2013
    23:37 1.6K
    <hr></pre>
    <address>Apache/2.2.16 (Ubuntu) Server at archive.cloudera.com Port
    80</address>
    </body></html>



    *Here is error log:*

    [sheypak@utility1 ~]$ sudo cobbler reposync --only="Impala"
    task started: 2013-05-17_103853_reposync
    task started (id=Reposync, time=Fri May 17 10:38:53 2013)
    hello, reposync
    run, reposync, run!
    running: rsync -rltDv --copy-unsafe-links --delete-after --delete
    --exclude-from=/etc/cobbler/rsync.exclude rsync://
    archive.cloudera.com/impala/parcels/latest//var/www/cobbler/repo_mirror/Impala
    *received on stdout: *
    *received on stderr: rsync: failed to connect to archive.cloudera.com:
    Connection timed out (110)*
    *rsync error: error in socket IO (code 10) at clientserver.c(124)
    [receiver=3.0.6]*

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
       File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
         raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
       File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    125, in run
         self.sync(repo)
        File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    173, in sync
         return self.rsync_sync(repo)
        File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    247, in rsync_sync
         utils.die(self.logger,"cobbler reposync failed")
        File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
         raise CX(msg)

    reposync failed, tries left: 0
    reposync failed, retry limit reached, skipping
    running: chown -R root:apache /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    running: chmod -R 755 /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
       File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
         raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
       File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 89, in run
         rc = self._run(self)
        File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 248, in
    runner
         3), name=name, nofail=nofail, logger=self.logger)
        File "/usr/lib/python2.6/site-packages/cobbler/api.py", line 758, in
    reposync
         return reposync.run(name)
        File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    154, in run
         utils.die(self.logger,"overall reposync failed, at least one repo
    failed to synchronize")
        File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
         raise CX(msg)

    !!! TASK FAILED !!!
    [sheypak@utility1 ~]$






    2013/5/17 Mark Grover <mgrover@cloudera.com>
    Hi Serega,
    I am not very familiar with Cobbler but here are a couple of thoughts:
    1. Can you please verify that the name of the repository is correct?
    In line with what's mentioned at

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_30.html
    ,
    the repo name you use in your reposync command should be
    "cloudera-impala". And, you would need to download the cloudera-impala
    repo file from archive.cloudera.com/impala/redhat before you can run
    that command.

    I tried the command on "reposync -r cloudera-impala" on my CentOS6 box
    and it worked.

    2. The above only downloads the RPMs. If you want to create your own
    personal repository for parcels, you need to copy down the parcels
    from the parcel repo
    (http://archive.cloudera.com/impala/parcels/latest/) along with the
    manifest.

    A command like this should do the trick:
    wget -r --cut-dirs=3 --accept "*.parcel" --accept="manifest.json"
    http://archive.cloudera.com/impala/parcels/latest/

    Please let me know how it goes!

    Mark

    On Wed, May 15, 2013 at 7:21 AM, Serega Sheypak
    wrote:
    Hi, we have a local miror for CDH parcels.
    We would like to create the same mirror for Impala parcels. It doesn't work:
    task started: 2013-05-15_181756_reposync
    task started (id=Reposync, time=Wed May 15 18:17:56 2013)
    hello, reposync
    run, reposync, run!
    creating: /var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    running: /usr/bin/reposync -l -m -d
    --config=/var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    --repoid=Impala --download_path=/var/www/cobbler/repo_mirror -a x86_64
    received on stdout:
    received on stderr: Error setting up repositories: Cannot retrieve
    repository metadata (repomd.xml) for repository: Impala. Please verify its
    path and try again

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)


    The same thing for CDH parcels works fine. What do I do wrong? How can I
    create mirror for impala parcels?
  • Mark Grover at May 17, 2013 at 7:06 pm
    Hi Serega,
    I talked to one of the IT folks here at Cloudera and it was mentioned
    to me that it isn't possible to rsync stuff from archive.cloudera.com
    from an external IP. That might explain your timeout errors.
    May I suggest you try curl or wget instead?

    Thanks!
    Mark

    On Thu, May 16, 2013 at 11:49 PM, Serega Sheypak
    wrote:
    I'm doing something wrong :(
    Here are settings for CDH parcels:
    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/cdh4/parcels/latest/

    And it works.

    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/impala/parcels/latest/

    It woesn't work.

    There is no problem with connection:

    [sheypak@utility1 ~]$ curl /archive.cloudera.com/impala/parcels/latest/
    curl: (3) <url> malformed
    [sheypak@utility1 ~]$ curl archive.cloudera.com/impala/parcels/latest/
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
    <html>
    <head>
    <title>Index of /impala/parcels/latest</title>
    </head>
    <body>
    <h1>Index of /impala/parcels/latest</h1>
    <pre><img src="/icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>
    <a href="?C=M;O=A">Last modified</a> <a href="?C=S;O=A">Size</a> <a
    href="?C=D;O=A">Description</a><hr><img src="/icons/back.gif" alt="[DIR]">
    <a href="/impala/parcels/">Parent Directory</a>
    -
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el5.parcel">IMPALA-1.0-1.p0.371-el5.parcel</a>
    28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el6.parcel">IMPALA-1.0-1.p0.371-el6.parcel</a>
    28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-lucid.parcel">IMPALA-1.0-1.p0.371-lucid.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-precise.parcel">IMPALA-1.0-1.p0.371-precise.parcel</a>
    28-Apr-2013 23:37 228M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-sles11.parcel">IMPALA-1.0-1.p0.371-sles11.parcel</a>
    28-Apr-2013 23:37 212M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-squeeze.parcel">IMPALA-1.0-1.p0.371-squeeze.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="manifest.json">manifest.json</a> 28-Apr-2013
    23:37 1.6K
    <hr></pre>
    <address>Apache/2.2.16 (Ubuntu) Server at archive.cloudera.com Port
    80</address>
    </body></html>



    Here is error log:

    [sheypak@utility1 ~]$ sudo cobbler reposync --only="Impala"
    task started: 2013-05-17_103853_reposync
    task started (id=Reposync, time=Fri May 17 10:38:53 2013)
    hello, reposync
    run, reposync, run!
    running: rsync -rltDv --copy-unsafe-links --delete-after --delete
    --exclude-from=/etc/cobbler/rsync.exclude
    rsync://archive.cloudera.com/impala/parcels/latest/
    /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr: rsync: failed to connect to archive.cloudera.com:
    Connection timed out (110)
    rsync error: error in socket IO (code 10) at clientserver.c(124)
    [receiver=3.0.6]

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    125, in run
    self.sync(repo)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    173, in sync
    return self.rsync_sync(repo)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    247, in rsync_sync
    utils.die(self.logger,"cobbler reposync failed")
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
    raise CX(msg)

    reposync failed, tries left: 0
    reposync failed, retry limit reached, skipping
    running: chown -R root:apache /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    running: chmod -R 755 /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 89, in run
    rc = self._run(self)
    File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 248, in
    runner
    3), name=name, nofail=nofail, logger=self.logger)
    File "/usr/lib/python2.6/site-packages/cobbler/api.py", line 758, in
    reposync
    return reposync.run(name)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    154, in run
    utils.die(self.logger,"overall reposync failed, at least one repo failed
    to synchronize")
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
    raise CX(msg)

    !!! TASK FAILED !!!
    [sheypak@utility1 ~]$






    2013/5/17 Mark Grover <mgrover@cloudera.com>
    Hi Serega,
    I am not very familiar with Cobbler but here are a couple of thoughts:
    1. Can you please verify that the name of the repository is correct?
    In line with what's mentioned at

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_30.html,
    the repo name you use in your reposync command should be
    "cloudera-impala". And, you would need to download the cloudera-impala
    repo file from archive.cloudera.com/impala/redhat before you can run
    that command.

    I tried the command on "reposync -r cloudera-impala" on my CentOS6 box
    and it worked.

    2. The above only downloads the RPMs. If you want to create your own
    personal repository for parcels, you need to copy down the parcels
    from the parcel repo
    (http://archive.cloudera.com/impala/parcels/latest/) along with the
    manifest.

    A command like this should do the trick:
    wget -r --cut-dirs=3 --accept "*.parcel" --accept="manifest.json"
    http://archive.cloudera.com/impala/parcels/latest/

    Please let me know how it goes!

    Mark

    On Wed, May 15, 2013 at 7:21 AM, Serega Sheypak
    wrote:
    Hi, we have a local miror for CDH parcels.
    We would like to create the same mirror for Impala parcels. It doesn't
    work:

    task started: 2013-05-15_181756_reposync
    task started (id=Reposync, time=Wed May 15 18:17:56 2013)
    hello, reposync
    run, reposync, run!
    creating: /var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    running: /usr/bin/reposync -l -m -d
    --config=/var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    --repoid=Impala --download_path=/var/www/cobbler/repo_mirror -a x86_64
    received on stdout:
    received on stderr: Error setting up repositories: Cannot retrieve
    repository metadata (repomd.xml) for repository: Impala. Please verify
    its
    path and try again

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in
    die
    raise CX(msg)


    The same thing for CDH parcels works fine. What do I do wrong? How can I
    create mirror for impala parcels?
  • Serega Sheypak at May 17, 2013 at 7:12 pm
    Yes, we did already it. It helped.


    2013/5/17 Mark Grover <mgrover@cloudera.com>
    Hi Serega,
    I talked to one of the IT folks here at Cloudera and it was mentioned
    to me that it isn't possible to rsync stuff from archive.cloudera.com
    from an external IP. That might explain your timeout errors.
    May I suggest you try curl or wget instead?

    Thanks!
    Mark

    On Thu, May 16, 2013 at 11:49 PM, Serega Sheypak
    wrote:
    I'm doing something wrong :(
    Here are settings for CDH parcels:
    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/cdh4/parcels/latest/

    And it works.

    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/impala/parcels/latest/

    It woesn't work.

    There is no problem with connection:

    [sheypak@utility1 ~]$ curl /archive.cloudera.com/impala/parcels/latest/
    curl: (3) <url> malformed
    [sheypak@utility1 ~]$ curl archive.cloudera.com/impala/parcels/latest/
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
    <html>
    <head>
    <title>Index of /impala/parcels/latest</title>
    </head>
    <body>
    <h1>Index of /impala/parcels/latest</h1>
    <pre><img src="/icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>
    <a href="?C=M;O=A">Last modified</a> <a href="?C=S;O=A">Size</a> <a
    href="?C=D;O=A">Description</a><hr><img src="/icons/back.gif"
    alt="[DIR]">
    <a href="/impala/parcels/">Parent Directory</a>
    -
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el5.parcel">IMPALA-1.0-1.p0.371-el5.parcel</a>
    28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el6.parcel">IMPALA-1.0-1.p0.371-el6.parcel</a>
    28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-lucid.parcel">IMPALA-1.0-1.p0.371-lucid.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-precise.parcel">IMPALA-1.0-1.p0.371-precise.parcel</a>
    28-Apr-2013 23:37 228M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-sles11.parcel">IMPALA-1.0-1.p0.371-sles11.parcel</a>
    28-Apr-2013 23:37 212M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-squeeze.parcel">IMPALA-1.0-1.p0.371-squeeze.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="manifest.json">manifest.json</a> 28-Apr-2013
    23:37 1.6K
    <hr></pre>
    <address>Apache/2.2.16 (Ubuntu) Server at archive.cloudera.com Port
    80</address>
    </body></html>



    Here is error log:

    [sheypak@utility1 ~]$ sudo cobbler reposync --only="Impala"
    task started: 2013-05-17_103853_reposync
    task started (id=Reposync, time=Fri May 17 10:38:53 2013)
    hello, reposync
    run, reposync, run!
    running: rsync -rltDv --copy-unsafe-links --delete-after --delete
    --exclude-from=/etc/cobbler/rsync.exclude
    rsync://archive.cloudera.com/impala/parcels/latest/
    /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr: rsync: failed to connect to archive.cloudera.com:
    Connection timed out (110)
    rsync error: error in socket IO (code 10) at clientserver.c(124)
    [receiver=3.0.6]

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    125, in run
    self.sync(repo)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    173, in sync
    return self.rsync_sync(repo)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    247, in rsync_sync
    utils.die(self.logger,"cobbler reposync failed")
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
    raise CX(msg)

    reposync failed, tries left: 0
    reposync failed, retry limit reached, skipping
    running: chown -R root:apache /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    running: chmod -R 755 /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 89, in run
    rc = self._run(self)
    File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 248, in
    runner
    3), name=name, nofail=nofail, logger=self.logger)
    File "/usr/lib/python2.6/site-packages/cobbler/api.py", line 758, in
    reposync
    return reposync.run(name)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    154, in run
    utils.die(self.logger,"overall reposync failed, at least one repo failed
    to synchronize")
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
    raise CX(msg)

    !!! TASK FAILED !!!
    [sheypak@utility1 ~]$






    2013/5/17 Mark Grover <mgrover@cloudera.com>
    Hi Serega,
    I am not very familiar with Cobbler but here are a couple of thoughts:
    1. Can you please verify that the name of the repository is correct?
    In line with what's mentioned at
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_30.html
    ,
    the repo name you use in your reposync command should be
    "cloudera-impala". And, you would need to download the cloudera-impala
    repo file from archive.cloudera.com/impala/redhat before you can run
    that command.

    I tried the command on "reposync -r cloudera-impala" on my CentOS6 box
    and it worked.

    2. The above only downloads the RPMs. If you want to create your own
    personal repository for parcels, you need to copy down the parcels
    from the parcel repo
    (http://archive.cloudera.com/impala/parcels/latest/) along with the
    manifest.

    A command like this should do the trick:
    wget -r --cut-dirs=3 --accept "*.parcel" --accept="manifest.json"
    http://archive.cloudera.com/impala/parcels/latest/

    Please let me know how it goes!

    Mark

    On Wed, May 15, 2013 at 7:21 AM, Serega Sheypak
    wrote:
    Hi, we have a local miror for CDH parcels.
    We would like to create the same mirror for Impala parcels. It doesn't
    work:

    task started: 2013-05-15_181756_reposync
    task started (id=Reposync, time=Wed May 15 18:17:56 2013)
    hello, reposync
    run, reposync, run!
    creating: /var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    running: /usr/bin/reposync -l -m -d
    --config=/var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    --repoid=Impala --download_path=/var/www/cobbler/repo_mirror -a x86_64
    received on stdout:
    received on stderr: Error setting up repositories: Cannot retrieve
    repository metadata (repomd.xml) for repository: Impala. Please verify
    its
    path and try again

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126,
    in
    die
    raise CX(msg)


    The same thing for CDH parcels works fine. What do I do wrong? How
    can I
    create mirror for impala parcels?
  • John Russell at May 17, 2013 at 8:16 pm
    I had just added an example to the docs showing how to use wget for archive.cloudera.com -- I'll add a note to clarify that wget and curl are OK, rsync won't work.

    John
    On May 17, 2013, at 12:12 PM, Serega Sheypak wrote:

    Yes, we did already it. It helped.


    2013/5/17 Mark Grover <mgrover@cloudera.com>
    Hi Serega,
    I talked to one of the IT folks here at Cloudera and it was mentioned
    to me that it isn't possible to rsync stuff from archive.cloudera.com
    from an external IP. That might explain your timeout errors.
    May I suggest you try curl or wget instead?

    Thanks!
    Mark

    On Thu, May 16, 2013 at 11:49 PM, Serega Sheypak
    wrote:
    I'm doing something wrong :(
    Here are settings for CDH parcels:
    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/cdh4/parcels/latest/

    And it works.

    arch=x86_64
    Breed=rsync
    mirror=rsync://archive.cloudera.com/impala/parcels/latest/

    It woesn't work.

    There is no problem with connection:

    [sheypak@utility1 ~]$ curl /archive.cloudera.com/impala/parcels/latest/
    curl: (3) <url> malformed
    [sheypak@utility1 ~]$ curl archive.cloudera.com/impala/parcels/latest/
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
    <html>
    <head>
    <title>Index of /impala/parcels/latest</title>
    </head>
    <body>
    <h1>Index of /impala/parcels/latest</h1>
    <pre><img src="/icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>
    <a href="?C=M;O=A">Last modified</a> <a href="?C=S;O=A">Size</a> <a
    href="?C=D;O=A">Description</a><hr><img src="/icons/back.gif" alt="[DIR]">
    <a href="/impala/parcels/">Parent Directory</a>
    -
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el5.parcel">IMPALA-1.0-1.p0.371-el5.parcel</a>
    28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-el6.parcel">IMPALA-1.0-1.p0.371-el6.parcel</a>
    28-Apr-2013 23:37 200M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-lucid.parcel">IMPALA-1.0-1.p0.371-lucid.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-precise.parcel">IMPALA-1.0-1.p0.371-precise.parcel</a>
    28-Apr-2013 23:37 228M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-sles11.parcel">IMPALA-1.0-1.p0.371-sles11.parcel</a>
    28-Apr-2013 23:37 212M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="IMPALA-1.0-1.p0.371-squeeze.parcel">IMPALA-1.0-1.p0.371-squeeze.parcel</a>
    28-Apr-2013 23:37 118M
    <img src="/icons/unknown.gif" alt="[ ]"> <a
    href="manifest.json">manifest.json</a> 28-Apr-2013
    23:37 1.6K
    <hr></pre>
    <address>Apache/2.2.16 (Ubuntu) Server at archive.cloudera.com Port
    80</address>
    </body></html>



    Here is error log:

    [sheypak@utility1 ~]$ sudo cobbler reposync --only="Impala"
    task started: 2013-05-17_103853_reposync
    task started (id=Reposync, time=Fri May 17 10:38:53 2013)
    hello, reposync
    run, reposync, run!
    running: rsync -rltDv --copy-unsafe-links --delete-after --delete
    --exclude-from=/etc/cobbler/rsync.exclude
    rsync://archive.cloudera.com/impala/parcels/latest/
    /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr: rsync: failed to connect to archive.cloudera.com:
    Connection timed out (110)
    rsync error: error in socket IO (code 10) at clientserver.c(124)
    [receiver=3.0.6]

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    125, in run
    self.sync(repo)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    173, in sync
    return self.rsync_sync(repo)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    247, in rsync_sync
    utils.die(self.logger,"cobbler reposync failed")
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
    raise CX(msg)

    reposync failed, tries left: 0
    reposync failed, retry limit reached, skipping
    running: chown -R root:apache /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    running: chmod -R 755 /var/www/cobbler/repo_mirror/Impala
    received on stdout:
    received on stderr:
    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die
    raise CX(msg)

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'overall reposync failed, at least one repo failed to
    synchronize'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 89, in run
    rc = self._run(self)
    File "/usr/lib/python2.6/site-packages/cobbler/remote.py", line 248, in
    runner
    3), name=name, nofail=nofail, logger=self.logger)
    File "/usr/lib/python2.6/site-packages/cobbler/api.py", line 758, in
    reposync
    return reposync.run(name)
    File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line
    154, in run
    utils.die(self.logger,"overall reposync failed, at least one repo failed
    to synchronize")
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in
    die
    raise CX(msg)

    !!! TASK FAILED !!!
    [sheypak@utility1 ~]$






    2013/5/17 Mark Grover <mgrover@cloudera.com>
    Hi Serega,
    I am not very familiar with Cobbler but here are a couple of thoughts:
    1. Can you please verify that the name of the repository is correct?
    In line with what's mentioned at

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_30.html,
    the repo name you use in your reposync command should be
    "cloudera-impala". And, you would need to download the cloudera-impala
    repo file from archive.cloudera.com/impala/redhat before you can run
    that command.

    I tried the command on "reposync -r cloudera-impala" on my CentOS6 box
    and it worked.

    2. The above only downloads the RPMs. If you want to create your own
    personal repository for parcels, you need to copy down the parcels
    from the parcel repo
    (http://archive.cloudera.com/impala/parcels/latest/) along with the
    manifest.

    A command like this should do the trick:
    wget -r --cut-dirs=3 --accept "*.parcel" --accept="manifest.json"
    http://archive.cloudera.com/impala/parcels/latest/

    Please let me know how it goes!

    Mark

    On Wed, May 15, 2013 at 7:21 AM, Serega Sheypak
    wrote:
    Hi, we have a local miror for CDH parcels.
    We would like to create the same mirror for Impala parcels. It doesn't
    work:

    task started: 2013-05-15_181756_reposync
    task started (id=Reposync, time=Wed May 15 18:17:56 2013)
    hello, reposync
    run, reposync, run!
    creating: /var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    running: /usr/bin/reposync -l -m -d
    --config=/var/www/cobbler/repo_mirror/Impala/.origin/Impala.repo
    --repoid=Impala --download_path=/var/www/cobbler/repo_mirror -a x86_64
    received on stdout:
    received on stderr: Error setting up repositories: Cannot retrieve
    repository metadata (repomd.xml) for repository: Impala. Please verify
    its
    path and try again

    Exception occured: <class 'cobbler.cexceptions.CX'>
    Exception value: 'cobbler reposync failed'
    Exception Info:
    File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in
    die
    raise CX(msg)


    The same thing for CDH parcels works fine. What do I do wrong? How can I
    create mirror for impala parcels?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedMay 16, '13 at 10:52p
activeMay 17, '13 at 8:16p
posts5
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase