FAQ
Hi folks,

we are looking for a way to manage our puppet modules in a convenient way.
At the moment we have about 30 modules which are versionized with git or
mercurial. Most of them are self-made, some are from github.
We thought it would be a good idea to use mercurial subrepositories to
manage the versions of those modules. Well, it is not. Everytime you push
your commits to the server all subrepositories are checked for updates. It
takes minutes to push your changes! And if you have to merge something you
do not really know what happens. You just try every command which appears
in your mind - multiple times and in different order :D
It happened three times that every developer had to delete the whole module
tree and check it out again because something was broken.
git submodules do not seem to be fun either.

I have looked for an alternative to submodules/subrepositories and found
librarian-puppet (https://github.com/rodjek/librarian-puppet).
Unfortunately it does not really work good for me. A lot of useless error
messages if something is configured wrong. This is pretty important for us.
All developers should be able to use the versioning tool. It has to be easy
to use. Further the librarian-puppet does not seem to be backed by a
healthy community. Just every few months a commit and a lot of open issues
and pull-requests.

I have hacked some Python code to realize something similar: dyps -
https://github.com/MalteJ/dyps
You have a yaml file which defines your modules: name, source, version and
DVCS system (git/hg):

- dep: mysql
source: https://github.com/puppetlabs/puppetlabs-mysql.git
type: git
version: af4b8bff0ad59f21211be2a98d31588066c41e0c

All modules are downloaded to the folder "dependencies" when executing
"dyps run".
You can decide if you want to get a specific changeset at the next run or
if you want to get the latest changes - just discard the version property
or execute "dyps release your-module-name". If you want to keep the current
version execute "dyps keep your-module". You can update one or all
versionized modules via "dyps update your-module" or "dyps update-all". The
modules without a defined versioned are updated at every dyps run.

I have added an import function for mercurial subrepository .hgsub files.
Feel free to extend it with an importGitSubmodules function ;-)


What do you think about this approach?
How are you managing your modules/dependencies?

Best,
Malte

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/wXsxrloQTVYJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Benjamin Priestman at Dec 20, 2012 at 1:36 pm
    I've tried using https://github.com/azimux/externals (referenced in Version
    Control with Git <http://shop.oreilly.com/product/9780596520137.do>) which
    kind of works, but is a bit buggy and I've yet to come up with an easily
    understood workflow.
    On Tuesday, 18 December 2012 17:59:48 UTC, Malte Janduda wrote:

    Hi folks,

    we are looking for a way to manage our puppet modules in a convenient way.
    At the moment we have about 30 modules which are versionized with git or
    mercurial. Most of them are self-made, some are from github.
    We thought it would be a good idea to use mercurial subrepositories to
    manage the versions of those modules. Well, it is not. Everytime you push
    your commits to the server all subrepositories are checked for updates. It
    takes minutes to push your changes! And if you have to merge something you
    do not really know what happens. You just try every command which appears
    in your mind - multiple times and in different order :D
    It happened three times that every developer had to delete the whole
    module tree and check it out again because something was broken.
    git submodules do not seem to be fun either.

    I have looked for an alternative to submodules/subrepositories and found
    librarian-puppet (https://github.com/rodjek/librarian-puppet).
    Unfortunately it does not really work good for me. A lot of useless error
    messages if something is configured wrong. This is pretty important for us.
    All developers should be able to use the versioning tool. It has to be easy
    to use. Further the librarian-puppet does not seem to be backed by a
    healthy community. Just every few months a commit and a lot of open issues
    and pull-requests.

    I have hacked some Python code to realize something similar: dyps -
    https://github.com/MalteJ/dyps
    You have a yaml file which defines your modules: name, source, version and
    DVCS system (git/hg):

    - dep: mysql
    source: https://github.com/puppetlabs/puppetlabs-mysql.git
    type: git
    version: af4b8bff0ad59f21211be2a98d31588066c41e0c

    All modules are downloaded to the folder "dependencies" when executing
    "dyps run".
    You can decide if you want to get a specific changeset at the next run or
    if you want to get the latest changes - just discard the version property
    or execute "dyps release your-module-name". If you want to keep the current
    version execute "dyps keep your-module". You can update one or all
    versionized modules via "dyps update your-module" or "dyps update-all". The
    modules without a defined versioned are updated at every dyps run.

    I have added an import function for mercurial subrepository .hgsub files.
    Feel free to extend it with an importGitSubmodules function ;-)


    What do you think about this approach?
    How are you managing your modules/dependencies?

    Best,
    Malte
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/VZjQExuTnvAJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Mason Turner at Dec 20, 2012 at 6:28 pm
    We are still using svn, but looking on adapt this to git next year.

    This is a simplified view of our svn repo:

    - nodes
    -- dev
    --- trunk
    --- tags
    ---- 20121220-121212
    -- prod
    --- trunk
    --- tags
    - modules
    -- apache
    --- trunk
    --- tags
    ---- 20121220-121212
    -- something_else
    --- trunk
    --- tags
    ---- 20121220-121212

    We then have a utility (link_puppet) that:
    * takes a module, tag and environment as command line options
    * Updates the svn sandbox in /opt/puppet/svn/modules/$module
    * links /opt/puppet/svn/modules/$module/tags/$tag to /etc/puppet/environments/$env/modules/$module

    It also validates the module before linking, logs who did what when. Pretty handy.

    — Mason Turner
    On Dec 20, 2012, at 8:36 AM, Benjamin Priestman wrote:

    I've tried using https://github.com/azimux/externals (referenced in Version Control with Git) which kind of works, but is a bit buggy and I've yet to come up with an easily understood workflow.
    On Tuesday, 18 December 2012 17:59:48 UTC, Malte Janduda wrote:

    Hi folks,

    we are looking for a way to manage our puppet modules in a convenient way. At the moment we have about 30 modules which are versionized with git or mercurial. Most of them are self-made, some are from github.
    We thought it would be a good idea to use mercurial subrepositories to manage the versions of those modules. Well, it is not. Everytime you push your commits to the server all subrepositories are checked for updates. It takes minutes to push your changes! And if you have to merge something you do not really know what happens. You just try every command which appears in your mind - multiple times and in different order :D
    It happened three times that every developer had to delete the whole module tree and check it out again because something was broken.
    git submodules do not seem to be fun either.

    I have looked for an alternative to submodules/subrepositories and found librarian-puppet (https://github.com/rodjek/librarian-puppet). Unfortunately it does not really work good for me. A lot of useless error messages if something is configured wrong. This is pretty important for us. All developers should be able to use the versioning tool. It has to be easy to use. Further the librarian-puppet does not seem to be backed by a healthy community. Just every few months a commit and a lot of open issues and pull-requests.

    I have hacked some Python code to realize something similar: dyps - https://github.com/MalteJ/dyps
    You have a yaml file which defines your modules: name, source, version and DVCS system (git/hg):

    - dep: mysql
    source: https://github.com/puppetlabs/puppetlabs-mysql.git
    type: git
    version: af4b8bff0ad59f21211be2a98d31588066c41e0c

    All modules are downloaded to the folder "dependencies" when executing "dyps run".
    You can decide if you want to get a specific changeset at the next run or if you want to get the latest changes - just discard the version property or execute "dyps release your-module-name". If you want to keep the current version execute "dyps keep your-module". You can update one or all versionized modules via "dyps update your-module" or "dyps update-all". The modules without a defined versioned are updated at every dyps run.

    I have added an import function for mercurial subrepository .hgsub files. Feel free to extend it with an importGitSubmodules function ;-)


    What do you think about this approach?
    How are you managing your modules/dependencies?

    Best,
    Malte
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/VZjQExuTnvAJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Nikola Petrov at Dec 24, 2012 at 12:17 pm

    On Thu, Dec 20, 2012 at 01:28:04PM -0500, Mason Turner wrote:
    We are still using svn, but looking on adapt this to git next year.

    This is a simplified view of our svn repo:

    - nodes
    -- dev
    --- trunk
    --- tags
    ---- 20121220-121212
    -- prod
    --- trunk
    --- tags
    - modules
    -- apache
    --- trunk
    --- tags
    ---- 20121220-121212
    -- something_else
    --- trunk
    --- tags
    ---- 20121220-121212

    We then have a utility (link_puppet) that:
    Can you please post the link_puppet utility. I will be happy if we can
    burrow some code and not start from scratch on almost the same setup(we
    are doing it manually at the moment)

    Best, Nikola

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedDec 18, '12 at 5:59p
activeDec 24, '12 at 12:17p
posts4
users4
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase