FAQ
We use it every day to handle our automatic deploy tarballs, once it's got
the bucket down initially it just pulls down the deltas there after. So the
first pull is slow, but the next ones are as fast as the changes made. But
if the bucket is truly huge and the contents aren't relevant to the minions
I would move what you want to a separate bucket.

-A.

On Thu, Aug 21, 2014 at 10:01 AM, Stephen Wood wrote:

Ah, that makes sense.

My guess is that the master is busy trying to synchronize a bucket that
contains a few million objects containing many hundreds of TB of data.
Checking /var/cache/salt/master/s3cache I can see that the master has
download just a fraction of the bucket. Maybe that's why I'm getting the
"source file not found" error? The file I'm looking for does not yet exist
in the cache.

I'd be very surprised if it can check md5s for all of the objects in this
bucket in an hour, let alone 60 seconds. Has this module been tested with
large buckets?


On Wed, Aug 20, 2014 at 4:49 PM, Alan Garfield <
alan.garfield@learnosity.com> wrote:
Yes. So don't use a bucket with stuff in it you're not expecting to
deliver to minions.

-A.

On Thu, Aug 21, 2014 at 9:45 AM, Stephen Wood wrote:

Does it cache all objects in the bucket, even if they aren't requested?

For example, if I have a bucket that contains 100 gigs of objects, will
all of those files be downloaded locally before they are available to
minions?

--
You received this message because you are subscribed to the Google
Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Stephen Wood
www.heystephenwood.com

--
You received this message because you are subscribed to the Google Groups
"Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 8 of 8 | next ›
Discussion Overview
groupsalt-users @
postedAug 20, '14 at 11:00p
activeAug 21, '14 at 12:08a
posts8
users2

2 users in discussion

Stephen Wood: 4 posts Alan Garfield: 4 posts

People

Translate

site design / logo © 2022 Grokbase