[bosh-users] Migrating cf-release blobs to S3

Dr Nic Williams

at Jan 16, 2013 at 10:57 pm

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We have been
able to move all of the blobs in cf-release/blobs over. That part was easy.
When we try to create a release doing 'bosh create release' we get errors
saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
---------------------------------

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Mike Heath

at Jan 16, 2013 at 11:24 pm

⇧

Adding --force eliminates the warnings about my changes not being committed
in git but doesn't prevent bosh from trying to download packages from the
blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath <elc...@gmail.com<javascript:>

wrote:
We're trying to fork cf-release and migrate our blobs to S3. We have been
able to move all of the blobs in cf-release/blobs over. That part was easy.
When we try to create a release doing 'bosh create release' we get errors
saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
---------------------------------

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Dr Nic Williams

at Jan 17, 2013 at 12:07 am

⇧

What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath wrote:

Adding --force eliminates the warnings about my changes not being
committed in git but doesn't prevent bosh from trying to download packages
from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We have
been able to move all of the blobs in cf-release/blobs over. That part was
easy. When we try to create a release doing 'bosh create release' we get
errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
------------------------------**---

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Nicholas Terry

at Jan 17, 2013 at 12:32 am

⇧

There is no simple way to do what you want. We dealt with this issue a
little while back.The best way to accomplish this is to modify the source
to use two blobstores. (Its how we had to do it)

There may be a hacky way to get the blobstore populated...
First, make sure the final.yml and provate.yml files in the config dir are
the vanilla copies and do a bosh sync blobs

1. Recreate the non-submodule directories in the src/ folder to the blobs
directory (not to be confused with the (dot)blobs directory)
2. For each symlink in each folder in the src dir, copy the referenced blob
to its corresponding folder in the blobs directory. For example:

Given: src/debian_nfs_server/whatever.tgz ->
/path/to/.blobs/a44b3f8319984bace61f21bb6d15cee1419e5bf3 (THIS IS AN
EXAMPLE)

Do the following:

cp /path/to/.blobs/a44b3f8319984bace61f21bb6d15cee1419e5bf3
blobs/debian_nfs_server/whatever.tgz

3. Empty out blobs.yml in the config folder so that is only contains --- {}
4. Then change your private and final yaml files to you appropriate setup
for your s3 store
5. Then do a bosh upload blobs. You should be prompted to upload each blob
in turn and blobs.yml will be populated with your blobs and their ids and
stuff.

If you are unsure of how to format you final and provate yml files, go here:

https://github.com/cloudfoundry/bosh/blob/master/cli/lib/cli/release.rb#L72-L86

make sure you have provisioned your s3 bucket correctly you need to follow
these directions:

https://github.com/cloudfoundry/oss-docs/blob/master/bosh/documentation/bosh_releases.md#setting-up-a-new-release-repository-using-s3

(The bucket policy referenced there is incorrect. Use the IAM builder to
make it)

Let me know if this works for you

reply | permalink

Martin Englund

at Jan 17, 2013 at 12:35 am

⇧

Excellent write-up! Thanks for taking the time to do it

/M

Sent from my iPhone

On Jan 16, 2013, at 16:25, Nicholas Terry wrote:

There is no simple way to do what you want. We dealt with this issue a little while back.The best way to accomplish this is to modify the source to use two blobstores. (Its how we had to do it)

There may be a hacky way to get the blobstore populated...
First, make sure the final.yml and provate.yml files in the config dir are the vanilla copies and do a bosh sync blobs

1. Recreate the non-submodule directories in the src/ folder to the blobs directory (not to be confused with the (dot)blobs directory)
2. For each symlink in each folder in the src dir, copy the referenced blob to its corresponding folder in the blobs directory. For example:

Given: src/debian_nfs_server/whatever.tgz -> /path/to/.blobs/a44b3f8319984bace61f21bb6d15cee1419e5bf3 (THIS IS AN EXAMPLE)

Do the following:

cp /path/to/.blobs/a44b3f8319984bace61f21bb6d15cee1419e5bf3 blobs/debian_nfs_server/whatever.tgz

3. Empty out blobs.yml in the config folder so that is only contains --- {}
4. Then change your private and final yaml files to you appropriate setup for your s3 store
5. Then do a bosh upload blobs. You should be prompted to upload each blob in turn and blobs.yml will be populated with your blobs and their ids and stuff.

If you are unsure of how to format you final and provate yml files, go here:

https://github.com/cloudfoundry/bosh/blob/master/cli/lib/cli/release.rb#L72-L86

make sure you have provisioned your s3 bucket correctly you need to follow these directions:

https://github.com/cloudfoundry/oss-docs/blob/master/bosh/documentation/bosh_releases.md#setting-up-a-new-release-repository-using-s3

(The bucket policy referenced there is incorrect. Use the IAM builder to make it)

Let me know if this works for you

reply | permalink

Mike Heath

at Jan 17, 2013 at 12:54 am

⇧

It uploads all the blobs in my blobs/* directory just fine. I have those
uploaded to S3 and it works great. I have a script that keeps the
cf-release blobs synced with our blobs in S3. The other devs on my team can
checkout our cf-release fork and do a 'bosh sync blobs' and all the blobs
in the in the blobs directory get downloaded. No problems there.

The problem we're seeing (that I did not anticipate) is that it appears
that jobs and packages get cached in the blobstore as well. The index.yml
files in /.final_builds are full of blobstore_id fields that reference what
appear to be tarballs in the http://blob.cfblob.com blobstore.

These job and package blobs are not listed in the config/blobs.yml. Which
is why I was surprised to find this problem. I like the idea of caching
these, especially to ensure consistency for future deployments. I don't
like that I can't do a "bosh add blob ..." to get them in S3. :)

I've been chatting with Nick Terry about his patch. We'll likely go down
that route. I like the idea of being able to chain blobstores.

-Mike

On Wed, Jan 16, 2013 at 5:07 PM, Dr Nic Williams wrote:

What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath wrote:

Adding --force eliminates the warnings about my changes not being
committed in git but doesn't prevent bosh from trying to download packages
from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We have
been able to move all of the blobs in cf-release/blobs over. That part was
easy. When we try to create a release doing 'bosh create release' we get
errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
------------------------------**---

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Dr Nic Williams

at Jan 17, 2013 at 1:00 am

⇧

Chaining/overriding blob stores sounds very nice. I look forward to seeing
any finish products/repos/gists.

On Wednesday, January 16, 2013, Mike Heath wrote:

It uploads all the blobs in my blobs/* directory just fine. I have those
uploaded to S3 and it works great. I have a script that keeps the
cf-release blobs synced with our blobs in S3. The other devs on my team can
checkout our cf-release fork and do a 'bosh sync blobs' and all the blobs
in the in the blobs directory get downloaded. No problems there.

The problem we're seeing (that I did not anticipate) is that it appears
that jobs and packages get cached in the blobstore as well. The index.yml
files in /.final_builds are full of blobstore_id fields that reference what
appear to be tarballs in the http://blob.cfblob.com blobstore.

These job and package blobs are not listed in the config/blobs.yml. Which
is why I was surprised to find this problem. I like the idea of caching
these, especially to ensure consistency for future deployments. I don't
like that I can't do a "bosh add blob ..." to get them in S3. :)

I've been chatting with Nick Terry about his patch. We'll likely go down
that route. I like the idea of being able to chain blobstores.

-Mike

On Wed, Jan 16, 2013 at 5:07 PM, Dr Nic Williams ({}, 'cvml', 'drnicwilliams@gmail.com');>

wrote:
What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath ({}, 'cvml', 'elcapo@gmail.com');>

wrote:
Adding --force eliminates the warnings about my changes not being
committed in git but doesn't prevent bosh from trying to download packages
from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We have
been able to move all of the blobs in cf-release/blobs over. That part was
easy. When we try to create a release doing 'bosh create release' we get
errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
------------------------------**---

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Nterry

at Jan 17, 2013 at 11:00 pm

⇧

YAY! We committed the composite blobstore client to be accepted!!! See it
here:

http://reviews.cloudfoundry.org/#/c/13445/

Also whats the process for submitting fixes/additions to the oss-docs
project (we need to document the format for the private and final yml files
to use the composite blobstore client)? Do I just create a pull request in
github?

On Wednesday, January 16, 2013 6:00:40 PM UTC-7, Dr Nic Williams wrote:

Chaining/overriding blob stores sounds very nice. I look forward to seeing
any finish products/repos/gists.

On Wednesday, January 16, 2013, Mike Heath wrote:

It uploads all the blobs in my blobs/* directory just fine. I have those
uploaded to S3 and it works great. I have a script that keeps the
cf-release blobs synced with our blobs in S3. The other devs on my team can
checkout our cf-release fork and do a 'bosh sync blobs' and all the blobs
in the in the blobs directory get downloaded. No problems there.

The problem we're seeing (that I did not anticipate) is that it appears
that jobs and packages get cached in the blobstore as well. The index.yml
files in /.final_builds are full of blobstore_id fields that reference what
appear to be tarballs in the http://blob.cfblob.com blobstore.

These job and package blobs are not listed in the config/blobs.yml. Which
is why I was surprised to find this problem. I like the idea of caching
these, especially to ensure consistency for future deployments. I don't
like that I can't do a "bosh add blob ..." to get them in S3. :)

I've been chatting with Nick Terry about his patch. We'll likely go down
that route. I like the idea of being able to chain blobstores.

-Mike

On Wed, Jan 16, 2013 at 5:07 PM, Dr Nic Williams <drnicwilliams@gmail.com

wrote:
What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath wrote:

Adding --force eliminates the warnings about my changes not being
committed in git but doesn't prevent bosh from trying to download packages
from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We have
been able to move all of the blobs in cf-release/blobs over. That part was
easy. When we try to create a release doing 'bosh create release' we get
errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
------------------------------**---

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Dr Nic Williams

at Jan 17, 2013 at 11:28 pm

⇧

CONGRATS!
Well done for submitting a (first?) patch!

Nic

On Thu, Jan 17, 2013 at 3:00 PM, wrote:

YAY! We committed the composite blobstore client to be accepted!!! See it
here:

http://reviews.cloudfoundry.org/#/c/13445/

Also whats the process for submitting fixes/additions to the oss-docs
project (we need to document the format for the private and final yml files
to use the composite blobstore client)? Do I just create a pull request in
github?

On Wednesday, January 16, 2013 6:00:40 PM UTC-7, Dr Nic Williams wrote:

Chaining/overriding blob stores sounds very nice. I look forward to
seeing any finish products/repos/gists.

On Wednesday, January 16, 2013, Mike Heath wrote:

It uploads all the blobs in my blobs/* directory just fine. I have those
uploaded to S3 and it works great. I have a script that keeps the
cf-release blobs synced with our blobs in S3. The other devs on my team can
checkout our cf-release fork and do a 'bosh sync blobs' and all the blobs
in the in the blobs directory get downloaded. No problems there.

The problem we're seeing (that I did not anticipate) is that it appears
that jobs and packages get cached in the blobstore as well. The index.yml
files in /.final_builds are full of blobstore_id fields that reference what
appear to be tarballs in the http://blob.cfblob.com blobstore.

These job and package blobs are not listed in the config/blobs.yml.
Which is why I was surprised to find this problem. I like the idea of
caching these, especially to ensure consistency for future deployments. I
don't like that I can't do a "bosh add blob ..." to get them in S3. :)

I've been chatting with Nick Terry about his patch. We'll likely go down
that route. I like the idea of being able to chain blobstores.

-Mike

On Wed, Jan 16, 2013 at 5:07 PM, Dr Nic Williams <
drnicwilliams@gmail.com> wrote:

What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath wrote:

Adding --force eliminates the warnings about my changes not being
committed in git but doesn't prevent bosh from trying to download packages
from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We have
been able to move all of the blobs in cf-release/blobs over. That part was
easy. When we try to create a release doing 'bosh create release' we get
errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
------------------------------****---

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

Dr Nic Williams

at Jan 17, 2013 at 11:28 pm

⇧

AFAIK, oss-docs is also on gerrit at the moment.

On Thu, Jan 17, 2013 at 3:20 PM, Dr Nic Williams wrote:

CONGRATS!
Well done for submitting a (first?) patch!

Nic

On Thu, Jan 17, 2013 at 3:00 PM, wrote:

YAY! We committed the composite blobstore client to be accepted!!! See it
here:

http://reviews.cloudfoundry.org/#/c/13445/

Also whats the process for submitting fixes/additions to the oss-docs
project (we need to document the format for the private and final yml files
to use the composite blobstore client)? Do I just create a pull request in
github?

On Wednesday, January 16, 2013 6:00:40 PM UTC-7, Dr Nic Williams wrote:

Chaining/overriding blob stores sounds very nice. I look forward to
seeing any finish products/repos/gists.

On Wednesday, January 16, 2013, Mike Heath wrote:

It uploads all the blobs in my blobs/* directory just fine. I have
those uploaded to S3 and it works great. I have a script that keeps the
cf-release blobs synced with our blobs in S3. The other devs on my team can
checkout our cf-release fork and do a 'bosh sync blobs' and all the blobs
in the in the blobs directory get downloaded. No problems there.

The problem we're seeing (that I did not anticipate) is that it appears
that jobs and packages get cached in the blobstore as well. The index.yml
files in /.final_builds are full of blobstore_id fields that reference what
appear to be tarballs in the http://blob.cfblob.com blobstore.

These job and package blobs are not listed in the config/blobs.yml.
Which is why I was surprised to find this problem. I like the idea of
caching these, especially to ensure consistency for future deployments. I
don't like that I can't do a "bosh add blob ..." to get them in S3. :)

I've been chatting with Nick Terry about his patch. We'll likely go
down that route. I like the idea of being able to chain blobstores.

-Mike

On Wed, Jan 16, 2013 at 5:07 PM, Dr Nic Williams <
drnicwilliams@gmail.com> wrote:

What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath wrote:

Adding --force eliminates the warnings about my changes not being
committed in git but doesn't prevent bosh from trying to download packages
from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams
wrote:

bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:

We're trying to fork cf-release and migrate our blobs to S3. We
have been able to move all of the blobs in cf-release/blobs over. That part
was easy. When we try to create a release doing 'bosh create release' we
get errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
------------------------------****---

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

James Bayer

at Jan 18, 2013 at 1:51 am

⇧

Actually we're in the middle of transitioning some processes for contributing to be a bit simpler, can you try a standard github pull request and see if it gets auto-closed? I think it's not going to do that any longer.

Thank you,

James Bayer

On Jan 17, 2013, at 3:20 PM, Dr Nic Williams wrote:

AFAIK, oss-docs is also on gerrit at the moment.

On Thu, Jan 17, 2013 at 3:20 PM, Dr Nic Williams wrote:
CONGRATS!
Well done for submitting a (first?) patch!

Nic

On Thu, Jan 17, 2013 at 3:00 PM, wrote:
YAY! We committed the composite blobstore client to be accepted!!! See it here:

http://reviews.cloudfoundry.org/#/c/13445/

Also whats the process for submitting fixes/additions to the oss-docs project (we need to document the format for the private and final yml files to use the composite blobstore client)? Do I just create a pull request in github?

On Wednesday, January 16, 2013 6:00:40 PM UTC-7, Dr Nic Williams wrote:
Chaining/overriding blob stores sounds very nice. I look forward to seeing any finish products/repos/gists.

On Wednesday, January 16, 2013, Mike Heath wrote:
It uploads all the blobs in my blobs/* directory just fine. I have those uploaded to S3 and it works great. I have a script that keeps the cf-release blobs synced with our blobs in S3. The other devs on my team can checkout our cf-release fork and do a 'bosh sync blobs' and all the blobs in the in the blobs directory get downloaded. No problems there.

The problem we're seeing (that I did not anticipate) is that it appears that jobs and packages get cached in the blobstore as well. The index.yml files in /.final_builds are full of blobstore_id fields that reference what appear to be tarballs in the http://blob.cfblob.com blobstore.

These job and package blobs are not listed in the config/blobs.yml. Which is why I was surprised to find this problem. I like the idea of caching these, especially to ensure consistency for future deployments. I don't like that I can't do a "bosh add blob ..." to get them in S3. :)

I've been chatting with Nick Terry about his patch. We'll likely go down that route. I like the idea of being able to chain blobstores.

-Mike

On Wed, Jan 16, 2013 at 5:07 PM, Dr Nic Williams wrote:
What does "bosh upload blobs" do?

On Wed, Jan 16, 2013 at 3:24 PM, Mike Heath wrote:
Adding --force eliminates the warnings about my changes not being committed in git but doesn't prevent bosh from trying to download packages from the blobstore, unfortunately.

On Wednesday, January 16, 2013 3:57:19 PM UTC-7, Dr Nic Williams wrote:
bosh create release --force - should skip the check

or
bosh upload blobs
bosh create release

On Wed, Jan 16, 2013 at 2:19 PM, Mike Heath wrote:
We're trying to fork cf-release and migrate our blobs to S3. We have been able to move all of the blobs in cf-release/blobs over. That part was easy. When we try to create a release doing 'bosh create release' we get errors saying that it can't find packages in the blobstore.

Specifically we're getting:

Building DEV release
---------------------------------

Building packages
-----------------
Building daylimit...
Final version: NOT FOUND
Dev version: FOUND LOCAL

Building mysql55...
Final version: Downloading `mysql55 (4)'...
Final version of `mysql55' not found in blobstore

Is there an easy way to migrate the package blobs to S3?

-Mike

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

--
Dr Nic Williams
http://drnicwilliams.com
cell +1 (415) 860-2185

reply | permalink

group	bosh-users @
posted	Jan 16, '13 at 10:19p
active	Jan 18, '13 at 1:51a
posts	12
users	6

responses	expanded	Hotkey: s
font	variable	Hotkey: f
user style	avatars	Hotkey: a

[bosh-users] Migrating cf-release blobs to S3

Search Discussions

Related Discussions

6 users in discussion