Brandon Ooi (br...@hotornot.com)

Profile | Posts (6)

User Information

Display Name:

Brandon Ooi

Partial Email Address:

br...@hotornot.com

Posts:

6	total
6	in MogileFS

5 Most Recent

All Posts

1) Brandon Ooi over-replicated files / fsck behavior

+1 vote

Hi, Again, love mogile, it's the cheap and robust alternative to pricey storage engines (EMC). I...

MogileFS

[ Profile | Reply to group ]

[ Flat Thread Threaded ]

Hi,

Again, love mogile, it's the cheap and robust alternative to pricey 
storage engines (EMC). I recently bumped my mindevcount from 2 to 3 and 
things seemed to go haywire. This may be related to another post about 
overreplicated files.

I have two tracker machines each with replicate_jobs 5. Things seemed to 
be okay at a mindevcount of 2, but once at 3 lots of files (maybe 25k) 
get replicated more than 3 times (up to 10 times!). I've restarted my 
trackers with only 1 replicate_job total, sacrificing the redundancy. 
This seemed to bring it under control.

My question is, what is the correct setup to prevent this from happening 
and keep the system redundant.

Also, what is the behavior of fsck? Documentation is a bit lacking in 
this area and I would love to help fill in those blanks but I don't 
really know. Does it fix this kind of damage? Will it bump the number of 
replicas of old files from 2 to 3?

2) Brandon Ooi Re: Mogile Startup scripts?

+1 vote

sorry sudo not su b ...

MogileFS

[ Profile | Reply to group ]

[ Flat Thread Threaded ]

sorry sudo not su

b


Brandon Ooi wrote:
> try
>
> > su -u mogile mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon
>
> b
>
> [email protected: drpr0ctol...@gmail.com] wrote:
>> I've been able to get MogileFS up and running. Now I need to put 
>> these services into system startup scripts so that they run when the 
>> system is started.  Are there such startup scripts or do I just put 
>> them in /etc/rc.d/rc.local?
>>  
>> The only item I'm having trouble with is the tracker.  Right now I 
>> start it by doing a "su mogile", then running "mogilefsd -c 
>> /etc/mogilefs/mogilefsd.conf --daemon".  How would I put that into 
>> the /etc/rc.d/rc.local?
>

3) Brandon Ooi Re: Mogile Startup scripts?

+1 vote

try b > /etc/rc.d/rc.local?...

MogileFS

[ Profile | Reply to group ]

[ Flat Thread Threaded ]

try

 > su -u mogile mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon

b


[email protected: drpr0ctol...@gmail.com] wrote:
> I've been able to get MogileFS up and running. Now I need to put these 
> services into system startup scripts so that they run when the system 
> is started.  Are there such startup scripts or do I just put them in 
> /etc/rc.d/rc.local?
>  
> The only item I'm having trouble with is the tracker.  Right now I 
> start it by doing a "su mogile", then running "mogilefsd -c 
> /etc/mogilefs/mogilefsd.conf --daemon".  How would I put that into the 
> /etc/rc.d/rc.local?

4) Brandon Ooi disk migration to a new host?

+1 vote

Hi, We've been using Mogilefs for a long time now and it's been working out great! (the "new"...

MogileFS

[ Profile | Reply to group ]

[ Flat Thread Threaded ]

Hi,

We've been using Mogilefs for a long time now and it's been working out 
great! (the "new" drain/rebalance stuff is awesome!). Quick question for 
anybody who has tried this.

We are migrating datacenters and I was wondering if it would be possible 
to "migrate" disks. For example:

* mark dev1 as down
* copy (or physically move) the dev1 disk to the other datacenter.
* change the host that dev1 is on (devices table)
* mark dev1 as up?

Any experience or input would be appreciated. Thanks!

Brandon

5) Brandon Ooi Re: Analysis of the over-replication issue

+1 vote

oh interesting, that's good to know. thx! brandon ...

MogileFS

[ Profile | Reply to group ]

[ Flat Thread Threaded ]

oh interesting, that's good to know. thx!

brandon


On 8/6/07, Brad Fitzpatrick <brad@danga.com> wrote:
>
> On Tue, 24 Jul 2007, Brandon Ooi wrote:
>
> > Perhaps it's because you have the replicate workers running on more than
> > one mogilefsd instance? Maybe they will run into each other and
> > replicate more than once?
> >
> > We have it set up so only one machine runs the replicate, delete
> workers.
> >
> > I have no idea how that all works but that's how we have it set up.
>
> They shouldn't get in each other's way.  It's designed to have them all
> running replicators (otherwise if you lose your mogilefsd doing replicate,
> you have a single point of failure!).
>
> There might be a bug, but I've never seen it, and every installation I've
> seen has multiple mogilefsd, all doing replicate.
>
> - Brad
>

Highlight

Profile | Posts (6)

Home > People > Brandon Ooi