FAQ
Hi,

I have a big server with 24 Disk on 2 3ware card.

When i write data on my nfs server everything is fine but when i want to read data i have a lot of cpu wait.

[root at NFS /]# vmstat 1
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 1 112 9592 7140 1879496 0 0 621 1180 6 2 0 2 86 11 0
0 9 112 10668 7148 1877760 0 0 84 21600 1926 454 0 3 49 49 0
1 3 112 9308 7156 1879092 0 0 1400 1580 2123 2009 0 2 50 48 0
0 0 112 9432 7156 1879476 0 0 3664 0 2275 2862 0 3 81 17 0
0 1 112 9212 7172 1879588 0 0 2196 32 2132 2293 0 2 66 32 0
1 0 112 9848 7180 1878864 0 0 1268 4168 1852 1934 0 2 84 14 0
0 7 112 11208 7180 1876936 0 0 92 13124 1339 355 0 2 50 48 0
0 0 112 10708 7188 1877512 0 0 628 16 1552 1014 0 1 84 16 0
0 1 112 10344 7188 1878044 0 0 472 0 1334 721 0 1 89 10 0
0 1 112 10372 7196 1878136 0 0 1936 32 2268 2850 0 2 50 48 0
0 0 112 10336 7192 1878312 0 0 2220 0 2308 2966 0 2 51 47 0
0 0 112 9716 7200 1879012 0 0 696 28 1484 885 0 1 89 11 0
0 2 112 9400 7200 1879280 0 0 236 0 2112 5819 0 8 75 18 0
0 0 112 9980 7196 1878816 0 0 1360 20 1840 1743 0 1 83 15 0
0 15 112 9168 7196 1879580 0 0 768 0 1423 728 0 1 91 9 0
0 0 112 10756 7200 1877752 0 0 4068 32 2788 4225 0 4 70 26 0

I have tuned nfs server with this option :

RPCNFSDARGS="-N 2"
RPCNFSDARGS="-N 4"
MOUNTD_NFS_V3="yes"
RPCNFSDCOUNT0

about the file system, i use ext4 on LVM partition.

Do you have any idea about that.

Fabien FAYE
RHCE
www.generationip.com
Free network tools & HOWTO for centos and Redhat

Search Discussions

  • Jim Perrin at Jan 25, 2010 at 1:27 pm

    On Mon, Jan 25, 2010 at 8:39 AM, fabien faye wrote:
    Hi,

    I have a big server with 24 Disk on 2 3ware card.
    Define 'big'. You're not giving much in terms of memory or cpu specs.
    When i write data on my nfs server everything is fine but when i want to read data i have a lot of cpu wait.
    Okay. Have you looked at any performance tuning or elevator/scheduler
    adjustments? File system tuning? What have you tried on your own so
    far to debug this?
    about the file system, i use ext4 on LVM partition.
    Why? Is there a performance reason for you to use ext4 vs ext3?
    Do you have any idea about that.
    Not based on the little information you've provided. It would help for
    you to supply more detail, and the things you've already looked at.

    --
    During times of universal deceit, telling the truth becomes a revolutionary act.
    George Orwell
  • Fabien faye at Jan 25, 2010 at 3:03 pm
    Hi,

    You can find below the spec of my server:

    cat /proc/cpuinfo
    processor : 0
    vendor_id : AuthenticAMD
    cpu family : 15
    model : 37
    model name : AMD Opteron(tm) Processor 248
    stepping : 1
    cpu MHz : 2200.034
    cache size : 1024 KB
    fpu : yes
    fpu_exception : yes
    cpuid level : 1
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm
    bogomips : 4400.06
    TLB size : 1024 4K pages
    clflush size : 64
    cache_alignment : 64
    address sizes : 40 bits physical, 48 bits virtual
    power management: ts fid vid ttp

    processor : 1
    vendor_id : AuthenticAMD
    cpu family : 15
    model : 37
    model name : AMD Opteron(tm) Processor 248
    stepping : 1
    cpu MHz : 2200.034
    cache size : 1024 KB
    fpu : yes
    fpu_exception : yes
    cpuid level : 1
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm
    bogomips : 4399.42
    TLB size : 1024 4K pages
    clflush size : 64
    cache_alignment : 64
    address sizes : 40 bits physical, 48 bits virtual
    power management: ts fid vid ttp

    free
    total used free shared buffers cached
    Mem: 2059236 2049896 9340 0 9800 1874828
    -/+ buffers/cache: 165268 1893968
    Swap: 4095992 112 4095880


    Linux 1-NFS.domain.com 2.6.18-164.9.1.el5 #1 SMP Tue Dec 15 20:57:57 EST 2009 x86_64 x86_64 x86_64 GNU/Linux

    --- Physical volume ---
    PV Name /dev/sdc1
    VG Name vgspace
    PV Size 4.55 TB / not usable 1.97 MB
    Allocatable yes
    PE Size (KByte) 4096
    Total PE 1192067
    Free PE 795267
    Allocated PE 396800

    --- Physical volume ---
    PV Name /dev/sdd1
    VG Name vgspace
    PV Size 4.55 TB / not usable 1.97 MB
    Allocatable yes
    PE Size (KByte) 4096
    Total PE 1192067
    Free PE 808067
    Allocated PE 384000

    I need ext4 for file bigger than 2TB.

    /dev/mapper/vgspace-vol1 on /vol/vol1 type ext4 (rw,noatime)
    /dev/mapper/vgspace-vol2 on /vol/vol2 type ext4 (rw,noatime)

    # tune2fs -l /dev/mapper/vgspace-vol1
    tune2fs 1.39 (29-May-2006)
    tune2fs: Filesystem has unsupported feature(s) while trying to open /dev/mapper/vgspace-vol1
    Couldn't find valid filesystem superblock.
    # tune2fs -l /dev/mapper/vgspace-vol2
    tune2fs 1.39 (29-May-2006)
    tune2fs: Filesystem has unsupported feature(s) while trying to open /dev/mapper/vgspace-vol2
    Couldn't find valid filesystem superblock.

    export nfs :

    /vol/vol1 192.168.*.0/255.255.255.0(async,no_subtree_check,rw,no_root_squash)
    /vol/vol2 192.168.*.0/255.255.255.0(async,no_subtree_check,rw,no_root_squash)

    Best regards

    Fabien FAYE






    ----- Original Message -----
    From: "Jim Perrin" <jperrin at gmail.com>
    To: "CentOS mailing list" <centos at centos.org>
    Sent: Monday, January 25, 2010 2:27:41 PM
    Subject: Re: [CentOS] Too much cpu wait on nfs server when we need to read data on it
    On Mon, Jan 25, 2010 at 8:39 AM, fabien faye wrote:
    Hi,

    I have a big server with 24 Disk on 2 3ware card.
    Define 'big'. You're not giving much in terms of memory or cpu specs.
    When i write data on my nfs server everything is fine but when i want to read data i have a lot of cpu wait.
    Okay. Have you looked at any performance tuning or elevator/scheduler
    adjustments? File system tuning? What have you tried on your own so
    far to debug this?
    about the file system, i use ext4 on LVM partition.
    Why? Is there a performance reason for you to use ext4 vs ext3?
    Do you have any idea about that.
    Not based on the little information you've provided. It would help for
    you to supply more detail, and the things you've already looked at.

    --
    During times of universal deceit, telling the truth becomes a revolutionary act.
    George Orwell
    _______________________________________________
    CentOS mailing list
    CentOS at centos.org
    http://lists.centos.org/mailman/listinfo/centos

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcentos @
categoriescentos
postedJan 25, '10 at 1:27p
activeJan 25, '10 at 3:03p
posts3
users2
websitecentos.org
irc#centos

2 users in discussion

Fabien faye: 2 posts Jim Perrin: 1 post

People

Translate

site design / logo © 2022 Grokbase