Messages in this thread | | | Date | Wed, 26 Aug 2009 06:44:42 -0700 (PDT) | From | david@lang ... | Subject | Re: [patch] document flash/RAID dangers |
| |
On Wed, 26 Aug 2009, Ric Wheeler wrote:
> On 08/26/2009 08:40 AM, Theodore Tso wrote: >> On Wed, Aug 26, 2009 at 07:58:40AM -0400, Ric Wheeler wrote: >>>> Drive in raid 5 failed; hot spare was available (no idea about >>>> UPS). System apparently locked up trying to talk to the failed drive, >>>> or maybe admin just was not patient enough, so he just powercycled the >>>> array. He lost the array. >>>> >>>> So while most people will not agressively powercycle the RAID array, >>>> drive failure still provokes little tested error paths, and getting >>>> unclean shutdown is quite easy in such case. >>> >>> Then what we need to document is do not power cycle an array during a >>> rebuild, right? >> >> Well, the softwar raid layer could be improved so that it implements >> scrubbing by default (i.e., have the md package install a cron job to >> implement a periodict scrub pass automatically). The MD code could >> also regularly check to make sure the hot spare is OK; the other >> possibility is that hot spare, which hadn't been used in a long time, >> had silently failed. > > Actually, MD does this scan already (not automatically, but you can set up a > simple cron job to kick off a periodic "check"). It is a delicate balance to > get the frequency of the scrubbing correct.
debian defaults to doing this once a month (first sunday of each month), on some of my systems this scrub takes almost a week to complete.
David Lang
| |