[PATCH] md: fix is_mddev_idle calculation now that disk/sector accounting...

[PATCH] md: fix is_mddev_idle calculation now that disk/sector accounting happens when request completes

md needs to monitor the rate of requests to its devices when doing
resync/recovery so that it can back-off when there is non-resync IO.  It
does this by comparing resync IO, which it counts, with total IO which is
taken from disk_stats.

disk_stats were recently changed to account sectors when a request
completes instead of when it is queued.  This upsets md's calculations.

We could do the sync_io accounting at the end of requests too, but that has
problems.  If an underlying device is an md array, the accounting will
still be done when the request is submitted.  This could be changed for
some raid levels, but it cannot be changed for raid0 or linear without
substantial code changes.

So instead, we increase the error that is_mddev_idle allows, up to the
maximum amount of resync IO that can be in flight at any time.  The
calculation is current fragile as each personality as different limits for
in-flight resync.  This should be fixed up.

For now, this simple patch fixes the problem.

Increasing the error margin decreases the sensitivity to non-resync IO.  To
partially compensate for this, the time to wait when non-resync IO is
detected is increased so that less steady IO is required to keep the resync
at bay.
Signed-off-by: NNeil Brown <neilb@suse.de>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

drivers/md/md.c

@@ -3846,11 +3846,20 @@ static int is_mddev_idle(mddev_t *mddev)
 		curr_events = disk_stat_read(disk, sectors[0]) + 
 				disk_stat_read(disk, sectors[1]) - 
 				atomic_read(&disk->sync_io);
-		/* Allow some slack between valud of curr_events and last_events,
-		 * as there are some uninteresting races.
+		/* The difference between curr_events and last_events
+		 * will be affected by any new non-sync IO (making
+		 * curr_events bigger) and any difference in the amount of
+		 * in-flight syncio (making current_events bigger or smaller)
+		 * The amount in-flight is currently limited to
+		 * 32*64K in raid1/10 and 256*PAGE_SIZE in raid5/6
+		 * which is at most 4096 sectors.
+		 * These numbers are fairly fragile and should be made
+		 * more robust, probably by enforcing the
+		 * 'window size' that md_do_sync sort-of uses.
+		 *
 		 * Note: the following is an unsigned comparison.
 		 */
-		if ((curr_events - rdev->last_events + 32) > 64) {
+		if ((curr_events - rdev->last_events + 4096) > 8192) {
 			rdev->last_events = curr_events;
 			idle = 0;
 		}
@@ -4109,7 +4118,7 @@ static void md_do_sync(mddev_t *mddev)
 		if (currspeed > sysctl_speed_limit_min) {
 			if ((currspeed > sysctl_speed_limit_max) ||
 					!is_mddev_idle(mddev)) {
-				msleep(250);
+				msleep(500);
 				goto repeat;
 			}
 		}