From 395793a8ecf41cb52eb6bdd150ad31c1fb447963 Mon Sep 17 00:00:00 2001 From: "Daniel P. Berrange" Date: Mon, 22 Nov 2010 16:44:13 +0000 Subject: [PATCH] Add initial docs about the lock managers --- docs/internals/locking.html.in | 257 +++++++++++++++++++++++++++++++++ docs/sitemap.html.in | 4 + 2 files changed, 261 insertions(+) create mode 100644 docs/internals/locking.html.in diff --git a/docs/internals/locking.html.in b/docs/internals/locking.html.in new file mode 100644 index 0000000000..3790ef042e --- /dev/null +++ b/docs/internals/locking.html.in @@ -0,0 +1,257 @@ + + +

Resource Lock Manager

+ + + +

+ This page describes the design of the resource lock manager + that is used for locking disk images, to ensure exclusive + access to content. +

+ +

Goals

+ +

+ The high level goal is to prevent the same disk image being + used by more than one QEMU instance at a time (unless the + disk is marked as sharable, or readonly). The scenarios + to be prevented are thus: +

+ +
    +
  1. + Two different guests running configured to point at the + same disk image. +
    2. +
  2. + One guest being started more than once on two different + machines due to admin mistake +
    3. +
  3. + One guest being started more than once on a single machine + due to libvirt driver bug on a single machine. +
    4. +
+ +

Requirements

+ +

+ The high level goal leads to a set of requirements + for the lock manager design +

+ +
    +
  1. + A lock must be held on a disk whenever a QEMU process + has the disk open +
    2. +
  2. + The lock scheme must allow QEMU to be configured with + readonly, shared write, or exclusive writable disks +
    3. +
  3. + A lock handover must be performed during the migration + process where 2 QEMU processes will have the same disk + open concurrently. +
    4. +
  4. + The lock manager must be able to identify and kill the + process accessing the resource if the lock is revoked. +
    5. +
  5. + Locks can be acquired for arbitrary VM related resources, + as determined by the management application. +
    6. +
+ +

Design

+ +

+ Within a lock manager the following series of operations + will need to be supported. +

+ + + +

Plugin Implementations

+ +

+ Lock manager implementations are provided as LGPLv2+ + licensed, dlopen()able library modules. The plugins + will be loadable from the following location: +

+ + 
+/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
+
+ +

+ The lock manager plugin must export a single ELF + symbol named virLockDriverImpl, which is + a static instance of the virLockDriver + struct. The struct is defined in the header file +

+ + 
+      #include <libvirt/plugins/lock_manager.h>
+
+ +

+ All callbacks in the struct must be initialized + to non-NULL pointers. The semantics of each + callback are defined in the API docs embedded + in the previously mentioned header file +

+ +

QEMU Driver integration

+ +

+ With the QEMU driver, the lock plugin will be set + in the /etc/libvirt/qemu.conf configuration + file by specifying the lock manager name. +

+ + 
+      lockManager="sanlock"
+
+ +

+ By default the lock manager will be a 'no op' implementation + for backwards compatibility +

+ +

Lock usage patterns

+ +

+ The following psuedo code illustrates the common + patterns of operations invoked on the lock + manager plugin callbacks. +

+ +

Lock acquisition

+ +

+ Initial lock acquisition will be performed from the + process that is to own the lock. This is typically + the QEMU child process, in between the fork+exec + pairing. When adding further resources on the fly, + to an existing object holding locks, this will be + done from the libvirtd process. +

+ + 
+      virLockManagerParam params[] = {
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
+          .key = "uuid",
+        },
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
+          .key = "name",
+          .value = { .str = dom->def->name },
+        },
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
+          .key = "id",
+          .value = { .i = dom->def->id },
+        },
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
+          .key = "pid",
+          .value = { .i = dom->pid },
+        },
+      };
+      mgr = virLockManagerNew(lockPlugin,
+                              VIR_LOCK_MANAGER_TYPE_DOMAIN,
+                              ARRAY_CARDINALITY(params),
+                              params,
+                              0)));
+
+      foreach (initial disks)
+          virLockManagerAddResource(mgr,
+                                    VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
+                                    $path, 0, NULL, $flags);
+
+      if (virLockManagerAcquire(lock, NULL, 0) < 0);
+        ...abort...
+
+ +

Lock release

+ +

+ The locks are all implicitly released when the process + that acquired them exits, however, a process may + voluntarily give up the lock by running +

+ + 
+      char *state = NULL;
+      virLockManagerParam params[] = {
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
+          .key = "uuid",
+        },
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
+          .key = "name",
+          .value = { .str = dom->def->name },
+        },
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
+          .key = "id",
+          .value = { .i = dom->def->id },
+        },
+        { .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
+          .key = "pid",
+          .value = { .i = dom->pid },
+        },
+      };
+      mgr = virLockManagerNew(lockPlugin,
+                              VIR_LOCK_MANAGER_TYPE_DOMAIN,
+                              ARRAY_CARDINALITY(params),
+                              params,
+                              0)));
+
+      foreach (initial disks)
+          virLockManagerAddResource(mgr,
+                                    VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
+                                    $path, 0, NULL, $flags);
+
+      virLockManagerRelease(mgr, & state, 0);
+
+ +

+ The returned state string can be passed to the + virLockManagerAcquire method to + later re-acquire the exact same locks. This + state transfer is commonly used when performing + live migration of virtual machines. By validating + the state the lock manager can ensure no other + VM has re-acquire the same locks on a different + host. The state can also be obtained without + releasing the locks, by calling the + virLockManagerInquire method. +

+ + + diff --git a/docs/sitemap.html.in b/docs/sitemap.html.in index ad8dc7b0f4..db2963e81d 100644 --- a/docs/sitemap.html.in +++ b/docs/sitemap.html.in @@ -284,6 +284,10 @@ Spawning commands Spawning commands from libvirt driver code +
  • + Lock managers + Use lock managers to protect disk content +
  • -- GitLab