/usr/share/doc/libvirt-doc/locking.html is in libvirt-doc 0.9.8-2ubuntu17.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 | <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<!--
This file is autogenerated from locking.html.in
Do not edit this file. Changes will be lost.
-->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="stylesheet" type="text/css" href="main.css" />
<link rel="SHORTCUT ICON" href="32favicon.png" />
<title>libvirt: Virtual machine disk locking</title>
<meta name="description" content="libvirt, virtualization, virtualization API" />
</head>
<body>
<div id="header">
<div id="headerLogo"></div>
<div id="headerSearch">
<form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><div>
<input id="query" name="query" type="text" size="12" value="" />
<input id="submit" name="submit" type="submit" value="Search" />
</div></form>
</div>
</div>
<div id="body">
<div id="menu">
<ul class="l0"><li>
<div>
<a title="Front page of the libvirt website" class="inactive" href="index.html">Home</a>
</div>
</li><li>
<div>
<a title="Details of new features and bugs fixed in each release" class="inactive" href="news.html">News</a>
</div>
</li><li>
<div>
<a title="Applications known to use libvirt" class="inactive" href="apps.html">Applications</a>
</div>
</li><li>
<div>
<a title="Get the latest source releases, binary builds and get access to the source repository" class="inactive" href="downloads.html">Downloads</a>
</div>
</li><li>
<div>
<a title="Information for users, administrators and developers" class="active" href="docs.html">Documentation</a>
<ul class="l1"><li>
<div>
<a title="How to compile libvirt" class="inactive" href="compiling.html">Compiling</a>
</div>
</li><li>
<div>
<a title="Information about deploying and using libvirt" class="active" href="deployment.html">Deployment</a>
<ul class="l2"><li>
<div>
<a title="The URI formats used for connecting to libvirt" class="inactive" href="uri.html">URI format</a>
</div>
</li><li>
<div>
<a title="Enable remote access over TCP" class="inactive" href="remote.html">Remote access</a>
</div>
</li><li>
<div>
<a title="Configure authentication for the libvirt daemon" class="inactive" href="auth.html">Authentication</a>
</div>
</li><li>
<div>
<a title="Migrating guests between machines" class="inactive" href="migration.html">Migration</a>
</div>
</li><li>
<div>
<a title="Access the libvirt daemon from a native Windows client" class="inactive" href="windows.html">Windows port</a>
</div>
</li><li>
<div>
<a title="The library and the daemon logging support" class="inactive" href="logging.html">Logging</a>
</div>
</li><li>
<div>
<a title="Firewall and network filter configuration" class="inactive" href="firewall.html">Firewall</a>
</div>
</li><li>
<div>
<span class="active">Disk locking</span>
</div>
</li><li>
<div>
<a title="Hooks for system specific management" class="inactive" href="hooks.html">Hooks</a>
</div>
</li></ul>
</div>
</li><li>
<div>
<a title="Overview of the logical subsystems in the libvirt API" class="inactive" href="intro.html">Architecture</a>
</div>
</li><li>
<div>
<a title="Description of the XML formats used in libvirt" class="inactive" href="format.html">XML format</a>
</div>
</li><li>
<div>
<a title="Hypervisor specific driver information" class="inactive" href="drivers.html">Drivers</a>
</div>
</li><li>
<div>
<a title="Reference manual for the C public API" class="inactive" href="html/index.html">API reference</a>
</div>
</li><li>
<div>
<a title="Bindings of the libvirt API for other languages" class="inactive" href="bindings.html">Language bindings</a>
</div>
</li><li>
<div>
<a title="Working on the internals of libvirt API, driver and daemon code" class="inactive" href="internals.html">Internals</a>
</div>
</li><li>
<div>
<a title="A guide and reference for developing with libvirt" class="inactive" href="devguide.html">Development Guide</a>
</div>
</li><li>
<div>
<a title="Command reference for virsh" class="inactive" href="virshcmdref.html">Virsh Commands</a>
</div>
</li></ul>
</div>
</li><li>
<div>
<a title="User contributed content" class="inactive" href="http://wiki.libvirt.org">Wiki</a>
</div>
</li><li>
<div>
<a title="Frequently asked questions" class="inactive" href="http://wiki.libvirt.org/page/FAQ">FAQ</a>
</div>
</li><li>
<div>
<a title="How and where to report bugs and request features" class="inactive" href="bugs.html">Bug reports</a>
</div>
</li><li>
<div>
<a title="How to contact the developers via email and IRC" class="inactive" href="contact.html">Contact</a>
</div>
</li><li>
<div>
<a title="Available test suites for libvirt" class="inactive" href="testsuites.html">Test suites</a>
</div>
</li><li>
<div>
<a title="Miscellaneous links of interest related to libvirt" class="inactive" href="relatedlinks.html">Related Links</a>
</div>
</li><li>
<div>
<a title="Overview of all content on the website" class="inactive" href="sitemap.html">Sitemap</a>
</div>
</li></ul>
</div>
<div id="content">
<h1>Virtual machine disk locking</h1>
<ul><li>
<a href="#plugins">Lock manager plugins</a>
</li><li>
<a href="#sanlock">Sanlock daemon setup</a>
</li><li>
<a href="#sanlockplugin">libvirt sanlock plugin configuration</a>
</li><li>
<a href="#sanlockstorage">libvirt sanlock storage configuration</a>
</li><li>
<a href="#qemuconfig">QEMU/KVM driver configuration</a>
</li></ul>
<p>
This page describes how to ensure a single disk cannot be
used by more than one running VM at a time, across any
host in a network. This is critical to avoid data corruption
of guest files systems that are not cluster aware.
</p>
<h2>
<a name="plugins" id="plugins">Lock manager plugins</a>
</h2>
<p>
libvirt includes a pluggable framework for lock managers,
which hypervisor drivers can use to ensure safety for
guest domain disks, and potentially other resources.
At this time there are only two plugin implementations,
a "no op" implementation which does absolutely nothing,
and a <a href="https://fedorahosted.org/sanlock/">sanlock</a> implementation which uses
the Disk Paxos algorithm to ensure safety.
</p>
<h2>
<a name="sanlock" id="sanlock">Sanlock daemon setup</a>
</h2>
<p>
On many operating systems, the <strong>sanlock</strong> plugin
is distributed in a sub-package which needs to be installed
separately from the main libvirt RPM. On a Fedora/RHEL host
this can be done with the <code>yum</code> command
</p>
<pre>
$ su - root
# yum install libvirt-lock-sanlock
</pre>
<p>
The next step is to start the sanlock daemon. For maximum
safety sanlock prefers to have a connection to a watchdog
daemon. This will cause the entire host to be rebooted in
the event that sanlock crashes / terminates abnormally.
To start the watchdog daemon on a Fedora/RHEL host
the following commands can be run:
</p>
<pre>
$ su - root
# chkconfig wdmd on
# service wdmd start
</pre>
<p>
Once the watchdog is running, sanlock can be started
as follows
</p>
<pre>
# chkconfig sanlock on
# service sanlock start
</pre>
<p>
<em>Note:</em> if you wish to avoid the use of the
watchdog, add the following line to <code>/etc/sysconfig/sanlock</code>
before starting it
</p>
<pre>
SANLOCKOPTS="-w 0"
</pre>
<p>
The sanlock daemon must be started on every single host
that will be running virtual machines. So repeat these
steps as neccessary.
</p>
<h2>
<a name="sanlockplugin" id="sanlockplugin">libvirt sanlock plugin configuration</a>
</h2>
<p>
Once the sanlock daemon is running, the next step is to
configure the libvirt sanlock plugin. There is a separate
configuration file for each libvirt driver that is using
sanlock. For QEMU, we will edit <code>/etc/libvirt/qemu-sanlock.conf</code>
There is one mandatory parameter that needs to be set,
the <code>host_id</code>. This is a integer between
1 and 2000, which must be set to a <strong>unique</strong>
value on each host running virtual machines.
</p>
<pre>
$ su - root
# augtool -s set /files/etc/libvirt/qemu-sanlock.conf/host_id 1
</pre>
<p>
Repeat this on every host, changing <strong>1</strong> to a
unique value for the host.
</p>
<h2>
<a name="sanlockstorage" id="sanlockstorage">libvirt sanlock storage configuration</a>
</h2>
<p>
The sanlock plugin needs to create leases in a directory
that is on a filesystem shared between all hosts running
virtual machines. Obvious choices for this include NFS
or GFS2. The libvirt sanlock plugin expects its lease
directory be at <code>/var/lib/libvirt/sanlock</code>
so update the host's <code>/etc/fstab</code> to mount
a suitable shared/cluster filesystem at that location
</p>
<pre>
$ su - root
# echo "some.nfs.server:/export/sanlock /var/lib/libvirt/sanlock nfs hard,nointr 0 0" >> /etc/fstab
# mount /var/lib/libvirt/sanlock
</pre>
<p>
In terms of storage requirements, if the filesystem
uses 512 byte sectors, you need to allow for <code>1MB</code>
of storage for each guest disk. So if you have a network
with 20 virtualization hosts, each running 50 virtual
machines and an average of 2 disks per guest, you will
need <code>20*50*2 == 2000 MB</code> of storage for
sanlock.
</p>
<p>
On one of the hosts on the network is it wise to setup
a cron job which runs the <code>virt-sanlock-cleanup</code>
script periodically. This scripts deletes any lease
files which are not currently in use by running virtual
machines, freeing up disk space on the shared filesystem.
Unless VM disks are very frequently created + deleted
it should be sufficient to run the cleanup once a week.
</p>
<h2>
<a name="qemuconfig" id="qemuconfig">QEMU/KVM driver configuration</a>
</h2>
<p>
The QEMU/KVM driver is fully integrated with the lock
manager framework as of release <span>0.9.3</span>.
The out of the box configuration, however, currently
uses the <strong>nop</strong> lock manager plugin.
To get protection for disks, it is thus necessary
to reconfigure QEMU to activate the <strong>sanlock</strong>
driver. This is achieved by editing the QEMU driver
configuration file (<code>/etc/libvirt/qemu.conf</code>)
and changing the <code>lock_manager</code> configuration
tunable.
</p>
<pre>
$ su - root
# augtool -s set /files/etc/libvirt/qemu.conf/lock_manager sanlock
# service libvirtd restart
</pre>
<p>
If all went well, libvirtd will have talked to sanlock
and created the basic lockspace. This can be checked
by looking for existance of the following file
</p>
<pre>
# ls /var/lib/libvirt/sanlock/
__LIBVIRT__DISKS__
</pre>
<p>
Every time you start a guest, additional lease files will appear
in this directory, one for each virtual disk. The lease
files are named based on the MD5 checksum of the fully qualified
path of the virtual disk backing file. So if the guest is given
a disk backed by <code>/var/lib/libvirt/images/demo.img</code>
expect to see a lease <code>/var/lib/libvirt/sanlock/bfa0240911bc17753e0b473688822159</code>
</p>
<p>
It should be obvious that for locking to work correctly, every
host running virtual machines should have storage configured
in the same way. The easiest way to do this is to use the libvirt
storage pool capability to configure any NFS volumes, iSCSI targets,
or SCSI HBAs used for guest storage. Simply replicate the same
storage pool XML across every host. It is important that any
storage pools exposing block devices are configured to create
volume paths under <code>/dev/disks/by-path</code> to ensure
stable paths across hosts. An example iSCSI configuration
which ensures this is:
</p>
<pre>
<pool type='iscsi'>
<name>myiscsipool</name>
<source>
<host name='192.168.254.8'/>
<device path='your-iscsi-target-iqn'/>
</source>
<target>
<path>/dev/disk/by-path</path>
</target>
</pool>
</pre>
</div>
</div>
<div id="footer">
<p id="sponsor">
Sponsored by:<br /><a href="http://et.redhat.com/"><img src="et.png" alt="Project sponsored by Red Hat Emerging Technology" /></a></p>
</div>
</body>
</html>
|