0 Голосов

Fix Proxmox (PVE) “can’t lock file ‘/var/lock/qemu-server/lock-xxx.conf’ – got timeout”

Редактировал(а) Сергей Коршунов 2022/07/01 16:40

Fix Proxmox (PVE) “can’t lock file ‘/var/lock/qemu-server/lock-xxx.conf’ – got timeout” (Proxmox can’t shutdown/stop virtual machine) (Proxmox kill/force stop virtual machine)

The Issue

When trying to “Stop” or “Shutdown” virtual machine from Proxmox (PVE) web gui, the “Cluster log” shows

end task UPID:pve:xxxxxxxx:xxxxxxxx:xxxxxxx:qmstop:xxx:root@pam: can’t lock file ‘/var/lock/qemu-server/lock-xxx.conf’ -got timeout

end task UPID:pve:xxxxxxxx:xxxxxxxx:xxxxxxx:qmreboot:xxx:root@pam: VM quit/powerdown failed

Error: can’t lock file ‘/var/lock/qemu-server/lock-202.conf’ – got timeout

Related Question

  • How to manually (Using terminal/console to) unlock a VM

The Fix

Manually

We can manually delete the lock from following path

/run/lock/qemu-server
# The file will be
/run/lock/qemu-server/lock-100.conf
/run/lock/qemu-server/lock-102.conf
...
# Make sure only delete the correct one
# Manually unlock again (100 is the VM id)
qm unlock 100

Automated via bash script

1 Open Proxmox web gui

2 Click on “>_ Shell” to launch Shell for Proxmox

3 Type following command to create a “killvm.sh” file

nano killvm.sh

4 Select which script you want to use, copy paste the script to nano editor (Here we use the Simple Script as example, you can use Interactive script or Loop script based on your needs)

Copy paste script to nano editor 

Copy paste script to nano editor

5 Use Ctrl + X, Y, Enter key to Exit and Save the file

6 Allow the script executing as program

chmod +x killvm.sh

7 Now we can use the bash script to remove locks easily by using following command

# ./killvm.sh VMID
#e.g.
./killvm.sh 202

Lock for VM ID 201 removed 

Lock for VM ID 201 removed

8 If you would like to use other bash scripts or all of them, save each of them to separate .sh files e.g. killvm-select.sh killvm-loop.sh, the rest steps will be the same as above

Simple script is suitable to normal usage and can be used with other custom scripts

Interactive script is verbose

Copy one or all of following bash scripts to separate name.sh files to use them

Simple script

#!/bin/sh
echo
echo '-----AUTHOR: https://dannyda.com-----'
echo
echo '---Existing locks---'
qm unlock $1
ls -l /run/lock/qemu-server
rm -f /run/lock/qemu-server/lock-$1.conf
qm unlock $1
echo
echo '---Remaining locks---'
ls -l /run/lock/qemu-server

Usage: Assume saved as “killvm.sh” in “/root” folder on PVE, 202 is the VM ID we want to delete

./killvm.sh 202

Interactive script

#!/bin/sh
echo
echo '-----AUTHOR: https://dannyda.com-----'
echo
echo 'Existing lock files'
ls -l /run/lock/qemu-server
read -p 'Enter the VM ID here to delete corresponding lock e.g. 101: ' vmid
qm unlock $vmid
rm -f /run/lock/qemu-server/lock-$vmid.conf
qm unlock $vmid
echo
echo '---Remaining locks---'
ls -l /run/lock/qemu-server

Usage: Assume saved as “killvmi.sh” in “/root” folder on PVE, 203 is the VM ID we want to delete

./killvmi.sh
Enter the VM ID here to delete corresponding lock e.g. 101: 203

Interactive script to rm lock 

Interactive script to rm lock

Loop script for deleting multiple lock files with ease

(Double click on the following script to select all, then copy)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

#!/bin/bash

echo

echo '-----AUTHOR: https://dannyda.com-----'

echo

echo 'Existing lock files'

ls -l /run/lock/qemu-server

while read -p 'Enter the VM ID here to delete corresponding lock, press Enter key to exit e.g. 101: ' vmid; do

echo

echo '-----AUTHOR: https://dannyda.com-----'

echo

echo 'Existing lock files'

ls -l /run/lock/qemu-server

if [[ "$vmid" = "" ]] || [[ "$vmid" = "q" ]] ;

then       

exit

elif [[ "$vmid" -gt 0 ]] && [[ "$vmid" -lt 1000000000 ]];

then

qm unlock $vmid

rm -f /run/lock/qemu-server/lock-$vmid.conf

qm unlock $vmid

ls -l /run/lock/qemu-server

else

echo 'Input error, please enter correct VM ID.'

fi

done

Usage: Assume saved as “rmlock.sh” in “/root” folder on PVE, 201, 202 is the VM ID we want to delete

https://dannyda.com/myupload/2020/05/image-32.png

./rmlock.sh
# Enter the VM ID which we want to delete but locked, then hit Enter key to unlock it
# Note
# Press Enter key without entering anything to exit the script, or type q then press Enter key to exit.

Last step

Now we can stop/shutdown or reboot the virtual machine from Proxmox web gui without problem!

Bonus

Simple script (Remove the lock and stop the VM)

#!/bin/sh
echo
echo '-----AUTHOR: https://dannyda.com-----'
echo
echo '---Existing locks---'
qm unlock $1
ls -l /run/lock/qemu-server
rm -f /run/lock/qemu-server/lock-$1.conf
qm unlock $1
echo
echo '---Remaining locks---'
ls -l /run/lock/qemu-server
echo
qm stop $1 && qm status $1

Interactive script (Remove the lock and stop the VM)

#!/bin/sh
echo
echo '-----AUTHOR: https://dannyda.com-----'
echo
echo 'Existing lock files'
ls -l /run/lock/qemu-server
read -p 'Enter the VM ID here to delete corresponding lock e.g. 101: ' vmid
qm unlock $vmid
rm -f /run/lock/qemu-server/lock-$vmid.conf
qm unlock $vmid
echo
echo '---Remaining locks---'
ls -l /run/lock/qemu-server
echo
qm stop $vmid && qm status $vmid

Loop script for deleting multiple lock files with ease (Remove the lock and stop the VM)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

#!/bin/bash

echo

echo '-----AUTHOR: https://dannyda.com-----'

echo

echo 'Existing lock files'

ls -l /run/lock/qemu-server

while read -p 'Enter the VM ID here to delete corresponding lock, press Enter key to exit e.g. 101: ' vmid; do

echo

echo '-----AUTHOR: https://dannyda.com-----'

echo

echo 'Existing lock files'

ls -l /run/lock/qemu-server

if [[ "$vmid" = "" ]] || [[ "$vmid" = "q" ]] ;

then       

exit

elif [[ "$vmid" -gt 0 ]] && [[ "$vmid" -lt 1000000000 ]];

then

qm unlock $vmid

rm -f /run/lock/qemu-server/lock-$vmid.conf

qm unlock $vmid

ls -l /run/lock/qemu-server

qm stop $vmid && qm status $vmid

else

echo 'Input error, please enter correct VM ID.'

fi

done

Using command to unlock VM

qm unlock <VMID>
# e.g.
qm unlock 100