Ispconfig3 certbot is not renewing certificates (Ubuntu 20)

When creating a new site, a valid certificate was issued but when expiring they where never renewed. Investigating /etc/letsencrypt directories was missing the usual subfolders, like live for example.

It turned out the server had both acme.sh and certbot installed. The solution was to remove certbot. Investigate if the directory /root/.acme.sh exists and it’s contents.

apt remove certbot
ispconfig_update.sh --force

Then in Ispconfig go to Tools -> Sync, select Web sites and the server you just removed.

apt update error message “Could not execute ‘apt-key’ to verify signature”

It turned out the reason for this was changed permissions on the /tmp folder (caused by restoring a folder with BackupPC to /tmp instead of it’s original location).

Solution:

chown root:root /tmp
chmod 1777 /tmp

After this apt update worked as normal.

How can I use a PC to recover data when my Synology NAS malfunctions?

Purpose

This article will guide you to recover data on your PC when Synology NAS malfunctions.

Notes:

The drives may not be able to mount the volume again after being migrated to a new NAS.

Environment

  • Available on DSM version 6.2.x and above.
  • Only applicable to ext4 or Btrfs file system.
  • Ubuntu version should be 18.04 and above.

Resolution

  1. Make sure your PC has sufficient drive slots for drive installation.
  2. Remove the drives from your Synology NAS and install them in your PC. For RAID or SHR configurations, you must install all the drives (excluding hot spare drives) in your PC at the same time.
  3. Prepare an Ubuntu environment by following the instructions in this tutorial.
  4. Go to the Files on the left bar and select Home.
  5. Right-click and select New Folder, and create one or more folders as mount points for accessing data.1
  6. Right-click on the new folder(s), click Properties, the parent folder with folder name is ${mount_point}.
    Example: If the parent folder is /home/ubuntu/ and the folder name is Test, the mount point will be /home/ubuntu/Test/.
  7. Go to Show Application in the lower-left corner > Type to search….
  8. Enter Terminal in the search bar and select Terminal.
  9. Enter the following command to obtain the root privileges.

sudo -i

  1. Enter the following commands to install mdadm and lvm2, both of which are RAID management tools. lvm2 must be installed or vgchange will not work.

apt-get update
apt-get install -y mdadm lvm2

  1. Enter the following command to assemble all the drives removed from your Synology NAS. The results may differ according to the storage pool configurations on your Synology NAS.

mdadm -Asf && vgchange -ay

  1. Enter the following command to get the information of ${device_path}.

cat /proc/mdstat
lvs

According to the output of pvs/vgs/lvs, the device paths are as follows:

${device_path}
No lvs output/dev/${md}2
With lvs output/dev/${VG}/${LV}3

Below are the samples of md status corresponding to its RAID and volume type:

Device PathsClassic RAID with single volume
cat /proc/mdstatroot@ubuntu:~# cat /proc/mdstat Personalities : [raid1] md4 : active raid1 sdc3[0] 73328704 blocks super 1.2 [1/1] [U] unused devices:<none>
lvsNo output
${device_path}/dev/md4
Device PathsSHR with single volume
cat /proc/mdstatroot@ubuntu:~# cat /proc/mdstat Personalities : [raid1] md3 : active raid1 sda5[0] 73319616 blocks super 1.2 [1/1] [U] unused devices:<none>
lvsroot@ubuntu:~# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lv vg1000 -wi-a—– 69.92g
${device_path}/dev/vg1000/lv
Device PathsClassic RAID/SHR with multiple volume
cat /proc/mdstatPersonalities : [raid1] md3 : active raid1 sdc3[0] sdd3[1] 73328704 blocks super 1.2 [2/2] [UU] unused devices:<none>
lvsroot@ubuntu:~# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy% Sync Convert syno_vg_reserved_area vg1 -wi-a—– 12.00m volume_1 vg1 -wi-a—– 30.00g volume_3 vg1 -wi-a—– 30.00g
${device_path}/dev/vg1/volume_1
/dev/vg1/volume_3
  1. Enter the following commands to mount all the drives as read-only to access your data. Enter your device path (according to RAID and volume type in Step 12) in ${device_path} and mount point (created in Step 6)in ${mount_point}. Your data will be placed under the mount point.

$ mount ${device_path} ${mount_point} -o ro

  1. Check the data in Files > Home > the folders you created in Step 5.

If you still cannot recover the data through the above steps, refrain from trying any other methods to repair because it may cause more damage to your data. As your last option, please seek the help of a local data rescue company. Kindly understand that data retrieval is still not guaranteed.

Notes:

  1. A mount point is equal to one volume. If you have multiple volumes that need to be recovered, please create the same number of folders as the number of volumes.
  2. The number of md (array) will be listed in the result of cat /proc/mdstat.
  3. syno_vg_reserved_area can be ignored, the number of volume_x is equal to the number of volumes.

ERROR 1067 (42000) at line xxx: Invalid default value for ‘field’

This is beacuse mysql server 5.7 have changed the the date time default field 0000-00-00 00:00:00 options. You can only have one field with the value 0000-00-00 00:00:00. Where fore you have to change the default indata mask to.

datetime NOT NULL DEFAULT ‘1000-01-01 00:00:00’
or
timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP

exampel
ALTER TABLE testdate CHANGE datestart datestart DATETIME NOT NULL DEFAULT ‘1000-01-01 00:00:00’;

How to change mysql table engine MyISAM to InnoDB

login to mysql shell as root.
locate the database where the tables are situated.
SET @DATABASE_NAME = ‘name_of_your_db’; // name_of_your_db = the database you want to change table engine.
SELECT CONCAT(‘ALTER TABLE ', table_name, ' ENGINE=InnoDB;’) AS sql_statements FROM information_schema.tables AS tb WHERE table_schema = @DATABASE_NAME AND ENGINE = ‘MyISAM’ AND TABLE_TYPE = ‘BASE TABLE’ ORDER BY table_name DESC;
The result will end up in a list of the tables that needs to be changed. Copy the list and do the following.
Shift to the database involved:

USE name_of_your_db
START TRANSACTION;
insert the copied list
COMMIT;
You have now changed the engine of the tables.

[ERROR] Fatal error: Can’t open and lock privilege tables: Table ‘mysql.user’ doesn’t exist

This error ocurs then the database mysql is missing or corrupt.
Stop the mysql server “service mysql-server stop”
Make a backup of /var/db/mysql “mv /var/db/mysql /var/db/mysql.old”.
To rebuild the database execute “/usr/local/libexec/mysqld –initialize”
You will get a temporary password. Remember the password for later use.
Start the mysql server “service mysql-server start”
To start upp the new configuration “mysql_secure_installation”. Use the Password to start the configuration and step thru the wizzard.
Restore the mysql backup and the server is good as new.

Prevent Mac OSX ssh from disconnecting (also on any Linux/BSD/*nix system)

To prevent ssh from disconnecting while idle, add the following to ~/.ssh/config:

Host *
    ServerAliveInterval 30
    TCPKeepAlive no

This solution is alse useable in any Linux/BSD/*nix environment. If you want to implement this not only on your own user, as a sysadmin, add the above to /etc/ssh/ssh_config instead.

WordPress shows white page after site move

WordPress just showing a white page means it is experienceing PHP errors. When it occurs after a site has been moved from one hosting to another it can be caused by several reasons.

To make it possible to track down the cause you can temporarily add the following to the .htaccess file in the root. If this gives you a internal server error 500 it means it is not allowed by the host. If allowed, you will get error messages from PHP dispayed on screen when accessing the site:

php_flag display_startup_errors on
php_flag display_errors on

There are visbible PHP ending tags ?> on the site

If you see more or less normal output from the site but once in a while you see lines ending with ?>. This can happen if the site has been running on PHP 5 where the short PHP start tag <? is allowed (instead of the longer <?php.

If the site has been moved to PHP 7 where this is not longer allowed, it can cause this kind of strange behaviour.

If you don’t want to go through all the code and find all the occurrences of <? and replace them with <?php (which is one solution), you can simply add the following to your php.ini:

short_open_tag=On

You need to restart Apache after this.

Wordfence plugin is installed

You need to manually edit the system specific paths in the following files:

  • .user.ini in the site root, change the path after auto_prepend_file
  • wordfence-waf.php in the site root, change the paths on all occurrences

WP Rocket plugin is installed

The file wp-content/advanced-cache.php contains system specific paths. You can manually update them or just simply delete the file and login to wp-admin of the site and it will be recreated automatically with the correct paths.

MySQL Error 1215: Cannot add foreign key constraint

Check that both tables have the same engine type, so there isn’t a mix of for example MyISAM and InnoDB.

rm unable to delete file even as root or sudo [Solved]

When it is not possible to delete a file using rm even when logged in as root or by using sudo, check the file attributes with:

lsattr filename.ext

If it has the “i” attribute, this is preventing the file from being deleted, even by super users. When you have verified the reason for this and still want to delete the file then remove the “i” attribute with:

chattr -i filename.ext

Then delete the file using rm as you normally would.