Systemd and SysVinit
Unix-based systems used SysV-style start-stop-scripts to control services and so did most Linux distributions. Systemd has become the default for most Linux distributions since 2011, and it is the default for most recent versions of Ubuntu. This chapter covers primarly systemd or how to convert to systemd as well as how to use systemd to manage services.
The original SysVinit was a replacement for initscripts and was used on most Linux distributions. Systemd was started by Lennart Poettering as a replacement for initd and SysVinit to manage services like on other operating systems as Microsoft Windows for example. With a dependency graph systemd can make your system more reliable and stable as you define the dependencies between services and the order in which they are started.
Available unit types
Systemd defines every service as a unit. A unit is a collection of configuration files and a set of binaries that are used to start and stop the service. The unit name is the name of the service. The example below shows how to list all available units on your system as they depend on the systemd version on your system.
$ systemctl -t help
Available unit types:
service
mount
swap
socket
target
device
automount
timer
path
slice
scope
By knowing the unit types you can list all available units of that type as we can see below where the command systemctl --type=service lists all available services.
$ systemctl --type=service
UNIT LOAD ACTIVE SUB DESCRIPTION
abrt-journal-core.service loaded active running Creates ABRT problems from …
abrt-oops.service loaded active running ABRT kernel log watcher
abrt-xorg.service loaded active running ABRT Xorg log watcher
abrtd.service loaded active running ABRT Automated Bug Reportin…
accounts-daemon.service loaded active running Accounts Service
alsa-state.service loaded active running Manage Sound Card State (re…
atd.service loaded active running Deferred execution scheduler
auditd.service loaded active running Security Auditing Service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
Systemd also can act like a scheduler and run services on a specific time schedule, but also verifies if all requirements are satisfied before execution the schedule. In a later section we will see how to use systemd to manage timers.
$ systemctl --type=timer
UNIT LOAD ACTIVE SUB DESCRIPTION
dnf-makecache.timer loaded active waiting dnf makecache --timer
fstrim.timer loaded active waiting Discard unused blocks once …
logrotate.timer loaded active waiting Daily rotation of log files
mlocate-updatedb.timer loaded active waiting Updates mlocate database ev…
rdiff-backup.timer loaded active running Execute rdiff-backup
snapper-cleanup.timer loaded active waiting Daily Cleanup of Snapper Sn…
snapper-timeline.timer loaded active waiting Timeline of Snapper Snapsho…
sysstat-collect.timer loaded active waiting Run system activity account…
sysstat-summary.timer loaded active waiting Generate summary of yesterd…
systemd-tmpfiles-clean.tim… loaded active waiting Daily Cleanup of Temporary …
unbound-anchor.timer loaded active waiting daily update of the root tr…
Unit type mount is another example and can be created for mounting filesystems, but systemd itself generates during the boot procedure of the system all required unit files based on the content of /etc/fstab. This makes the transition smooth for a lot people working with Linux.
$ systemctl --type=mount
UNIT LOAD ACTIVE SUB DESCRIPTION
-.mount loaded active mounted Root Mount
boot-efi.mount loaded active mounted /boot/efi
boot.mount loaded active mounted /boot
dev-hugepages.mount loaded active mounted Huge Pages File System
dev-mqueue.mount loaded active mounted POSIX Message Queue File Sy…
home.mount loaded active mounted /home
run-user-1000-doc.mount loaded active mounted /run/user/1000/doc
run-user-1000-gvfs.mount loaded active mounted /run/user/1000/gvfs
run-user-1000.mount loaded active mounted /run/user/1000
srv.mount loaded active mounted /srv
sys-fs-fuse-connections.mo… loaded active mounted FUSE Control File System
sys-kernel-config.mount loaded active mounted Kernel Configuration File S…
sys-kernel-debug.mount loaded active mounted Kernel Debug File System
sys-kernel-tracing.mount loaded active mounted Kernel Trace File System
tmp.mount loaded active mounted Temporary Directory /tmp
var-lib-docker.mount loaded active mounted /var/lib/docker
var-lib-libvirt.mount loaded active mounted /var/lib/libvirt
var-lib-nfs-rpc_pipefs.mou… loaded active mounted RPC Pipe File System
Managing services
One of the benefits of systemd is that it can monitor the state of services and make sure that they are running as expected. This is done by using the systemctl command. In the section Available unit types it was shown how to list all services on the system. The example below shows how to request the status of a service, but also if it is enabled or not to be started on boot.
$ systemctl status sshd.service
● sshd.service - OpenSSH server daemon
Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2022-04-17 09:36:55 CEST; 20s ago
Docs: man:sshd(8)
man:sshd_config(5)
Main PID: 663271 (sshd)
Tasks: 1 (limit: 18476)
Memory: 2.1M
CPU: 14ms
CGroup: /system.slice/sshd.service
└─663271 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups"
apr 17 09:36:55 server01 systemd[1]: Starting OpenSSH server daemon...
apr 17 09:36:55 server01 sshd[663271]: Server listening on 0.0.0.0 port 22.
apr 17 09:36:55 server01 sshd[663271]: Server listening on :: port 22.
apr 17 09:36:55 server01 systemd[1]: Started OpenSSH server daemon.
Note
By default systemd will assume the unit type service if no unit type is specified. In the case for the example above systemctl status sshd.service and systemctl status sshd are the same.
Enabling the sshd service with command systemctl enable <service name> will tell systemd to start the service when the system boots, but the service isn’t automatically started. To start the service we need to use the systemctl start sshd command.
$ systemctl enable sshd
Disabling the sshd service with command systemctl diable <service name> has the same behavior as systemctl enable <service name> and will only make sure the services isn’t started when the system boots.
$ systemctl disable sshd
Note
The --now option is used to start or stop the service immediately after enabling or disabling it. This option is available since Red Hat Enterprise Linux 7.2 and CentOS 7.2. Older versions require two commands to both enable and start a service.
Starting the sshd service with command systemctl start <service name> will start the service immediately. The service doesn’t have to be enabled for this and when the system system reboots the service will be stopped automatically, but not started if it isn’t enabled.
$ systemctl start sshd
The same rules as with starting a service also apply to stopping a service. This makes troubleshooting easier as it is easy to see if the service is running or not and should be running or not.
$ systemctl stop sshd
Like most SysVinit scripts systemd also has the option to restart a service by stopping and starting if for you. This is done by using the systemctl restart <service name> command.
$ systemctl restart sshd
$ systemctl reload sshd
With the command systemctl is-active <service name> we can check if the service is running or not as systemd will return active if the service is running and inactive if it isn’t. This make scripting and automation easier.
$ systemctl is-active sshd
active
The same rules apply to verify if a service is enabled or not as well with the command systemctl is-enabled <service name>.
$ systemctl is-enabled sshd
enabled
List active services
$ systemctl list-units --type=service
UNIT LOAD ACTIVE SUB DESCRIPTION
abrt-journal-core.service loaded active running Creates ABRT problems from …
abrt-oops.service loaded active running ABRT kernel log watcher
abrt-xorg.service loaded active running ABRT Xorg log watcher
abrtd.service loaded active running ABRT Automated Bug Reportin…
accounts-daemon.service loaded active running Accounts Service
alsa-state.service loaded active running Manage Sound Card State (re…
atd.service loaded active running Deferred execution scheduler
auditd.service loaded active running Security Auditing Service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
$ systemctl list-units --type=service --all
UNIT LOAD ACTIVE SUB DESCRIPTION
abrt-journal-core.service loaded active running Creates ABRT problems fr…
abrt-oops.service loaded active running ABRT kernel log watcher
abrt-vmcore.service loaded inactive dead Harvest vmcores for ABRT
abrt-xorg.service loaded active running ABRT Xorg log watcher
abrtd.service loaded active running ABRT Automated Bug Repor…
accounts-daemon.service loaded active running Accounts Service
alsa-restore.service loaded inactive dead Save/Restore Sound Card …
alsa-state.service loaded active running Manage Sound Card State …
● apparmor.service not-found inactive dead apparmor.service
atd.service loaded active running Deferred execution sched…
auditd.service loaded active running Security Auditing Service
auth-rpcgss-module.servi… loaded inactive dead Kernel Module supporting…
● auto-cpufreq.service not-found inactive dead auto-cpufreq.service
● autofs.service not-found inactive dead autofs.service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
List enabled services
$ systemctl list-unit-files --type=service
UNIT FILE STATE VENDOR PRESET
abrt-journal-core.service enabled enabled
abrt-oops.service enabled enabled
abrt-pstoreoops.service disabled disabled
abrt-vmcore.service enabled enabled
abrt-xorg.service enabled enabled
abrtd.service enabled enabled
accounts-daemon.service enabled enabled
alsa-restore.service static -
alsa-state.service static -
anaconda-direct.service static -
anaconda-fips.service static -
anaconda-nm-config.service static -
anaconda-noshell.service static -
anaconda-pre.service static -
anac[email protected] static -
anaconda-sshd.service static -
anac[email protected] static -
anaconda.service static -
arp-ethers.service disabled disabled
atd.service enabled enabled
auditd.service enabled enabled
auth-rpcgss-module.service static -
auto[email protected] alias -
avahi-daemon.service enabled enabled
$ systemctl --failed --type=service
UNIT LOAD ACTIVE SUB DESCRIPTION
Dependencies
$ systemctl list-dependencies sshd.service
sshd.service
● ├─system.slice
○ ├─sshd-keygen.target
○ │ ├─[email protected]
○ │ ├─[email protected]
○ │ └─[email protected]
● └─sysinit.target
● ├─dev-hugepages.mount
● ├─dev-mqueue.mount
● ├─dracut-shutdown.service
○ ├─fedora-import-state.service
○ ├─iscsi.service
● ├─kmod-static-nodes.service
○ ├─ldconfig.service
○ ├─lvm2-lvmetad.socket
$ systemctl list-dependencies sshd.service --reverse
sshd.service
Masking services
With systemd it is possible to define the state of a service by enabling or disabling it. It still allows the service to be started or enabled even. Another option exists and systemd allows a service to be masked. This means that the service is a symbolic link to /dev/null and stops systemd from starting it.
$ systemctl mask iptables
Created symlink from /etc/systemd/system/iptables.service to /dev/null.
Masked services can also be unmasked so they can be enabled and started again.
$ systemctl unmask iptables
Note
The main purpose of masking services is to prevent administrative errors when competing services like iptables and firewalld are installed on the same system.
Managing bootlevels
$ systemctl list-dependencies graphical.target | grep target
graphical.target
● └─multi-user.target
● ├─basic.target
● │ ├─paths.target
● │ ├─slices.target
● │ ├─sockets.target
● │ ├─sysinit.target
● │ │ ├─cryptsetup.target
● │ │ ├─local-fs.target
● │ │ ├─swap.target
● │ │ └─veritysetup.target
● │ └─timers.target
● ├─getty.target
● ├─nfs-client.target
● │ └─remote-fs-pre.target
● └─remote-fs.target
● └─nfs-client.target
● └─remote-fs-pre.target
$ systemctl list-units --type=target --all
$ systemctl list-unit-files --type=target --all
$ systemctl isolate graphical.target
$ systemctl get-default
graphical.target
$ systemctl set-default multi-user.target
To be used with Grub:
systemd.unit=rescue.target (emergency.target)
Managing temporary files
A generic mechanism for creating, adjusting, and deleting temporary files is provided by systemd-tmpfiles. It is mainly used in combinantion of volatile and temporary files and directories located under /run/, /tmp/, and /var/tmp/ for example. A standard set of rules is shipped with the systemd-tmpfiles package, but it can be customized to suit your needs by adding additional configuration files under /etc/tmpfiles.d/. A full list of locations with configuration files is shown in the table below.
Location |
Description |
|---|---|
|
Manual configuration |
|
Generated configuration |
|
System configuration |
In unit file /usr/lib/systemd/system/systemd-tmpfiles-clean.service, shown below, it is defined which command is runned to clean up the temporary files. Secondly, it is defined when the unit file has to be executed and that is right after both dependencies local-fs and time-set are met, but also just before the system shutdown. If required the service can also be executed manually by running systemctl start systemd-tmpfiles-clean.service.
# SPDX-License-Identifier: LGPL-2.1-or-later
#
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
[Unit]
Description=Cleanup of Temporary Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
DefaultDependencies=no
Conflicts=shutdown.target initrd-switch-root.service
After=local-fs.target time-set.target
Before=shutdown.target
[Service]
Type=oneshot
ExecStart=systemd-tmpfiles --clean
SuccessExitStatus=DATAERR
IOSchedulingClass=idle
The previous unit file describes the service, but is only executed during the boot and shutdown process. Running it regulary is done by the timer unit file /usr/lib/systemd/system/systemd-tmpfiles-clean.timer. This unit file is defined to run the service every day and at least 15 minutes after the system has been booted. This to not affect the system performance when the system is booted.
#
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
[Unit]
Description=Daily Cleanup of Temporary Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
ConditionPathExists=!/etc/initrd-release
[Timer]
OnBootSec=15min
OnUnitActiveSec=1d
Creating a new configuration file /etc/tmpfiles.d/local.conf with the examples below and running command systemd-tmpfiles --create to create the directory defined in the configuration file. It also cleans up the defined folder after 10 days.
d /var/local/appdump 2750 apprun appgrp 10d -
With option d the systemd-tmpfiles --clean triggers the removal of all files passed the 10 day mark, but command systemd-tmpfiles --remove doesn’t remove anything. If the configuration file is changed to start with D the files are still removed after the 10 days, but also all files are removed if the command systemd-tmpfiles --remove is run.
D /var/local/appdump 2750 apprun appgrp 10d -
Another example is to create symbolic links and directories in /run/ with the example below.
L /run/mtab - root root - /proc/self/mounts
D /run/myapp 2750 apprun appgrp - -
More options do exist, but are not shown here and can be found with man tmpfiles.d.