Systemd and SysVinit
Unix-based systems used SysV-style start-stop-scripts to control services and so did most Linux distributions. Systemd has become the default for most Linux distributions since 2011, and it is the default for most recent versions of Ubuntu. This chapter covers primarly systemd or how to convert to systemd as well as how to use systemd to manage services.
The original SysVinit was a replacement for initscripts and was used on most Linux distributions. Systemd was started by Lennart Poettering as a replacement for initd and SysVinit to manage services like on other operating systems as Microsoft Windows for example. With a dependency graph systemd can make your system more reliable and stable as you define the dependencies between services and the order in which they are started.
Available unit types
Systemd defines every service as a unit. A unit is a collection of configuration files and a set of binaries that are used to start and stop the service. The unit name is the name of the service. The example below shows how to list all available units on your system as they depend on the systemd version on your system.
$ systemctl -t help Available unit types: service mount swap socket target device automount timer path slice scope
By knowing the unit types you can list all available units of that type as we can see below where the command
systemctl --type=service lists all available services.
$ systemctl --type=service UNIT LOAD ACTIVE SUB DESCRIPTION abrt-journal-core.service loaded active running Creates ABRT problems from … abrt-oops.service loaded active running ABRT kernel log watcher abrt-xorg.service loaded active running ABRT Xorg log watcher abrtd.service loaded active running ABRT Automated Bug Reportin… accounts-daemon.service loaded active running Accounts Service alsa-state.service loaded active running Manage Sound Card State (re… atd.service loaded active running Deferred execution scheduler auditd.service loaded active running Security Auditing Service avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
Systemd also can act like a scheduler and run services on a specific time schedule, but also verifies if all requirements are satisfied before execution the schedule. In a later section we will see how to use systemd to manage timers.
$ systemctl --type=timer UNIT LOAD ACTIVE SUB DESCRIPTION dnf-makecache.timer loaded active waiting dnf makecache --timer fstrim.timer loaded active waiting Discard unused blocks once … logrotate.timer loaded active waiting Daily rotation of log files mlocate-updatedb.timer loaded active waiting Updates mlocate database ev… rdiff-backup.timer loaded active running Execute rdiff-backup snapper-cleanup.timer loaded active waiting Daily Cleanup of Snapper Sn… snapper-timeline.timer loaded active waiting Timeline of Snapper Snapsho… sysstat-collect.timer loaded active waiting Run system activity account… sysstat-summary.timer loaded active waiting Generate summary of yesterd… systemd-tmpfiles-clean.tim… loaded active waiting Daily Cleanup of Temporary … unbound-anchor.timer loaded active waiting daily update of the root tr…
mount is another example and can be created for mounting filesystems, but systemd itself generates during the boot procedure of the system all required unit files based on the content of
/etc/fstab. This makes the transition smooth for a lot people working with Linux.
$ systemctl --type=mount UNIT LOAD ACTIVE SUB DESCRIPTION -.mount loaded active mounted Root Mount boot-efi.mount loaded active mounted /boot/efi boot.mount loaded active mounted /boot dev-hugepages.mount loaded active mounted Huge Pages File System dev-mqueue.mount loaded active mounted POSIX Message Queue File Sy… home.mount loaded active mounted /home run-user-1000-doc.mount loaded active mounted /run/user/1000/doc run-user-1000-gvfs.mount loaded active mounted /run/user/1000/gvfs run-user-1000.mount loaded active mounted /run/user/1000 srv.mount loaded active mounted /srv sys-fs-fuse-connections.mo… loaded active mounted FUSE Control File System sys-kernel-config.mount loaded active mounted Kernel Configuration File S… sys-kernel-debug.mount loaded active mounted Kernel Debug File System sys-kernel-tracing.mount loaded active mounted Kernel Trace File System tmp.mount loaded active mounted Temporary Directory /tmp var-lib-docker.mount loaded active mounted /var/lib/docker var-lib-libvirt.mount loaded active mounted /var/lib/libvirt var-lib-nfs-rpc_pipefs.mou… loaded active mounted RPC Pipe File System
One of the benefits of systemd is that it can monitor the state of services and make sure that they are running as expected. This is done by using the
systemctl command. In the section Available unit types it was shown how to list all services on the system. The example below shows how to request the status of a service, but also if it is enabled or not to be started on boot.
$ systemctl status sshd.service ● sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2022-04-17 09:36:55 CEST; 20s ago Docs: man:sshd(8) man:sshd_config(5) Main PID: 663271 (sshd) Tasks: 1 (limit: 18476) Memory: 2.1M CPU: 14ms CGroup: /system.slice/sshd.service └─663271 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups" apr 17 09:36:55 server01 systemd: Starting OpenSSH server daemon... apr 17 09:36:55 server01 sshd: Server listening on 0.0.0.0 port 22. apr 17 09:36:55 server01 sshd: Server listening on :: port 22. apr 17 09:36:55 server01 systemd: Started OpenSSH server daemon.
By default systemd will assume the unit type
service if no unit type is specified. In the case for the example above
systemctl status sshd.service and
systemctl status sshd are the same.
sshd service with command
systemctl enable <service name> will tell systemd to start the service when the system boots, but the service isn’t automatically started. To start the service we need to use the
systemctl start sshd command.
$ systemctl enable sshd
sshd service with command
systemctl diable <service name> has the same behavior as
systemctl enable <service name> and will only make sure the services isn’t started when the system boots.
$ systemctl disable sshd
--now option is used to start or stop the service immediately after enabling or disabling it. This option is available since Red Hat Enterprise Linux 7.2 and CentOS 7.2. Older versions require two commands to both
start a service.
sshd service with command
systemctl start <service name> will start the service immediately. The service doesn’t have to be enabled for this and when the system system reboots the service will be stopped automatically, but not started if it isn’t enabled.
$ systemctl start sshd
The same rules as with starting a service also apply to stopping a service. This makes troubleshooting easier as it is easy to see if the service is running or not and should be running or not.
$ systemctl stop sshd
Like most SysVinit scripts systemd also has the option to restart a service by stopping and starting if for you. This is done by using the
systemctl restart <service name> command.
$ systemctl restart sshd
$ systemctl reload sshd
With the command
systemctl is-active <service name> we can check if the service is running or not as systemd will return
active if the service is running and
inactive if it isn’t. This make scripting and automation easier.
$ systemctl is-active sshd active
The same rules apply to verify if a service is enabled or not as well with the command
systemctl is-enabled <service name>.
$ systemctl is-enabled sshd enabled
List active services
$ systemctl list-units --type=service UNIT LOAD ACTIVE SUB DESCRIPTION abrt-journal-core.service loaded active running Creates ABRT problems from … abrt-oops.service loaded active running ABRT kernel log watcher abrt-xorg.service loaded active running ABRT Xorg log watcher abrtd.service loaded active running ABRT Automated Bug Reportin… accounts-daemon.service loaded active running Accounts Service alsa-state.service loaded active running Manage Sound Card State (re… atd.service loaded active running Deferred execution scheduler auditd.service loaded active running Security Auditing Service avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
$ systemctl list-units --type=service --all UNIT LOAD ACTIVE SUB DESCRIPTION abrt-journal-core.service loaded active running Creates ABRT problems fr… abrt-oops.service loaded active running ABRT kernel log watcher abrt-vmcore.service loaded inactive dead Harvest vmcores for ABRT abrt-xorg.service loaded active running ABRT Xorg log watcher abrtd.service loaded active running ABRT Automated Bug Repor… accounts-daemon.service loaded active running Accounts Service alsa-restore.service loaded inactive dead Save/Restore Sound Card … alsa-state.service loaded active running Manage Sound Card State … ● apparmor.service not-found inactive dead apparmor.service atd.service loaded active running Deferred execution sched… auditd.service loaded active running Security Auditing Service auth-rpcgss-module.servi… loaded inactive dead Kernel Module supporting… ● auto-cpufreq.service not-found inactive dead auto-cpufreq.service ● autofs.service not-found inactive dead autofs.service avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
List enabled services
$ systemctl list-unit-files --type=service UNIT FILE STATE VENDOR PRESET abrt-journal-core.service enabled enabled abrt-oops.service enabled enabled abrt-pstoreoops.service disabled disabled abrt-vmcore.service enabled enabled abrt-xorg.service enabled enabled abrtd.service enabled enabled accounts-daemon.service enabled enabled alsa-restore.service static - alsa-state.service static - anaconda-direct.service static - anaconda-fips.service static - anaconda-nm-config.service static - anaconda-noshell.service static - anaconda-pre.service static - [email protected] static - anaconda-sshd.service static - [email protected] static - anaconda.service static - arp-ethers.service disabled disabled atd.service enabled enabled auditd.service enabled enabled auth-rpcgss-module.service static - [email protected] alias - avahi-daemon.service enabled enabled
$ systemctl --failed --type=service UNIT LOAD ACTIVE SUB DESCRIPTION
$ systemctl list-dependencies sshd.service sshd.service ● ├─system.slice ○ ├─sshd-keygen.target ○ │ ├─[email protected] ○ │ ├─[email protected] ○ │ └─[email protected] ● └─sysinit.target ● ├─dev-hugepages.mount ● ├─dev-mqueue.mount ● ├─dracut-shutdown.service ○ ├─fedora-import-state.service ○ ├─iscsi.service ● ├─kmod-static-nodes.service ○ ├─ldconfig.service ○ ├─lvm2-lvmetad.socket
$ systemctl list-dependencies sshd.service --reverse sshd.service
With systemd it is possible to define the state of a service by enabling or disabling it. It still allows the service to be started or enabled even. Another option exists and systemd allows a service to be masked. This means that the service is a symbolic link to
/dev/null and stops systemd from starting it.
$ systemctl mask iptables Created symlink from /etc/systemd/system/iptables.service to /dev/null.
Masked services can also be unmasked so they can be enabled and started again.
$ systemctl unmask iptables
The main purpose of masking services is to prevent administrative errors when competing services like
firewalld are installed on the same system.
$ systemctl list-dependencies graphical.target | grep target graphical.target ● └─multi-user.target ● ├─basic.target ● │ ├─paths.target ● │ ├─slices.target ● │ ├─sockets.target ● │ ├─sysinit.target ● │ │ ├─cryptsetup.target ● │ │ ├─local-fs.target ● │ │ ├─swap.target ● │ │ └─veritysetup.target ● │ └─timers.target ● ├─getty.target ● ├─nfs-client.target ● │ └─remote-fs-pre.target ● └─remote-fs.target ● └─nfs-client.target ● └─remote-fs-pre.target
$ systemctl list-units --type=target --all
$ systemctl list-unit-files --type=target --all
$ systemctl isolate graphical.target
$ systemctl get-default graphical.target
$ systemctl set-default multi-user.target
To be used with Grub:
Managing temporary files
A generic mechanism for creating, adjusting, and deleting temporary files is provided by
systemd-tmpfiles. It is mainly used in combinantion of volatile and temporary files and directories located under
/var/tmp/ for example. A standard set of rules is shipped with the
systemd-tmpfiles package, but it can be customized to suit your needs by adding additional configuration files under
/etc/tmpfiles.d/. A full list of locations with configuration files is shown in the table below.
In unit file
/usr/lib/systemd/system/systemd-tmpfiles-clean.service, shown below, it is defined which command is runned to clean up the temporary files. Secondly, it is defined when the unit file has to be executed and that is right after both dependencies
time-set are met, but also just before the system shutdown. If required the service can also be executed manually by running
systemctl start systemd-tmpfiles-clean.service.
# SPDX-License-Identifier: LGPL-2.1-or-later # # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. [Unit] Description=Cleanup of Temporary Directories Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8) DefaultDependencies=no Conflicts=shutdown.target initrd-switch-root.service After=local-fs.target time-set.target Before=shutdown.target [Service] Type=oneshot ExecStart=systemd-tmpfiles --clean SuccessExitStatus=DATAERR IOSchedulingClass=idle
The previous unit file describes the service, but is only executed during the boot and shutdown process. Running it regulary is done by the timer unit file
/usr/lib/systemd/system/systemd-tmpfiles-clean.timer. This unit file is defined to run the service every day and at least 15 minutes after the system has been booted. This to not affect the system performance when the system is booted.
# # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. [Unit] Description=Daily Cleanup of Temporary Directories Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8) ConditionPathExists=!/etc/initrd-release [Timer] OnBootSec=15min OnUnitActiveSec=1d
Creating a new configuration file
/etc/tmpfiles.d/local.conf with the examples below and running command
systemd-tmpfiles --create to create the directory defined in the configuration file. It also cleans up the defined folder after 10 days.
d /var/local/appdump 2750 apprun appgrp 10d -
systemd-tmpfiles --clean triggers the removal of all files passed the 10 day mark, but command
systemd-tmpfiles --remove doesn’t remove anything. If the configuration file is changed to start with
D the files are still removed after the 10 days, but also all files are removed if the command
systemd-tmpfiles --remove is run.
D /var/local/appdump 2750 apprun appgrp 10d -
Another example is to create symbolic links and directories in
/run/ with the example below.
L /run/mtab - root root - /proc/self/mounts D /run/myapp 2750 apprun appgrp - -
More options do exist, but are not shown here and can be found with