Systemd

Cron

As a Cron substitute:

$ cat /usr/local/etc/foo.service
[Unit]
Description=Execute Date

[Service]
Type=simple
ExecStart=/bin/date > /tmp/foo
Restart=no

$ cat /usr/local/etc/foo.timer
[Unit]
Description=Execute Date every hour

[Timer]
OnCalendar=hourly
RandomizedDelaySec=7200
Persistent=true

[Install]
WantedBy=timers.target

Enable, and start right away:

systemctl enable       /usr/local/etc/foo.service
systemctl enable --now /usr/local/etc/foo.timer

That OnCalendar directive is explained in systemd.time(7), and systemd-analyze can be used to validate its values:

$ grep ^On /lib/systemd/system/phpsessionclean.timer
OnCalendar=*-*-* *:09,39:00

$ systemd-analyze calendar '*-*-* *:09,39:00'
Normalized form: *-*-* *:09,39:00
   Next elapse: Mon 2023-01-02 22:09:00 CET
      (in UTC): Mon 2023-01-02 21:09:00 UTC
      From now: 14min left

Debug

Some notable Debug Parameters when things go south:

systemd.log_level=debug systemd.journald.forward_to_console=1 console=ttyS0,115200 console=tty1

Or, to use the kernel log buffer instead:

systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M printk.devkmsg=on

To enable a debug shell:

systemd.debug-shell=1

.service files

Create a custom .service file, for example to start miscellaneous programs:

$ cat /usr/local/etc/rc-local.service 
[Unit]
Description=/etc/rc.local Compatibility

[Service]
Type=idle
ExecStart=/etc/rc.local start
#ExecStop=/etc/rc.local stop
StandardOutput=journal+console

[Install]
WantedBy=multi-user.target

Start with:

$ sudo ln -s /usr/local/etc/rc-local.service /etc/systemd/system/rc-local.service
$ sudo systemctl start rc-local

Override .service files

Override .service files with:

$ sudo systemctl edit nrpe.service
[Service]
ExecStart=
ExecStart=/usr/bin/nrpe -c /etc/nrpe/nrpe.cfg -f -n

After that, an override is produced in /etc/systemd/system/nrpe.service.d containing these lines. Note: we had to clear ExecStart first before we could re-define it, for mysterious reasons.

More examples:

$ cat /etc/systemd/system/autossh@example.service.d/override.conf
[Service]
Restart=always
RestartSec=30
UMask=0066

Or, with scripts executed upon start/stop:

$ cat /etc/systemd/system/redis.service.d/override.conf 
[Service]
ExecStartPost=/usr/bin/sleep 1
ExecStartPost=/usr/bin/setfacl -m u:http:rwx /var/run/redis/ /var/run/redis/redis.sock
ExecStop=
ExecStop=/usr/bin/redis-cli -s /var/run/redis/redis.sock shutdown

We can also define pre-conditions:

[Service]
ExecStartPre=mountpoint /nfs

Disable journald

We may have Syslog in place and don't need another set of log files on disk:

$ grep ^St /etc/systemd/journald.conf
Storage=volatile                                                      # Storage=none would also do the trick

Restart systemd-journald and also zap already written log files:

sudo journalctl --vacuum-size=1 --vacuum-files=0 --vacuum-time=1
sudo rm -r /var/log/journal/