Link to home
Start Free TrialLog in
Avatar of Jeff swicegood
Jeff swicegood

asked on

Using init script with Systemd: need help crafting unit file

Hi,

I am just trying to get my service to start at boot. My system, Ubuntu 16.10, uses systemd but the installer for the service,  airControl2Server just installed an init script. A systemd unit, airControl2Server.service, was auto-generated but it doesn't start the service.

I can start it with /etc/init.d/airControl2Server start,
but systemctl airControl2Server does not start the service, but reports it does.


Here is the unit file
jaga@jaga-Desktop:~$ cat /lib/systemd/system/airControl2Server.service 
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/airControl2Server
Description=LSB: airControl2Server
Before=multi-user.target
Before=multi-user.target
Before=multi-user.target
Before=graphical.target
After=postgresql-9.5.service

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
ExecStart=/etc/init.d/airControl2Server start
ExecStop=/etc/init.d/airControl2Server stop

Open in new window


airControl2Server
SOLUTION
Avatar of gheist
gheist
Flag of Belgium image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Jeff swicegood
Jeff swicegood

ASKER

Ok I got it to enable. I realized the reason it won't start with systemctl start is because it thinks its already running since the script exits without error even when does have an error. So systemctl restart always starts it. I guess it doesn't start at boot because it requires postgresql-9.5 to be running. I thought I specified that but I guess not correctly. How do I troubleshoot that?
Avatar of Duncan Roe
Here's a general method I use to troubleshoot scripts. Near the top of your script, insert the following lines
exec >>/tmp/scriptname.out 2>&1
date
set -x

Open in new window

where you substitute whatever the name of your script is for scriptname
This creates activity logs in /tmp which you can check for unexpected behaviour
My theory is that because the script calls the final command line with nohup instead of exec that the command is returning 0 even though its not starting fully. This script is supplied by the software vendor though and should work. My other theory is that by running the script with the stop param (/etc/init.d/airControl2Server stop) it somehow initializes java so that when called with the "start" param it works.
Ask ubiquiti why it does not work.... I dont think there is anything about 'initializing java' that can persist after script shutdown.
Anything in nohup.out? Else modify the vendor's script as in https:#a41912477
Is the nohup backgrounded with an ampersand? If not, it will wait until the command completes and then return the exit status of the command. See info nohup (more comprehensive than man nohup)
Ok this is helpful. I changed the output of nohup form /dev/null to hohup.out. Rebooted and looked at nohup.out:

It just contains two lines:

Error: Could not find or load main class 0
Error: Could not find or load main class 0

Open in new window


Maybe stopping the service without it being started somehow tells it where it's main class is, Like maybe stopping it sets the environment variable or something.
Oops, I think that was a false nohup.out. Here is the real one:

jaga@jaga-Desktop:~$ cat nohup.out 
java.lang.RuntimeException: org.postgresql.util.PSQLException: FATAL: semctl(425990, 3, SETVAL, 0) failed: Invalid argument
        at com.ubnt.ac.backend.AirServerLauncher.main(SourceFile:299)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:62)
        at com.install4j.runtime.launcher.UnixLauncher.main(UnixLauncher.java:60)
Caused by: org.postgresql.util.PSQLException: FATAL: semctl(425990, 3, SETVAL, 0) failed: Invalid argument
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:427)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:203)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:65)
        at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:149)
        at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:35)
        at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
        at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:47)
        at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:30)
        at org.postgresql.Driver.makeConnection(Driver.java:414)
        at org.postgresql.Driver.connect(Driver.java:282)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:77)
        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:256)
        at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:868)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
        at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
        at mO.a(SourceFile:71)
        at nK.a(SourceFile:391)
        at nK.a(SourceFile:101)
        at I.a(SourceFile:24471)
        at E.b(SourceFile:269)
        at com.ubnt.ac.backend.AirServerLauncher.a(SourceFile:170)
        at com.ubnt.ac.backend.AirServerLauncher.main(SourceFile:289)
        ... 6 more

Open in new window


What is it trying to do to postgresql that it can't and is giving and error?
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Got it! It's working now. I guess postresql wasn't ready for my service to start, so airControl2 failed to connect to the database. I had to put

Requires=postgreql-9.5.service

Open in new window

in my unit file.

I also put in Restart=on-failure just to make sure. Thank you guys for the education, I really appreciate it!
I suspect Invalid argument is because the semaphore has not been created yet. To check this hypothesis
  • When airControl2Server is  running (i.e. after you started it manually), save the output from entering "ipcs -s"
  • Insert "ipcs -s" in the vendor's script immediately before the 'nohup' line
  • Compare the semaphores in nohup.out with what you saved previously
If you do find that semaphores are missing, it means you are running /etc/init.d/airControl2Server out of sequence (too early). You need to run some other rc script first.