Tuesday, April 8, 2014

Upstart - Syntax error: Bad fd number upstart

Debugging upstart scripts is hard and cumbersome. I've been writing some very simple and straight forward upstart configuration files. The problems began when I tried to start my service using one of them.

Undescriptive error messages

I've experienced two problems I didn't find anything on the internet about:
  • Errors in the service standard error output: : not foundfd/9: 3: /proc/self/fd/9:
  • Errors in /var/log/upstart/{service-name}/logSyntax error: Bad fd number upstart
What happened to be the problem wasn't the actual configuration files, but their formatting. I've been preparing the .conf files on Windows and have uploaded them on some Linux machines. What I haven't thought of was these hidden symbols, which are not visible by default - the line endings. 

Line endings

Windows uses '\r\n' or CRLF at the end of each line and Linux uses only '\n' or LF (Newline explained in Wikipedia). So,  my configuraton files were all with CRLF at the end of each line, which wasn't accepted by the Linux and Upstart respectively. Upstart failed to start my services and didn't give any descriptive errors. Changing CRLF to LF in the .conf file fixed these issues.

TIP: You can use vim -b filename to open a file name in binary mode, which shows all special characters in the file.

I hope this saves some hours someone out there.