Adding a JS9 Server-side Helper

Do You Need a Server-side Helper? Which One?

JS9 supports server-side analysis on FITS data, allowing you to execute virtually any command-line analysis program from JS9, run that analysis command on the server, and view results on your web page. It also can support external communication with JS9 (via the shell and Python) and handle the display of large files.

The server-side analysis capability is useful for archive centers and individual users who want to integrate their own data analysis programs into JS9. You configure a JS9 server-side helper by adding additional switches to the configure command described in Installing JS9.

JS9 supports two mechanisms for server-side analysis: you can run a separate Node.js-based server to process requests using socket.io, or use CGI calls to your own web server. The Node.js server-side helper is the recommended method: it is faster and more flexible than the CGI helper: in addition to analysis tasks, Node-js-based helper also supports:

Add Switches to Configure the Server-side Helper

JS9 is built using the standard GNU ./configure; make; make install method. To configure server-side analysis, you add switches to the configure command line. The most important switches are:

--with-helper=[type]    type of helper: nodejs (for Node.js) , get (for CGI)
--prefix=[path]         location to install the non-web programs and scripts
--with-cfitsio=[path]   cfitsio location for building js9helper, e.g., /usr/local
Other less-used switches are:
--with-cgidir=[path]     cgi install directory e.g., /var/www/public_html/cgi-bin
--with-cgiurl=[url]      cgi url relative to the web root, e.g.. ./cgi-bin/js9
--with-cgixpath=[dirs]   directories to add to cgi path

Both the Node.js and CGI helpers make use of a C program called js9helper to handle certain types of JS9 requests. To support extraction of FITS representation files from large FITS files, js9helper must be linked against the cfitsio library (version 3.39 or later is preferred). Cfitsio is the de facto standard FITS library and is available from NASA/HEASARC at the Goddard Space Flight Center:

    https://heasarc.gsfc.nasa.gov/fitsio/fitsio.html
To tell configure where cfitsio libraries are located, use the --with-cfitsio=[dir] switch, e.g., --with-cfitsio=/usr/local will find the cfitsio libraries in /usr/local/lib and include files in /usr/local/include. If cfitsio is not used in the build, the js9helper will still support server-side analysis, but will not support representation files.

Note that the non-web files, programs, and scripts (e.g., js9helper) will be installed in accordance with the standard GNU --prefix=[dir] switch. The default is /usr/local.

Example configure commands are shown below for a typical Linux Apache setup (using CGI) and for a personal Mac setup (using Node.js):

  linux-cgi)
  ./configure 	--prefix=/soft/saord					\
		--with-helper=get					\
		--with-webdir=/var/www/htdocs/js9			\
		--with-cgidir=/var/www/cgi-bin/js9			\
		--with-cgiurl=./cgi-bin/js9				\
		--with-cgixpath=/soft/saord/bin				\
		--with-cfitsio=/soft/saord				\
  		CC=gcc $*

  mac-nodejs)
  ./configure 	--prefix=$HOME						\
		--with-helper=nodejs					\
		--with-webdir=/Users/me/Sites/js9			\
		--with-cfitsio=/Users/me/soft				\
		CC=gcc $*

In these examples, the JS9 web files (JavaScript, CSS etc.) will be installed in a js9 sub-directory of htdocs in the /var/www directory or in the Mac user's personal Sites directory. JS9 CGI files will be installed in a subdirectory of the cgi-bin directory. C programs and shell scripts will be installed in sub-directories of the user's home directory or the system-wide /soft/saord directory.

Build the JS9 System

Once the configure arguments are set up, you run configure, etc. as described in the basic install instructions at: Installing JS9.

Some additional notes on getting the server-side helpers working are added below.

Adding Analysis Tasks

When configuring server-side analysis, you add server-side analysis tasks to JS9 (or change the default analysis tasks) by adding or modifying json files in the analysis-plugins directory (and optionally, by adding wrapper files to the analysis-wrappers directory.) See Server-based Tasks for a description of how to configure server-side analysis tasks for your site.

Notes on Installing the Node.js-based Server-side Helper

The Node.js server-side helper is noticeably faster than CGI. It also offers more power and flexibility, e.g., the node server supports external control of the browser from the command line via the js9 script and it's underlying js9Msg facility (see External Messaging with JS9 for more information). However, use of the Node.js helper requires that you open another internet port to the outside world or configure your web server to proxy all server-side helper communication to a non-public-facing Node.js server. Node is available at:

    https://nodejs.org/
Please install the most up-to-date version of Node.js available. Once installed, you must start the server-side JS9 helper from the JS9 web install directory, i.e. the location where you installed (among other things) js9.js and the node_modules directory:
    # in the bash shell:
    node js9Helper.js 1>~/logs/js9node.log 2>&1 &
    # or, in the tcsh shell:
    node js9Helper.js >& ~/logs/js9node.log &
Unlike CGI, Node can be run as any user: it does not have to be run as the http daemon user. It certainly should not be run as root! We run it as a non-privileged user with sufficient permissions to run analysis programs. In any case, it should be run with its Unix PATH set to include the directory containing the JS9 helper program, as well as any analysis programs that will be executed by the JS9 server-side helper when requested by the client. The PATH also should include the directories housing standard Unix tools such as awk, sed, grep, and echo.

Communication between the JS9 helper and the browser (or desktop) utilizes the socket.io protocol and library (see socket.io for more information).

For security reasons, the log file should be created outside the web server directory structure.

Clients will automatically connect to the Node.js-based JS9 helper when an image is loaded into JS9. The server-side helper listens on a port specified in the js9Prefs.json file by the globalOpts.helperPort value. The default is 2718, but you can change this to any open port.

For example, some institutions do not allow non-standard ports to be open on public-facing hosts. In such a case, the Node.js-based helper can change its port to a standard port allowed by that institution (e.g., 8080 or 8000). This can be done by simply changing the helperPort property in js9prefs.js (for the client) and js9Prefs.json (for the helper).

Installing in a Secure Environment

If you are running in a secure environment (i.e. your web pages use https protocol instead of http), then node server also must be run securely. This is because modern browsers block "active mixed content". To run the Node.js helper using https protocol instead of http, create a js9Secure.json file containing path names to your private key, certificate, and certificate authority files:

  {
      "key":  "/path/to/private.key",
      "cert": "/path/to/server.certificate",
      "ca":   "/path/to//certifcate.authority"
  }
This file will reside in the directory in which the Node.js helper is started.

It is imperative that you keep your private key file secure! If this key is accessed by unauthorized parties, they can masquerade as your site. The standard recommendation is to change files permission so that only the Node.js user can read it:

   chmod 400 private.key
and keep that user's password safe!

Setting up a Web Proxy for the Node.js-based Helper

Instead of opening the JS9 helper port on a public-facing server, you can set up your web server as a reverse proxy to pass requests to a helper running on a non-public host. Reverse proxy is a standard web server technique, and is used on the main JS9 web site.

Documentation concerning setting up a proxy for the helper's underlying socket.io library can be found in the behind a reverse proxy page. Additional information can be found in these (slightly out of date) socket.io issue and stackoverflow pages.

A variation of the following httpd.conf (or ssl.conf for https) configuration should work on Apache web servers:

  # js9 client connects to main server => rewrites to js9 helper on 2718
  # NB: helper does not use ssl (ssl-based proxy doesn't work, afaik)
  RewriteEngine    on

  # new style:
  # https://socket.io/docs/v4/reverse-proxy/
  RewriteCond      %{HTTP:Upgrade}    websocket                 [NC]
  RewriteCond      %{HTTP:Connection} upgrade                   [NC]
  RewriteRule      ^/?(.*)            ws://localhost:2718/$1    [P]

  # old style:
  # https://stackoverflow.com/questions/36472920/apache-proxy-configuration-for-socket-io-project-not-in-root
  # RewriteCond    %{REQUEST_URI}     ^/socket.io               [NC]
  # RewriteCond    %{QUERY_STRING}    transport=websocket       [NC]
  # RewriteRule    ^/?(.*)            ws://localhost:2718/$1    [P]

  ProxyPass        /socket.io         http://localhost:2718/socket.io
  ProxyPassReverse /socket.io         http://localhost:2718/socket.io

  # required to handle js9msg requests to the helper's httpd support:
  RewriteCond      %{REQUEST_URI}     ^/msg                     [NC]
  RewriteRule      ^/?(.*)            http://localhost:2718/$1  [P]
The optional ProxyPass of msg allows you to send commands using wget or curl to the helper.

When configuring a proxy, set the "helperPort" in js9prefs.js to 80 or 443 (or whatever port your web server listens on) so that the browser connects to the helper via the public-facing web server. Continue to set the "helperPort" in js9Prefs.json to 2718 (or whatever port you choose), so that the helper listens on its own port. This is a special situation where the ports will be different for the client and the helper.

Notes on the CGI-based Server-side helper

If you want to utilize a CGI-based helper, you must specify the --with-cgidir and --with-cgiurl switches on the configure command line. Optionally, you also can specify the --with-cgixpath switch if you want to add directories to the CGI path:

After the build is complete, the globalOpts.helperType variable in the js9Prefs.json file should have a value of "get" and JS9 will send messages via the specified CGI command, utilizing the js9Helper.cgi script.

Of course, you must now ensure that your web server can run the js9Helper.cgi script successfully to execute JS9 CGI commands. Normal web debugging techniques come into play here: check the CGI section of your web server's configuration file and review the web access and error logs. If you continue to have problems, please let us know and we will try to help.

Last updated: July 13, 2021