JS9: image display beyond the desktop
Eric Mandel
Alexey Vikhlinin
Center for Astrophysics | Harvard & Smithsonian
AAS 238: June 7, 2021
In the beginning ...
- SAOimage rose from the ashes of the Challenger disaster
- SAOtng added a connection to external analysis via XPA
- DS9 has long been the gold standard on the desktop
Steve Murray (aka the boss): Einstein/ROSAT HRI, Chandra HRC, ADS, 1944-2015
The world has moved beyond the desktop, as have our data analysis needs:
- Web/cloud technology makes data available everywhere
- Just in time for the upcoming avalanche of optical/IR data
- [hears about Smart-X] ... OMG in X-ray astronomy too??
How much can we put into the browser? All of DS9? More?
- Responsive colormaps, scaling, contrast/bias, zoom
- Create, edit, load, store regions of interest
- Load and run external analysis tasks
- Support a stable API for scripting
- Utilize browser capabilities, such as blending
- Multiple displays, synchronization, plugins, callbacks ...
- For more demos, visit https://js9.si.edu
https://emscripten.org: run C/C++ code in the browser
- Open-source compiler toolchain compiles to WebAssembly or JavaScript
- Drop-in replacements for configure, make, gcc ...
- Runtime environment with memory-based file system
- Often requires only a thin layer to call from JavaScript
- Wasm binary format runs at near-native speed
https://socket.io: communicate with Python or the shell
- The browser connects to the JS9 helper via socket.io
- Then upgrades to a persistent websocket connection
- The browser sends analysis requests to the helper and gets back results for display
- pyjs9 and js9msg send API requests to the helper, which are relayed to the browser ...
- ... browser responds back to the helper, which in turn sends results to the requester
https://www.electronjs.org/: Chrome-based desktop apps
- Today, we still need desktop apps that foster "unconscious
ease of use"
- Electron combines Chrome's UI libraries with Node.js
- App generation even handles Mac notarization, signing
- This talk utilizes a web page loaded into Electron
- We are working on a desktop app called Voyager, the next in the SAOimage line
Of course, there are challenges ...
- Most Browsers limit the amount of available memory
- Firefox seems to be semi-unlimited
- Chrome's limit seems to be 4Gb, iPad much less
- Sounds reasonable, but we're used to unlimited
- Even worse, FITS files get stored in memory ...
- Sandboxing and other browser security restrictions
- Electron: no default access to underlying file system
- To access file system, turn off some security ... hmmm
- The ubiquitous problem of downloading large data files has not
gone away ...
- Yeah, what about Python?
- Mozilla working on it: Pyodide is Python in a browser
- https://github.com/pyodide/pyodide
- Not surprisingly, they use Emscripten
- Can we actually use Pyodide, or is it just a toy?
- And of course, there's always CSS to contend with ...
We need much deeper experience with web techniques
- Especially if we want to increase analysis capabilities
in browsers
- E.g., I'm exploring an Emscripten-based cfitsio driver to
access a file without downloading it completely
- Can we make a serious run at Python-based analysis in the browser?
- A web-based future requires more heads and hands: please join me!
Conclusion: the web community is doing amazing things,
thanks to efforts by groups large and small. We should join them.
|
Disclaimer: my career as a tech prophet has been a bit uneven:
|
Sun-2 workstation, circa 1983
|
"Why would you want to display two terminal windows at the same time?"
|
"No one is going to write a socket-based server just to display status reports!"
|
Mosaic browser, circa 1993
|
|