Managing Apache Spark#
Querona ships with a single, pre-configured instance of Apache Spark that starts automatically. Spark is a separate entitity from Querona and works independently in a dedicated Java process. However, Querona manages Spark configuration and lifecycle, and uses it for the following purposes:
a default federation engine,
a ready-made materialization target for end users, available for use via a pre-configured connection.
Local Spark instance management options can be found under the.
Viewing instance status#
Selecting the instance brings up the details blade with all configuration values in read-only mode.
Click the Refresh button to refresh the instance list and to ensure the State column shows the latest information. The view has not auto-refreshing functionality so it might be necessary to use it a few times during state transition.
Starting and stopping Spark instance#
The following buttons control the instance state:
Starts the instance, if not already started.
Stops the instance, if not already stopped, by sending a “self-shutdown” command, allowing the instance to free all resources and close itself gracefully. This may take up from one minute to several minutes, depending on the load and instance size.
Kills the instance without waiting for a clean shutdown, by sending an OS-level kill signal which terminates the underlying JVM process (not recommended, unless a graceful shutdown fails).