Release Notes 4.4

Release Notes for Incorta Analytics Release 4.4

New Features

The following new features were released in Release 4.4.

Data Sources & Data Files

Added the ability to load data from multiple files into a single Incorta table. The following are now supported for file share systems like Box, Dropbox, and Google Drive: * Wildcard union of files within a directory * Loading all or matching files from a directory of folders or a subfolder into a single Incorta table * Extract metadata from filenames and add it as columns when creating a table

  • Added the ability to do the following when importing a flat file:

    • Enable or disable the header row. The header row defaults to “enabled.” If the header row is disabled, Incorta automatically labels the columns as “Column <number>.”
    • Skip a number of leading rows that you select. For example, if the header row is on row 10 and the first 9 rows are descriptive text, you can choose to skip the first 9 rows. * Enhanced the new connector SDK framework so that it’s externalized in a separate jar file.
  • Made enhancements to the Freshservice connector API.

External BI Tools

  • All published Incorta datasets can now be discovered from external BI tools for whom Incorta is a PostgreSQL database.

Security

  • Added support for Auth0, an OAuth based provider for authentication and authorization.

CMC

  • Added the ability to manage loader and analytics log files in the Cluster Management Service (CMC), without requiring back end command line access to retrieve the logs. Now you can filter and see log files based on:

    • Tenant
    • Service
    • Date
    • Service type
  • Simplified the configurations related to tuning and data loading in the Admin UI. Removed parameters like Max concurrent queries and enable compaction spill to disk from the Admin UI and is now internally managed within Incorta.
  • Tenant status in the cluster was changed to “enabled” and “disabled” instead of “Starting Tenant” and “Stopping Tenant.”

  • The following Admin UI options were removed in 4.4 because they are now calculated internally, no longer used, or were configurable in another location:

    • Under Server Configs > Clustering

      • Zookeeper Connect String

    • Under Server Configs > Tuning

      • Max In-Memory Data (%)
      • Max Concurrent Queries
      • Max CPU cores (%)

    • Under Default Tenant Configurations > Tuning

      • Table Pool Size
      • Compaction Pool Size
      • Java Compaction Memory (MB)

    • Under Tenant Configurations > Schema Pool Size

      • Compaction Pool Size
      • Compaction Files Writer Pool Size

    • Under Default Tenant Configs > Data Loading

      • Enable Compaction Spill To Disk
      • Compaction Algorithm Type

  • The following Admin UI option names changed:

    • Under Default Tenant Configs > Advanced

      • Maximum Groups Default was renamed to Insight Max Groups UI Default.
      • Maximum Groups Limit was renamed to Insight Max Groups Limit.
      • Maximum Values in Filter Subquery was renamed to Insight Max Values in Filter Subquery.

Loader

  • Incremental data extracts from source tables, especially from replicated source systems, could result in certain transactions being updated in source systems after the incremental extract in Incorta is complete. Subsequent incremental loads will miss these records. You can now choose the desired timestamp column for incremental extracts. If you choose to perform an incremental extract using the desired timestamp column, Incorta extracts records based on the MAX of the incremental timestamp column. By default, incremental extracts are based on the last successful extract time for a table. Assumptions:

    • Incremental column support is currently limited to timestamp columns only.
    • Support in the first phase is limited to Oracle and mySQL databases.
  • For large table extracts, if you use a chunking strategy, Incorta now uses the size of the chunks instead of the number of chunks to extract large tables. Incorta determines the best number of chunks to use based on the number of available threads that it can run in parallel.

Installer

  • You can now use IPv4, IPv6, cname, and domain name in the Public IP configuration during installation.
  • Introduced a new property in the engine.properties file called “engine.allow_formula_sorting_in_flat_table”. This is disabled by default. If you enable this property and use a formula to sort a list table on the dashboard, your performance could be impacted. The recommended best practice is to create and materialize this formula in the physical schema instead.

Schemas

  • Added the ability to run a schema table as incremental-only to avoid running a full load on snapshot tables. Snapshot tables are typically daily, weekly, or monthly snapshots that are more granular than in the source data system The option to run a schema table as incremental-only is disabled by default. If enabled, this query and this table will only run in incremental mode.
  • Added a feature where the files created after exporting tenants, schemas, business schemas, and dashboards have a prefix identifying what they are to better track and manage the different entities. For example, tenant_<filename>.zip, dashboard_<filename>.zip, schema_<filename>.zip, and business_schema_<filename>.zip.

Compaction

  • Added a log notification that compaction finished and the duration of time to complete compaction for all tables.

Resolved Issues

The following issues were fixed in this release.

Component Release Notes
AdminUI Changed the Excel option in the Admin UI > Default Tenants Config > External Visualization Tools Tab to Excel Add-in.
AdminUI, Server Fixed an issue where Incorta displayed different values for the Engine and Spark ports than the Admin UI.
Analyzer Fixed an issue with list table support in an Incorta table.
Charts, Dashboards Fixed an issue where dashboards were formatted incorrectly after using the “Send Now” or “Schedule Dashboard” features.
CMC Fixed an issue where log files did not download as expected in Windows.
CMC Fixed an issue where a local cluster was not created as expected after installation without Spark.
CMC Updated log files so you can see all log files in a cluster during a specified date range.
CMC Fixed an issue where a cluster did not start or stop other services on the node when the node was stopped.
CMC Fixed an issue where selecting the “tenants” column did not sort as expected.
CMC Fixed an issue where log files did not download in the Firefox browser.
CMC Fixed an issue where filters retrieved files for a tenant that was not selected.
CMC Fixed an issue where a log file did not delete from the CMC as expected.
CMC Fixed an issue where a disabled tenant name in a cluster was hyperlinked, but should not have been.
Dashboard Fixed an issue where scatter plots did not render as expected when sent in an HTML email.
Dashboard Fixed an issue where KPIs with conditional formatting did not render as expected when sent in an HTML email.
Data Alerts Fixed an issue where data alerts were suspended when an invalid email was used.
Engine Fixed an issue with applied filters on a column that did not render the selected values as expected.
Engine Fixed an issue where the stacked column chart displayed incomplete data labels.
Engine Fixed an issue where changing a formula in the business schema did not change the output in the dashboard.
Engine Fixed an issue where a session and presentation variable display correctly in the dashboard, but the catalina.out log file shows an error for the session variable.
Front-End Web Fixed an issue where date and timestamp format rendered incorrectly when used as a prompt on a dashboard.
HA Added better handling of HA when loader service becomes unresponsive.
Insights Line and bar cart column colors are retained when drilling from primary insights into secondary insights in the drilldown dashboards.
Insights Fixed an issue where no progress indicator displayed on an insight after applying filters when the max groups warning was displayed.
Loader Fixed an issue where an invalid load filter logged an error message for every row, increasing the size of the catalina.out file.
Loader Fixed an issue where adding a comment in the SQL caused a SQL format error.
Loader Added the ability to see the number of extracted rows in a log file.
Loader Previously, Incorta asked users to specify the data type of the change data capture (CDC) column during an incremental load. Now Incorta asks users to specify the CDC table and column name and queries the database metadata to get the data type.
Loader Improved the way dates are ingested into Incorta using Spark.
Script Fixed an issue where dashboards migrated from v3.3.1 to v4.2.6 did not load and displayed an error.
Server Fixed an issue where trying to reset a user password caused an EWS error, even though SMTP was configured
Server Fixed an issue where the KPI HTML did not export when the KPI value was empty.
Server After you upgrade to Release 4.3.5 and before you start the cluster or the analytics and loader services: 1. In a terminal, run the command <cmc_installation_path>/tmt, 2. Run the command ./tmt.sh -clnm <cluster_name> -umd, then 3. After the metadata updates finish, start the cluster or services.
Server Fixed an issue where you could not add a bookmark to a filter using a formula with a boolean condition.
SQL Interface Fixed an issue where if a join filter uses the IN operator on two tables, the reference SQL on the dashboard generates an error.
SQL Interface Fixed an issue where selecting large amounts of data from a large table using the SQL application port caused an error.
© Incorta, Inc. All Rights Reserved.