SlashDB ver. 1.8 (Winter 2024) /w Updates

SlashDB ver. 1.8

SlashDB version 1.8 is now available for download or for instant launch on Azure and AWS clouds.

This release adds 4 new features and a number of improvements to performance, software developer’s kits, security and documentation.

For upgrading instructions please see chapter Upgrading in our User Guide.

 

0
New features
0
Enhancements & fixes
0
Automated tests
0%
Test code coverage
GET /db
Schedule a Demo

Version 1.8.90, updated August 20th, 2024

This is small maintenance release, which includes a fix to only one issue, which one of our clients has spotted. Setting json_sdb.href to False in slashdb.ini unexpectedly breaks one of the screens in the UI. This setting is True by default.

Version 1.8.89, updated August 8th, 2024

This is another maintenance release. Included are fixes to certain non-critical security findings from recent penetration test. One notable functionality change is that a user who was granted a user management privilege will not be able to create new users with privileges they themselves do not have.

Security

  • Added userdef_enforce_privilege_hierarchy configuration option to slashdb.ini. When enabled, it will disallow users to create new users or modify other users’ privileges that they themselves don’t have. This allows for creating a hierarchy of privileged accounts.
    • Note that his will become a default behavior in the next major release.
  • Added allowed_connection_string_parameters configuration option to slashdb.ini. Only parameters so whitelisted will be allowed in database connection strings.
    • This fixes several actual and potential security vulnerabilities (none critical).
  • Fixed possibility of a denial of service attack in the database test connection feature.
    • Test connection endpoint is now only available to users authorized to manage database connections.
    • Databases on localhost must have ports white listed in the new slashdb.ini configuration option allowed_localhost_database_ports.
  • Fixed client-side template injection.
  • Fixed a possibility to create object with another user set as the creator. The creator field has little to no significance in authorization logic. Now creator will be assigned from the active request’s user session.
  • Removed comments from compiled code.

User Interface/User Experience

  • Fixed regression in the single record editing screen, which would fail to update numerical values due to stricter payload validation, which was enabled by default in version 1.8.76.

Version 1.8.76, updated May 31st, 2024

This is a maintenance release, which provides improvements and fixes for 76 items.

Security

  • Require change of default username/password in virtual machines for development.
  • Prevent a theoretically possible ReDoS vulnerability in AngularJS
  • Fix an error when trying to create a user without apikey, which can be allowed
  • Remove sensitive data from query parameters
  • Improve SAML configuration – remove name_id_value_req as settable configuration option
  • Remove password hash from the userdef API response
  • Fix SAML authentication with Azure IDP and to two factor authentication
  • Fix known vulnerabilities in the components of the Docker image

Performance & Stability

  • Fix regression when connection was being tested on each request
  • Fix bulk insert for tables without primary keys in certain database backends.
  • Fix limitation of memcache for very large configs that prevent multiprocess synchronization
  • Fix issues with POST (insert) in XML
  • Sorting on invalid column will now return HTTP code 400 (Bad Request)
  • Fix XML PUT/POST requests silent fail when only invalid fields sent in request body
  • Fix regression where MySQL stored procedure could proceed data truncated or error raised when using limit/offset and stream
  • Fix return code for a delete by a non-existing alternate key (returned 204 instead of 404
  • Query string parameters now validated before validation of the full payload
  • Normalize handling of the missing fields in POST requests
  • Fix error 500 when filtering for wildcard * on a binary column
  • Fix error 500 with multi-statement SQL Pass-thru queries on SQLite
  • Fix possible truncate of the config files on kill -9
  • Normalize treatment of boundary values in float type (nan, -inf, inf)
  • Fix HTML/JSON/CSV schema requests raising 500 error on an empty result from a SQLPass-thru query
  • Fix raising of the code 500 when ambiguous column names are used in SQLite
  • Fix JSON validation failing if “required” field is not present in the schema
  • Fix JSON schema when db.limit parameter is set in slashdb.ini
  • Payload validation is now enabled by default, which can be overridden in the ini file
  • Refactoring of `db_to_python_map` extending for plugins
  • Upgrade pyodbc and other python dependencies

User Interface/User Experience

  • Fix display and rounding of very small floating point numbers in the GUI
  • Remove user “admin” from a dropdown in privilege in configuration.
  • Upon reconnecting an abandoned will now redirect to the same page instead of the home page
  • Improve unclear/misleading error message in SQL Pass-thru
  • Fix display of error messages in user definition screen
  • Fix opening of the SQL Pass-thru query output in separate window under Firefox
  • Fix display and rounding of very large floating point numbers in HTML format
  • Fix persistent error message when deleting a single record in the GUI
  • Fix the order of table columns in the SQL Pass-thru Query Studio schema view
  • Fix duplicate warning and error message boxes after session timeout
  • Add formal schema in HTML representation

Databricks

  • Fix session status in the GUI for Databricks after timeout
  • Fix unhandled 500 exception when user connects without credentials to Databricks
  • Fix offset without a limit not working in Databricks
  • Fix error 500 on GET to Parquet table in Databricks
  • Fix the POST to table without PK does not respecting the requested output format
  • Allow data updates in Databricks using Data Discovery
  • Review and add automated tests against Databricks

Snowflake

  • Fix connection connection setup for Snowflake to use users’ default role and warehouse, unless specified
  • Fix Snowflake’s `TIMESTAMP_NTZ` is not mapped to a Python type
  • Fix `distinct&offset` not working in Snowflake
  • Improved an error message for Snowflake on connect if invalid account identifier is used
  • Improve test connection to Snowflake with support for role and warehouse
  • Improve schema reflection in Snowflake by supporting quoted and mixed case schemas and database names
  • Fix incorrect mapping of Snowflake’s `INT` to `Decimal`
  • Fix filtering for null or empty in Snowflake
  • Add read-only functional tests for Snowflake

DevOps, Testing and Tools

  • Fix a typo in slashdb-diag help
  • slashdb-diag tool now adds header to log files
  • Refactoring of the slashdb CLI tools to reusable parts
  • Fix failing builds for vagrant, Azure and AWS
  • Prevent startup in multi-process mode without memcache defined
  • Add support for Python 3.11
  • Review Azure Marketplace configuration

What’s new in version 1.8?

Schema (metadata) in JSON and CSV (in addition to XML)

SlashDB is unique in providing database APIs in multiple representations: JSON, XML, CSV and HTML. Until now, the schema or metadata has been supported for XML only. We now have support for JSON and CSV. The user friendly HTML representation is coming soon too.

We are deprecating the .xsd extension in favor of a generic ?schema query string.

For example, this URL leads to the JSON representation of data in the response table in the pystreet database:

https://demo.slashdb.com/db/pystreet/response.json?limit=5

Accordingly, the URL for the schema (metadata) will be:

https://demo.slashdb.com/db/pystreet/response.json?schema

For complete documentation on this feature, please see our User Guide at https://docs.slashdb.com/user-guide/developing-with-slashdb/schemas/

Uniform log format across all components

Like many web service applications, SlashDB’s architecture is layered, using components such as a reverse proxy (by default NGINX) and a web application server (uWSGI). The core application and the third party components all log request activity, which can be helpful in diagnosing any issues or performance monitoring.

From this release on, we configured the log format for each component in the same way. This allows users to concatenate log files from all components for analysis (e.g. to trace the full request lifecycle across all stages). We are also shipping a command line tool for gathering all relevant logs in any given time period.

Here’s how it works:

Screenshot of slashdb-diag tool

For complete command line, options please use –help.

# /opt/slashdb/bin/slashdb-diag --help
Usage: slashdb-diag [OPTIONS] LOG_FILES

  Tool for collecting log files and config files to help with troubleshooting.

  Examples:
  $ slashdb_diag
  Will try to find configs and logs in standard locations
  and will save them in a zip file.

  $ slashdb_diag --ini /path/to/slashdb.ini --nginx-conf /path/to/nginx.conf
  Will get configs and logs based on the paths provided.

  $ slashdb_diag /var/log/slashdb/slashdb.log /var/log/slashdb/uwsgi.log --no-combined --no-zip
  Will get only these two log files and save them in a folder.

  $ slashdb_diag --start 2023-11-17T16:06:10 --end 2023-11-17T17:06:10
  Will get only logs in the given time frame.

Options:
  --ini FILE                   Path to slashdb.ini.
  --nginx-conf FILE            Path to nginx.conf.
  --start [%Y-%m-%dT%H:%M:%S]  Start date of the logs, YYYY-MM-DDTHH:mm:ss
                               format.
  --end [%Y-%m-%dT%H:%M:%S]    End date of the logs, YYYY-MM-DDTHH:mm:ss
                               format.
  --configs / --no-configs     Whether to include config files.  [default:
                               configs]
  --zip / --no-zip             Whether to zip the resulting directory.
                               [default: zip]
  --combined / --no-combined   Whether to combine log files to a single file.
                               [default: combined]
  --help                       Show this message and exit.

Encrypt database credentials

Storing credentials that need to be passed to downstream services is challenging. Encryption is an obvious solution, but the application needs to be able to decrypt them when needed and do that unattended on restart. SlashDB now supports passing the secret key from cloud infrastructure to the instance. Please contact our technical support team for more details on using this capability.

Single Sign-on capability for JavaScript and ReactJS SDK

One of the uses for SlashDB is to provide a data backend to web and mobile applications without doing any coding. Authentication is almost always a critical requirements for such applications. In case you did not know, we have open source SDKs for JavaScript and ReactJS. The SDBClient class has new features to enable single sign on. For technical documentation, please see https://slashdb.github.io/js-slashdb/docs/SlashDBClient.html

For a fully functional interactive application, which showcases how to use SlashDB as a backend, including the single sign-on feature, please visit this GitHub project:

https://github.com/SlashDB/taskapp-demo

What’s improved?

Security

  • Supports encryption of database credentials
  • Prevents sending user’s credentials in HTTP responses
  • Adds optional security headers for controlling browser policies for cross-origin resources
  • Fixed tests for SAML 2.0 when running on local machine
  • Reviewed and fix remaining items from the Netspark test
  • Fixed hostname validation

Performance and Core Functionality

Of note are:

  • Schema generation, including in streaming mode
  • Optional schema validation for posted data
  • Enhanced SlashDB JDK for JS/React with SSO OAuth2 (PKCE)
  • Ensured there is a unique request identifier in logs where possible
  • Fixed problems with GEOGRAPHY type in Microsoft SQL Server
  • Updated Snowflake driver due to end of support
  • Updated Databricks driver
  • Fixed Decimal and Timestamp issues in the new Databricks driver
  • Code refactoring in MySQL driver
  • Fixed a license validation issue
  • Fixed endpoint for database model loading, which could result with unintentional model unload
  • Fixed test connection to include all optional settings
  • Added database type level TDS_VERSION and trust connection settings for MS SQL
  • Fixed JSON rendering of time and timedelta types in SQL Pass-thru
  • Fixed limit/offset use with stored procedures in MySQL

User Experience

Various fixes and feature improvements including:

  • Allows clicking to filter on cells which contain only spaces
  • Normalized error codes related to features that are not implemented in certain contexts
  • Displays warning when user could remove a privilege to a query from themselves
  • Displays warning about duplicate email when viewing user definition
  • Fixed checkbox rules in the Administrative Privileges section of the user configuration screen
  • Allows copying of SQL Pass-thru query for users with permission to see the query definition
  • Fixed hyperlinks for download
  • Fixed filtering links on NULL value
  • Fixed minor display anomaly for single column results with empty cells
  • Fixed display of values with curly braces

Documentation, Dev/Ops and Testing

  • Made Azure distribution compatible with Trusted Launch
  • Rewrote instructions for installation on AWS
  • Automated about 75% of UI tests
  • Updated documentation on slashdb.ini settings, schema
  • Code cleanup and refactoring
  • Automatically clean up docker images after test runs
  • Deprecated URL substitution feature

Connect with us

We invite you to get in touch with us. You can reach us via:

About the image

AI generated by Microsoft Bing powered by DALL-E with manual modifications in GIMP.

Back To Top