JDBC Driver for Databricks
Overview
This Databricks JDBC driver is provided by Databricks and can be used in Collibra Catalog.
To register Databricks-provided sources, use the section ‘Collibra provided drivers’ in Collibra Catalog. Leveraging this driver, Collibra Catalog will be able to register database information and extract the structure of the source into its schemas, tables and columns.
JDBC drivers can retrieve the following database components:
- Schemas
- Tables
- Views
- Columns
More details
Release Notes
Enhancements & New Features
- Added `DatabaseMetaData.getProcedures()` and `DatabaseMetaData.getProcedureColumns()` to discover stored procedures and their parameters. Queries `information_schema.routines` and `information_schema.parameters` using parameterized SQL for both SEA and Thrift transports.
- Added connection property `OAuthWebServerTimeout` to configure the OAuth browser authentication timeout for U2M (user-to-machine) flows, and also updated hardcoded 1-hour timeout to default 120 seconds timeout.
- Added connection property `UseQueryForMetadata` to use SQL SHOW commands instead of Thrift RPCs for metadata operations (getCatalogs, getSchemas, getTables, getColumns, getFunctions). This fixes incorrect wildcard matching where `_` was treated as a single-character wildcard in Thrift metadata pattern filters.
- Added connection property `TreatMetadataCatalogNameAsPattern` to control whether catalog names are treated as patterns in Thrift metadata RPCs. When disabled (default), unescaped `_` in catalog names is escaped to prevent single-character wildcard matching.
- Bumped `com.fasterxml.jackson.core:jackson-core` from 2.18.3 to 2.18.6.
- Fat jar now routes SDK and Apache HTTP client logs through Java Util Logging (JUL), removing the need for external logging libraries.
- Added Apache Arrow on-heap memory management for processing Arrow query results. Previously, Arrow result processing was unusable on JDK 16+ without passing the `–add-opens=java.base/java.nio=ALL-UNNAMED` JVM argument, due to stricter encapsulation of internal APIs. With this change, there is no JVM argument required – the driver automatically falls back to an on-heap memory path that uses standard JVM heap allocation instead of direct memory access.
- Log timestamps now explicitly display timezone.
- [Breaking Change] `PreparedStatement.setTimestamp(int, Timestamp, Calendar)` now properly applies Calendar timezone conversion using LocalDateTime pattern. Previously Calendar parameter was ineffective.
- `DatabaseMetaData.getColumns()` with null catalog parameter now retrieves columns from all available catalogs when using SQL Execution API.
Compatibility
- Collibra Data Intelligence On-Prem
- Collibra Data Intelligence Cloud
Dependency
License and Usage Requirements
Release History
Release Notes
Enhancements & New Features
Resolved Issues
The following issues have been resolved in Databricks JDBC Driver 3.0.3.
* Fixed excessive calls to token endpoint during getToken.
Compatibility
- Collibra Data Intelligence On-Prem
- Collibra Data Intelligence Cloud
Dependency
License and Usage Requirements
Release Notes
-
Added support for Azure Managed Identity (OAuth 2.0), OAuth token exchange, and token caching on Linux/Mac.
-
Introduced support for VOID, Variant, and TIMESTAMP_NTZ data types.
-
Enhanced metadata handling for unknown column types.
-
Updated Netty libraries and expanded domain support (e.g.,
cloud.databricks.mil). -
Resolved issues with timestamp conversion, proxy documentation, and property handling.
-
Known issues include Arrow deserialization with Java 11+, date handling before 1582, and TLS 1.1 incompatibility on Java 6.
Compatibility
- Collibra Data Intelligence On-Prem
- Collibra Data Intelligence Cloud
Dependency
License and Usage Requirements
Release Notes
- Added a new
OAuthEnabledIPAddressRangesproperty that allows clients to override the default OAuth callback port(s), facilitating OAuth token acquisition in environments with network port restrictions. - Refresh token support is now available. This enables the driver to automatically refresh authentication tokens using the
Auth_RefreshTokenproperty. - Added support to use the system’s trusted store with a new
UseSystemTrustStoreproperty. When enabled (UseSystemTrustStore=1), the driver verifies connections using certificates from the system’s trusted store. - Added
UseServerSSLConfigsForOAuthEndPointproperty that when it is enabled, it allows clients to share the driver’s SSL configuration for the OAuth endpoint. - BASIC authentication is now disabled by default. To re-enable it, set the
allowBasicAuthenticationproperty to 1.
Compatibility
- Collibra Data Intelligence On-Prem
- Collibra Data Intelligence Cloud
Dependency
License and Usage Requirements
Release Notes
The driver now supports SQLPrimaryKeys and SQLForeignKeys catalog functions when connecting to a server of a supported version.
Compatibility
- Collibra Data Intelligence Cloud
- Collibra Data Intelligence On-Prem
Dependency
License and Usage Requirements
Release Notes
The driver has been certified with the username/password authentication scheme for the following:
- Ingestion of technical metadata from the source.
- Profiling of the registered data source and calculation of statistics that describe the data.
Compatibility
- Collibra Data Intelligence Cloud
- Collibra Data Intelligence On-Prem
Dependency
- N/A
License and Usage Requirements
Community
See existing Q&A in the Collibra Community
Browse discussions with customers who also use this app.
Start a New Topic in the Collibra Community
Collibra-hosted discussions will connect you to other customers who use this app.
Technical Support
Support is provided for the template as downloaded from the Marketplace, without changes to the way it works. These drivers are designed to only work with Collibra Catalog and cannot be used for other purposes.
Partner supported resources
Already a Databricks customer? Click here if you are encountering a technical or licensing issue
Discuss, share and network with Databricks users and experts in the Databricks Community.
Use of this solution requires a license to Collibra Data Intelligence Cloud and the purchase of Metadata Integrations at a volume commensurate with desired use. Please contact your sales representative or Customer Advisory Manager for more information.
