We are excited to unveil Open Data Contract Standard (ODCS) Version 3, featuring significant improvements and new functionalities to better support modern data management needs. This release introduces updates to the fundamentals, schema, and data quality, as well as adding a dedicated Support & Communication Channels section. Our goal with these changes is to streamline the data contract process, making it more adaptable, scalable, and easier to use in various contexts. OCDS v3 is now available on Bitol’s GitHub.

Highlights of ODCS v3

One of the key advancements in ODCS v3 is the modification of the schema structure, which now enables support not only for relational databases (as in v2) but also for hierarchical and complex data storage, streaming messages, and unstructured data. This greatly expands the types of data environments that can seamlessly integrate with ODCS, facilitating better management of complex data types and structures.

 

Additionally, the definition of infrastructure has been extended, improving integration by encompassing metadata around systems, applications, and platforms. This enhancement enables ODCS to serve as an even more powerful source of truth, capturing essential context from a broader range of system metadata, which helps create a unified view of the data and the systems interacting with it.

Data quality support has also been significantly boosted in v3 to enable compatibility with virtually any data quality tool, making it easier for organizations to integrate their preferred tools. Looking ahead, these capabilities will be further extended in the upcoming ODCS v3.1.

 

Visit OCDS v3 on Bitol’s GitHub. Don’t forget to star it while you are there!

ODCS v3 is now divided into 11 sections:

• Fundamentals.
• Schema.
• Data quality.
• Pricing.
• Team (fka stakeholders).
• Security.
• Service-Level Agreement (SLA).
• Infrastructure.
• Support.
• Business rules.
• Custom properties.

Link to full-size PNG and SVG. Available under the Apache 2 license.

Changes in ODCS v3

ODCS v3 brings some changes, including breaking ones, to ODCS v2. Here is a list of the changes. Details can be found in the changelog.

Support & Communication Channels

We’ve added a new section dedicated to providing a more structured approach to support, including Slack channels, feedback URLs, and distribution lists, which were previously scattered throughout the contract.

Changes to Fundamentals

The fundamentals have been streamlined.

• UUID renamed to id
• Name was added for easier identification.
• quantumName was renamed to dataProduct and made optional.
• datasetDomain renamed to domain, removing the dataset prefix.
• Dropped unused fields: datasetKind, userConsumptionMode, sourceSystem, sourcePlatform, productSlackChannel, productFeedbackUrl, productDl, username, password, driverVersion, driver, server, project, datasetName, database, schedulerAppName.

Schema Overhaul

Major changes have been applied to the schema, including support for non-table formats, hierarchies, and arrays.

• New fields: name, items.
• Deprecated: priorTableName, table (consider using name instead).
• columns renamed to properties.
• dataGranularity renamed to dataGranularityDescription.
• Other field changes:
  • encryptedColumnName → encryptedName
  • partitionStatus → partitioned
  • sampleValues → examples
  • isNullable → required
  • isUnique → unique
  • isPrimaryKey → primaryKey
  • criticalDataElementStatus → criticalDataElement

We also introduced restrictions to schema.*.logicalType, limiting it to string, date, number, integer, object, array, and boolean. New logicalTypeOptions have been added to offer more flexibility.

Data Quality Improvements

ODCS v3 has reimagined its Data Quality section, offering more support for tools and use cases. Key changes include:

• Deprecated toolName, replaced by type=custom; engine: <engine name>.
• scheduleCronExpression replaced with a simplified schedule and scheduler.

Role and Team Updates

They stay mostly the same in their structure but clarify names.

• Stakeholders were renamed to Team, with the content remaining the same.
• Role updates: Descriptions have been added, and access is no longer required.

SLA and Security

In response to the new schema structure, we’ve adjusted the SLA to reflect that data is no longer solely based on tables and columns.

• Columns are renamed to elements across the SLA.
• The SLA now includes explicit references to Data QoS.

Custom Properties

Several elements, such as systemInstance, are no longer supported but can be included as custom properties where necessary. Custom properties can be used in more sections as well.

No Changes in Pricing and Security

While much has evolved, our pricing and security models remain unchanged.

 

The Team Behind ODCS v3

ODCS v3 represents the collaborative effort of a global team of experts and volunteers who worked tirelessly to make this version a reality:

• Andrew Jones, Principal Engineer and Independent Consultant, UK
• Andy Petrella, CEO at Kensu, Belgium
• Atanas Iliev, System Architect at ALH Gruppe, Germany
• Christian Foyer, Data Engineer, Denmark
• Dirk Van de Poel, Co-founder of Klarrio, Belgium
• Eugene (Gene) Stakhov, Information Governance Practice Lead at enChoice, USA
• Georges Kopp, Economist and Business Analyst, France
• Jean-Georges Perrin (Chair), Chief Innovation Officer at AbeaData, USA
• Jochen Christ, Co-founder of Data Contract Manager, Germany
• Manuel Destouesse, Software Engineer, USA
• Martin Meermeyer, Head of Global Data Governance at Lidl e-commerce, Germany
• Peter Flook, Data Engineer, Taiwan
• Sandro Pugliese, Head of Enterprise Data Engineering at TelevisaUnivision, USA
• Simon Harrer, Co-founder of Data Contract Manager, Germany
• Todd Nemanich, Founder and CTO at AbeaData, USA
• Tom Baeyens, CTO at Soda, Belgium

To avoid missing anyone, this list does not mention the volunteers outside the TSC who contributed in v3.

 

About Bitol

Bitol is an incubation project from the Linux Foundation, focused on creating open standards for data management. This initiative fosters industry-wide collaboration and innovation, allowing organizations to adopt a unified approach to handling data contracts, metadata, and system interoperability.

The Technical Steering Committee (TSC), which oversees the development and direction of Bitol, is chaired by Jean-Georges Perrin (USA) and includes Andrew Jones (UK), Andy Petrella (Belgium), Atanas Iliev (Germany), Christian Foyer (Denmark), Dirk Van de Poel (Belgium), Eugene (Gene) Stakhov (USA), Georges Kopp (France), Jochen Christ (Germany), Manuel Destouesse (USA), Martin Meermeyer (Germany), Peter Flook (Taiwan), Sandro Pugliese (USA), Simon Harrer (Germany), Todd Nemanich (USA), and Tom Baeyens (Belgium).

Currently, Bitol’s primary focus is on advancing the Open Data Contract Standard (ODCS) and the Open Data Product Standard (ODPS). These standards aim to provide comprehensive frameworks for data contracts and data products, supporting organizations in managing data with greater consistency, transparency, and collaboration across systems.

The Bitol project seeks to provide a shared framework that promotes consistency, transparency, and collaboration across diverse data systems and environments. For more information, visit bitol.io.