Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.10.1 #45

Merged
merged 19 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
903e061
displaying red border on use_ai check if user has unpaid version
psychologianauki Nov 4, 2024
fef015d
displaying columns tab correctly
psychologianauki Nov 4, 2024
4e07851
opening PageTabs corectly on sources without nested /:tab
psychologianauki Nov 4, 2024
06a2891
Merged PR 3129: 13570 displaying red border on use_ai check if user h…
Nov 4, 2024
18efc7c
Merged PR 3130: 13237 opening all source tabs from url
Nov 4, 2024
551cb89
Version number change to 1.10.1
piotrczarnas Nov 4, 2024
3d3e24b
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documa…
piotrczarnas Nov 4, 2024
f46de76
Added new environment for Moving configured check between environments
Andrzej1984 Nov 5, 2024
3e3dead
Added section about automatic table import configuration to Schedule …
Nov 5, 2024
9f295e4
Updated description in the documentation of the time when the check a…
Nov 5, 2024
9ffe1f2
Added information about deleting incidents to the documentation.
Nov 5, 2024
6c68c8c
Minor update in the delete incidents documentation.
Nov 5, 2024
c24eb20
Added example in the documentation. Moving configured checks between …
Andrzej1984 Nov 5, 2024
78e4b7b
upening single checks from url correctly
psychologianauki Nov 8, 2024
cceeaef
Merged PR 3131: 13579 opening single checks from url correctly
Nov 8, 2024
8599eda
Fix NRE when configuring timestamp columns.
piotrczarnas Nov 9, 2024
1bb143b
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documa…
piotrczarnas Nov 9, 2024
06d9726
Fix installation on Windows when the Python site packages folder is i…
piotrczarnas Nov 12, 2024
d5972a4
Update change log.
piotrczarnas Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .run/dqo run.run.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<option name="region" />
<option name="useCurrentConnection" value="false" />
</extension>
<option name="JAR_PATH" value="$PROJECT_DIR$/dqops/target/dqo-dqops-1.10.0.jar" />
<option name="JAR_PATH" value="$PROJECT_DIR$/dqops/target/dqo-dqops-1.10.1.jar" />
<option name="VM_PARAMETERS" value="-XX:MaxRAMPercentage=60.0 --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED" />
<option name="PROGRAM_PARAMETERS" value="--server.port=8888 --dqo.python.debug-mode=silent" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
Expand Down
12 changes: 3 additions & 9 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,5 @@
# 1.9.1
# 1.10.1

* Data lineage editor suggests similar source and target tables
* Fixes in the management of tabs
* ClickHouse connector
* Teradata connector
* Small UI fixes
* Performance improvements in the anomaly detection code
* Other performance optimizations
* Run Python rules in parallel to speed up anomaly detection when tables have a different number of enabled checks
* Small UI fixes to open pages directly from an URL.
* Fix problems when installing on Windows using pip, when Python was installed from Windows Store and uses a deeply nested folder structure

2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.10.0
1.10.1
2 changes: 1 addition & 1 deletion distribution/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

<groupId>com.dqops</groupId>
<artifactId>dqo-distribution</artifactId>
<version>1.10.0</version> <!-- DQOps Version, do not touch (changed automatically) -->
<version>1.10.1</version> <!-- DQOps Version, do not touch (changed automatically) -->
<name>dqo-distribution</name>
<description>DQOps Data Quality Operations Center final assembly</description>
<packaging>pom</packaging>
Expand Down
4 changes: 3 additions & 1 deletion distribution/python/dqops/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,9 @@ def install_dqo(dest: str, dqo_tag: str, dqo_version: str):
download_to_file(dqo_tag, dqo_version, distribution_local_name)

with zipfile.ZipFile(distribution_local_name, "r") as zip_ref:
zip_ref.extractall(dest)
os_platform = sys.platform.lower()[0:3]
dest_path = ("\\\\?\\" if os_platform == "win" else "") + os.path.abspath(dest)
zip_ref.extractall(dest_path)

if os.path.exists(distribution_local_name):
os.remove(distribution_local_name)
Expand Down
4 changes: 2 additions & 2 deletions distribution/python/dqops/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
# limit

# WARNING: the next two lines with the version numbers (VERSION =, PIP_VERSION =) should not be modified manually. They are changed by a maven profile at compile time.
VERSION = "1.10.0"
PIP_VERSION = "1.10.0"
VERSION = "1.10.1"
PIP_VERSION = "1.10.1"
GITHUB_RELEASE = "v" + VERSION + ""
JAVA_VERSION = "17"

Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/athena.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/clickhouse.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/csv.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/databricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/db2.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/deltalake.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/gcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/hana.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/iceberg.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/mariadb.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ and profiling data by running default data profiling checks. Simply click on the

Once new tables are imported, DQOps automatically activates [profiling and monitoring checks](../dqo-concepts/definition-of-data-quality-checks/index.md) which are which are pre-enabled by [data quality policies](../dqo-concepts/data-observability.md#automatic-activation-of-checks).
These checks detect volume anomalies, data freshness anomalies, empty tables, table availability, schema changes, anomalies in the count of distinct values, and null percent anomalies. The profiling checks are scheduled
to run at 1:00 a.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.
to run at 12:00 p.m. on the 1st day of every month, and the monitoring checks are scheduled to run daily at 12:00 p.m.

[**Profiling checks**](../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md) are designed to assess
the initial data quality score of a data source. Profiling checks are also useful for exploring and experimenting with
Expand Down
Loading
Loading