Question about Decimal Type Limitations in Iceberg

[rice668]

Query engine

Spark

Question

Hello Iceberg Community,

I would like to ask why Iceberg imposes the restriction on changing the scale for the decimal type. According to the SQL standard, the scale can indeed be changed. For instance, the SQL standard states that:

Given this, I am curious about the rationale behind Iceberg's decision to restrict the change of scale for decimal types.

Thank you!

[hashmapybx]

Hi,
This issue can be considered from two perspectives: storage layout and compatibility.

1.Iceberg typically uses a columnar storage format (such as Parquet or Avro), and data is stored in files according to the original decimal(P,S) layout. Modifying P without changing S may require rewriting the storage file.

2.Iceberg tables may be used by multiple systems or applications, and modifying P may affect compatibility with other systems because different systems may handle decimal types differently. Even if S remains unchanged, increasing P may break compatibility with data exchange and sharing with other systems.