Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dcache-billing-indexer seems to be broken with Java 17 and dCache 10.2 #7738

Closed
VilleS1 opened this issue Jan 28, 2025 · 2 comments
Closed
Assignees
Labels

Comments

@VilleS1
Copy link

VilleS1 commented Jan 28, 2025

/usr/sbin/dcache-billing-indexer -index /var/lib/dcache/billing/2025/01/billing-2025.01.27.bz2
ERROR - Uncaught exception
java.lang.NoClassDefFoundError: org/apache/commons/io/input/CloseShieldInputStream
at org.dcache.services.billing.text.Indexer$7.openStream(Indexer.java:526)
at com.google.common.io.ByteSource$AsCharSource.openStream(ByteSource.java:474)
at com.google.common.io.CharSource.readLines(CharSource.java:371)
at org.dcache.services.billing.text.Indexer.produceIndex(Indexer.java:510)
at org.dcache.services.billing.text.Indexer.index(Indexer.java:396)
at org.dcache.services.billing.text.Indexer.(Indexer.java:211)
at org.dcache.services.billing.text.Indexer.main(Indexer.java:686)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.CloseShieldInputStream
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 7 common frames omitted

commons.apache.org says:
"CloseShieldInputStream(InputStream inputStream) Deprecated."
"Using this constructor prevents IDEs from warning if the underlying input stream is never closed. Use wrap(InputStream) instead."

https://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/input/CloseShieldInputStream.html

@kofemann kofemann added the bug label Jan 28, 2025
@kofemann
Copy link
Member

@VilleS1 I have no idea how it was working in the past. Did you use only uncompressed files?

The workaround until we release the fix:

update /usr/share/dcache/lib/billing.sh as (add commons-io into classpath):

 billing_indexer() # $* = arguments
 {
     CLASSPATH="$(printLimitedClassPath slf4j-api logback-classic logback-core logback-console-config \
-        jul-to-slf4j commons-compress gson spring-core guava dcache-common common-cli cells dcache-core \
+        jul-to-slf4j commons-compress commons-io gson spring-core guava dcache-common common-cli cells dcache-core \
         curator-client)" \

@VilleS1
Copy link
Author

VilleS1 commented Jan 30, 2025

I just started using compressed logs to save disk space and noticed dcache-billing-indexer has problems. Indexing compressed log works with your patch and it also fixes another bug which I did not report. "dcache billing" command was unable to show logs from compressed log files. With your patch it works as it should.

kofemann added a commit that referenced this issue Jan 30, 2025
Motivation:
the dcache-billing-indexer uses commons-compress to handle bz2 files,
and has non-optional dependency on commons-io, which is missing. So we
get:

```
/usr/sbin/dcache-billing-indexer -index /var/lib/dcache/billing/2025/01/billing-2025.01.27.bz2
ERROR - Uncaught exception
java.lang.NoClassDefFoundError: org/apache/commons/io/input/CloseShieldInputStream
at org.dcache.services.billing.text.Indexer$7.openStream(Indexer.java:526)
at com.google.common.io.ByteSource$AsCharSource.openStream(ByteSource.java:474)
at com.google.common.io.CharSource.readLines(CharSource.java:371)
at org.dcache.services.billing.text.Indexer.produceIndex(Indexer.java:510)
at org.dcache.services.billing.text.Indexer.index(Indexer.java:396)
at org.dcache.services.billing.text.Indexer.(Indexer.java:211)
at org.dcache.services.billing.text.Indexer.main(Indexer.java:686)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.CloseShieldInputStream
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 7 common frames omitted
```

Modification:
explicitly add commons-io into dcache-billing-indexer classpath.

Result:
dcache-billing-indexer can process compressed files.

Fixes: #7738
Acked-by: Lea Morschel
Tested-by: Ville Salmela
Target: master, 10.2
Require-book: no
Require-notes: yes
(cherry picked from commit 153a018)
Signed-off-by: Tigran Mkrtchyan <[email protected]>
@kofemann kofemann self-assigned this Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants