API, Core: Add toString to StructProjection and StructLikeUtil - #18342
Conversation
StructProjection and the struct returned by StructLikeUtil.copy represent partition tuples but relied on the default Object.toString, which prints an unreadable identity hash when debugging. Render field names and values instead. Followup from apache#18108
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Thanks for this, I agree we need a more human readable name for these structs
| StringBuilder sb = new StringBuilder(); | ||
| sb.append("StructProjection{"); | ||
| List<Types.NestedField> fields = type.fields(); | ||
| for (int i = 0; i < fields.size(); i += 1) { | ||
| if (i > 0) { | ||
| sb.append(", "); | ||
| } | ||
| sb.append(fields.get(i).name()).append("=").append(get(i, Object.class)); | ||
| } | ||
| sb.append("}"); | ||
| return sb.toString(); |
There was a problem hiding this comment.
Looks like we already used StringJoiner before in PartitionSet
, so it shall save some manual stringBuilder logic. Might worth considering something like below| StringBuilder sb = new StringBuilder(); | |
| sb.append("StructProjection{"); | |
| List<Types.NestedField> fields = type.fields(); | |
| for (int i = 0; i < fields.size(); i += 1) { | |
| if (i > 0) { | |
| sb.append(", "); | |
| } | |
| sb.append(fields.get(i).name()).append("=").append(get(i, Object.class)); | |
| } | |
| sb.append("}"); | |
| return sb.toString(); | |
| List<Types.NestedField> fields = type.fields(); | |
| StringJoiner joiner = new StringJoiner(", ", "StructProjection{", "}"); | |
| for (int pos = 0; pos < fields.size(); pos += 1) { | |
| joiner.add(fields.get(pos).name() + "=" + get(pos, Object.class)); | |
| } | |
| return joiner.toString(); |
On a separate note, I noticed that if we use get(pos, Object.class) on deeply nested struct, it might have side effect of invoking javaClass.cast(nestedProjections[pos].wrap(nestedStruct)), so worth double checking whether we want the behavior as part of toString call.
There was a problem hiding this comment.
it might have side effect of invoking javaClass.cast(nestedProjections[pos].wrap(nestedStruct)),
This is a great callout, I missed this originally but any reason we can't just read from the actual struct? Otherwise logging one of these may lead to really unexpected results after the logging as a result of the side effect.
There was a problem hiding this comment.
Thanks for the feedback. I changed it such that we are reading it from the original struct.
There was a problem hiding this comment.
I am wondering if helps to add a few tests in https://github.com/apache/iceberg/blob/main/api/src/test/java/org/apache/iceberg/util/TestStructProjection.java
Given the current form, the structProjection toString only show the top level field, which might not help much for the human-readable debug-ability. I do feel that the toString is effectively a nested materialization for struct projection, so probing is not free.
StructProjection.create(PROJECTED_STRUCT, PROJECTED_STRUCT).wrap(Row.of(1L, Row.of("John", "Q", "Doe"))).toString()
//return "StructProjection{id=1, person=org.apache.iceberg.TestHelpers$Row@447e510}"
// instead of desired
// "StructProjection{id=1, person=StructProjection{first=John, middle=Q, last=Doe}}"There was a problem hiding this comment.
I think both are good points. I initially hesitated to add tests because currently we don't have many toString tests in the project.
I made toString descend into nested projections. To avoid the wrap() side effect we discussed above, the nested descent uses copyFor, which wraps a fresh projection instead of mutating the shared nested one. This is OK, because we don't expect toString() to be used in the inner loop.
| sb.append(values[i]); | ||
| } | ||
| sb.append("]"); | ||
| return sb.toString(); |
There was a problem hiding this comment.
This is similar to StructTransform::toString with values is already object array, so can probably simplify to
| return sb.toString(); | |
| return Arrays.stream(values) | |
| .map(String::valueOf) | |
| .collect(Collectors.joining(", ", "[", "]")); |
| } | ||
|
|
||
| @Test | ||
| void toStringRendersNullNestedStructAsNull() { |
There was a problem hiding this comment.
nit: would be great to have coverage on StructProjection.createAllowMissing(DATA_STRUCT_MISSING_NESTED_FIELD, PROJECTED_STRUCT) as well.
There was a problem hiding this comment.
Added toStringRendersMissingNestedFieldAsNull
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Thanks @anoopj and thank you @dramaticlly for the reviews!
StructProjection and the struct returned by StructLikeUtil.copy represent partition tuples but relied on the default Object.toString, which prints an unreadable identity hash when debugging. Render field names and values instead.
Followup from #18108