[AURON #1716] fix string cast array missing null value

[cxzl25]

Which issue does this PR close?

Closes #1716

Rationale for this change

set spark.auron.enable=true;
select cast(array(c1) as string ) from test_t1 ;

[]

In Spark3.1 and above, it should be [null]

What changes are included in this PR?

Are there any user-facing changes?

How was this patch tested?

Add UT

[Copilot]

Pull request overview

This PR fixes a bug in array-to-string casting where null values within arrays were not being properly rendered. Previously, cast(array(c1) as string) would return [] for an array containing a null element, but in Spark 3.1+, it should return [null]. The implementation adds a custom Spark-compatible cast operation for List to Utf8 types.

Key Changes:

• Added custom array-to-string cast implementation in Rust that properly handles null elements
• Added comprehensive test coverage with 5 integration tests and 5 unit tests
• Implementation follows existing Spark-compatible patterns for complex type casting

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
native-engine/datafusion-ext-commons/src/arrow/cast.rs	Implements array-to-string casting with proper null handling, mirroring the existing struct-to-string pattern. Includes 5 unit tests covering edge cases (nulls, empty arrays, nested arrays, string arrays).
spark-extension-shims-spark/src/test/scala/org.apache.auron/AuronQuerySuite.scala	Adds 5 integration tests to verify Spark compatibility for various array casting scenarios (basic arrays, nested arrays, arrays with nulls, string arrays, empty arrays).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[richox]

we should think about a better way to handle inconsistent behaviors of different spark versions. for example we can expose @sparkver to rust code?

[cxzl25]

we should think about a better way to handle inconsistent behaviors of different spark versions. for example we can expose @sparkver to rust code?

After this PR merge, I plan to refactor array / map / struct cast to string to be compatible with versions less than Spark3.1.0

[yew1eb]

Hi @richox @cxzl25
I'd like to help review Pull Requests in this repo. Could you please add me as a collaborator with write access so I can approve reviews?
Thanks!