[ARUON #1541] Add work flow for iceberg #1541 #1536#1936
[ARUON #1541] Add work flow for iceberg #1541 #1536#1936guixiaowen wants to merge 2 commits intoapache:masterfrom
Conversation
ShreyeshArangath
left a comment
ShreyeshArangath
left a comment
There was a problem hiding this comment.
nit: there is a typo in the PR title
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| test-flink: |
There was a problem hiding this comment.
| test-flink: | |
| test-iceberg: |
There was a problem hiding this comment.
Pull request overview
This PR adds a new GitHub Actions workflow for testing Iceberg integration in the Auron project. The workflow is configured to run on pushes and pull requests to the master and branch-* branches.
Changes:
- Added a new GitHub Actions workflow file for automated Iceberg module testing
- Configured matrix testing for multiple Scala versions (2.12, 2.13) and Spark versions (3.4, 3.5)
- Set up automated test report uploads on failure
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Java and Maven cache | ||
| uses: actions/setup-java@v4 |
There was a problem hiding this comment.
The workflow is using an outdated version of the setup-java action. Other similar workflows in the repository (flink.yml, paimon.yml) use actions/setup-java@v5, while this workflow uses actions/setup-java@v4. For consistency and to use the latest features and security updates, this should be updated to v5.
| uses: actions/setup-java@v4 | |
| uses: actions/setup-java@v5 |
|
|
||
| - name: Upload reports | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 |
There was a problem hiding this comment.
The workflow is using an outdated version of the upload-artifact action. Other similar workflows in the repository (flink.yml, paimon.yml) use actions/upload-artifact@v6, while this workflow uses actions/upload-artifact@v4. For consistency and to use the latest features and security updates, this should be updated to v6.
| uses: actions/upload-artifact@v4 | |
| uses: actions/upload-artifact@v6 |
| cache: 'maven' | ||
|
|
||
| - name: Test Iceberg Module | ||
| run: ./build/mvn -B test -X -pl ${{ matrix.module }} -am -Pscala-${{ matrix.scalaver }} -Piceberg-${{ matrix.iceberg }} -P${{ matrix.sparkver }} -Prelease |
There was a problem hiding this comment.
The Maven test command includes the -X flag which enables debug output. This flag is not used in other similar workflows (flink.yml, paimon.yml) in the repository and will produce verbose output that can clutter the CI logs. Unless debug output is specifically needed for troubleshooting, this flag should be removed for consistency with other workflows.
| run: ./build/mvn -B test -X -pl ${{ matrix.module }} -am -Pscala-${{ matrix.scalaver }} -Piceberg-${{ matrix.iceberg }} -P${{ matrix.sparkver }} -Prelease | |
| run: ./build/mvn -B test -pl ${{ matrix.module }} -am -Pscala-${{ matrix.scalaver }} -Piceberg-${{ matrix.iceberg }} -P${{ matrix.sparkver }} -Prelease |
| matrix: | ||
| iceberg: [ "1.10.1" ] | ||
| javaver: [ "17"] | ||
| scalaver: [ "2.12", "2.13"" ] |
There was a problem hiding this comment.
There's a syntax error in the YAML array definition. The line has two double quotes after "2.13" which will cause the workflow to fail parsing. The correct syntax should have only one double quote at the end of the array.
| scalaver: [ "2.12", "2.13"" ] | |
| scalaver: [ "2.12", "2.13" ] |
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| test-flink: |
There was a problem hiding this comment.
The job name is "test-flink" but this workflow is testing Iceberg modules, not Flink. The job name should be updated to accurately reflect what is being tested, such as "test-iceberg" or "test-iceberg-module".
| test-flink: | |
| test-iceberg: |
| @@ -0,0 +1,68 @@ | |||
| # | |||
There was a problem hiding this comment.
The PR title contains a spelling error. "ARUON" should be spelled "AURON" to match the project name used throughout the workflow file (see line 50 "Checkout Auron").
|
|
||
| steps: | ||
| - name: Checkout Auron | ||
| uses: actions/checkout@v4 |
There was a problem hiding this comment.
The workflow is using an outdated version of the checkout action. Other similar workflows in the repository (flink.yml, paimon.yml) use actions/checkout@v6, while this workflow uses actions/checkout@v4. For consistency and to use the latest features and security updates, this should be updated to v6.
| uses: actions/checkout@v4 | |
| uses: actions/checkout@v6 |
2 participants
Which issue does this PR close?
Closes #1541
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?
How was this patch tested?