GH-51007: [C++] Make uriparser an external dependency - #51244
imtherealnaska wants to merge 6 commits into
Conversation
| packages+=("${MINGW_PACKAGE_PREFIX}-gflags") | ||
| packages+=("${MINGW_PACKAGE_PREFIX}-grpc") | ||
| packages+=("${MINGW_PACKAGE_PREFIX}-gtest") | ||
| packages+=("${MINGW_PACKAGE_PREFIX}-uriparser") |
There was a problem hiding this comment.
Can we revert this because we already have it in this list?
| if(URIPARSER_ROOT) | ||
| find_library(URIPARSER_LIB | ||
| NAMES uriparser | ||
| PATHS ${URIPARSER_ROOT} | ||
| PATH_SUFFIXES ${ARROW_LIBRARY_PATH_SUFFIXES} | ||
| NO_DEFAULT_PATH) | ||
| find_path(URIPARSER_INCLUDE_DIR | ||
| NAMES uriparser/Uri.h | ||
| PATHS ${URIPARSER_ROOT} | ||
| NO_DEFAULT_PATH | ||
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) | ||
| else() |
There was a problem hiding this comment.
We don't need this check. We need to check only CMake package and pkg-config package.
| NO_DEFAULT_PATH | ||
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) | ||
| else() | ||
| find_package(PkgConfig QUIET) |
There was a problem hiding this comment.
Could you add QUIET only when uriparserAlt_FIND_QUIETLY?
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) | ||
| else() | ||
| find_package(PkgConfig QUIET) | ||
| pkg_check_modules(URIPARSER_PC liburiparser) |
There was a problem hiding this comment.
Could you add QUIET when uriparserAlt_FIND_QUIETLY?
Could you add IMPORTED_TARGET?
| NO_DEFAULT_PATH | ||
| PATH_SUFFIXES ${ARROW_LIBRARY_PATH_SUFFIXES}) | ||
| set(URIPARSER_VERSION "${URIPARSER_PC_VERSION}") | ||
| else() |
There was a problem hiding this comment.
We don't need this because we want to use CMake target instead of variables.
There was a problem hiding this comment.
We don't need this because we want to use CMake target instead of variables.
Didnt really get this. I have made some changes , please see if they work.
| find_library(URIPARSER_LIB | ||
| NAMES uriparser | ||
| PATH_SUFFIXES ${ARROW_LIBRARY_PATH_SUFFIXES}) | ||
| find_path(URIPARSER_INCLUDE_DIR | ||
| NAMES uriparser/Uri.h | ||
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) |
| externalproject_add(uriparser_ep | ||
| ${EP_COMMON_OPTIONS} | ||
| CMAKE_ARGS ${URIPARSER_CMAKE_ARGS} | ||
| INSTALL_DIR ${URIPARSER_PREFIX} | ||
| URL ${ARROW_URIPARSER_SOURCE_URL} | ||
| URL_HASH "SHA256=${ARROW_URIPARSER_BUILD_SHA256_CHECKSUM}" | ||
| BUILD_BYPRODUCTS "${URIPARSER_STATIC_LIB}") |
There was a problem hiding this comment.
Could you use FetchContent instead of ExternalProject?
| HAVE_ALT | ||
| TRUE | ||
| REQUIRED_VERSION | ||
| "1.0.2" |
There was a problem hiding this comment.
Can we require 0.9.6 or later for Ubuntu 22.04?
There was a problem hiding this comment.
Specifically for ubuntu ? or do you want me to have the distribution bundled minimum for other distros as well? fedora has 1.0 and likewise.
There was a problem hiding this comment.
No. I think that Ubuntu 22.04 ships the oldest uriparser in our supported (CI checked) platforms. If we don't require 1.0.2 or later features, I want to accept older versions in supported platforms such as Ubuntu 22.04:
| "1.0.2" | |
| "0.9.6" |
If we accept older versions, we can reduce our CI time because we don't need to build bundled uriparser.
There was a problem hiding this comment.
I will update with >= 0.9.6
|
|
||
| arrow_util_deps = [threads_dep] | ||
| uriparser_dep = dependency( | ||
| 'liburiparser', |
There was a problem hiding this comment.
Does this search both of CMake package and pkg-config package?
There was a problem hiding this comment.
added uriparser as well so we can check for both the names
|
@github-actions crossbow submit -g r |
|
Revision: c885134 Submitted crossbow builds: ursacomputing/crossbow @ actions-f0e03a3a56 |
|
Could you check CI results and update the PR description? |
raulcd
left a comment
There was a problem hiding this comment.
@imtherealnaska could you rebase main, several of the CI failures are old and have been fixed on main. I want to assess the state of the PR as we would prefer to merge this before the 26.0.0 release and the feature freeze is expected for tomorrow.
|
The lint failure requires fixing, see the diff on the job failure: diff --git a/cpp/cmake_modules/FinduriparserAlt.cmake b/cpp/cmake_modules/FinduriparserAlt.cmake
index 7ef10f1841..0dd810d4fc 100644
--- a/cpp/cmake_modules/FinduriparserAlt.cmake
+++ b/cpp/cmake_modules/FinduriparserAlt.cmake
@@ -53,9 +53,10 @@ if(uriparser_PC_FOUND)
set(uriparserAlt_VERSION "${uriparser_PC_VERSION}")
endif()
-find_package_handle_standard_args(uriparserAlt
- REQUIRED_VARS uriparser_PC_FOUND
- VERSION_VAR uriparserAlt_VERSION)
+find_package_handle_standard_args(
+ uriparserAlt
+ REQUIRED_VARS uriparser_PC_FOUND
+ VERSION_VAR uriparserAlt_VERSION) |
…u and distro bundled version
9b47937 to
525b820
Compare
I had some more changes for Dockerfiles. I have tested them . MinIO failures should be fixed from rebase. |
|
of the 3 jobs failing 2 are MINGW at parquet-reader-test and are fixed by this PR . Conda forge is not sure |
|
I think the conda job failure is also because of simdjson 5.0.1 bug , but the env change in this PR forces a fresh solve so it's exposed i think . Should I pin simdjson in ci/conda_env_cpp.txt ? (on a different PR ? ) |
|
@github-actions crossbow submit -g r |
|
Revision: 525b820 Submitted crossbow builds: ursacomputing/crossbow @ actions-a46fa27440 |
Rationale for this change
Updating the uriparser version so as to include security fixes. Rather than refreshing vendored copy , this makes uriparser and external dependency.
What changes are included in this PR?
uriparseradded toARROW_THIRDPARTY_DEPENDENCIES, with abuild_uriparser()using FetchContent pinned to 1.0.2 (docs, tests, toolsand
wchar_tsupport disabled; static;URI_STATIC_BUILDon theinterface; registered in
ARROW_BUNDLED_STATIC_LIBS).REQUIRED_VERSION "0.9.6"for SYSTEM builds, so supported platforms canuse the uriparser their distro already ships instead of building from
source. The bundled build still uses 1.0.2.
cmake_modules/FinduriparserAlt.cmake. Upstream ships a CMake packageconfig, but Debian/Ubuntu's
liburiparser-devinstalls onlyliburiparser.pc, so the module triesfind_package(CONFIG)first, thenpkg-config, then a plain library search.
ARROW_STATIC_INSTALL_INTERFACE_LIBSgainsuriparser::uriparserforSYSTEM builds, so static consumers link correctly.
subprojects/uriparser.wrap(method = cmake, same tarballand checksum as the CMake pin) plus a
cmake.subprojectfallback, sincemeson has no BUNDLED equivalent.
cpp/src/arrow/vendored/uriparser/(28 files) and itsLICENSE.txtsection removed;
util/uri.ccnow includes<uriparser/Uri.h>.PKGBUILDandmsys2_setup.sh,r_windows_build.sh, the-Duriparser_SOURCEpassthrough incpp_build.sh, and the dependencylist in
building.rst.NOTICE : The floor is 0.9.6 rather than 1.0.2, so supported platforms
such as Ubuntu 22.04 can use their distro package and CI does not have to
build uriparser from source.
Are these changes tested?
All existing tests pass.
Verified on Ubuntu 22.04:
Are there any user-facing changes?
Now Arrow requires uriparser or builds it . No user facing changes as such because no API changes .
This PR contains a "Critical Fix". It has security fixes that has gone in uriparser.
AI Usage: