Skip to content

Conversation

@Alex-PLACET
Copy link
Member

No description provided.

@Alex-PLACET Alex-PLACET self-assigned this Dec 22, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for the Arrow canonical extension type arrow.fixed_shape_tensor, which enables storing fixed-shape tensors in Arrow format. The implementation follows the Apache Arrow specification for fixed shape tensor arrays.

Key Changes

  • Introduces fixed_shape_tensor_array class wrapping fixed_sized_list_array with tensor-specific metadata
  • Implements JSON serialization/deserialization for tensor metadata (shape, dimension names, permutation)
  • Replaces nlohmann_json dependency with simdjson for improved JSON parsing performance
  • Adds comprehensive test coverage with 752 lines of tests

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
include/sparrow_extensions/fixed_shape_tensor.hpp Defines the fixed_shape_tensor_array class and metadata structures with comprehensive documentation
src/fixed_shape_tensor.cpp Implements metadata validation, JSON serialization, and array construction logic
tests/test_fixed_shape_tensor.cpp Provides extensive test coverage for metadata operations, array construction, and spec compliance
cmake/external_dependencies.cmake Adds simdjson dependency and updates doctest version
environment-dev.yml Replaces nlohmann_json with simdjson in conda dependencies
CMakeLists.txt Registers new source files and headers in build system
include/sparrow_extensions.hpp Includes the new fixed_shape_tensor header in main library header
tests/CMakeLists.txt Adds test file to test suite

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant