Apache Arrow Data

Apache arrow is a cross-platform binary format for in-memory data storage. Being much more compact than JSON it is suitable for larger datasets as well as for interoperability with other software that generates apache-arrow format files:

To use Apache Arrow data with Vega-Lite (or Vega) you need to link to the vega-loader-arrow and apache-arrow javascript packages in the head of your HTML along with the normal links to the vega and vega-lite runtimes:

  <script src="https://cdn.jsdelivr.net/npm/apache-arrow"></script>
  <script src="https://cdn.jsdelivr.net/npm/vega-loader-arrow"></script>
    

Flight Data Example

Data source contains 10,000 flight records as a binary arrow file.

arrow1 : Spec
arrow1 =
    let
        data =
            dataFromUrl "https://gist.githubusercontent.com/domoritz/0f53a5abde95564c36dfaac623a7a922/raw/cce3719b853e25d5dfff97a270283ba83af3c0e6/flights-10k.arrow"
                [ arrow ]
        enc =
            encoding
                << position X [ pName "DEP_TIME", pMType Quantitative ]
                << position Y [ pName "ARR_DELAY", pMType Quantitative ]
                << color [ mName "DISTANCE", mMType Quantitative ]
    in
    toVegaLite [ width 500, height 300, data, enc [], circle [ maSize 4 ] ]

Scrabble Data Example

10,000 random samples from a 1.5 million record scrabble dataset that is 157Mb in CSV format and 47Mb in arrow format.