HomeAboutArchivesTools & ResourcesSupport me
Hi! πŸ‘‹ I'm Szymon

I help you become a better software developer.

  • 1.15kFollowers
  • 2.36kSubscribers
  • 110Followers
  • 30.2kReputation

Latest posts from the jq cookbook

Merging JSON files recursively in the command-line
How to convert JSON to CSV from the command-line?
Parsing JSON in command-line with jq: basic filters and funct...
More

Popular categories

  • Groovy Cookbook28
  • Jenkins Pipeline Cookbook9
  • Programmer's Bookshelf6
  • Micronaut Cookbook4
  • Ratpack Cookbook4
  • Learning Java4
  • jq cookbook3
  • How to3
  • Blog Reports2

Don't miss anything - join my newsletter

Additional materials and updates for subscribers. No spam guaranteed.
Unsubscribe at any time.

Did you like this article?

Spread the !
  • β˜•οΈ
  1. e.printstacktrace.blog
  2. jq cookbook

How to convert JSON to CSV from the command-line?

  • August 25, 2020
  • 3 min. read
  • 0 Comments

Are you looking for an easy and fast way to convert a JSON file to a CSV one? In this blog post, I will show you how you can use jq command-line tool to do exactly that.

Table of Contents
  • Install jq
  • Prepare the JSON file
  • Convert a JSON file to the CSV one

Install jq

First, you need to install jq in your operating system - https://stedolan.github.io/jq/download/

Prepare the JSON file

To convert JSON file to the CSV format we will need a JSON file that is represented by an array of objects. In this tutorial, I use Github’s Jobs API that returns JSON in that desired format. Below you can find a curl command that searches for open kotlin job positions and stores the result in the jobs.json file.

$ curl -s "https://jobs.github.com/positions.json?description=kotlin" > jobs.json

Convert a JSON file to the CSV one

And here is how we can convert input JSON file to the CSV format.

$ jq -r 'map({id,title,url,company,location}) | (first | keys_unsorted) as $keys | map([to_entries[] | .value]) as $rows | $keys,$rows[] | @csv' jobs.json > jobs.csv

$ cat jobs.csv
"id","title","url","company","location"
"2ededc50-3c18-4d2c-ba54-95151ca4209a","Android Engineer","https://jobs.github.com/positions/2ededc50-3c18-4d2c-ba54-95151ca4209a","Trade Republic Bank GmbH","Berlin"
"2cc06700-d9ac-45ad-9bd7-a7e5d3c29b6f","Software Engineer - Android","https://jobs.github.com/positions/2cc06700-d9ac-45ad-9bd7-a7e5d3c29b6f","AiCure","Remote"
"242132e0-f129-4a1b-8bfa-30f2fed13123","Senior Backend Developer (Data Science)","https://jobs.github.com/positions/242132e0-f129-4a1b-8bfa-30f2fed13123","komoot","Remote Europe"
"fd8c3b7c-ccfe-4cc1-882c-ef9c0522903e","Software Engineer","https://jobs.github.com/positions/fd8c3b7c-ccfe-4cc1-882c-ef9c0522903e","HBM nCode Federal LLC","Starkville, MS"
"7d84945b-893d-493e-957a-4ed9e6f84607","Software Engineer","https://jobs.github.com/positions/7d84945b-893d-493e-957a-4ed9e6f84607","HBM nCode Federal LLC","Southfield, MI"
"2231d64f-e79b-4036-8601-24b2717b2896","Senior Fullstack / Flutter Developer (m/f/d)","https://jobs.github.com/positions/2231d64f-e79b-4036-8601-24b2717b2896","Superlist","Remote"
"caa90907-8252-4732-a815-06d96f1348bb","Senior Software Engineer - Mobile (m/f/d)","https://jobs.github.com/positions/caa90907-8252-4732-a815-06d96f1348bb","BASF Digital Farming GmbH","KΓΆln"

Let’s deconstruct it piece by piece.

  • We run jq -r to output raw strings (without double quotes.)

  • In the filter part, we pipe multiple filters together, starting with map({id,title,url,company,location}). This filter instructs jq which keys we want to extract from the input JSON file.

  • Then we use (first | keys_unsorted) as $keys filter which takes the first object, extracts its keys and stores them under the $keys variable as an array.

  • Next, we use map([to_entries[] | .value]) as $rows filter which converts every key-value entry like "foo": "bar" into an object like {"key":"foo","value":"bar"} so we can extract only values as an array and store it in a $rows variable.

  • Once we do it, we can use $keys,$rows[] filter to put keys and rows together and then pipe it with the @csv filter to convert JSON objects to CSV rows.

How to merge two JSON files in the command-line using jq? | #jq #json #curl

In this jq tutorial video, I show you how you can merge two JSON files in the command-line using jq JSON processor. I explain how to merge those files using a simple "add" filter, then I explain how to merge nested objects, as well as how to merge arrays of different objects. Watch now »

  • curl
  • jq
  • json
  • csv
  • command-line
  • how-to

Groovy Cookbook

Groovy Ecosystem Usage Report (2020)

  • September 13, 2020
  • 3 min. read
  • 0 Comments

What is the most popular Groovy library, framework, or a tool? I surveyed 308 Groovy community members, and here are the results.

jq cookbook

Parsing JSON in command-line with jq: basic filters and functions (part 1)

  • May 24, 2020
  • 3 min. read
  • 0 Comments

Have you ever wondered, what is the most convenient way to parse JSON data in the Unix/Linux com...

Jenkins Pipeline Cookbook

How to catch curl response in Jenkins Pipeline?

  • June 22, 2020
  • 3 min. read
  • 0 Comments

In this blog post, I explain why you may want to use curl command in your Jenkinsfile, how to ca...

jq cookbook

Merging JSON files recursively in the command-line

  • November 5, 2020
  • 3 min. read
  • 0 Comments

Have you ever need to merge two (or more) JSON files and you wondered if you can do it in the co...

Any thoughts or ideas?

Let's talk in the comment's section πŸ’¬

Want to put a code sample in the comment? Read the Syntax highlighting guide for more information.
Empty
Latest blog posts
  • Merging JSON files recursively in the command-line
  • Jenkins Declarative Pipeline with the dynamic agent - how to configure it?
  • Groovy Ecosystem Usage Report (2020)
  • How to convert JSON to CSV from the command-line?
  • 5 Common Jenkins Pipeline Mistakes
  • How to merge two maps in Groovy?
  • Building stackoverflow-cli with Java 11, Micronaut, Picocli, and GraalVM
  • How to catch curl response in Jenkins Pipeline?
  • Atomic Habits - book review
  • Parsing JSON in command-line with jq: basic filters and functions (part 1)
Trending videos
5 Common Jenkins Pipeline Mistakes πŸ”₯

In this Jenkins Pipeline tutorial video, I reveal five common Jenkins Pipeline mistakes and I explain how those mistakes can ...

Jenkins Declarative Pipeline vs Scripted Pipeline - 4 key differences | #jenkinspipeline

Jenkins Pipeline as a code is a new standard for defining continuous integration and delivery pipelines in Jenkins. The scrip...

Useful links
  • Start here
  • About
  • Archives
  • Resources
  • Privacy Policy
  • Merchandise
  • My Kit
  • RSS
  • Support the blog
Popular categories
  • Groovy Cookbook28
  • Jenkins Pipeline Cookbook9
  • Programmer's Bookshelf6
  • Micronaut Cookbook4
  • Ratpack Cookbook4
  • Learning Java4
  • jq cookbook3
  • How to3
  • Blog Reports2
Popular tags
affiliate async benchmark blogging book career cicd continuous-integration curl devops docker git github graalvm gradle grails groovy haskell hexo java java-8 jenkins jenkins-pipeline jenkinsfile jmh jq json junit learning maven metaprogramming micronaut native-image non-blocking progress ratpack reactive-programming reading recursion review rxjava sdkman session split spock stackoverflow string tail-call tail-recursion unit-test
  • Designed by @wololock
  • Created with Hexo
  • Hosted on GitHub
  • Deployed with Circle CI
  • License CC BY-NC-SA 4.0