dnedev/pdi-fastjsoninput-plugin

FastJsonInput step plugin for Pentaho Data Integration

Download

Step 1. Add the JitPack repository to your build file

Add it in your root settings.gradle at the end of repositories:

	dependencyResolutionManagement {
		repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
		repositories {
			mavenCentral()
			maven { url 'https://jitpack.io' }
		}
	}

Add it in your settings.gradle.kts at the end of repositories:

	dependencyResolutionManagement {
		repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
		repositories {
			mavenCentral()
			maven { url = uri("https://jitpack.io") }
		}
	}

Add to pom.xml

	<repositories>
		<repository>
		    <id>jitpack.io</id>
		    <url>https://jitpack.io</url>
		</repository>
	</repositories>

Add it in your build.sbt at the end of resolvers:

 
    resolvers += "jitpack" at "https://jitpack.io"

Add it in your project.clj at the end of repositories:

 
    :repositories [["jitpack" "https://jitpack.io"]]

Step 2. Add the dependency

	dependencies {
		implementation 'com.github.dnedev:pdi-fastjsoninput-plugin:v1.1.7'
	}
	dependencies {
		implementation("com.github.dnedev:pdi-fastjsoninput-plugin:v1.1.7")
	}
	<dependency>
	    <groupId>com.github.dnedev</groupId>
	    <artifactId>pdi-fastjsoninput-plugin</artifactId>
	    <version>v1.1.7</version>
	</dependency>

                            
    libraryDependencies += "com.github.dnedev" % "pdi-fastjsoninput-plugin" % "v1.1.7"
        
        

                            
    :dependencies [[com.github.dnedev/pdi-fastjsoninput-plugin "v1.1.7"]]

Readme

Fast JSON Input - PDI Plugin

Build Status

This is an alternate version of the "JSON Input" step that uses Jayway JsonPath instead of a parser based on JavaScript. It is intended to be a drop-in replacement for the "JSON Input" step but should be much faster and memory efficient.

Features over PDI JSON Input

  • [PDI-10344] Replaced JavaScript parsing engine with Jayway JsonPath
  • [PDI-10858] Checkbox to "Remove source field from output stream"
  • Checkbox to enable JsonPath's DEFAULT_PATH_LEAF_TO_NULL option which returns null for missing leafs:
[
    {
        "name": "Jesse Adametz",
        "gender": "male"
    },
    {
        "name": "Etienne Dube"
    }
]
  • Tests! There are currently 4 test cases which test the permutations of "Ignore missing path" and "Default path leaf to null"

Help

Help for the step and its usage can be found in the Help documentation.

Development

Build

To build (requires Apache Maven 3 or later):

mvn package

Install

  1. Run mvn install -Dpdi.home=/path/to/local/data-integration (e.g. - mvn install -Dpdi.home=/ssd/graphiq/gq-kettle/dist)

Authors: