confluentinc/kafka-connect-elasticsearch

Kafka Connect Elasticsearch connector

Download

Step 1. Add the JitPack repository to your build file

Add it in your root settings.gradle at the end of repositories:

	dependencyResolutionManagement {
		repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
		repositories {
			mavenCentral()
			maven { url 'https://jitpack.io' }
		}
	}

Add it in your settings.gradle.kts at the end of repositories:

	dependencyResolutionManagement {
		repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
		repositories {
			mavenCentral()
			maven { url = uri("https://jitpack.io") }
		}
	}

Add to pom.xml

	<repositories>
		<repository>
		    <id>jitpack.io</id>
		    <url>https://jitpack.io</url>
		</repository>
	</repositories>

Add it in your build.sbt at the end of resolvers:

 
    resolvers += "jitpack" at "https://jitpack.io"

Add it in your project.clj at the end of repositories:

 
    :repositories [["jitpack" "https://jitpack.io"]]

Step 2. Add the dependency

	dependencies {
		implementation 'com.github.confluentinc:kafka-connect-elasticsearch:v14.1.2'
	}
	dependencies {
		implementation("com.github.confluentinc:kafka-connect-elasticsearch:v14.1.2")
	}
	<dependency>
	    <groupId>com.github.confluentinc</groupId>
	    <artifactId>kafka-connect-elasticsearch</artifactId>
	    <version>v14.1.2</version>
	</dependency>

                            
    libraryDependencies += "com.github.confluentinc" % "kafka-connect-elasticsearch" % "v14.1.2"
        
        

                            
    :dependencies [[com.github.confluentinc/kafka-connect-elasticsearch "v14.1.2"]]

Readme

Kafka Connect Elasticsearch Connector

FOSSA Status

Changelog for this connector can be found here.

kafka-connect-elasticsearch is a Kafka Connector for copying data between Kafka and Elasticsearch.

Development

To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the FAQ for guidance on this process.

You can build kafka-connect-elasticsearch with Maven using the standard lifecycle phases.

Configuring

Creating an Elasticsearch user and assigning required privileges

Create an Elasticsearch role

curl -u elastic:elastic -X POST "localhost:9200/_security/role/es_sink_connector_role?pretty" -H 'Content-Type: application/json' -d'
{
  "indices": [
    {
      "names": [ "*" ],
      "privileges": ["create_index", "read", "write", "view_index_metadata"]
    }
  ]
}'

Create an Elasticsearch user

curl -u elastic:elastic -X POST "localhost:9200/_security/user/es_sink_connector_user?pretty" -H 'Content-Type: application/json' -d'
{
  "password" : "seCret-secUre-PaSsW0rD",
  "roles" : [ "es_sink_connector_role" ]
}'

Contribute

  • Source Code: https://github.com/confluentinc/kafka-connect-elasticsearch
  • Issue Tracker: https://github.com/confluentinc/kafka-connect-elasticsearch/issues
  • Learn how to work with the connector's source code by reading our Development and Contribution guidelines.

License

This project is licensed under the Confluent Community License.

FOSSA Status