I have the following set up:

1) kafka server
2) druid server (historical, broker, coordinator, overlord, middleManager, tranquility, zookeeper)

To get data from kafka I use kafka indexing service (supervisor), which is managed by overlord.

The problem is that recently I started seeing these errors and no data published to dataset:

2018-10-17T04:22:05,547 WARN [task-runner-0-priority-0] io.druid.indexing.kafka.KafkaIndexTask - OffsetOutOfRangeException with message [Offsets out of range with no configured reset policy for partitions: {request-staging-0=4774}]
2018-10-17T04:22:05,547 WARN [task-runner-0-priority-0] io.druid.indexing.kafka.KafkaIndexTask - Retrying in 30000ms
2018-10-17T04:22:35,550 WARN [task-runner-0-priority-0] io.druid.indexing.kafka.KafkaIndexTask - OffsetOutOfRangeException with message [Offsets out of range with no configured reset policy for partitions: {request-staging-0=4774}]
2018-10-17T04:22:35,550 WARN [task-runner-0-priority-0] io.druid.indexing.kafka.KafkaIndexTask - Retrying in 30000ms
2018-10-17T04:23:05,553 WARN [task-runner-0-priority-0] io.druid.indexing.kafka.KafkaIndexTask - OffsetOutOfRangeException with message [Offsets out of range with no configured reset policy for partitions: {request-staging-0=4774}]

After some research I found that I need to set this property in kafka consumer properties

auto.offset.reset=latest

And also I see in the log file of indexing task that property is set to none:

2018-10-17T03:01:27,769 INFO [task-runner-0-priority-0] org.apache.kafka.clients.consumer.ConsumerConfig - ConsumerConfig values: 
 auto.commit.interval.ms = 5000
 auto.offset.reset = none
 bootstrap.servers = [kafka.myhost.com:9092]

Unfortunately I didn't find how to modify it in kafka, because on kafka server I run only kafka server like this:

/home/centos/kafka/bin/kafka-server-start.sh -daemon /home/centos/devops/kafka/config/server.properties

and I don't know where to specify consumer properties. Please let me know if you know.

But I found another way how to fix this problem. You can also specify one setting in supervisor config. Here is what you need to set:

{
  "type": "kafka",
  "dataSchema": {
    ...
  },
  "tuningConfig" : {
    "type": "kafka",
    "resetOffsetAutomatically": true
  },
  "ioConfig": {
   ...
  }
}

and resubmit it to overlord:

curl -XPOST -H'Content-Type: application/json' -d @/fullpath/supervisor-config.json http://localhost:8090/druid/indexer/v1/supervisor