Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Consumer looses partitons after restart connection  #1305

Copy link
Copy link
Open
@gladkikhtutu

Description

@gladkikhtutu
Issue body actions

Description

At the beginning consumer connects and reads records from topic as should. At one moment (after several hours) connection with broker restarts, partitions unassign and all the other time consumer.assignment() returns None in loop.
I have two questions:
What could be the reason for such a crash, in which the consumer cannot get partitions?
How to fix it?

Here is Consumer class code with configuration:

class ConsumerProxy:
    WAIT_ASSIGNMENT_MINUTES = 1

    def __init__(self, host: str, group: str, topic: str):
        self.__consumer = self.__create_consumer(host, group, topic)

    def __create_consumer(self, host: str, group: str, topic: str) -> Consumer:
        consumer = Consumer(
            {
                'bootstrap.servers': host,
                'group.id': group,
                'enable.auto.commit': False,
                'auto.offset.reset': 'earliest'
                'partition.assignment.strategy': 'roundrobin',
                'debug': 'broker,consumer,cgrp,topic,fetch'
            }
        )
        topics = [topic]
        consumer.subscribe(topics)
        return consumer

    def commit(self):
        counter = 0
        while not self.__consumer.assignment():
            time.sleep(self.WAIT_ASSIGNMENT_MINUTES * 60)
            counter += 1
            logging.debug(f"Waiting assignment for {counter} minutes")

        self.__consumer.commit(asynchronous=False)

    def poll(self) -> dict:
        while True:
            msg = self.__consumer.poll(1.0)
            if msg is None:
                logging.debug("Wait new message")
                yield None
            elif msg.error():
                raise Exception(msg.error())
            else:
                logging.debug(f"New message from {msg.topic()}")
                yield json.loads(msg.value())

Checklist

Please provide the following information:

  • confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): ('1.7.0', 17235968), ('1.7.0', 17236223)
  • Apache Kafka broker version: kafka_version: 2.8.0, kafka_scala_version: 2.13
  • Client configuration: { 'bootstrap.servers': host, 'group.id': group, 'enable.auto.commit': False, 'auto.offset.reset': 'earliest' 'partition.assignment.strategy': 'roundrobin', 'debug': 'broker,consumer,cgrp,topic,fetch' }

Logs around moment when all crashes with some extra logs from program:

{"time": "2022-03-22 19:01:12,761", "level": "DEBUG", "message": "New message from topic_name"}
{"time": "2022-03-22 19:01:12,827", "level": "DEBUG", "message": "Schema validation OK"}
{"time": "2022-03-22 19:01:12,827", "level": "DEBUG", "message": "New message from topic_name"}
%7|1647975677.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975683.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975689.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975695.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975701.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975707.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975713.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975719.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975725.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975731.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975737.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975743.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975749.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975755.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 0 Leader 300
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 1 Leader 400
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 2 Leader 200
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 3 Leader 300
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 4 Leader 400
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 5 Leader 200
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 6 Leader 300
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 7 Leader 400
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 8 Leader 200
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 9 Leader 300
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 10 Leader 400
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 11 Leader 200
%7|1647975760.440|METADATA|rdkafka#consumer-1| [thrd:main]: kafka-400.event-store.common.tutu.ru:9092/400: 1/1 requested topic(s) seen in metadata
%7|1647975761.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975767.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975767.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state steady, 1 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975767.929|REBALANCE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" is rebalancing (EAGER) in state up (join-state steady) with 1 assigned partition(s): rebalance in progress
%7|1647975767.929|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" changed join state steady -> wait-unassign-call (state up)
%7|1647975767.929|ASSIGN|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": delegating revoke of 1 partition(s) to application on queue rd_kafka_cgrp_new: rebalance in progress
%7|1647975767.929|PAUSE|rdkafka#consumer-1| [thrd:main]: Pausing fetchers for 1 assigned partition(s): rebalance
%7|1647975767.929|PAUSE|rdkafka#consumer-1| [thrd:main]: Library pausing 1 partition(s)
%7|1647975767.929|BARRIER|rdkafka#consumer-1| [thrd:main]: topic_name [1]: rd_kafka_toppar_op_pause_resume:2434: new version barrier v5
%7|1647975767.929|PAUSE|rdkafka#consumer-1| [thrd:main]: Pause topic_name [1] (v5)
%7|1647975767.929|ASSIGNMENT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": clearing group assignment
%7|1647975767.929|OP|rdkafka#consumer-1| [thrd:main]: topic_name [1] received op PAUSE (v5) in fetch-state active (opv4)
%7|1647975767.929|PAUSE|rdkafka#consumer-1| [thrd:main]: Pause topic_name [1]: at offset 91122 (state active, v5)
%7|1647975768.447|FETCHDEC|rdkafka#consumer-1| [thrd:kafka-400.event-store.common.tutu.ru:9092/400]: Topic topic_name [1]: fetch decide: updating to version 5 (was 4) at offset 91122 (was 177500)
%7|1647975773.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975773.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975779.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975779.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975785.928|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975785.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975791.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975791.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975797.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975797.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975803.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975803.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975809.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975809.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975815.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975815.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975821.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975821.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975827.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975827.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975833.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975833.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
%7|1647975839.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 8
%7|1647975839.929|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" heartbeat error response in state up (join-state wait-unassign-call, 0 partition(s) assigned): Broker: Group rebalance in progress
{"time": "2022-03-22 19:04:03,094", "level": "DEBUG", "message": "Last consumed data: {...}"}
%7|1647975843.098|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-unassign-call)
%7|1647975843.098|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op OFFSET_COMMIT in state up (join-state wait-unassign-call)
%7|1647975843.098|OFFSET|rdkafka#consumer-1| [thrd:main]: Topic topic_name [1]: stored offset 91122, committed offset 90719: setting stored offset 91122 for commit
%7|1647975843.098|COMMIT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Committing offsets for 1 partition(s) with generation-id 8 in join-state wait-unassign-call: manual
%7|1647975843.098|OFFSET|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Enqueue OffsetCommitRequest(v7, 1/1 partition(s))): manual
%7|1647975843.102|COMMIT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: OffsetCommit for 1 partition(s) in join-state wait-unassign-call: manual: returned: Success
%7|1647975843.102|DUMP|rdkafka#consumer-1| [thrd:main]: Assignment dump (started_cnt=1, wait_stop_cnt=0)
%7|1647975843.102|DUMP_ALL|rdkafka#consumer-1| [thrd:main]: List with 1 partition(s):
%7|1647975843.102|DUMP_ALL|rdkafka#consumer-1| [thrd:main]:  topic_name [1] offset STORED
%7|1647975843.102|DUMP_PND|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.102|DUMP_QRY|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.102|DUMP_REM|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.102|ASSIGNDONE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": assignment operations done in join-state wait-unassign-call (rebalance rejoin=false)
{"time": "2022-03-22 19:04:03,102", "level": "DEBUG", "message": "consumer commit result: [TopicPartition{topic=topic_name,partition=1,offset=91122,error=None}]"}

{"time": "2022-03-22 19:04:03,116", "level": "DEBUG", "message": "Schema validation OK"}
{"time": "2022-03-22 19:04:03,116", "level": "DEBUG", "message": "Topic topic_name revoked from partition 1"}
%7|1647975843.116|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_REBALANCE_PROTOCOL in state up (join-state wait-unassign-call)
%7|1647975843.116|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op ASSIGN in state up (join-state wait-unassign-call)
%7|1647975843.116|CLEARASSIGN|rdkafka#consumer-1| [thrd:main]: Clearing current assignment of 1 partition(s)
%7|1647975843.116|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" changed join state wait-unassign-call -> wait-unassign-to-complete (state up)
%7|1647975843.116|DUMP|rdkafka#consumer-1| [thrd:main]: Assignment dump (started_cnt=1, wait_stop_cnt=0)
%7|1647975843.116|DUMP_ALL|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|DUMP_PND|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|DUMP_QRY|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|DUMP_REM|rdkafka#consumer-1| [thrd:main]: List with 1 partition(s):
%7|1647975843.116|DUMP_REM|rdkafka#consumer-1| [thrd:main]:  topic_name [1] offset STORED
%7|1647975843.116|BARRIER|rdkafka#consumer-1| [thrd:main]: topic_name [1]: rd_kafka_toppar_op_fetch_stop:2375: new version barrier v6
%7|1647975843.116|CONSUMER|rdkafka#consumer-1| [thrd:main]: Stop consuming topic_name [1] (v6)
%7|1647975843.116|BARRIER|rdkafka#consumer-1| [thrd:main]: topic_name [1]: rd_kafka_toppar_op_pause_resume:2434: new version barrier v7
%7|1647975843.116|RESUME|rdkafka#consumer-1| [thrd:main]: Resume topic_name [1] (v7)
%7|1647975843.116|DESP|rdkafka#consumer-1| [thrd:main]: Removing (un)desired topic topic_name [1]
%7|1647975843.116|REMOVE|rdkafka#consumer-1| [thrd:main]: Removing topic_name [1] from assignment (started=true, pending=false, queried=false, stored offset=91122)
%7|1647975843.116|REMOVE|rdkafka#consumer-1| [thrd:main]: Served 1 removed partition(s), with 1 offset(s) to commit
%7|1647975843.116|ASSIGNMENT|rdkafka#consumer-1| [thrd:main]: Current assignment of 0 partition(s) with 0 pending adds, 0 offset queries, 1 partitions awaiting stop and 0 offset commits in progress
%7|1647975843.116|OP|rdkafka#consumer-1| [thrd:main]: topic_name [1] received op FETCH_STOP (v6) in fetch-state active (opv5)
%7|1647975843.116|FETCH|rdkafka#consumer-1| [thrd:main]: Stopping fetch for topic_name [1] in state active (v6)
%7|1647975843.116|PARTSTATE|rdkafka#consumer-1| [thrd:main]: Partition topic_name [1] changed fetch state active -> stopping
%7|1647975843.116|STORETERM|rdkafka#consumer-1| [thrd:main]: topic_name [1]: offset store terminating
%7|1647975843.116|PARTSTATE|rdkafka#consumer-1| [thrd:main]: Partition topic_name [1] changed fetch state stopping -> stopped
%7|1647975843.116|OP|rdkafka#consumer-1| [thrd:main]: topic_name [1] received op PAUSE (v7) in fetch-state stopped (opv6)
%7|1647975843.116|RESUME|rdkafka#consumer-1| [thrd:main]: Not resuming stopped topic_name [1]: at offset 91122 (state stopped, v7)
%7|1647975843.116|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op PARTITION_LEAVE in state up (join-state wait-unassign-to-complete) for topic_name [1]
%7|1647975843.116|PARTDEL|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": delete topic_name [1]
%7|1647975843.116|STOPSERVE|rdkafka#consumer-1| [thrd:main]: All partitions awaiting stop are now stopped: serving assignment
%7|1647975843.116|DUMP|rdkafka#consumer-1| [thrd:main]: Assignment dump (started_cnt=0, wait_stop_cnt=0)
%7|1647975843.116|DUMP_ALL|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|DUMP_PND|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|DUMP_QRY|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|DUMP_REM|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1647975843.116|ASSIGNDONE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": assignment operations done in join-state wait-unassign-to-complete (rebalance rejoin=false)
%7|1647975843.116|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": unassign done in state up (join-state wait-unassign-to-complete)
%7|1647975843.116|REJOIN|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": Rejoining group without an assignment: Unassignment done
%7|1647975843.116|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" changed join state wait-unassign-to-complete -> init (state up)
%7|1647975843.116|JOIN|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": join with 1 subscribed topic(s)
%7|1647975843.116|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription is up to date (82676ms old)
%7|1647975843.116|JOIN|rdkafka#consumer-1| [thrd:main]: kafka-300.event-store.common.tutu.ru:9092/300: Joining group "consumer_group_name" with 1 subscribed topic(s)
%7|1647975843.116|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" changed join state init -> wait-join (state up)
{"time": "2022-03-22 19:04:13,116", "level": "DEBUG", "message": "Wait new message"}
-> {"time": "2022-03-22 19:04:13,970", "level": "DEBUG", "message": "Last consumed data: None. Buffer to commit: {...}"}
%7|1647975853.971|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
{"time": "2022-03-22 19:05:14,019", "level": "DEBUG", "message": "Waiting assignment for 1 minutes"}
%7|1647975914.020|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
{"time": "2022-03-22 19:06:14,080", "level": "DEBUG", "message": "Waiting assignment for 2 minutes"}
%7|1647975974.080|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
{"time": "2022-03-22 19:07:14,140", "level": "DEBUG", "message": "Waiting assignment for 3 minutes"}
%7|1647976034.140|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 0 Leader 300
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 1 Leader 400
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 2 Leader 200
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 3 Leader 300
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 4 Leader 400
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 5 Leader 200
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 6 Leader 300
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 7 Leader 400
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 8 Leader 200
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 9 Leader 300
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 10 Leader 400
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]:   Topic topic_name partition 11 Leader 200
%7|1647976060.440|METADATA|rdkafka#consumer-1| [thrd:main]: kafka-400.event-store.common.tutu.ru:9092/400: 1/1 requested topic(s) seen in metadata
%7|1647976060.927|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: kafka-400.event-store.common.tutu.ru:9092/400: Group "consumer_group_name": querying for coordinator: intervaled in state up
%7|1647976060.929|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: kafka-400.event-store.common.tutu.ru:9092/400: Group "consumer_group_name" coordinator is kafka-300.event-store.common.tutu.ru:9092 id 300
{"time": "2022-03-22 19:08:14,194", "level": "DEBUG", "message": "Waiting assignment for 4 minutes"}
%7|1647976094.195|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
{"time": "2022-03-22 19:09:14,255", "level": "DEBUG", "message": "Waiting assignment for 5 minutes"}
%7|1647976154.255|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
{"time": "2022-03-22 19:10:14,315", "level": "DEBUG", "message": "Waiting assignment for 6 minutes"}
%7|1647976214.315|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" received op GET_ASSIGNMENT in state up (join-state wait-join)
%7|1647976237.998|JOINGROUP|rdkafka#consumer-1| [thrd:main]: JoinGroup response: GenerationId 9, Protocol roundrobin, LeaderId rdkafka-bec0968e-08cf-41b5-9494-978e521a2a93, my MemberId rdkafka-2abe21a1-7740-4726-84d9-f10219a51e3e, member metadata count 0: (no error)
%7|1647976237.998|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" changed join state wait-join -> wait-sync (state up)
%7|1647976238.004|SYNCGROUP|rdkafka#consumer-1| [thrd:main]: SyncGroup response: Success (43 bytes of MemberState data)
%7|1647976238.004|ASSIGNMENT|rdkafka#consumer-1| [thrd:main]: List with 1 partition(s):
%7|1647976238.004|ASSIGNMENT|rdkafka#consumer-1| [thrd:main]:  topic_name [1] offset INVALID
%7|1647976238.004|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name" changed join state wait-sync -> wait-assign-call (state up)
%7|1647976238.004|ASSIGN|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": delegating assign of 1 partition(s) to application on queue rd_kafka_cgrp_new: new assignment
%7|1647976238.004|ASSIGNMENT|rdkafka#consumer-1| [thrd:main]: Group "consumer_group_name": setting group assignment to 1 partition(s)
%7|1647976238.004|GRPASSIGNMENT|rdkafka#consumer-1| [thrd:main]: List with 1 partition(s):
%7|1647976238.004|GRPASSIGNMENT|rdkafka#consumer-1| [thrd:main]:  topic_name [1] offset INVALID
%7|1647976238.004|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 9
%7|1647976244.431|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 9
%7|1647976250.431|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 9
%7|1647976256.431|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 9
%7|1647976262.432|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 9
%7|1647976268.432|HEARTBEAT|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/300: Heartbeat for group "consumer_group_name" generation id 9
{"time": "2022-03-22 19:11:14,369", "level": "DEBUG", "message": "Waiting assignment for 7 minutes"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.