Module: Kafka::FFI

Extended by:
FFI::Library
Defined in:
lib/kafka/ffi.rb,
lib/kafka/ffi/admin.rb,
lib/kafka/ffi/error.rb,
lib/kafka/ffi/event.rb,
lib/kafka/ffi/queue.rb,
lib/kafka/ffi/topic.rb,
lib/kafka/ffi/client.rb,
lib/kafka/ffi/config.rb,
lib/kafka/ffi/opaque.rb,
lib/kafka/ffi/message.rb,
lib/kafka/ffi/consumer.rb,
lib/kafka/ffi/metadata.rb,
lib/kafka/ffi/producer.rb,
lib/kafka/ffi/group_info.rb,
lib/kafka/ffi/group_list.rb,
lib/kafka/ffi/topic_config.rb,
lib/kafka/ffi/message/header.rb,
lib/kafka/ffi/opaque_pointer.rb,
lib/kafka/ffi/topic_metadata.rb,
lib/kafka/ffi/broker_metadata.rb,
lib/kafka/ffi/topic_partition.rb,
lib/kafka/ffi/group_member_info.rb,
lib/kafka/ffi/partition_metadata.rb,
lib/kafka/ffi/topic_partition_list.rb

Overview

Module FFI provides both a (mostly) complete set of low level function calls into librdkafka as well as a set of slightly higher level abstractions and objects that make working with the API easier. It is still required to know enough about using librdkafka to use the abstractions safely (see the introduction below).

All exposed functions on Kafka::FFI are named to match the functions exposed by librdkafka (see rdkafka.h).

See: https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md See: https://github.com/edenhill/librdkafka/blob/master/src/rdkafka.h

Defined Under Namespace

Modules: Admin Classes: BrokerMetadata, Client, Config, ConfigError, Consumer, Error, Event, GroupInfo, GroupList, GroupMemberInfo, InvalidConfigValue, Message, Metadata, Opaque, OpaquePointer, PartitionMetadata, Producer, Queue, Topic, TopicAlreadyConfiguredError, TopicConfig, TopicMetadata, TopicPartition, TopicPartitionList, UnknownConfigKey

Constant Summary collapse

RD_KAFKA_RESP_ERR__TIMED_OUT =

Response Errors

-185
RD_KAFKA_RESP_ERR__ASSIGN_PARTITIONS = 
-175
RD_KAFKA_RESP_ERR__REVOKE_PARTITIONS = 
-174
RD_KAFKA_RESP_ERR__NO_OFFSET = 
-168
RD_KAFKA_RESP_ERR__NOENT = 
-156
RD_KAFKA_RESP_ERR__FATAL = 
-150
RD_KAFKA_OFFSET_BEGINNING = 
-2
RD_KAFKA_OFFSET_END = 
-1
RD_KAFKA_OFFSET_STORED = 
-1000
RD_KAFKA_OFFSET_INVALID = 
-1001
RD_KAFKA_EVENT_NONE = 
0x00
RD_KAFKA_EVENT_DR = 
0x01
RD_KAFKA_EVENT_FETCH = 
0x02
RD_KAFKA_EVENT_LOG = 
0x04
RD_KAFKA_EVENT_ERROR = 
0x08
RD_KAFKA_EVENT_REBALANCE = 
0x10
RD_KAFKA_EVENT_OFFSET_COMMIT = 
0x20
RD_KAFKA_EVENT_STATS = 
0x40
RD_KAFKA_EVENT_CREATETOPICS_RESULT = 
100
RD_KAFKA_EVENT_DELETETOPICS_RESULT = 
101
RD_KAFKA_EVENT_CREATEPARTITIONS_RESULT = 
102
RD_KAFKA_EVENT_ALTERCONFIGS_RESULT = 
103
RD_KAFKA_EVENT_DESCRIBECONFIGS_RESULT = 
104
RD_KAFKA_EVENT_OAUTHBEARER_TOKEN_REFRESH = 
0x100
RD_KAFKA_MSG_STATUS_NOT_PERSISTED = 
0
RD_KAFKA_MSG_STATUS_POSSIBLY_PERSISTED = 
1
RD_KAFKA_MSG_STATUS_PERSISTED = 
2
RD_KAFKA_MSG_F_FREE =

Flags for rd_kafka_produce, rd_kafka_producev, and rd_kafka_produce_batch.

See Also:

  • Kafka::FFI.rdkafkardkafka.h
0x01
RD_KAFKA_MSG_F_COPY = 
0x02
RD_KAFKA_MSG_F_BLOCK = 
0x04
RD_KAFKA_MSG_F_PARTITION = 
0x04
RD_KAFKA_PURGE_F_QUEUE =

Flags for rd_kafka_purge

See Also:

  • Kafka::FFI.rdkafkardkafka.h
0x01
RD_KAFKA_PURGE_F_INFLIGHT = 
0x02
RD_KAFKA_PURGE_F_NON_BLOCKING = 
0x04
RD_KAFKA_VTYPE_END =

rd_kafka_producev va-arg vtype constants.

0
RD_KAFKA_VTYPE_TOPIC = 
1
RD_KAFKA_VTYPE_RKT = 
2
RD_KAFKA_VTYPE_PARTITION = 
3
RD_KAFKA_VTYPE_VALUE = 
4
RD_KAFKA_VTYPE_KEY = 
5
RD_KAFKA_VTYPE_OPAQUE = 
6
RD_KAFKA_VTYPE_MSGFLAGS = 
7
RD_KAFKA_VTYPE_TIMESTAMP = 
8
RD_KAFKA_VTYPE_HEADER = 
9
RD_KAFKA_VTYPE_HEADERS = 
10
RD_KAFKA_PARTITION_UA =

Use for partition when it should be assigned by the configured partitioner.

-1
RD_KAFKA_ADMIN_OP_ANY = 
0
RD_KAFKA_ADMIN_OP_CREATETOPICS = 
1
RD_KAFKA_ADMIN_OP_DELETETOPICS = 
2
RD_KAFKA_ADMIN_OP_CREATEPARTITIONS = 
3
RD_KAFKA_ADMIN_OP_ALTERCONFIGS = 
4
RD_KAFKA_ADMIN_OP_DESCRIBECONFIGS = 
5
RD_KAFKA_RESOURCE_UNKNOWN = 
0
RD_KAFKA_RESOURCE_ANY = 
1
RD_KAFKA_RESOURCE_TOPIC = 
2
RD_KAFKA_RESOURCE_GROUP = 
3
RD_KAFKA_RESOURCE_BROKER = 
4
RD_KAFKA_CONFIG_SOURCE_UNKNOWN_CONFIG = 
0
RD_KAFKA_CONFIG_SOURCE_DYNAMIC_TOPIC_CONFIG = 
1
RD_KAFKA_CONFIG_SOURCE_DYNAMIC_BROKER_CONFIG = 
2
RD_KAFKA_CONFIG_SOURCE_DYNAMIC_DEFAULT_BROKER_CONFIG = 
3
RD_KAFKA_CONFIG_SOURCE_STATIC_BROKER_CONFIG = 
4
RD_KAFKA_CONFIG_SOURCE_DEFAULT_CONFIG = 
5

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.features ⇒ Array<String>

Returns the set of features available from librdkafka

Returns:

  • (Array<String>) —

    List of available features



36
37
38
39
40
41
 
# File 'lib/kafka/ffi.rb', line 36

def self.features
  conf = Config.new
  conf.get("builtin.features").split(",")
ensure
  conf.destroy
end

.version ⇒ String

Returns the loaded version of librdkafka

Returns:

  • (String) —

    Version string



29
30
31
 
# File 'lib/kafka/ffi.rb', line 29

def self.version
  rd_kafka_version_str
end

Instance Method Details

#rd_kafka_AdminOptions_new ⇒ Object

AdminOptions



549
 
# File 'lib/kafka/ffi.rb', line 549

attach_function :rd_kafka_AdminOptions_new, [Client, :admin_operation], Admin::AdminOptions

#rd_kafka_AlterConfigs ⇒ Object

AlterConfigs



562
 
# File 'lib/kafka/ffi.rb', line 562

attach_function :rd_kafka_AlterConfigs, [Client, :pointer, :size_t, Admin::AdminOptions, Queue], :void

#rd_kafka_ConfigEntry_name ⇒ Object

ConfigEntry



580
 
# File 'lib/kafka/ffi.rb', line 580

attach_function :rd_kafka_ConfigEntry_name, [Admin::ConfigEntry], :string

#rd_kafka_ConfigResource_new ⇒ Object

ConfigResource



569
 
# File 'lib/kafka/ffi.rb', line 569

attach_function :rd_kafka_ConfigResource_new, [:resource_type, :string], Admin::ConfigResource

#rd_kafka_ConfigSource_name ⇒ Object

ConfigSource



590
 
# File 'lib/kafka/ffi.rb', line 590

attach_function :rd_kafka_ConfigSource_name, [:config_source], :string

#rd_kafka_consume_start ⇒ Object

Legacy Simple Consumer API



440
 
# File 'lib/kafka/ffi.rb', line 440

attach_function :rd_kafka_consume_start, [Topic, :partition, :offset], :int

#rd_kafka_CreateTopics ⇒ Object

Create Topics / NewTopic



593
 
# File 'lib/kafka/ffi.rb', line 593

attach_function :rd_kafka_CreateTopics, [Client, :pointer, :size_t, Admin::AdminOptions, Queue], :void, blocking: true

#rd_kafka_DescribeConfigs ⇒ Object

DescribeConfigs



558
 
# File 'lib/kafka/ffi.rb', line 558

attach_function :rd_kafka_DescribeConfigs, [Client, :pointer, :size_t, Admin::AdminOptions, Queue], :void

#rd_kafka_ResourceType_name ⇒ Object

Resource Type (enum)



566
 
# File 'lib/kafka/ffi.rb', line 566

attach_function :rd_kafka_ResourceType_name, [:resource_type], :string

#rd_kafka_subscribe ⇒ Object

High Level Consumer API



418
 
# File 'lib/kafka/ffi.rb', line 418

attach_function :rd_kafka_subscribe, [Consumer, TopicPartitionList.by_ref], :error_code