# Ansible Playbooks for Apache Kafka in production

## 前言

参考了[confluentinc/cp-ansible](https://github.com/confluentinc/cp-ansible)playbook批量安装部署Apache Kafka 2.2.0. 已在[Github](https://github.com/Junyangz/ansible-kafka)上开源

> 注：集群的zookeeper集群已通过Cloudera Manager安装了，所以不包括Zookeeper的安装部分

## Apache Kafka & Systemd

### Requirements

* Ansible setup on your terminal
* rhel7/CentOS7
* Zookeeper cluster
* Ansible playbook([repo](https://github.com/Junyangz/ansible-kafka))
* offline dist([Apacher Kafka](https://kafka.apache.org/downloads))

### Source tree

![Source tree](https://img.junyangz.com/picGo/20190524102658.png)

> 注：仓库里不含Kafka发行包`kafka_2.12-2.2.0.tgz`,需要自行下载并放置在上图的位置中

* `hosts.yml` 里列出所有的主机角色，包括`Zookeeper`和`Kafka broker`
* `all.yml` 里列出对位于`broker`分组中的主机执行`task`任务

### 理解和使用

运行

```bash
#ansible-playbook -C all.yml #运行之前可以先检查下,检查没问题后再运行
ansible-playbook -i hosts.yml all.yml
```

`roles/kafka-broker/defaults/main.yml`

```
配置安装过程中的所需变量即Zookeeper端口，Kafka broker配置，Systemd等相关内容
```

`roles/kafka-broker/handlers/main.yml`

```
配置完Systemd服务后执行daemon-reload并重启Kafka broker
```

`roles/kafka-broker/tasks/main.yml`

```
任务执行流，创建Kafka用户及用户组，创建Kafka数据目录，分发Kafka发行包，更改权限，拷贝broker配置文件并修改，创建systemd文件并启动Kafka
```

`roles/kafka-broker/templates/`

```
该文件夹下放置了broker的配置文件和Systemd文件，执行的时候将defaults设定的变量写入到文件中
默认配置文件参考了confluent Kafka的配置，请自行修改成自己所需的配置。
```

部署后可以使用 `journalctl -xefu kafka`检查Kafka的运行日志,部署目录/opt/apps/kafka/logs下也有相应的日志文件，此外还可以在Syslog中自行配置Identifier=kafka的日志处理。例如传输到各种监控程序中。

## Reference

* [Ansible Playbooks for Confluent Platform](https://docs.confluent.io/current/tutorials/cp-ansible/docs/index.html)
* [Running Kafka in Production](https://docs.confluent.io/current/kafka/deployment.html#cp-production-parameters)
* [cp-ansible](https://github.com/confluentinc/cp-ansible)
* [Ansible playbooks for Kafka and Zookeeper](https://blog.insightdatascience.com/ansible-playbooks-for-kafka-and-zookeeper-with-ec2-dynamic-inventory-8f317d4d2bfc)
* [HowTo: Organize Ansible Playbook to install, uninstall, start and stop Kafka and Kafka Connect](https://medium.com/@mykidong/howto-organize-ansible-playbook-to-install-uninstall-start-and-stop-kafka-and-kafka-connect-e7250c5def9d)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.junyangz.com/ops/ansible-kafka.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.