elasticsearch向量数据库服务搭建

elasticsearch_0">elasticsearch向量数据库服务搭建

文章目录

elasticsearch向量数据库服务搭建
- - 一、es安装步骤
  - 二、es可视化kibana安装步骤
  - 三、安装elasticsearch-head
- 四、数据测试，kibana脚本，操作es
- 五、向量数据测试，kibana脚本，操作es
- 参考资料

一、es安装步骤

1、创建网络

docker network create es-net

2、拉取镜像

docker pull elasticsearch:8.15.0

docker pull docker.elastic.co/elasticsearch/elasticsearch:8.15.0

3、创建挂载目录

mkdir -p /home/data/elasticsearch/config
mkdir -p /home/data/elasticsearch/data
mkdir -p /home/data/elasticsearch/logs
mkdir -p /home/data/elasticsearch/plugins

4、赋予权限

chmod -R 777 /home/data/elasticsearch

5、创建配置（可选项）


vi /home/data/elasticsearch/config/elasticsearch.yml

# 可访问IP
cluster.name: "my-es"
network.host: 127.0.0.1

# 集群的节点列表（ElasticSearch8新配置）
# discovery.seed_hosts: ["es-ip:9300"]

# 跨域开启密码
http.cors.enabled: true
http.cors.allow-origin: "*"
# http.cors.allow-headers: Authorization
xpack.security.enabled: false

6、创建脚本

vim startEs.sh

sudo docker run -d \
--restart=always \
--name elasticsearch \
--network es-net \
-p 9200:9200 \
-p 9300:9300 \
--privileged \
-v /home/data/elasticsearch/data:/usr/share/elasticsearch/data \
-v /home/data/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
-e "discovery.type=single-node" \
-e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \
elasticsearch:8.15.0


docker exec -it elasticsearch /bin/bash

# 关闭 密码安全验证

echo 'xpack.security.enabled: false' >> elasticsearch.yml

# 重启容器
sudo docker restart elasticsearch

7、测试Elasticsearch是否安装成功

http://192.168.198.197:9200

8、容器操作

# 进入容器
docker exec -it elasticsearch /bin/bash

sudo docker exec -it elasticsearch /bin/bash

# 重启容器
docker restart elasticsearch

二、es可视化kibana安装步骤

1、拉取镜像


docker pull kibana:8.10.2

2、创建配置（可选项）

vi /home/data/kibana/config/kibana.yml

# 主机地址，可以是ip,主机名
server.host: 0.0.0.0
# 提供服务的端口，监听端口
server.port: 5601
# 该 kibana 服务的名称，默认 your-hostname
server.name: "bolei-kibana"
server.shutdownTimeout: "5s"
 
#####----------elasticsearch相关----------#####
# kibana访问es服务器的URL,就可以有多个，以逗号","隔开
elasticsearch.hosts: [ "http://192.168.198.197:9200" ]
monitoring.ui.container.elasticsearch.enabled: true

3、启动镜像


sudo docker run -d \
--name kibana2 \
-e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 \
--network=es-net \
-p 5601:5601  \
kibana:8.6.0

4、容器炒作

# 进入容器
docker exec -it kibana2 /bin/bash

sudo docker exec -it kibana2 /bin/bash

# 重启容器
docker restart kibana1

5、访问地址

http://192.168.198.197:5601

elasticsearchhead_162">三、安装elasticsearch-head

直接使用浏览器插件Multi ElasticSearch Head

使用教程：https://blog.csdn.net/qq_50854662/article/details/135967448

四、数据测试，kibana脚本，操作es

dev tool控制台处理

# 创建索引
PUT goods_index
        
# 查询索引
GET goods_index

# 添加映射
PUT goods_index/_mapping
{
  "properties":{
    "age":{
      "type":"integer"
    },
    "name":{
      "type":"keyword"
    },
    "desc":{
      "type":"text"
    }
  }
}
        
# 创建索引并且添加映射
PUT goods_index1
{
  "mappings": {
    "properties": {
       "age":{
      "type":"integer"
      },
      "name":{
        "type":"keyword"
      },
      "desc":{
        "type":"text"
      }
    }
  }
}

# 添加文档，指定id
PUT goods_index/_doc/1
{
  "name":"张三",
  "age":34,
  "desc":"北京人深圳来"
}

# 添加文档，不指定id
POST goods_index/_doc
{
  "name":"李四",
  "age":25,
  "desc":"四川妹子真辣"
}
        
# 修改文档
PUT goods_index/_doc/1
{
  "name":"张三1111",
  "age":34,
  "desc":"北京人深圳来"
}

# 删除文档
DELETE goods_index/_doc/1

# 查询文档 指定id
GET goods_index/_doc/56YoJ5IBpXr2Jk-kqx2q
# 查询所有文档
GET goods_index/_search

# 删除索引
DELETE goods_index

五、向量数据测试，kibana脚本，操作es

1. 创建索引

PUT /vector_index
{
  "mappings": {
    "properties": {
      "my_vector_field": {
        "type": "dense_vector",
        "dims": 128
      },
      "my_text": {
        "type": "text"
      }
    }
  }
}

2. 写入数据
PUT /vector_index/_doc/1
{
  "my_text": "示例文本",
  "my_vector_field": [0.1, 0.2, ..., 0.128]
}

3. 搜索查询
POST /vector_index/_search
{
  "query": {
    "script_score": { // 使用脚本评分来计算相似度得分
      "query": {
        "match_all": {}
      },
      "script": {
        "source": "cosineSimilarity(params.queryVector, 'my_vector_field') + 1.0", // 计算查询向量与存储向量的余弦相似度，并加 1.0 以确保得分非负
        "params": {
          "queryVector": [0.1, 0.2, ..., 0.128]
        }
      }
    }
  }
}