代码拉取完成，页面将自动刷新
同步操作将从 src-openEuler/iSulad 强制同步，此操作会覆盖自 Fork 仓库以来所做的任何修改，且无法恢复！！！
确定后同步将在后台操作，完成时将刷新页面，请耐心等待。
From 6911c68b96f3a0fc35b40e5c387da51cd84e5461 Mon Sep 17 00:00:00 2001
From: "Neil.wrz" <[email protected]>
Date: Mon, 13 Feb 2023 03:34:31 -0800
Subject: [PATCH 44/53] doc: add document about support remote ro directory
---
docs/design/README.md | 4 +-
docs/design/README_zh.md | 6 +-
.../design/detailed/Image/remote_ro_design.md | 333 ++++++++++++++++++
3 files changed, 340 insertions(+), 3 deletions(-)
create mode 100644 docs/design/detailed/Image/remote_ro_design.md
diff --git a/docs/design/README.md b/docs/design/README.md
index a9d357b4..cf29c0a1 100644
--- a/docs/design/README.md
+++ b/docs/design/README.md
@@ -34,6 +34,8 @@ This section contains some design documents for users who want to learn more abo
- You can see how the isula search are designed in [image_search_design](./detailed/Image/image_search_design_zh.md)
+- You can see how the remote ro are designed in [remote_ro_design](./detailed/Image/remote_ro_design.md)
+
## Network
- You can see how the cni operator modules are designed in [cni_operator_design](./detailed/Network/cni_operator_design.md).
@@ -54,4 +56,4 @@ This section contains some design documents for users who want to learn more abo
## Volume
-- You can see how the local volume modules are designed in [local_volume_design](./detailed/Volume/local_volume_design.md).
\ No newline at end of file
+- You can see how the local volume modules are designed in [local_volume_design](./detailed/Volume/local_volume_design.md).
diff --git a/docs/design/README_zh.md b/docs/design/README_zh.md
index f51930e4..1f1c94b5 100644
--- a/docs/design/README_zh.md
+++ b/docs/design/README_zh.md
@@ -32,7 +32,9 @@
- 查看 registry 模块的设计文档： [registry_degisn](./detailed/Image/registry_degisn_zh.md) 。
-- 查看 isula search 的设计文档：[image_search_design](./detailed/Image/image_search_design_zh.md)
+- 查看 isula search 的设计文档：[image_search_design](./detailed/Image/image_search_design_zh.md) 。
+
+- 查看 ro目录分离的设计文档： [remote_ro_design](./detailed/Image/remote_ro_design.md) 。
## Network
@@ -54,4 +56,4 @@
## Volume
-- 查看 local volume 模块的设计文档： [local_volume_design](./detailed/Volume/local_volume_design_zh.md).
\ No newline at end of file
+- 查看 local volume 模块的设计文档： [local_volume_design](./detailed/Volume/local_volume_design_zh.md).
diff --git a/docs/design/detailed/Image/remote_ro_design.md b/docs/design/detailed/Image/remote_ro_design.md
new file mode 100644
index 00000000..fee33835
--- /dev/null
+++ b/docs/design/detailed/Image/remote_ro_design.md
@@ -0,0 +1,333 @@
+| Author | 王润泽 |
+| ------ | ---------------------- |
+| Date | 2023-2-13 |
+| Email | [email protected] |
+
+# 1. 方案目标
+目标有两个：
+1. 把isulad当前的layer store里的RO层分离出来，把RW layer和RO layer分开到不同到目录存储。
+2. isulad在运行时如果在相关目录里恢复了正确的镜像数据(image和layer数据), 可实现不重启isulad, 直接使用新恢复的镜像。如果移除当前没有容器正在使用的镜像数据，可实现不重启isulad，isulad更新当前管理的镜像列表去除该镜像。只考虑完全正确的新增和删除。
+
+
+## 1.1 用法说明
+通过源码编译打开编译选项来开启功能，cmake添加`cmake -DENABLE_REMOTE_LAYER_STORE=ON ..`, 然后`make -j`即可。启动iSulad之前，还需要在配置文件`/etc/isulad/daemon.json`里面添加`"storage-enable-remote-layer": true`来打开开关。
+
+# 2. 总体设计
+
+*Modules Dependencies*
+```
+=> New Added Module:
+
+ +===================================+
+ | Remote Supporter Module |
+ +===================================+
+ | |
+ | +-------------------------+ |
+ | | maintainer submod | |
+ | +-------------------------+ |
+ | | global data initer | |
+ | | symbol link maintainer | |
+ | | global data getter | |
+ | +-------------------------+ |
+ | |
+ | +-------------------------+ |
+ | | Supporter submod | |
+ | +-------------------------+ |
+ | | supporter interface | |
+ | | overlay supporter impl | |
+ | | layer supporter impl | |
+ | | image supporter impl | |
+ | | remote refresh thread | |
+ | +-------------------------+ |
+ +-----------------------------------+
+
+
+=> Modified Modules:
+
+ +===================================+
+ | Storage Module |
+ +===================================+
+ | |
+ | +-------------------------+ |
+ | | Image Store submod | |
+ | | Added Functions | |
+ | +-------------------------+ |
+ | | add image in memory | |
+ | | delete image in memory | |
+ | | get image top layer | |
+ | | valid image manifest | |
+ | +-------------------------+ |
+ | |
+ | +-------------------------+ |
+ | | Layer Store submod | |
+ | | Added Functions | |
+ | +-------------------------+ |
+ | | add layer in memory | |
+ | | delete layer in memory | |
+ | +-------------------------+ |
+ | |
+ | +-------------------------+ |
+ | | Driver Overlay submod | |
+ | | Added Functions | |
+ | +-------------------------+ |
+ | | - | |
+ | +-------------------------+ |
+ +-----------------------------------+
+
+
+=> Modules Dependencies:
+
+ +-------------------------+ +---------------+
+ | Supporter submod | | Image Store |
+ +-------------------------+ +----▶| submod ---------+
+ | supporter interface | | +---------------+ |
+ | overlay supporter impl -----+ |
+ | layer supporter impl -----+ |
++----------------+ | image supporter impl | | +----------------+ |
+| storage module |-------▶| remote refresh thread | +----▶| Layer Store | | init +-----------------------+
++----------------+ +-------------------------+ | submod --------+--------▶| maintainer submod |
+ | +----------------+ | +-----------------------+
+ | | ▲
+ | | |
+ | +----------------+ | |
+ | | Layer Store | | |
+ | | submod --------+ |
+ | +----------------+ |
+ | |
+ +------------------------------------------------------------------------+
+ get global data
+```
+
+
+总体来说有两部分的功能：
+- iSulad原有的image storage适配分离的RO目录结构，*分离的RO目录*可用于远程挂载
+- iSulad实例同步内存数据，镜像数据和layer数据*定期更新*，不通过`isula pull` 和 `isula rmi` 等命令，直接通过分离目录里面的数据来更新镜像数据。
+
+*分离RO目录*
+修改前后storage目录结构对比：
+
+```
+old:
+overlay-layer
+├── b703587
+│ └── layer.json
+└── b64792c
+ └── layer.json
+ └── b64792.tar.gz
+
+new:
+overlay-layer
+├── b64792c -> ../RO/b64792c
+├── b703587
+│ └── layer.json
+└── RO
+ └── b64792c
+ └── layer.json
+ └── b64792.tar.gz
+```
+
+以overlay-layers目录为例，创建新layer时，如果是只读层，就把层数据放到RO目录下，在RO上层目录创建软连接指向真实数据。删除layer时需要额外删除软连接。
+
+
+*定期更新*
+定期更新通过启动一个线程周期扫描`overlay`, `overlay-layers`, `overlay-image`这三个目录，通过比较当前时刻与上一时刻的目录差异，来获取镜像和层的删减情况，进而同步isulad的storage内存数据和维护软链接。
+
+```
++---------------------+ +---------------------+ +---------------------+ +-----------------------+
+| refresh thread loop | | overlay remote impl | | layer remote impl | | image remote impl |
++---------------------+ +---------------------+ +---------------------+ +-----------------------+
+ | | | |
+ | refresh start | | |
+ |-----------------------------▶| | |
+ | | overlay dir scan | |
+ | | | |
+ | | to added layers | |
+ | | memory and symlink add | |
+ | | to deleted layers | |
+ | | memory and symlink del | |
+ | | valid overlay layers | |
+ | |---------------------------------▶| |
+ | | next scan | |
+ | | | |
+ | | | |
+ | | | |
+ | | | overlay-layers dir scan |
+ | | check overlay layer ready | |
+ | |◀---------------------------------| to added layers |
+ | |---------------------------------▶| filter invalid layers |
+ | | result | memory and symlink add |
+ | | | to deleted layers |
+ | | | memory and symlink del |
+ | | | valid overlay layers |
+ | | |---------------------------------▶|
+ | | | next scan |
+ | | | |
+ | | | |
+ | | | |
+ | | | | overlay-image dir scan
+ | | | check layers ready |
+ | | |◀---------------------------------| to added images
+ | | |---------------------------------▶| filter invalid images
+ | | | result | memory add images
+ | | | | to deleted images
+ | | | | memory del images
+ | | | |
+ |◀---------------------------------------------------------------------------------------------------|
+ | refresh end | | |
+ | | | |
++---------------------+ +---------------------+ +---------------------+ +-----------------------+
+| refresh thread loop | | image remote module | | layer remote module | | overlay remote module |
++---------------------+ +---------------------+ +---------------------+ +-----------------------+
+
+```
+
+# 3. 接口描述
+
+```c
+// 初始化remote模块里的layer data
+int remote_layer_init(const char *root_dir);
+
+// 初始化remote模块里的overlay data
+int remote_overlay_init(const char *driver_home);
+
+// 清理remote模块的资源
+void remote_maintain_cleanup();
+
+// 启动 定期更新的thread
+int start_refresh_thread(void);
+
+// 创建新layer目录
+int remote_layer_build_ro_dir(const char *id);
+
+// 创建新overlay目录
+int remote_overlay_build_ro_dir(const char *id);
+
+// 删除layer目录
+int remote_layer_remove_ro_dir(const char *id);
+
+// 删除overlay目录
+int remote_overlay_remove_ro_dir(const char *id);
+```
+
+# 4. 详细设计
+分离RO目录的关键在于适配原来的代码逻辑，原先的代码在操作镜像和层的时候，不管是RO层还是RW层，从创建到删除都是在当前目录下进行的，这就是我们额外创建一个软连接的作用:
+- RO目录的作用是为了支持远程挂载
+- 软连接的作用是模拟原来的目录结构
+
+这样以来，image module的逻辑几乎不需要改动，除了以下几点需要注意：
+- 创建和删除的时候需要处理一个额外的资源：软连接，之前只需要关注目录即可，现在如果创建的是只读层，就需要额外创建软连接，如果删除的是只读层，就需要额外删除软连接
+- 以`overlay-layers`目录为例，isulad启动时会以正则规则扫描当前目录下的子目录是否合法，所以需要屏蔽`RO`目录
+
+定时刷新的逻辑如下：
+以`overlay-image`目录的刷新为例，通过维护两个集合`new` 和 `old`, 这两个集合初始都为空，通过扫描目录里面所有的子目录，把合法的image id 加入`new`集合， 通过计算两个集合的差， 在集合`new`里面存在而在集合`old`里面不存在的id则为新增加的镜像， 在集合`old`里面存在而在集合`new`里面不存在的id则为删除的镜像。处理新增加的镜像还需要额外的一个判断，就是判断镜像的层数据是否已经加载，如果没加载则该镜像本轮不加载。`overlay-layers` 和 `overlay` 目录的处理逻辑类似。
+
+
+*可能的使用场景*
+一个可能的使用场景就是通过远程文件共享(nfs)让多台启动的isulad实例共享某些只读的数据；具体来说，在两个host A和B上都启动了iSulad, 如果A pull或者load了镜像busybox, 那么B上的isulad同样可以使用这个镜像。
+
+```
+operations:
+
++--------------------+ +--------------------+ +--------------------+
+| isula pull busybox | | without pull | | without pull |
+| isula pull nginx | | isula run busybox | | isula run ngxinx |
+| isula pull ... | | isula run ... | | isula run centos |
++--------------------+ +--------------------+ +--------------------+
+ | | |
+ ▼ ▼ ▼
++======================+ +======================+ +======================+
+| isulad on Host A | | isulad on Host B | | isulad on Host C |
++======================+ +======================+ +======================+
+| image store module | | image store module | | image store module |
++----------------------+ +----------------------+ +----------------------+
+| refresh thread off | | refresh thread on | | refresh thread on |
++----------------------+ +----------------------+ +----------------------+
+| local rw | remote ro | | local rw | remote ro | | local rw | remote ro |
++----------------------+ +----------------------+ +----------------------+
+ | | |
+ | enable nfs | mounted on | mounted on
+ ▼ ▼ ▼
+ +=====================================================================+
+ | nfs directory over network |
+ +=====================================================================+
+ | image store |
+ +---------------------------------------------------------------------+
+ | image | image | image | image | image |
+ | busybox | nginx | my-app | ubuntu | centos |
+ | 4MB | 100MB | 1.2GB | 5MB | 6MB |
+ +---------------------------------------------------------------------+
+ | layers store |
+ +---------------------------------------------------------------------+
+ | layer | layer | layer |
+ | 05c361054 | 8a9d75caad | 789d605ac |
+ +---------------------------------------------------------------------+
+```
+
+*同步问题*
+共享资源并发使用发生竞争的条件是：`two process access the same resource concurrently and at least one of the access is a writer`。这里的共享资源有：
+
+```
++============================+ +=====================================+
+| Sharing Resource | | Storage |
++============================+ +=====================================+
+| read-only overlay layers | mounted | /var/lib/isulad/overlay/RO |
++----------------------------+ ======▶ +-------------------------------------+
+| reald-only layers metadata | shared | /var/lib/isulad/overlay-layers/RO |
++----------------------------+ +-------------------------------------+
+| reald-only images | | /var/lib/isulad/overlay-images |
++----------------------------+ +-------------------------------------+
+
+```
+而分布在不同host上的isulad进程通过网络共享这些资源，如果不考虑新增删除的情况，不会出现资源竞争，所有的节点都是reader。
+
+对于主节点pull镜像，其他节点使用镜像的情况，主节点是writer，其他节点是reader。这时候可能出现的问题是主节点pull镜像的流程没有完全结束，但是其他节点开始使用这个不完整的镜像。对于这个问题的解决方案是通过扫描image目录来新增镜像，通过这种方式能确保一定该新增镜像的信息完整。
+
+```
++---------------------+ +--------------------+ +-----------------------+ +-----------------------+
+| registry | | image module | | layer store module | | driver overlay module |
++---------------------+ +--------------------+ +-----------------------+ +-----------------------+
+ | | | |
+ | | | |
+ | registry_pull | | |
+ | fetch_manifest | | |
+ | check reuse and fetch | | |
+ | +----------------+ | | |
+ | | register_layer | | | |
+ | +----------------+ | | |
+ |-----------------------------------------------------------------▶| |
+ | | | layer_store_create |
+ | | |--------------------------------▶|
+ | | | | driver_create_layer
+ | | | | +--------------------+
+ | | | | | setup overlay dir |
+ | | | | +--------------------+
+ | | | | driver_create_layer done
+ | | |◀--------------------------------|
+ | | | +------------+ |
+ | | | | save layer | |
+ | | | +------------+ |
+ | | | layer create done |
+ |◀-----------------------------------------------------------------| |
+ | all layer setup | | |
+ | +----------------+ | | |
+ | | register image | | | |
+ | +----------------+ | | |
+ |-------------------------------▶| | |
+ | | storage_img_create | |
+ | | set img top layer | |
+ | | img create | |
+ | | +------------+ | |
+ | | | save image | | |
+ | | +------------+ | |
+ | | create image done | |
+ |◀-------------------------------| | |
+ | pull img done | | |
+ | | | |
+ | | | |
++---------------------+ +--------------------+ +-----------------------+ +-----------------------+
+| registry | | image module | | layer store module | | driver overlay module |
++---------------------+ +--------------------+ +-----------------------+ +-----------------------+
+
+```
+
+至于主节点删除镜像的情况，主节点是writer，其他节点是reader，可能出现的情况是其他节点还有容器在使用镜像的时候，镜像被删除，但是根据需求场景暂不处理这种情况。其他的处理与新增镜像相同，依然以image dir作为入口，扫描发现删除的镜像。删除时需要关注layer和overlay目录下的软链接。
--
2.25.1
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
马建仓 AI 助手