Docker用法进阶
实用工具
15 分钟 读完 (大约 2181 个字)

Docker用法进阶

泡泡机器人SLAM与你一同成长❤

使用之前先看看本地都有哪些镜像和容器。

1
sudo docker ps -a

查看现有容器以及状态。

1
2
3
4
CONTAINER ID   IMAGE                                              COMMAND                  CREATED         STATUS                   PORTS                                       NAMES
0ac8c062fa10   pointclouddl:segmentation                          "/bin/sh -c 'service…"   8 minutes ago   Up 8 minutes             6006/tcp, 50051/tcp, 0.0.0.0:2224->22/tcp   brave_kilby
cd8b9c9c5b07   stereolabs/zed:3.0-gl-devel-cuda10.0-ubuntu18.04   "/bin/bash"              4 weeks ago     Exited (0) 5 hours ago                                               jolly_feynman
af07110a5157   stereolabs/kalibr                                  "/ros_entrypoint.sh …"   4 weeks ago     Exited (0) 4 weeks ago                                               modest_leakey

e.g.现有用于laser slam的镜像, 设置了界面，ssh，基于amazinghao/ubuntu:graph_slam_clean镜像。

1
2
CONTAINER ID   IMAGE                                COMMAND          CREATED         STATUS         PORTS                            NAMES
fe7323d95d8f   amazinghao/ubuntu:graph_slam_clean   "./startup.sh"   9 seconds ago   Up 7 seconds   22/tcp, 0.0.0.0:5900->5900/tcp   laser_slam_env

一些容器内部设置必须在创建的时候就设置好，不然就只能重新创建。比如端口映射，文件夹挂载，设置别名，虚拟内存设置，时区设置等。

使用进阶

端口映射

列举几个需要用到网络端口的情况：

  1. 使用Docker环境进行深度学习网络训练时，需要将对应的Pytorch日志通过浏览器的方式打开查看，但是Docker是没有图形化界面的，因此需要通过端口映射的方式将Docker环境中的端口映射到本地。在本地打开浏览器查看。
  2. 使用Docker环境使用ROS运行程序，需要RVIZ进行可视化。
  3. 使用Marsim等仿真环境，需要一个可视化界面进行渲染。
1
sudo docker run -it -p 5900:5900 paopaorobot/ubuntu-xfce-vnc
  • -p，容器内部的 5900 端口映射到本地主机的 5900 端口上。访问本地的
  • 可以同时开启多个端口映射。
  • 容器内的网络是隔离的，其中的127.0.0.1指向容器本身。也可以通过指定--network-host共用宿主机网络。

查看端口映射

查看一个容器中已经映射的端口

1
docker port containerId
  • 即可查看容器端口映射情况。

文件夹挂载

Docker中是固定了环境的，其中不包含数据和代码，避免容器关闭时代码和数据丢失。容器中修改相关的代码，在本地同时会被更新。

1
sudo docker run -i -t -v /Data:/data amazinghao/ubuntu
  • -v，将本地的Data路径挂载到容器中的/data路径下。
  • 同样可以同时挂载多个路径。

拷贝文件

这里不建议拷贝数据或者代码到容器中，Docker设计时是环境和代码分离的，代码和数据可以放在本地。容器只负责环境管理。

1
sudo docker cp host_path containerID:container_path
  • 从主机复制到容器
1
sudo docker cp containerID:container_path host_path
  • 从容器复制到主机

容器打包成镜像

将现有容器环境打包成镜像，方便后续使用或跨平台使用

Note that the code file and data file should not be packaged into the image and only environment should be in there

1
sudo docker commit -a "amazinghao" -m "for point cloud deep learning environment" dc1e2f6fcf8d pointclouddl:segmentation
  • -a, author info
  • -m, commit message
  • dc1e2f6fcf8d, container id, which is unique
  • the last parameter is the new image name and its tag. The new image name should be lowercase

在推送到云端时，需要和云端仓库中的镜像名字保持一致。那么可能就会用到tag命令。 例如:将462f5c932e5c重命名为amazinghao/ubuntu:dev

1
docker tag 462f5c932e5c amazinghao/ubuntu:dev
  • dev即为新的标签，这样做将会创建一个新的镜像。

也可以将hdl_graph_slam:graph_slam_updated重命名为amazinghao/ubuntu:graph_slam_updated

1
docker tag hdl_graph_slam:graph_slam_updated amazinghao/ubuntu:graph_slam_updated

再开始推送即可。

设置别名

设置容器的别名

1
docker run -itd --name=laser_slam_env
  • 注意别名的设置尽量放置在命令行中靠前的位置。

环境变量设置

1
sudo docker run -i -t -e TZ="Asia/Shanghai" -e SSHPW=xxx amazinghao/ubuntu bin/bash
  • -e TZ="Asia/Shanghai"，设置时区，使得容器内部的时间和宿主机时间保持一致，提升日志的可读性。
  • -e SSHPW=xxx，设置密码。
  • bin/bash，这个表示容器启动时执行的命令，必须有-it才能生效。 这样容器在退出时容器转为后台运行。

虚拟内存设置

常见的情况是，在深度学习环境Pytorch的Dataloader中num_works设置较大时，太多数据加载到内存中时就会报错。e.g.

RuntimeError: DataLoader worker (pid 43257) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.

1
sudo docker run -i -t -shm_size=32g amazinghao/ubuntu
  • -shm_size=32g

检查镜像

检查镜像中的现有环境，包括镜像生成时的一些设置等

1
sudo docker image inspect pointclouddl:segmentaton

检查容器

同理检查容器的内部细节

1
docker inspect containerID
  • 它会返回一个 JSON 文件记录着 Docker 容器的配置和状态信息。

导出镜像

export & import

1
2
3
4
# export current image
docker export image_id > current.tar
# import current image
docker import image_name < current.tar

save & load

1
2
3
docker save 0fdf2b4c26d3 > hangge_server.tar
docker save -o images.tar postgres:9.6 mongo:3.4
docker load < hangge_server.tar

​ Differences from image sizes, image rename, multiple-containers into one single image, application scenarios, more detail could be seen at here

使用Dockerfile构建进行

可以先参考这里，我一般都用官方镜像，然后手动创建。主要涉及到Dockerfile内部的一些命令的使用，可以配合xfce的相关内容构建，又可可视化又可深度学习的环境。

使用实例

New a container for SLAM

The images for SLAM are mostly built from the paopaorobot/ros-vnc, so the visualization module is embedded into the image. Any thing related to the visualization could be set as you wish.

1
2
3
4
5
6
7
8
9
# open a new container using the xfce&vnc for visualization, host port is 5900, host mount dir is /Data, extra setting is the resolution which is 1080P
sudo docker run -i -t -p 5900:5900 -v /Data:/data -e RESOLUTION=1920x1080 paopaorobot/ubuntu-xfce-vnc
# similar to the upper one, amazinghao/ubuntu is build based on the ros-vnc image
sudo docker run -i -t -p 5900:5900 -v /Data:/data -e RESOLUTION=1920x1080 amazinghao/ubuntu
# similar to the upper two
sudo docker run -i -t -p 5900:5900 -v /Data:/data -e RESOLUTION=1920x1080 paopaorobot/ros-vnc:latest
# windows docker container creation commond
# Note that the alias name should be placed frontier and the dir mount should be carefully set
docker run -itd --name=laser_slam_env -p 5900:5900 -e SSHPW=usmn9a -v /D/data/Publicdatasets/:/data -v /E/Vs15WorkSpace/catkin_ws_lidar:/workspsace_slam  -e RESOLUTION=1920x1080 amazinghao/ubuntu:graph_slam_clean

New a container for Deep learning with CUDA

1
sudo docker run -itd --gpus all --privileged=true -v /home/amazinghao/Data:/data-lyh -v /home/amazinghao/PointSeg:/pointcloud_seg_code -p 2224:22 -e SSHPW=xxx pointclouddl:segmentation
  • --gpus all , only this the Nvidia driver could be accessed, check this
  • --privileged=true , to obtain the 'real' permission to mount file or others. This one is very important
  • -v , mount the host dir or file to the container. The dir should begin with '/' and connected by ':'
  • -e SSHPW , set the ssh connect password
  • -p , two ports are opened for both ssh or visualize. Only if the visualization module is activated, the VNC-Viewer is available.
  • -d , the container should be closed manually, and the local command will not convert to the container.

After using run to new container

Using "run" to new a new container and once you quit the container, the container will immediately be closed. The container that you new right now will stay started and STATUS will be like: Up 8 minutes. Only thing you should do, if you are using VNC-Viewer for visualization, is as followings:

1
sudo docker start 0ac8c062fa10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
amazinghao@xixi-ubuntu:~$ sudo docker attach 0ac8c062fa10
root@0ac8c062fa10:~# nvidia-smi
Mon Jan 18 04:55:55 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:01:00.0  On |                  N/A |
| 24%   46C    P5    18W / 250W |    271MiB / 11175MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+
root@0ac8c062fa10:~# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

When you exit the container, the correspond container's STATUS will be some thing like: Exited (0) 4 seconds ago. But if you only close vscode (connnet by Remote Container), the container will not be closed.

Also you can using exec to new a container

The container will not be stopped, while exec command.

1
2
docker exec -it af07110a5157 /bin/bash
docker attach previous_container_id

New a container for zed sdk

1
2
# Open a container for zed stereo camera. The image contains a visualization module using "X11"
sudo docker run --gpus all -it --privileged --network=host stereolabs/zed:3.0-runtime-cuda10.0-ubuntu18.04

ZED docker display support

1
xhost +si:localuser:root
1
2
sudo docker run --gpus all -it --privileged -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix stereolabs/zed:3.0-gl-devel-cuda10.0-ubuntu18.04
sudo docker attach containerID
  • The camera connection can be verified using lsusb:
1
lsusb -d 2b03: -vvv
  • in case of replug/hotplug, mount all the dev folder
1
docker run --gpus all -it -v /dev:/dev --privileged stereolabs/zed:3.0-runtime-cuda10.0-ubuntu18.04

problem

  • the ZED_Depth_Viewer didn't work somehow.

zed_ros_wapper & cuda for a ros bag

add a ros melodic to zed-sdk and build the zed_ros_wapper in it ID:cd8b9c9c5b07

stereolabs/zed:3.0-gl-devel-cuda10.0-ubuntu18.04

New a container for kalibr

container ID:af07110a5157 is the one which rebuild the ros_kalibr_system ubuntu version:16.04 ros version:kinetic

--show-extraction

------ copyright ------

本博客所有文章除特别声明外，均采用 许可协议

title:Docker用法进阶

author:AmazingHao

link:http://whu-lyh.github.io/blogs/2021/01/18/Docker%E7%94%A8%E6%B3%95%E8%BF%9B%E9%98%B6/

publish time:2021-01-18

update time:2023-12-10

#
ROS basic usage and manager
VS开发Tricks
| visits
关注我

链接

分类

标签云

B-Spline CloudCompare Docker FFMPEG Geometics Geometry Hexo LiDAR_SLAM Linux MarkDown Mathemetics OSG PCL PaperReview Point Cloud Pytorch ROS VS WindowsScript Zotero 服务器 鱼眼图像

最新文章

related posts

归档

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×